The most annoying thing for me in B4A5 is the way info boxes or alerts pop up. They do not follow the standard windows design UI. Furthermore, fading out an important message box without clicking is a big no-no!
Well, it's your product of course, but I cannot understand why you reprogrammed the whole interface to look like Visual Studio and did not do the same thing with the message boxes... If you want your product to look like Visual Studio, I vote to use the Visual Studio popups/messageboxes as well... but that's just my idea.
I also prefer Windows-standard dialogs, but if you keep the custom one, it would be useful to let the dialog grow wider to make it easier to read longer messages.
I can live with the current one, the only drawback is when you hit compile and move away to do something else that you don't see the error and need to compile again.
Use F5 instead of Alt + 3. In debug mode it hides the compilation window after 3 seconds if there are no errors. If there is any error the dialog will stay.