I ran into a similar situation many years ago, when I was trying to write a software synthesizer using Visual Basic (version 4 at the time). The big problem is that if you're doing sample-by-sample processing of audio data in a loop (like doing pixel-by-pixel processing of images) and your chosen language's compiler can't compile to a native EXE or inline calls, then you end up suffering the performance hit of function calls that have to be made for each sample (or pixel). In many applications you're not making a lot of function calls and the overall performance hit is negligible, but when you're doing something where you're making hundreds of thousands or even millions of calls per second, you're screwed by the overhead of the function calls themselves - without there being any other sort of inefficiency going on.

In my case, I eventually offloaded the heavy sample processing to a compiled DLL I wrote in C, and I was able to keep using Visual Basic for what it did really well, which was quickly building a reliable Windows GUI.