Virtual machines and speed

Q: "Isn't a program running on a virtual machine slower than a native-code program, because the JVM program has to constantly decode the bytecodes?"

A: Not necessarily, in theory. Of course, a totally naive implementation, like the following pseudocode:

    byte[] bytecodeArray = readClassFile();
    while (!shouldHalt()) {
        Instruction current = getNextInstruction( bytecodeArray );
        current.execute();
    }

would be dog slow. However, a good JVM works nothing like this. Among other things, it will use a just-in-time compiler, or JIT. Unlike the Java front-end compiler (javac), the JIT:

• Runs immediately before the bytecode is loaded and executed---hence, "just in time".
• Optimizes and compiles from bytecode into native code on the underlying machine.

Hence, once the initial cost of loading and JIT-compiling the bytecode is done, the code can theoretically run as fast as (or faster than) code that was natively compiled ahead of time. Given a wizardly JVM, Java will run at speeds equal to native code.

In practice, the following constraints currently make JIT-compiled code, all else being equal, run slightly slower than code from a good native-code compiler for, e.g., C++:

• Java is a young language; writing good JITs/JVMs is a major engineering task. Only in the past couple of years have decent JITs been publicly available.
• JITs operate under severe time constraints---the user usually does not want to wait too long for code to compile every time (s)he runs a program! This limits optimization opportunities.

Making Java code run fast, and making JITs effective, is a major area of current research.

By the way, take Java vs. C++ benchmarks with a grain of salt. Benchmarks cannot separate performance loss due to JVM overhead from performance loss due to the following features:

• Java is garbage collected, and most Java implementations do not use the fastest, cutting-edge garbage collectors.
• The design of the Java standard libraries (e.g., String, and the collection classes) was optimized for rich functionality, programmer convenience, and conceptual cleanness. The C++ standard library design was optimized for speed.
• All Java instance methods are virtual, and good Java programmers make heavy use of multiple interface inheritance. Current Java implementations typically do not optimize these features as aggressively as they could be optimized.