Computer virtualization during the last five years has spread to standard x86 based servers and personal computers. Follow along in this article as we explore some virtual machine configurations running Java and various settings that affect its performance.

Virtual Machines

For those that might not be familiar with virtual machines, we’ll start with a short introduction A virtual machine (or “VM” for short) is essentially a simulated computer that runs on a real computer (the “host” machine) Several VMs can be run simultaneously on a host computer allowing the user to have several different operating systems (or several copies of the same one) running on the same hardware Using VMs can allow you to run several computers with the hardware expense of only having one computer By using VMs appropriately you can run several server applications at once (each in its own VM) without the fear that failure in one of the applications will bring down your entire server Software running on a VM does not know that it is not running on real hardware; the VM environment looks and acts like a real computer.

Two Kinds of VM Hosts

There are currently two kinds of VM software: one that installs on top of an OS, and one that doesn’t need an OS in order to configure and run VMs. For either type of VM software, the OSs that run in the VMs are called Guests. The first kind of VM software is said to run on a Host OS. This kind of VM software makes use of the Host OS services, such as file system services and network services, to provide virtual hardware (see Figure 1).

               Figure 1: Host OS – VM Software – Guest OS Configuration

The second kind of VM software is called a Hypervisor. A hypervisor does not require an OS to be pre-installed in order to run (see Figure 2). It has built-in basic hardware support that it uses to virtualize hardware for VMs. Hypervisors typically support only the most common hardware, and even then, only for the basic subsystems such as disk, network, and video interface. Another difference between a hypervisor host and an OS-based Host is that hypervisors sometimes provide APIs that allow guest OSs to recognize when they are running in VMs and communicate with the host – thus allowing certain types of activities to be accelerated.

                       Figure 2: Hypervisor – Guest OS Configuration

How Virtualization Works - Virtualized Hardware

VM technology works by mimicking certain parts of a computer’s hardware in software. And it mimics it in such a way that multiple virtual machines can be run simultaneously on the real hardware (the host computer). The microprocessors used in PCs can be programmed so that they trap certain activity. Virtual machine software sets up a PC’s microprocessor to trap access to certain hardware and microprocessor resources that need special handling if more than one OS is to be run simultaneously on the PC. Anytime software running in a VM tries to access resources that need to be shared between multiple OSs, the VM software catches these accesses and reroutes them to appropriate simulated hardware or arbitrates their use between the OSs. For example, when an OS running in a VM tries to access the virtual hard disk, its disk driver must read and write certain hardware ports to accomplish this. These hardware port accesses are filtered and trapped by the VM software which, in turn, uses calls to the host computer to read and write blocks in a file on the host computer’s filesystem – in this way mimicking a hard disk.

VM Performance and Overhead

As you can imagine, VMs tend to have some overhead associated with them. Trapping, arbitrating, and emulating hardware access in software is inherently slower than using the hardware itself. But this extra overhead only occurs when the software running in a VM does something that causes a trap -- typically less than 1% of the time. Normally (the other 99% of the time) the guest software isn’t doing anything that causes traps so it runs at regular speed, executing directly on the hardware.  However, when a trap finally does occur, it can take 10+ times as long as the real hardware to process and perform the ensuing emulation. Thus the performance penalty for running in a VM tends to be 10% to 20%, depending upon the applications and OS involved.

Java Virtual Machine (JVM)

A Java Virtual Machine or JVM is actually quite different than the virtual machines that were discussed in the previous sections of this article A JVM provides a virtual computing platform upon which the Java programming language can run regardless of the underlying hardware/OS combination The JVM emulates this virtual computing platform entirely in software A VM from the previous section is more like an exact duplicate of the underlying hardware while a JVM is in no way a duplicate of the underlying hardware, but rather its own distinct computing platform When a Java application runs, the JVM interprets each of the Java application’s instructions and performs the equivalent on the underlying hardware / OS This is different than a VM where most of the time, native code (including the guest OS) runs on the real hardware

Unlike in the VM diagrams above, a JVM does not have a guest OS that runs within it that then runs the Java application A Java application runs directly on the JVM (see Figure 3) The JVM does not provide a number of things that a normal OS environment would Instead it leaves these to be supplied by OS-specific Java libraries

                       Figure 3: The Java Virtual Machine

In order to run a Java application in a JVM, the Java source code is first compiled into “bytecode”, which is equivalent to machine instructions on a real machine. The bytecode is then interpreted by the JVM. Due to software doing the interpreting, Java bytecode is inherently slower than real code executing on real hardware The advantage of Java bytecode is that it can run on any machine with a JVM

For each different computer architecture type and for each OS on each different computer type, a JVM must be implemented Due to most JVMs being written in C/C++, this simply means doing a recompile on the platform it is intended for This is very different from a VM which is targeted to a specific processor and hardware platform and requires extensive understanding of the inner workings of the processor and computer architectural issues Creating a VM that runs on new computer architecture is a long and expensive process.

Java Virtual Machine Performance

In theory, a person could implement a JVM in hardware, thus creating a “JRM” – or Java “Real” Machine upon which the Java bytecode could execute directly This would provide optimal speed Since Java is often used in web based content and has to execute on any machine used to browse the web, having a JRM doesn’t make a lot of sense

Instead of trying to overcome the slowness of interpreted bytecode by creating specialized hardware, many JVM implementations convert sections of bytecode to native code the first time the section of bytecode is run and cache that converted section of code in case it is run again If during the same run of the Java application that section of bytecode gets run again, the native code is used in its place, thus running at full speed on the hardware directly Of course this all takes part of the JVM’s memory, leaving less for the Java application.

Once the run of the Java application ends, all of the caching of native code sections is lost The next time that application runs, it will have to be reinterpreted / converted to get the benefit of direct machine execution Therefore, due to its interpreted nature, Java runs at best marginally slower than native code and in many cases, it can be many times slower than native code.

JVM and VM Similarities

One thing that a JVM and a VM have in common is that they provide execution isolation – meaning that what happens inside each respective virtual machine does not affect stability of other virtual machines In other words, if a Java program goes bad it only takes down its JVM – not the machine it is running on, nor any other Java applications running in other JVMs on that machine In like manner, if software running a VM does something that crashes that VM, only that VM is affected Other VMs running on the same machine keep right on going The machine itself and any host OS and services it provides keep right on going as well This idea of isolation is an important part of virtualization both for JVMs and VMs.

Another similarity between VMs and JVMs is that they both require the host to provide certain support But the ways and level at which they do this are very different For example both a VM and a JVM require disk storage provided by the host But a VM requires virtual hard disk / block device storage to be provided by the host A JVM requires storage in the form of a file system, not just raw hard disk/block storage A VM requires virtual memory from its host, but can perform memory swapping if necessary to emulate a larger memory pool than is virtually present A JVM uses memory from its host to provide a memory heap and does garbage collection on it But when the JVM has exhausted its heap: game over!  It does not play virtual memory games to provide more memory than what it has been given by the host The underlying host may use virtual memory to back the heap, but the JVM is completely unaware of any of that In essence, a JVM relies on the host OS a lot more for various OS level services, while a VM relies on the host for limited types of hardware emulation – not OS services.

Java in a VM

Now that we know what both a VM and a JVM are, we can easily see that a JVM can run either on an OS running in a real computer or in a VM running a guest OS We should also recognize that the JVM should act and run the same regardless of running in either environment -- it won’t know any difference The following diagram in Figure 4 illustrates this.

                       Figure 4: JVM runs on either an OS or a VM running a guest OS

The same is true of a hypervisor-based VM system (see Figure 5).

                       Figure 5: A JVM running on a hypervisor based VM system

In the diagrams above, if the JVM and Java applications are all the same, then all of the Java applications should act the same as if they were running in a non-VM environment – with one exception: the performance of the Java applications running in the VMs will tend to be slower than the one running on the host OS.

Optimizing the VM environment for running Java applications

In order to get the most out of a Java application running in a VM we now explore various optimization methods that deal not with optimizations within the code, but with optimizations of the surrounding environment, including the JVM, the VM, and the host OS.

To do this we need a Java application that we can use as a benchmark We then will need to determine the baseline or normal speed at which that Java application runs on a regular (non-VM) system From there we will repeatedly run the benchmark while varying different VM, OS, and JVM settings in an effort to determine what gives optimal (closest to non-VM) performance.

For our Java application there happens to be a well developed, readily available Java benchmark application named SPECjvm®2008 It provides an easy-to-use overall score in ops/m (operations per minute) The benchmark runs in such a way as to exclude dependencies on network, video, file system, and disk speeds This allows us to have much more deterministic runs, with the delta in performance being a good measure of the effect of the settings changes we make

Disclaimer: The results of the SPECjvm®2008 runs mentioned in this article are NOT comparable to other SPEC® runs since these are not official runs Most of the benchmark runs used in the testing were “compliant” runs but are not considered “official” For further information regarding SPEC® policies on publishing run results, please visit the SPEC® web site, www.spec.org.

For our Java Virtual Machine the readily available Sun JVM was used This JVM normally runs in “client” mode, but in testing it was quickly discovered that running the benchmark with the JVM’s “-server” setting made a substantial performance improvement So, most of the tests were run using this setting Note that this was not an attempt to benchmark the JVM – and the results are not meant to indicate anything about the relative performance of the JVM since it is a constant in all of the tests The Sun JVM was used simply because it was a common, easily available, and well known JVM One other note about the JVM settings is that the heap was always set for the minimum and the maximum to be the same amount In other words, the JVM always started with the entire amount of heap memory pre-allocated.

The host system software was setup two different ways – one where a hypervisor was used as the host for the guest VMs, and one where a host OS was used with VM software running on it This was done to determine optimal settings for either VM host environment, not to directly compare the two For the hypervisor setup, VMware ESXi by VMware Inc. was used For the setup with a host OS, Microsoft® Windows Server® 2003 (32-bit) running VMware Server was used.

In the VMs the guest OS used in all of the tests was Microsoft® Windows Server ® 2003 (32-bit) And once again, the benchmark results are not meant to indicate anything about the relative performance of the OS, since once again it is a constant in all of the tests It is simply a readily available, very commonly used server OS.

For the hardware platform, a fairly new motherboard was chosen, along with a quad-core AMD Phenom™ processor, and 4GB of DDR2 800MHz RAM The other components such as disk drive, video card, and network adapters were common performance peripherals, but since the benchmark doesn’t measure them in any way, their details don’t need to be included.

In the end, the specific benchmark numbers themselves aren’t of interest as much as the deltas between the runs This allows us to determine the influence that various settings have on the overall performance.

Settings and Optimizations

The next sections outline what settings are applied in an attempt to achieve optimal performance of the Java benchmark in the VM environment All the scores shown will be rounded to whole numbers of SPECjvm®2008 ops/m And for those wondering about such things, most of the runs referenced here were SPEC® “compliant” runs The few that were labeled “not compliant” were still labeled “valid”.1

The benchmark was run from a command prompt using a command line similar to:

java.exe -server -Xms512m -Xmx512m -jar SPECjvm2008.jar -bt 2 -pf runprops.txt

The value “512” was modified from time-to-time in order to vary the JVM heap size and the “-bt 2” being changed to force the benchmark to run with a specific number of threads (without specifying the “-bt” option the benchmark defaults to using the same number of threads as processors/cores) The “-pf” option simply specified the file that contained the other textual information that was to be included in the results for the run, such as the motherboard type, the amount and type of RAM in the motherboard, etc.

First, we start with the performance of a real (non-VM) system In this case the benchmark ran and achieved a score that will be used as the 100% mark or goal for VMs (this will be referred to as Score A).  This original run on the real system was done without optimizing the system This provided us with a goal to try and achieve through optimizing the VM environment This initial JVM had an 800 MB heap using one thread for each available processor core (4 threads) Peak memory usage of the system during the test was just a little over 1GB.

The corresponding run of the benchmark in a VM with 4 virtual CPUs and 4 threads obtained a score that came in at 91% of Score A This was a VM with 2.5GB virtual RAM, and a JVM with an 800MB heap This was a decrease of 9% due to running in a VM The benchmark was run again on both the real machine and the VM, this time with only 2 threads being used The real machine achieved a score of 61% of Score A The VM configuration with only 2 threads achieved a score that put it at 59% of Score A.

Running a VM configuration where there is only one VM running on a host is somewhat out of the ordinary however In these first two tests, the VM was essentially using all of the processor resources and as much of the system’s RAM as it wanted. If that is all the system’s administrator wanted to do, they would probably just run the Java application / JVM directly on the host OS Typically a system’s administrator will use virtualization so that they can run several applications at the same time, each in their own VM to ensure good isolation So that our tests represent more real-world situations, the remaining tests focused on benchmarking multiple VMs concurrently running on the same host machine.

1. The runs were marked 'non-compliant' due to a mistyped command line argument that was ignored, but still allowed the run to take place.

Multiple VMs using single or double virtual processors

Under normal circumstances, multiple VMs would be run on a host computer, so it was decided to run two VMs concurrently, each with 2 virtual CPUs, with the benchmark running either single threaded or dual threaded This was done partially in a hypervisor environment and partially in a Host OS environment The results showed a trend.

When running two VMs on a hypervisor host, and running the benchmark in single threaded mode, the two VMs came in at 64% of Score A This was better than the double threaded run on the real machine or VM.

When the simultaneously running VMs were reconfigured to have only 1vCPU each and the single threaded benchmark was run again, the combined results came in at 59% of Score A - an 8% decrease from having 2vCPUs, even though only one was being used by the benchmark.

As testing continued it was found that this phenomenon remained consistent The best overall performance was achieved when a VM had an extra virtual CPU that the benchmark wasn’t using The extra virtual CPU didn’t get used very heavily, but having it available for use by the guest OS seems to have kept the OS from stealing the other virtual processors away from the benchmark.

Running two VMs each using 2 virtual CPUs also showed excellent results in a host OS environment After lots of testing and tuning and applying the best practices, this configuration ended up being the one of two configurations that beat Score A, achieving a combined score of 103% Yes - this was better then the original value achieved on the real system using all 4 cores and 4 threads A final test using three VMs, each with 2 vCPUs, two VMs using the single threaded benchmark and one using a two threaded benchmark also beat the best real machine run with a score that came in at 107% of Score A.

During these tests not every possible combination of options was tried and in particular there was one that might be able to offer some additional increase On the hypervisor based VMs it was possible to specify which VM used which CPU cores The theory is that by keeping a VM running only on certain cores, some savings in L1 & L2 cache re-fills can be achieved This was only tried once during the testing, but that test did show an increase of about 10% over the previous test Since this option wasn’t available in all the test configurations, it wasn’t pursued further, but it looks like a promising option for additional exploration.

How Size of JVM Heap affects Performance

A number of tests were made where only the amount of JVM heap was varied from 256MB up to 1500MB The benchmark would simply not succeed when running with only 256MB of heap It was found the benchmark would run with a heap size in the upper 300MB range in single or double threaded mode At 512MB it ran well as long as only 1 or 2 threads were used When 4 threads were used 512MB failed so the benchmark was often run with an 800MB heap To determine if there was anything to be gained by running with the larger heap in situations where 512MB was sufficient, the benchmark was run at both heap sizes The difference in performance was less than 1% And some runs with even larger heaps sizes showed slightly lower results than with an 800MB heap This showed that running with sufficient memory was important, but giving the JVM large amounts of extra memory accomplished little with respect to performance

Effects of not providing enough virtual RAM for the JVM Heap

Running with extra, unneeded heap did have a detrimental effect in one way though: unless the VM’s virtual RAM was increased to allow for the heap to fit within the virtual RAM, the VM’s OS began swapping to provide additional virtual memory for the process This actually slowed things down in our test by approximately 20% In a situation where multiple VMs are running and their combined virtual RAM settings are beyond the amount of real RAM, the host OS may have to swap to provide the backing for the virtual RAM If the guest OS also starts swapping to provide virtual memory for the JVM’s heap, the performance can become quite bad So giving sufficient JVM heap to accomplish the process, but not too much extra, and then tailoring the VM’s virtual RAM size to allow that entire heap to reside in the virtual RAM gives the best performance.

Turning off Filesystem Caching and Paging File

Several tests independent of the above configuration were also run to determine if the host OS’s use of memory for filesystem cache as well as paging file use made much of a difference with regard to VM performance The reasoning for this is if the host and guest OS are both doing filesystem caching there is a good chance that both OSs will end up using memory to cache the same disk blocks So the host OS preferences were changed to favor giving RAM to running programs instead of file system cache or services At the same time, the swap file was disabled in the host OS to ensure that the virtual RAM provided to the VMs could not be swapped out

There are many scenarios in which these changes might or might not make much VM performance difference, as well as some situations in which you wouldn’t want to turn off the swap file However, in this case, the test showed that there was a very substantial increase in performance of the VMs, especially in the situation where the amount of virtual RAM used by all simultaneously running VMs was a significant amount (50% to 75%) of the system’s physical memory.   Without making these host OS configuration changes, there were times when the host OS simply did not provide the amount of physical RAM backing for the virtual RAM of the VMs and lots of paging/disk I/O ensued and the VM and host OS performance went to a near stand-still.

There is also the option to disable both of these in the guest OS This was also tried and it also showed some performance increase (~5%) in part due to leaving all of the virtual RAM available to the JVM Heap without having to contend with the filesystem cache But this could also turn out to be a disadvantage under many circumstances (i.e. having the Java classes cached by the OS could offer significant speed improvements for some Java application startup times).

Other OS Optimizations

The other OS optimizations made during testing were fairly obvious, such as stopping any unneeded processes or services. This can be done both in the host as well as in the guest OS Shutting down the GUI shell improved performance a little Other services / processes, even if idle, over time add a little drag to the system You cannot get rid of all other processes and services, but some you can and this will help squeeze out a little more performance.

Optimization Results

The following graph in Figure 6 shows representative benchmark results Not all of the VM benchmark runs are shown in the graph This was done to keep the chart from being too cluttered and because many of the runs ended up with identical numbers.

                       Figure 6: The representative JVM benchmark results

The bars with white borders are from real hardware The remaining bars represent the results from benchmarks run in VMs The gradient colored bars represent the combined totals of running two or three VMs simultaneously The other colors are used to group similar runs The explanation of each column is shown in Table 1.

                       Table 1: Explanation of the JVM benchmark results

Conclusion

Based on the benchmark results there seem to be some general conclusions that can be drawn with regard to how to optimize Java performance in a VM First, make sure that the VM has enough virtual RAM for any concurrently running guest OS processes and the maximum JVM heap.  Some of the data also suggests that giving VMs a fairly significant amount of additional virtual RAM beyond what was needed could speed things up, especially if the paging file in the guest OS is disabled But this has to be balanced with the other needs of the host OS and other concurrent VMs

Second, make sure that the amount of virtual RAM required by all concurrently running VMs is less than the physical RAM of the machine and leave some for the hypervisor / host OS to use as well Third, if your Java application makes heavy use of the CPU, then give the VM one more virtual processor than is needed by the Java application. This virtual CPU will end up getting used to take care of non-Java processes in the guest OS Overcommitting the physical CPUs by having more virtual CPUs than physical CPUs is okay, as long as the extra virtual CPUs in the VMs are not being heavily used

Finally, if the work being done by a Java application can be split up into several single threaded tasks that are then done in separate VMs, this could work to your advantage Doing this in our testing resulted in overall higher performance than trying to get all the work done on one VM or even on one un-optimized real system This may require higher resource utilization overall, but if you are after performance, this may provide the final boost needed to achieve near native performance.

One last performance note that is more anecdotal than statistically sound was that the hypervisor-based VMs seemed to offer slightly better performance than host OS based virtualization, at least in certain configurations Single virtual CPU VMs seemed to run marginally faster under the hypervisor than in the host OS Since the purpose of hypervisor software is essentially to minimize VM overhead, this would be expected At the same time, when trying to find ways of tweaking performance, the host OS offered some additional avenues to view and tune and manage things (including running software directly on it) As stated earlier though, the tests weren’t really set up to directly measure the performance difference between these two configurations so the reader should perform their own investigations if they are interested.

* Note: AMD, the AMD Arrow logo, AMD Phenom, and combinations thereof are trademarks of Advanced Micro Devices, Inc Microsoft and Windows are registered trademarks of Microsoft Corporation in the United States and/or other jurisdictions SPEC and SPECjvm® are registered trademarks of the Standard Performance Evaluation Corporation All other names used in this article are for reference only and may be trademarks of their respective trademark owners