The year was 1999, and the reviewers who contributed to the software magazine I edited were ecstatic about a new technology straight out of Stanford that made their lives easier both as developers and as tool junkies: VMware. Finally, they could simultaneously run different operating system flavors in a virtual file system on a single machine, with each OS believing it had full control of system resources. No longer did they have to muck with bootable partitions or deal with disk erasure—guest OSes could be added, deleted or reverted to prior instances with ease.

"Virtualization solved a very big problem for me when I had to learn Unix," says John Sequeira, a Cambridge, Mass.-based software architect. "I bought VMware when it first came out. At the time, I wrote most of my apps on Windows, so from a knowledge acquisition standpoint, VMware made it easy for me to live within Unix and then go back to my familiar Windows if I got lost."

Over time, Sequeira found that Moore 's Law kept abreast of his computing demands—but he easily envisions how the high performance of hardware-level virtualization could be utilized. And though it's more often applied to parallel computing, this is where Amdahl's Law comes in, pouring cold water on the promise of double-digit performance improvements for today's processors without optimizing communication among subsystems such as memory controllers:

"I run a lot of VMs constantly—three or four, including my desktop—and it performs really well. The big push is, if you can keep ramping up the performance, you can start virtualizing smaller things: Instead of a whole box, you can just virtualize an application on top of your desktop. Or, from a deployment standpoint, virtualization now means it's simply now a download rather than a sys admin gauntlet."

Sequeira's also seen the demand for virtualization grow in the enterprise space, with nearly half his consulting clients now using the technique. As an architect for a financial ASP offering customer relationship management, for example, he works with a single server that runs 20 virtual machines. "Greater processor performance means they have to buy fewer servers. I'm looking forward to deploying the new Opteron processors for them when they outgrow the one box they're on. In larger enterprises, when you're talking about half of a football field of servers consolidated into just a few boxes—imagine dealing with all that heat. Think of all that electricity you don't have to pay for."

That's just one of the reasons IT managers, chipmakers and ISVs have virtualization on the brain. What began as a niche developer tool bringing the concept of classic mainframe virtualization to the desktop presaged an explosion of interest in how to maximize computing resources and security at the grid, application, OS and hardware level.

Forget RISC vs. CISC

Not long ago, debate raged over whether the x86's Complex Instruction Set Computing processor architecture could outpace its Reduced Instruction Set Computing sibling, found in Alpha, MIPS and SPARC workstations. While the battle burned, market forces were unmoved—and x86 became the platform of choice for consumers. Today, according to a February 2006 study by TheInfoPro, more than one third of server professionals believe that "virtualized x86 boxes will be the dominant platform being deployed in 2010, while only 3% believed virtualized RISC boxes would be dominant."

"Virtualization is not a new technology, but the development of high-performance processors, multi-core technology, and 64 bit have created a new scenario; some of characteristics found in a RISC-based processor have now moved into the x86 processor," says Margaret Lewis, director of commercial solutions for AMD. "Also, many x86 servers on market are underutilized. They're drawing power, needing cooling and space. All of these things are converging, creating a perfect storm for x86 virtualization."

AMD Virtualization (AMD-V) is now shipping in the Socket F Opteron processors, along with DDR2 memory. This is in spite of the x86 architecture's barriers to virtualization, namely multiple levels (or rings) of privileged operating instructions, ways for guest programs to call privileged system routines, memory protection mechanisms and asynchronous interrupts for I/O communication with the CPU. A University of Cambridge research team led by Paul Barham (of Xen hypervisor fame) puts it well in a paper entitled "Xen and the Art of Virtualization": "x86 represents a worst case in the areas where it differs significantly from RISC-style processors—for example, efficiently virtualizing hardware page tables is more difficult than virtualizing a software-managed TLB."

"In the x86 world, there are some impediments for doing effective virtualization," Lewis agrees. "The OS wants to own all of the resources. We have people like VMware who have done an excellent job of software virtualization, allowing hypervisor and guest OS to share resources effectively. The next generation of processors have enablement at the architectural level—such as the integrated, low-latency memory controller—to help that software be more efficient."

The Hypervisor Hop

Retooling the x86 instruction set for virtualization with extensions such as the device exclusion vector (to enforce security among VMs) and tagged TLBs (to allow the hypervisor and each guest OS to occupy separate address spaces for faster switching) has sent a shock wave through the industry. Could this change OS market share dynamics? Never one to wait and see, Microsoft has announced its own virtualization plans, with the purchase of Softricity, an application virtualization and streaming tool, and the forthcoming Windows Server "Longhorn," a hypervisor-based solution.

Meanwhile, virtualization frontrunner VMware finds itself at a curious crossroads: Suddenly, hypervisors are simultaneously hip and old-school. In a widely cited 2005 whitepaper, VMware engineers claimed hardware-enabled virtualization offered no significant performance improvements over software-only solutions such as theirs.

"VMware has been out there a long time, and their code is very efficient," says Tim Mueting, AMD's product manager for virtualization solutions. "Today, they are not directly using AMD-V. They've been waiting for nested page tables, which will give them significant performance improvements." In the same whitepaper, VMware engineers are "optimistic about the potential of future hardware assistance in the area of MMU virtualization. The three-way trade-off among trace costs, hidden page faults and context-switch costs was successfully addressed with appropriate hardware in IBM's s/370 machines," they write. AMD's nested page tables will do the trick as well.

While XenSource and Virtual Iron are coming out with their own versions of the open-source Xen virtual machine monitor, using nearly all the extensions provided by AMD-V, it appears Microsoft and VMware are not far behind in embracing the new hardware. Xen is currently found in most Linux distros as well as Sun's Solaris OS, while a market is blossoming at every level of virtualization—including the data center.

Rethinking Server Sprawl

With a different perspective is Rick Barnard, director of enterprise architecture at data center virtualization vendor Egenera in Marlborough , Mass. Founded in 2000, Egenera offers BladeFrame systems equipped with AMD Opteron processors as well as software to transform most of the I/O devices on a legacy server into software: Utility computing, at last.

"In a financial company with 10,000 Unix servers and 6,000 Windows servers, no amount of dollars spent on new servers or pieces of software or people to manage environment can fix the problems of low utilization, long time-to-market and low flexibility," Barnard says. "If companies want to leverage distributed environments on a server architecture that's 10 to 15 years old, you need to virtualize at the data center level."

While partitioning a single server lets you bump up utilization and run many small apps, companies with large databases, business intelligence or global trading applications need high availability, fast provisioning and disaster recovery. According to Barnard, "Virtualization at the data center level means abstracting the CPU and memory of processing resources into a processing area network (PAN) so they can be tapped when needed."

Egenera's PAN, the company claims, can reduce sys admin costs by 80 percent by eliminating many physical components and controlling the allocation of all network and storage resources via software. Commerce Bank US , an Egenera customer, was able to consolidate 140 servers down to 48 servers, for example. Others have reduced configuration times from days to hours.

"If you look inside a legacy server, you'll find disk drives, CD-ROMs and local network and storage connectivity. If you've got 24 servers, you need cables for networking, storage, power and ports. Creating these stateless environments solves that physical complexity." And you don't have to build it yourself. Using commodity components and software, vendors such as Egenera can create a high availability infrastructure that lets you do your own benchmark studies if you so desire. "You can run any app on any of those servers. If one gets better performance, you can easily just move it in a matter of minutes." Further, old servers aren't rendered worthless; simply reprovision them for departmental purposes or hypervisor-based consolidation.

Whatever your level of abstraction, x86 virtualization "is definitely a huge deal. AMD's extensions can be considered a breakthrough at the chip level—and it shows there's lots of demand for virtualization," says Barnard.

Wrap Those Apps

Clearly, a new ecosystem is emerging. Sequeira envisions virtualizing "smaller things," say, wrapping a program and its dependencies, creating new sandboxes or instrumenting applications for secure behavior.

Meanwhile, at the data center level, Google may be able to afford rocket scientists, but for most companies, "It's not a trivial exercise to make your standard app run on a grid. I think that's the promise developers see in the Egeneras and Virtual Irons of the world," Sequeira says.

And with increased processor performance, Mueting points to benchmarks that have run 26 VMs on the Opteron without maxing out the CPU, along with installations in the field of 40 to 50 VMs. While hypervisors may be memory and I/O intensive, developers using AMD technology shouldn't know or care, Lewis says, so long as the hardware is conducting a productive—but silent—conversation with the hypervisor. Amdahl might have to make a note of it.

An award-winning magazine writer and the former editor in chief of Software Development, Alexandra Weber Morales is also a Webmaster, singer-songwriter, and recovering auto mechanic.