Use Virtual 8086 Mode to Secure Virtual Servers

More on server virtualization

Protecting your IT infrastructure by running vital applications in virtual machines — or “security through virtualization” — is a strategy that’s bound to fail. That’s the conclusion of Jonathan Brossard, CTO of P1 Code Security, following research he carried out into ways of attacking virtualization technologies. Brossard presented his findings at the Hack In The Box Secconf 2010 in Amsterdam in July.

‘Security through virtualization’ is a strategy that’s bound to fail. So how do you protect your virtual machines? By testing virtualization systems methodically. One way to accomplish this is to break virtualization by switching to virtual 8086 mode.

Plenty of security problems are associated with virtualization that are already well known, Brossard points out. These include:

• Virtual machine (VM) hopping: The ability of the owner of one virtual machine to attack another guest machine on the same hypervisor (e.g., using the VMware workstation guest isolation vulnerability)
• VM DoS: The ability to crash the host machine and other guests VMs from a guest (e.g., using CVE-2007-4593)
• VM escape: The ability of a malicious guest VM user to write arbitrary files on the host using a directory traversal vulnerability (using CVE-2007-1744)

These security problems affect companies using virtual servers hosted by a service provider because in many cases virtual machines from different customers are hosted on the same physical hardware. The implication of this is that getting a shell on the same host machine as a given target is just a matter of paying a few dollars a month, Brossard said. “This means you don’t have to worry about finding a remote exploit. Sharing is very interesting if you are malicious because then seemingly minor local bugs do matter: Virtualization amplifies the consequences. Owning the host from the guest is practical, so security through virtualization is a failure.”

If the security of virtual machines is to be improved, what’s needed are new tools to test virtualization systems methodically. Without the source code — and, arguably, even with the source code — fuzzing is the most obvious path to explore. But how, for example, can you dynamically test a virtual hard drive by fuzzing it methodically to reveal any potential problems?

Using the standard API:

ssize_t read(int fd, void *buf, size_t count);
ssize_t write(int fd, const void *buf, size_t count);

is unlikely to be fruitful, Brossard says. “This would mostly fuzz the kernel, not the virtual machine. We need something much lower level.”

Addressing standard low-level attack vectors, such as I/O ports and Ioctls, is impractical, as there are simply too many possibilities to test them all. Brossard suggests looking at what might have been done in the past, “With MS DOS, we had direct access to the hardware through BIOS, hard drive and display interrupts and so on. Can we get back to this?”

To answer this, let’s take a quick look at the Intel x86 CPU. As you’ll probably recall, it supports three modes:

• Real address mode: This mode implements the programming environment of the Intel 8086 processor with extensions, such as the ability to switch to protected or system management mode. The processor is placed in real address mode following power-up or a reset.
• Protected mode: This is the native state of the processor
• System management mode: This mode is used for implementing platform specific functions, such as power management

What’s significant about protected mode for our purposes is that one of its features is the ability to directly execute real address mode 8086 software in a protected, multi-tasking environment. This feature, called “Virtual 8086 mode,” can be enabled for any task.

That’s interesting because Virtual 8086 mode has direct access to hardware via interrupts, and while there are a huge number of possible parameter combinations for each interrupt, this huge number is at least finite, making it feasible to fuzz comprehensively — perhaps using cloud based computing resources, for example.

However, one problem remains. How do you access this Virtual 8086 mode from inside a virtual machine? The answer, Brossard figured out, is actually quite easy. That’s because x86 Windows and Linux operating systems offer a way to run 16-bit applications under protected mode by emulating a switch to Virtual 8086 mode. This is exactly what happens when you run an old MS-DOS app in a DOS box under Windows. The Linux kernel can do something similar, providing an emulation of real address mode using two system calls:

#define __NR_vm86old 113

#define __NR_vm86 166

By using either of those methods:

• The switch to virtual 8086 mode is entirely emulated by the kernel, which means it will work inside a virtual machine
• Programming using “old style” interrupts is possible
• The interrupts will be delivered, both to emulated or physical hardware

In other words, says Brossard, you have a bare-metal virtualized hardware interface.

There is one further difficulty to overcome: 64-bit processors running in long mode (where 64-bit applications can access 64-bit instructions and registers) can’t switch to Virtual 8086 mode. At first glance this would seem to rule out fuzzing any 64-bit software, such as the latest VMware ESX or Microsoft Hyper-V hypervisors.

But, Brossard points out, under virtualization, the switch to Virtual 8086 mode is emulated by the kernel, so using kernel patches it is possible to add Virtual 8086 mode capabilities to an x64 Linux kernel. For example the V86-64 patch available on SourceForge allows legacy 8086 programs to run on modern x86-64 AMD and Intel processors. “What’s not possible in real hardware becomes possible under a virtualized environment!” Brossard said.

Using this technique, Brossard has so far fuzzed and found bugs in a number of virtualization systems including VirtualBox, Virtual PC, Parallels and Hyper-V, although he has not yet had time to investigate each one to establish which might be exploitable.

His conclusion? “Adding layers of virtualization is actually a bad idea that can lead to exploitability. The best way to get secure software is to properly test it for security bugs.”

Paul Rubens is a journalist based in Marlow on Thames, England. He has been programming, tinkering and generally sitting in front of computer screens since his first encounter with a DEC PDP-11 in 1979.

Follow ServerWatch on Twitter