Leveraging Virtualization Techniques for System Security

System virtualization is a technology to run multiple operating systems as so-called virtual machines on one physical host. The virtual machines run on a thin software layer, namely the virtual machine monitor (VMM, also called hypervisor). It manages and partitions the physical hardware resources and provides strong isolation between the virtual machines.

The VMM can also implement security features at a whole new level. Up to now, most host-based security mechanisms rely on the proper function of the underlying operating system. If it is compromised, these mechanisms can be disabled or bypassed. By contrast, a security mechanism implemented in the VMM remains unaffected if the guest operating system has been successfully attacked.

Virtual machine introspection (VMI) describes the method of monitoring and manipulating the state of a virtual machine from the level of a VMM. The focus of our virtualization security research is the detection of intrusions by means of VMI. We are exploring how state information of a guest operating system can be gathered without having to rely on its cooperation by using a-priory knowledge about the virtual hardware and guest software. In addition, we are analyzing different types of system-level attacks from the vantage point of the machine state to generate a classification of such attacks above und beyond current signature-based methods. The overall goal is to provide general guidelines on how to build a VMI-based intrusion detection system, to understand its potential and limitations, and to make it an established technique for future security applications.

Researchers: Thomas Kittel, Julian Kirsch, Sergej Proskurin, Peng XU