Using Hardware Performance Events for Instruction-Level Monitoring on the x86 Architecture

Full virtualization has become one of the basic technologies for the development of security applications. This is due to the fact that full virtualization provides important properties such as isolation and transparency that are essential for the development of robust security mechanisms. However, a fact that is often overlooked is that full virtualization also enables developers to make full use of the existing hardware features. By using these features in novel ways, it is possible to create new robust hardware-based security mechanisms. In this paper we make use of the Performance Monitoring Counters (PMCs), which are available on most mainstream processors, to provide PMC-based trapping, a general concept for trapping hardware performance events to the hypervisor. We make use of this concept by proposing a novel approach to monitoring applications running within a virtual machine on the instruction-level from the hypervisor. In contrast to existing approaches, this course of action allows us to not only monitor all instructions of a program, but also enables us to limit the monitoring to specific instruction types. To demonstrate the possibilities of such an approach we implemented a shadow stack that protects the return addresses of functions running within a virtual machine from the hypervisor by only trapping call and return instructions.