TUM Logo

Rethinking Process Isolation with Lightweight Memory Protection Extensions

Rethinking Process Isolation with Lightweight Memory Protection Extensions

Supervisor(s): Marius Momeu
Status: finished
Topic: Others
Author: Valentin Dimov
Submission: 2023-05-15
Type of Thesis: Masterthesis

Description

In modern operating systems, frequently switching the runtime context
between processes and privilege levels incurs a significant performance
overhead. In this thesis, we aim to reduce this overhead by enhancing
traditional process and privilege isolation mechanisms with MPK, a lightweight
memory protection extension shipped with recent Intel processors.
We introduce two novel process isolation mechanisms. Page table sharing
allows multiple mutually-distrusting processes to run in a shared address
space, and uses MPK to isolate them in memory, with the goal of speeding up
context switching. Kernel-mode tasks allow user processes to run in kernel mode,
using MPK to separate userspace and kernelspace memory, with the goal of speeding
up system calls. We identify the security challenges of implementing memory and privilege
isolation using MPK domains, and we discuss solving these challenges with available
techniques such as binary scanning, code instrumentation, and modern CFI extensions.
We develop prototype implementations of PT sharing and kernel-mode tasks based on
the Linux kernel, and empirically evaluate their performance.
With PT sharing, the latency of context switches between processes in the same shared
address space is significantly reduced. Using shared PTs also seems to speed up the
execution of instructions in userspace, which we speculate may be due to a reduction
in TLB misses. We estimate that the per-task memory overhead of PT sharing is on the
order of 1-3 KB. On the other hand, our implementation of kernel-mode tasks does not
achieve faster system call dispatching compared to the syscall instruction. We propose
several future directions for extending and improving the PT sharing mechanism, some
of which could also benefit from the insights gathered during the development of
kernel-mode tasks.