TUM Logo

Enhancing Software Fuzzing Through Dynamic Instrumentation

Enhancing Software Fuzzing Through Dynamic Instrumentation

Supervisor(s): Ferdinand Jarisch
Status: finished
Topic: Others
Author: Cosmin Banica
Submission: 2025-01-22
Type of Thesis: Masterthesis
Thesis topic in co-operation with the Fraunhofer Institute for Applied and Integrated Security AISEC, Garching

Description

Software fuzzing is an automated testing technique that feeds a program generated
inputs with the goal of finding vulnerabilities, such as crashes or memory corruption
issues. Address sanitization, a critical tool in detecting such issues, has some perfor-
mance penalties, which are harder to mitigate when the source code is not available,
since it misses out on any optimizations that could be done with prior knowledge of
the code. This thesis explores the enhancement of software fuzzing through dynamic
instrumentation, focusing on lowering the overhead associated with address sanitiza-
tion in binary-only targets. This study leverages QEMU, an emulator that can be used
to instrument targets, and QASan, an address sanitizer that can be used in conjunction
with QEMU, to dynamically sanitize blocks of code.
The main contribution of this work is the introduction of a novel dynamic sanitiza-
tion approach, leveraging QEMU and QASan to optimize performance by selectively
enabling address sanitization based on a hitcount-based heuristic. This methodology
ensures that only the most unsafe parts of the binary are instrumented, reducing the
overhead while maintaining similar bug detection. During this research, two strategies
for dynamic sanitization are compared with conventional address sanitization, with the
results showing that it is best to start by not sanitizing any blocks, and then gradually
begin sanitizing all of them. This approach is shown to be more effective in terms of
performance, while not sacrificing coverage or the ability to detect bugs, and also re-
quiring the least amount of initial configuration from the user. These findings represent
a promising advancement in the field of software fuzzing and lay the groundwork for
future research into dynamic heuristics for address sanitization.