ROSA: Finding Backdoors with Fuzzing

Abstract: A code-level backdoor is a hidden access, programmed and concealed within the code of a program. For instance, hard-coded credentials planted in the code of a file server application would enable maliciously logging into all deployed instances of this application. Confirmed software supply chain attacks have led to the injection of backdoors into popular open-source projects, and backdoors have been discovered in various router firmware. Manual code auditing for backdoors is challenging and existing semi-automated approaches can handle only a limited scope of programs and backdoors, while requiring manual reverse-engineering of the audited (binary) program. Graybox fuzzing (automated semi-randomized testing) has grown in popularity due to its success in discovering vulnerabilities and hence stands as a strong candidate for improved backdoor detection. However, current fuzzing knowledge does not offer any means to detect the triggering of a backdoor at runtime. In this work we introduce ROSA, a novel approach (and tool) which combines a state-of-the-art fuzzer (AFL++) with a new metamorphic test oracle, capable of detecting runtime backdoor triggers. To facilitate the evaluation of ROSA, we have created ROSARUM, the first openly available benchmark for assessing the detection of various backdoors in diverse programs. Experimental evaluation shows that ROSA has a level of robustness, speed and automation similar to classical fuzzing. It finds all 17 authentic or synthetic backdooors from ROSARUM in 1h30 on average. Compared to existing detection tools, it can handle a diversity of backdoors and programs and it does not rely on manual reverse-engineering of the fuzzed binary code.

An Expert Overview of "Rosa: Finding Backdoors with Fuzzing"

The paper "Rosa: Finding Backdoors with Fuzzing" presents an innovative methodology for detecting code-level backdoors in software by leveraging graybox fuzzing techniques. The authors, Dimitri Kokkonis, Michaël Marcozzi, Emilien Decoux, and Stefano Zacchiroli, tackle a significant challenge in software security: the identification of hidden backdoors which can offer unauthorized access or permit privilege escalation in software systems.

Context and Problem Formulation

Code-level backdoors are surreptitious access points embedded within software code, akin to concealed keys which can unlock system resources to those with knowledge of their existence. The insertion of backdoors, frequently seen in well-documented software supply-chain attacks, poses a persistent threat to security, requiring rigorous and often manual code audits to locate. Traditional methods of detection are limited in both scope and efficacy, necessitating manual reverse-engineering efforts which are time-consuming and error-prone.

Rosa and the Fuzzing Approach

Rosa is a framework that couples a state-of-the-art fuzzer, AFL++, with a newly developed metamorphic oracle to detect runtime backdoor triggers. Graybox fuzzing, a technique involving the automated generation and subsequent evaluation of test inputs to explore a program's execution paths, underpin the detection process in Rosa. The novel aspect of Rosa lies in its ability to discern anomalies suggestive of backdoors through observed deviations in expected program behavior.

Technical Implementation

The authors employ a two-phase approach in Rosa's operation. Initially, a representative inputs collection phase utilizes fuzzers to populate a database with varied inputs reflecting typical input family behaviors of the Program Under Test (PUT). In the backdoor detection phase, a freshly generated input from the fuzzer is compared against this database to identify any discrepancies in system call behaviors, flagged as potential backdoor triggers. This strategy is supported by the Rosarum benchmark, a first-of-its-kind dataset specifically designed for backdoor detection tool evaluation, comprising both authentic and synthetic backdoors.

Experimental Evaluation

Rosa demonstrates notable robustness and speed across diverse program contexts. Its metamorphic oracle enables a precision in detection that markedly reduces false-positive rates, necessitating limited manual validation of suspicious inputs. This attribute indicates a promising level of automation in Rosa's backdoor detection capability, making it suitable for large-scale application in complex software environments.

The benchmark testing, particularly against Stringer—an existing static analysis tool for backdoor detection—highlights Rosa's superior accuracy and broader applicability. While Stringer displays quicker detection times due to its static analysis nature, it is significantly limited in scope, detecting only a small subset of backdoors and producing higher false-positive rates compared to Rosa.

Implications and Future Directions

This research introduces a methodological shift in how backdoor detection can be effectively integrated into software auditing processes. By capitalizing on the dynamic exploration capabilities of graybox fuzzing and the introduction of metamorphic testing concepts, Rosa represents a significant advance in the domain of software security.

The implications of this work are profound, suggesting potential adaptations of the Rosa framework for broader types of security vulnerabilities beyond code-level backdoors, including its integration with static analysis techniques for enhanced detection capabilities. Additionally, the creation of Rosarum stands as a valuable resource for ongoing and future evaluations of backdoor detection technologies.

Overall, "Rosa: Finding Backdoors with Fuzzing" contributes a compelling perspective on the automatic detection of hidden software threats, advocating for a proactive stance in the defense of software integrity and security. As this work is integrated into practical security frameworks, ongoing developments in AI and software testing methodologies are anticipated to further enhance its effectiveness and scope.