From Sands to Mansions: Towards Automated Cyberattack Emulation with Classical Planning and Large Language Models (2407.16928v3)

Abstract: As attackers continually advance their tools, skills, and techniques during cyberattacks - particularly in modern Advanced Persistence Threats (APT) campaigns - there is a pressing need for a comprehensive and up-to-date cyberattack dataset to support threat-informed defense and enable benchmarking of defense systems in both academia and commercial solutions. However, there is a noticeable scarcity of cyberattack datasets: recent academic studies continue to rely on outdated benchmarks, while cyberattack emulation in industry remains limited due to the significant human effort and expertise required. Creating datasets by emulating advanced cyberattacks presents several challenges, such as limited coverage of attack techniques, the complexity of chaining multiple attack steps, and the difficulty of realistically mimicking actual threat groups. In this paper, we introduce modularized Attack Action and Attack Action Linking Model as a structured way to organizing and chaining individual attack steps into multi-step cyberattacks. Building on this, we propose Aurora, a system that autonomously emulates cyberattacks using third-party attack tools and threat intelligence reports with the help of classical planning and LLMs. Aurora can automatically generate detailed attack plans, set up emulation environments, and semi-automatically execute the attacks. We utilize Aurora to create a dataset containing over 1,000 attack chains. To our best knowledge, Aurora is the only system capable of automatically constructing such a large-scale cyberattack dataset with corresponding attack execution scripts and environments. Our evaluation further demonstrates that Aurora outperforms the previous similar work and even the most advanced generative AI models in cyberattack emulation. To support further research, we published the cyberattack dataset and will publish the source code of Aurora.

• The paper introduces AURORA, an LLM-driven framework that automates full-life-cycle cyberattack construction and expands TTP coverage by over 40%.
• It employs a four-component architecture—report analyzer, emulation planner, infrastructure builder, and attack executor—to streamline complex attack emulation.
• Evaluations show that AURORA constructs multi-step attacks in minutes with minimal human intervention, enhancing robust cybersecurity testing.

Overview of "From Sands to Mansions: Enabling Automatic Full-Life-Cycle Cyberattack Construction with LLM"

The paper "From Sands to Mansions: Enabling Automatic Full-Life-Cycle Cyberattack Construction with LLM" introduces AURORA, an end-to-end cyberattack construction and emulation framework leveraging LLMs. This system aims to streamline the construction of full-life-cycle cyberattacks, enhancing the efficiency and breadth of cyberattack emulation, an essential practice for testing and evaluating cybersecurity defenses.

Problem Statement

The construction and emulation of full-life-cycle cyberattacks present significant challenges in cybersecurity. Existing frameworks suffer from limited coverage of attack techniques and require substantial human intervention and domain expertise. Furthermore, attack emulations often fail to achieve the diversity and realism necessary to reflect real-world scenarios due to decentralized and unstructured attack knowledge. Existing systems cover only a fraction of relevant techniques, and the lack of comprehensive and standardized attack libraries impedes comprehensive emulation efforts.

AURORA's Architecture

AURORA addresses these challenges by utilizing the capabilities of LLMs to synthesize attack knowledge from diverse sources into executable procedures for emulated cyberattacks. The system’s architecture is composed of four main components:

1. Report Analyzer: This component utilizes LLMs to parse and extract attack techniques from unstructured CTI (Cyber Threat Intelligence) reports, generating a list of tactics, techniques, and procedures (TTP) that serve as a basis for constructing cyberattacks.
2. Emulation Planner: This planner leverages a newly developed attack procedure knowledge graph, which maps relationships between various attack procedures, to formulate a detailed attack plan. The LLM-integrated knowledge graph improves the planner's ability to integrate complex attack scenarios comprehensively.
3. Infrastructure Builder: By employing Infrastructure as Code (IaC) paradigms combined with LLMs, this component automatically configures the necessary environments for executing emulated attacks, thus significantly reducing the manual effort typically involved in creating test environments.
4. Attack Executor: AURORA uses this component to execute attack procedures, integrating both traditional attack procedures and those generated from LLM insights to simulate multi-step cyberattacks within the constructed infrastructure.

Major Contributions and Results

AURORA contributes significantly to the field by increasing the automation and accuracy of attack emulation. Key results from experiments and evaluation show:

• Improved TTP Coverage: AURORA integrates a wider range of attack techniques, enhancing coverage by over 40% compared to professional red teams.
• Efficiency: The system significantly reduces the time required to construct full-life-cycle attacks to several minutes with minimal human intervention.
• Dataset Contribution: The team open-sourced a dataset containing execution files and infrastructures for 20 emulated attacks, facilitating further research and enhancement of cybersecurity measures.

Implications and Future Directions

AURORA’s contributions extend both practical implications and theoretical advancements. Practically, it enables more robust testing environments for cybersecurity defenses, allowing researchers and practitioners to simulate complex and comprehensive attack patterns rapidly. Theoretically, it demonstrates the potential of LLMs in structuring and executing cybersecurity tasks which were previously labor-intensive and reliant on expertise.

Future research could explore enhancing LLM models to further increase accuracy in recognizing and synthesizing real-world cybersecurity scenarios. Additionally, the integration of more diverse data sources into the attack procedure knowledge graph could expand the capabilities of AURORA, including the potential to simulate an even broader array of attack techniques.

AURORA marks an important step towards automated, comprehensive cybersecurity evaluation frameworks, leveraging the cutting-edge capabilities of artificial intelligence to meet evolving cyber threats.