TACTIC Framework for CTI Extraction
- TACTIC Framework is an LLM-driven system that maps unstructured CTI texts to the standardized MITRE ATT&CK tactics and techniques.
- It utilizes a dual-layer Situational Knowledge Representation (SKR) and an evolvable memory system to ensure precise extraction and transparency.
- The framework enhances explainability and standard adherence by validating outputs through a two-stage retrieval, classification, and refinement process.
to=arxiv_search.search 平台总代理 大发快三彩票json {"query":"(2505.09261) Instantiating Standards Enabling Standard-Driven Text TTP Extraction with Evolvable Memory", "max_results": 5} to=arxiv_search.search ￣影音先锋 ахәыҷқәа ирызjson {"query":"(Legoy et al., 2020) Automated Retrieval of ATT&CK Tactics and Techniques for Cyber Threat Reports", "max_results": 5} to=arxiv_search.search ചികിത to=arxiv_search.search 天天彩票是json {"query":"(Lin et al., 2022) Attack Tactic Identification by Transfer Learning of LLM", "max_results": 5} to=arxiv_search.search _一本道 ахәыҷқәа ирызjson {"query":"(Roy et al., 2023) SoK: The MITRE ATT&CK Framework in Research and Practice", "max_results": 5} Within cyber threat intelligence, the “TACTIC Framework” can be understood as a standard-driven architecture for mapping unstructured CTI text to MITRE ATT&CK tactics, techniques, and procedures. In the formulation most closely associated with this label, the framework is not introduced under the literal name “TACTIC”; rather, it is presented as an LLM-driven framework with an Evolvable Memory System and a standard-driven TTP extraction framework based on a dual-layer Situational Knowledge Representation (SKR). Its central purpose is to convert abstract ATT&CK definitions into actionable, contextualized knowledge that can be generated, updated, retrieved, and audited during extraction, thereby enforcing faithful adherence to ATT&CK rather than relying on a purely black-box classifier (2505.09261).
1. Definition and position within ATT&CK-based CTI
The framework is centered on extracting MITRE ATT&CK Tactics, Techniques, and Procedures (TTPs) from unstructured CTI text such as technical reports and blogs. Its defining characteristic is that ATT&CK is treated not merely as a label inventory but as an explicit, evolving knowledge layer. In this design, the main prediction target is the technique; tactics are not predicted as a separate label in experiments, but are implied by ATT&CK’s established technique-to-tactic mapping, while procedures appear both as sentence-level behavioral text and as a baseline source drawn from ATT&CK procedure sections (2505.09261).
This orientation is consistent with broader uses of ATT&CK in research and practice. ATT&CK is widely employed as a behavior-centric repository of adversarial tactics and techniques across CTI, intrusion detection, and incident response, but practical implementation and evaluation remain uneven across the literature (Roy et al., 2023). Earlier ATT&CK extraction systems such as “Automated Retrieval of ATT&CK Tactics and Techniques for Cyber Threat Reports” (Legoy et al., 2020) and packet-level tactic identification systems such as “Attack Tactic Identification by Transfer Learning of LLM” (Lin et al., 2022) show that automated ATT&CK mapping is feasible, but the standard-driven framework places unusual emphasis on explicit standard instantiation, explainability, and post hoc verification rather than only end-task classification accuracy.
2. Motivation: standard adherence, consistency, and transparency
The motivating problem is not simply low performance on a benchmark. Traditional approaches optimize against labeled datasets, but do not explicitly enforce consistency with the evolving official ATT&CK standard. The consequences identified for this failure mode are intelligence silos, contradictory assessments, non-standard labels and drift, and low transparency. A canonical example is the same activity being labeled as T1132 in one system and T1071 in another, even when the underlying behavioral evidence is similar (2505.09261).
This motivates a framework in which the extraction process is mediated by explicit knowledge derived from official ATT&CK definitions and labeled CTI evidence. The aim is faithful, transparent, standard-driven mapping from text to ATT&CK TTPs. That differs materially from document-level classification systems such as rcATT, which framed the problem as multi-label classification over ATT&CK tactics and techniques at report level (Legoy et al., 2020), and from packet-level systems such as PELAT, which transfer tactic knowledge from ATT&CK lifecycle documents into HTTP payload classification (Lin et al., 2022). Those systems are important precursors, but the present framework prioritizes disambiguation between similar ATT&CK techniques and the inspectability of the reasons for assignment.
A plausible implication is that the framework functions as a harmonization layer across organizations: if different detectors or analysts produce divergent preliminary labels, a standard-driven verification stage can re-evaluate them against the same explicit ATT&CK-grounded criteria.
3. Core representation: dual-layer SKR and evolvable memory
The core knowledge unit is the Situational Knowledge Representation (SKR), a dual-layer structured object. Layer 1, called state, captures a situational context shared by semantically related techniques. Layer 2, called action, stores technique-specific manifestations that discriminate among techniques within that state. The representation is explicitly JSON-like rather than heavily formalized mathematically (2505.09261).
1 2 3 4 5 6 7 8 9 |
{
"state": "Communication with C2 using encoded subdomains",
"action": {
"T1132": "Uses base32 encoding for subdomains to obfuscate C2 communication",
"T1071": "Employs DNS as an application layer protocol for C2 communication",
"T1001": "Involves data obfuscation techniques like AES ciphertext within subdomains",
"T1008": "Indicates fallback to alternative protocols like HTTP if primary DNS fails"
}
} |
The state layer supports coarse-grained retrieval and groups semantically related techniques. The action layer supports precise classification, disambiguation, and explanation. The distinction is operationally important: retrieval over Layer 1 narrows the candidate set, while Layer 2 supplies the contrastive cues needed to decide, for example, whether the text is primarily about encoding methods or application-layer protocol usage (2505.09261).
These SKR instances are stored in an Evolvable Memory System, also described as a Life-long Memory System. The memory is initialized by generating SKRs from labeled text plus ATT&CK definitions, optimized by adding new Layer 2 manifestations when new labeled evidence appears, and pruned by forgetting low-quality or obsolete entries. This memory is “evolvable” because it can gain new situational contexts, gain new technique manifestations within existing contexts, and shed unhelpful knowledge over time (2505.09261).
4. Extraction workflow and treatment of tactics, techniques, and procedures
The extraction pipeline has two stages. Stage 1: Initial Retrieval & Classification takes a CTI text snippet, retrieves top- SKR instances by semantic similarity over Layer 1, and constructs a prompt containing the input text, retrieved situational contexts, their associated Layer 2 manifestations, and the candidate ATT&CK technique identifiers. The LLM then produces an initial technique prediction. Stage 2: Refinement & Verification takes the text plus an initial technique label—either from Stage 1 or from an external system—retrieves SKRs and Layer 2 entries relevant to that technique and its close alternatives, and constructs a contrastive prompt focused on the fine distinctions between similar techniques. The LLM then confirms or corrects the assignment (2505.09261).
This organization is central to how the framework handles the ATT&CK hierarchy. Techniques are the primary extraction target, and sub-techniques are resolved to parent techniques for evaluation. Tactics are inferred through the ATT&CK technique-to-tactic mapping rather than via a separate tactic classifier. Procedures are treated in two ways: as sentence-level procedural descriptions in a dataset and as a baseline prompt source using ATT&CK procedure sections. The framework therefore does not construct an explicit graph model of tactics, techniques, and procedures; instead, hierarchy is represented implicitly through SKR states and technique-specific actions (2505.09261).
This suggests a particular conception of a TACTIC-like system: its most important function is not direct tactic classification, but the adjudication of technique identity under standardized, contrastive ATT&CK semantics.
5. Knowledge instantiation, LLM roles, and empirical results
The framework’s central operation is instantiating the abstract ATT&CK standard into SKRs. Inputs for generating one SKR are a target sentence labeled with ATT&CK techniques, the official ATT&CK definitions for those identifiers, and top- contextually similar labeled sentences retrieved by semantic search. The LLM first derives a state that is technique-agnostic but behavior-specific, then derives actions that describe how each technique manifests within that state. Initialization bootstraps from the full pool of labeled sentences; update uses previously stored targets for the same state and adds refined or additional Layer 2 actions (2505.09261).
The LLM has three distinct roles. First, it performs knowledge instance generation. Second, it supports memory update, where existing SKR state and actions are combined with new textual evidence to produce refined or new actions while keeping the state stable. Third, it performs Stage 1 and Stage 2 extraction. The concrete experiments use Qwen2.5-32B as the main model, with comparisons to GPT-4o and DeepSeek-v3 (2505.09261).
The evaluation uses the labeled dataset from Nguyen et al. (2024), specifically a procedures dataset and an expert dataset, both sentence-level and technique-labeled. Metrics are Accuracy, Precision, Recall, and F1-score, all at technique level with sub-techniques mapped to parent techniques (2505.09261).
| System | Procedures F1 | Expert F1 |
|---|---|---|
| Base | 0.10 | 0.07 |
| official | 0.47 | 0.30 |
| procedures | 0.36 | 0.25 |
| SKR1 | 0.59 | 0.31 |
| SKR2 | 0.63 | 0.35 |
| GPT-4o | 0.45 | 0.15* |
| DeepSeek-v3 | 0.46 | — |
On the procedures dataset, SKR2 reaches F1 = 0.63, compared with 0.45 for GPT-4o and 0.46 for DeepSeek-v3. On the expert dataset, SKR2 reaches F1 = 0.35, compared with 0.15 for GPT-4o, which was evaluated on part of the test set. The abstract also reports that the framework boosts Technique F1 scores by 11% over GPT-4o (2505.09261).
The largest gains appear at the technique level, which is the hardest part of the ATT&CK hierarchy and the level at which standard-driven disambiguation is most valuable.
6. Explainability, standardization function, and limitations
Explainability follows directly from the representation. Layer 1 exposes the high-level scenario the model believes it is in, while Layer 2 exposes the technique-specific cues used to distinguish nearby ATT&CK labels. Because the decision knowledge is stored as human-editable state/action text associated with technique identifiers, analysts can inspect, refine, or prune underperforming entries. The Memory Forget mechanism further makes poor or obsolete SKRs visible as objects of review rather than opaque weights (2505.09261).
The framework’s Stage 2 also gives it a broader standardization function. It can be used not only to produce first-pass labels, but also to re-evaluate outputs of other TTP extraction systems against explicit ATT&CK-based contrastive criteria. This suggests a role as a shared adjudication layer across multi-source CTI pipelines, including SOC workflows, TIPs, and SIEM-linked TTP normalization, although such deployment extensions are presented as transferable design ideas rather than as reported experiments (2505.09261).
The stated limitations are substantial. Situational Context granularity remains somewhat ad hoc; overly broad states can reduce discriminative power, while overly narrow states fragment memory. Retrieval quality depends on embeddings, and the work explicitly notes the need for a specialized embedding model trained on cyber-attack text. Performance also depends on the quality and cost profile of powerful LLMs, and the method remains constrained by the quality of labeled CTI sentences used to initialize memory. Future work is directed toward refining the taxonomy of contexts, adding auto-curation and sanity checks for conflicting actions, aligning SKRs with new ATT&CK releases, and improving efficiency for large-scale, near-real-time CTI streams (2505.09261).
In that sense, the TACTIC Framework is best understood not as a single fixed product name, but as a design pattern for ATT&CK-grounded CTI extraction: a dual-layer knowledge schema, an evolvable memory, and a two-stage classify/verify loop that mediates LLM inference through explicit, inspectable standard knowledge rather than through unconstrained prompting alone.