---
title: Automatic Vulnerability Triaging
url: https://www.emergentmind.com/topics/automatic-vulnerability-triaging
type: topic
---

# Automatic Vulnerability Triaging

Searching arXiv for recent and directly relevant papers on automatic vulnerability triaging, severity assessment, exploit validation, and assignment.
Automatic vulnerability triaging is the automated transformation of vulnerability reports, scanner findings, firmware inventories, code findings, or issue tickets into structured decisions about applicability, ownership, type, severity, impact, and remediation priority. In the literature, this includes assigning vulnerabilities to developers or components as a multi-class text classification problem, classifying weaknesses and severity directly from descriptions, filtering irrelevant findings, generating VEX-style applicability judgments, ranking risk with composite scores, mapping CVEs to ATT&CK techniques, and validating exploitability through automatically generated proofs of concept [2310.06913] [2501.18908] [2407.14640] [2605.01739] [2508.18439].

## 1. Conceptual scope and task formulations

A central formulation treats triage as a supervised mapping from issue text to discrete labels. In bug triaging, the input is the concatenation of `Summary` and `Description`, and the outputs are `Assignee` or `Component`; the same formulation transfers directly to vulnerability–developer assignment and vulnerability–component assignment, where the labels become security engineers, feature teams, codebase components, microservices, or applications [2310.06913]. This establishes automatic vulnerability triaging as a family of single-label, multi-class classification tasks over textual or text-plus-metadata inputs.

Other formulations expand the output space beyond assignment. CASEY frames triage as jointly identifying CWE categories and severity from vulnerability descriptions and code context, while VLAI predicts severity classes directly from vulnerability descriptions before official CVSS scores are available [2501.18908] [2507.03607]. CVE-LLM defines the task as a mapping from an asset and a notification to `VEXCategory`, `VEXJustification`, an environmental `Vector`, and internal and customer comments, thereby treating triage as a structured generation problem over applicability, rationale, and context-aware risk [2407.14640].

A further expansion makes impact explicit. TRIAGE maps CVEs to ATT&CK techniques under the three impact phases defined in the CVE Mapping Methodology: exploitation technique, primary impact, and secondary impact [2508.18439]. In SBOM-driven firmware triage, the task is expressed as a pipeline from extraction and normalization to SBOM generation, vulnerability mapping, and a localized risk score, while exploit-centric systems such as AXE, AutoEG, and A2 redefine triage around the question of whether a reported vulnerability is actually exploitable and what practical effect it produces [2601.01308] [2602.14345] [2604.00704] [2508.21579].

## 2. Data sources and vulnerability representation

The dominant input modality is text. VLAI uses the full vulnerability description text and reports that titles and CPE fields provide negligible gains, whereas CASEY evaluates prompts built from the NVD description alone or augmented with buggy files, methods, or hunks extracted from linked commits [2507.03607] [2501.18908]. In assignment-oriented settings, the same reliance on textual issue reports appears in transformer-based bug triaging, where `Summary` and `Description` are concatenated and used to predict developers or components [2310.06913].

Operational systems broaden the evidence base. AgenticVM ingests scanner outputs from Trivy, Grype, and Snyk, correlates them with NVD, EUVD, and the CISA KEV catalog, and normalizes them into a canonical vulnerability record with fields such as `cve_id`, `description`, `cvss_vector`, `cvss_base_score`, `severity`, `source_db`, `scanner_origin`, `asset_context`, and `recommendation` [2605.01739]. CVE-LLM uses internal notifications and evaluations together with public NVD documents, assembling 320K documents for domain adaptive pretraining and 440K instructions for supervised fine-tuning; its asset representation includes product and software names, versions, sub-organization, and lists of software components, reaching approximately 1.6K assets and 152K components [2407.14640].

In software supply-chain settings, representation becomes SBOM-centric. The IoT firmware pipeline extracts Linux-based firmware with binwalk, normalizes filesystems, generates CycloneDX JSON SBOMs with Syft, and maps components through PURLs, CPEs, filename heuristics, and hashes to vulnerabilities from Grype, OSV-Scanner, NVD, GitHub Advisory Database, EPSS, and KEV [2601.01308]. The same literature emphasizes uncertainty markers such as the placeholder `0.0.0-unknown` when stripped binaries or static linking prevent reliable version inference [2601.01308].

Training data quality is itself a triage problem. CleanVul shows that vulnerability datasets derived from vulnerability-fixing commits often contain 40% to 75% noise because all changes in a commit are labeled as vulnerability-related; VulSifter addresses this by assigning function-level scores in \(\{0,1,2,3,4\}\) and combining LLM analysis with heuristics for test filtering, producing CleanVul with 11,632 functions and 90.6% Correctness at threshold 3, or 8,337 functions and 97.3% Correctness at threshold 4 [2411.17274].

## 3. Methodological families

One major family treats triage as text classification or sequence classification. Transformer-based bug triaging fine-tunes BERT, DistilBERT, RoBERTa, ALBERT, DeBERTa, and CodeBERT with a standard sequence-classification head over the final `[CLS]` representation and cross-entropy loss, while maintaining TF-IDF + SVM as a strong baseline [2310.06913]. VLAI uses `roberta-base` with a linear softmax head over four severity classes and a cross-entropy objective, retrained daily on approximately 550,000 training examples [2507.03607]. CASEY uses prompt-engineered and fine-tuned GPT-3.5 models to output perfect-match and top-candidate CWEs, plus severity labels and scores, from combinations of descriptions and code context [2501.18908].

A second family is structure-aware and code-centric. AIBugHunter builds a triage pipeline for C/C++ in which LineVul first detects and localizes vulnerable lines, then a multi-objective optimization formulation jointly predicts CWE-ID and CWE-Type with a shared CodeBERT encoder and two task-specific heads, and finally a CodeBERT regression model estimates CVSS v3.1 scores [2305.16615]. AutoVulnPHP uses a two-stage PHP pipeline: SIFT-VulMiner converts ASTs augmented with control-flow and data-flow edges into linearized sequences for a CodeT5 + GRU detector, SAFE-VulMiner verifies candidates with CodeBERT embeddings and a risk-biased self-attention term,
\[
\mathrm{Attn}(Q,K,V)=\mathrm{softmax}\!\left(\frac{QK^\top}{\sqrt{d}}+\mathbf{B}\right)V,
\]
and ISAL performs localization through syntax-guided tracing, template-constrained LLM generation, and causal consistency checks [2601.06177]. Older work on vulnerabilities found by compositional symbolic execution uses call-graph and LLVM-derived features such as node degree, distance to interface, clustering coefficient, node path length, number of vulnerabilities, infection-chain length, function size, basic-block count, and pointer parameters, then trains random forest and Naive Bayes models to predict each CVSS v3 base metric [1807.09160].

A third family is agentic and workflow-oriented. AgenticVM decomposes vulnerability management into Detection, Assessment, Prediction, Integration, Prioritisation, and Recommendation agents, combining rule-based logic, a BERT-small CVSS prediction model, and LLM-driven components under LangGraph orchestration [2605.01739]. AXE and AutoEG turn vulnerability validation into multi-agent exploit generation: AXE uses Strategist, Explorer, Exploiter, and PoC Gen modules under a grey-box setting with CWE and code-location hints, whereas AutoEG separates trigger-function construction from runtime exploit refinement under black-box conditions [2602.14345] [2604.00704]. A2 applies an analogous Planner–Executor–Validator pattern to Android, using multimodal tooling over UI, ICC, filesystem, and cryptographic attack surfaces [2508.21579].

## 4. Prioritization, scoring, and exploit-backed validation

Severity estimation is a core triage primitive. VLAI reports 82.8% accuracy on a held-out test set and about 85% agreement with eventual official severity on vulnerabilities that initially had no score, positioning text-only severity classification as a pre-CVSS assistant [2507.03607]. AgenticVM instead predicts the eight CVSS v3.1 base metrics with a BERT-small model and reports overall accuracy 89.3%, precision 89.3%, recall 89.4%, and F1 89.1%; once a complete vector is available, its Prioritisation Agent uses a configurable threshold, with CVSS \(\ge 7.0\) treated as high-priority in the experiments [2605.01739]. CASEY reports 68% accuracy for CWE identification, 73.6% severity identification accuracy, and 51.2% combined accuracy for identifying both, using an augmented NVD dataset and prompt variants that include descriptions and code hunks [2501.18908].

Risk-ranking systems often augment CVSS with exploitability and context. The SBOM-driven firmware pipeline defines the Risk Priority Score as
\[
\mathrm{RPS}=(B\times W_b)+(E\times W_e)+(C\times W_c),
\]
with \(W_b=3.0\), \(W_e=4.0\), and \(W_c=3.0\), so that \(\mathrm{RPS}=3.0B+4.0E+3.0C\); \(E\) is set to 10 for KEV-listed vulnerabilities and otherwise equals \(10\cdot P_{\mathrm{epss}}\), while \(C\) starts at 5 and is raised by evidence such as service configuration, init scripts, binary location, and inferred open ports [2601.01308]. CVE-LLM similarly makes context explicit by generating asset-specific environmental vectors and VEX judgments, with rule-based corrections that enforce consistency between `Affected`/`NotAffected`, `VEXJustification`, and vector output [2407.14640]. TRIAGE extends prioritization from severity to adversary behavior by producing ATT&CK techniques for exploitation, primary impact, and secondary impact, enabling triage based on attacker objectives rather than CVSS alone [2508.18439].

Exploit confirmation provides the strongest triage signal in this literature. AXE achieves a 30% exploitation success rate on CVE-Bench, a 3x improvement over state-of-the-art black-box baselines, and produces reproducible PoC artifacts with explicit verification oracles [2602.14345]. AutoEG evaluates 104 real-world vulnerabilities over 660 exploitation tasks and 55,440 exploit attempts, achieving an average success rate of 82.41%, with state-of-the-art baselines reaching only 32.88% [2604.00704]. A2 reduces Android findings to 82 speculative vulnerability findings on Ghera, then generates working PoCs for 51 of them, and on 169 production APKs uncovers 104 true-positive zero-day vulnerabilities, 57 of which are self-validated with automatically generated PoCs [2508.21579]. This suggests that exploit-backed validation functions as a late-stage triage mechanism that separates warnings from actionable findings.

## 5. Evaluation practices and representative empirical results

Evaluation protocols vary with the triage target. Assignment systems often use temporal splits: transformer-based bug triaging sorts reports chronologically, splits them into 11 folds, trains on folds \(1..n\), tests on fold \(n+1\), repeats up to \(n=10\), and averages over the 10 runs, with Top@K, MRR, Precision, Recall, F1, and paired \(t\)-tests used to assess ranking and component assignment quality [2310.06913]. Ranking-based CVE→ATT&CK mapping uses MAP, Recall@10, and Recall@5, while CVE-LLM evaluates `VEXCategory`, `VEXJustification`, and vector generation with micro-F1 and comments with ROUGE-L [2508.18439] [2407.14640]. Code-centric systems add localization rate, MSE/MAE for CVSS regression, and Correctness for dataset curation [2305.16615] [2601.06177] [2411.17274].

The literature reports a mix of component-level gains, noise reduction, and validation success. In developer/component assignment, DeBERTa is the strongest transformer and achieves the best F1 for component assignment across all four datasets, with F1 \(=0.815\) on Eclipse JDT, \(0.760\) on Eclipse Platform, \(0.637\) on Mozilla Core, and \(0.682\) on Mozilla SeaMonkey; however, TF-IDF + SVM remains competitive or best on some developer-assignment datasets, showing that simple baselines still matter [2310.06913]. AgenticVM reduces Train-Ticket scanner output from 3,983 raw detections to 155 unique CVEs and then to 82 prioritized items, corresponding to 97.9% reduction relative to the original detection count, while preserving a deterministic prioritization process [2605.01739].

The following examples illustrate the range of reported outcomes.

| System | Triaged output | Representative reported result |
|---|---|---|
| DeBERTa bug/vulnerability assignment | Developer/component shortlist | Eclipse Platform developer assignment Top@1 \(=0.317\), MRR \(=0.476\) [2310.06913] |
| AgenticVM | Prioritized queue from scanner findings | Train Ticket: 3,983 detections \(\rightarrow\) 155 unique CVEs \(\rightarrow\) 82 prioritized [2605.01739] |
| VLAI | Severity class from text | 82.8% test accuracy; about 85% agreement with eventual official severity [2507.03607] |
| CASEY | CWE + severity from description/code | 68% CWE accuracy; 73.6% severity accuracy; 51.2% combined accuracy [2501.18908] |
| CVE-LLM | VEX, environmental vector, comments | `VEXCategory` micro-F1 \(=0.93\), `Vector` micro-F1 \(=0.95\) in evaluation [2407.14640] |
| AutoVulnPHP | Detection + localization | 99.7% detection accuracy, 99.5% F1 score, 81.0% localization rate [2601.06177] |
| CleanVul | Function-level curation for downstream triage | 90.6% Correctness at threshold 3; 97.3% at threshold 4 [2411.17274] |
| A2 | Validated Android findings with PoCs | 78.3% coverage on Ghera; 51 validated PoCs; 104 true-positive zero-days on 169 APKs [2508.21579] |

These results suggest two recurring empirical patterns. First, triage quality improves substantially when systems move from flat detection to staged pipelines with filtering, ranking, and validation. Second, evaluation must distinguish between shortlist utility, exact label accuracy, and validation-backed actionability, since high-quality Top@K performance or validated PoCs can be operationally more important than exact Top-1 assignment [2310.06913] [2602.14345] [2508.21579].

## 6. Human oversight, failure modes, and research directions

The literature consistently treats automatic vulnerability triaging as human-in-the-loop rather than fully autonomous. AgenticVM uses confidence gating for low-confidence CVSS predictions, schema validation, and approval gates, stating that “human approval remains required for destructive or production-impacting actions” [2605.01739]. CVE-LLM prioritizes expert review for `Affected` evaluations and applies rule-based corrections after generation, while A2 separates Planner, Executor, and Validator roles so that exploit claims are independently checked against read-only oracles [2407.14640] [2508.21579].

Several failure modes recur across paradigms. In assignment settings, highly similar textual signatures induce systematic confusion between components or developers; the bug-triaging study reports positive correlation between component-text cosine similarity and confusion frequency, and shows that all models possess non-trivial orthogonality, meaning each solves some cases that others miss [2310.06913]. In LLM-based triage, prompt sensitivity, hallucinations, spurious entity substitutions, and degradation on long or heterogeneous inputs remain prominent; CVE-LLM reports incorrect version names and product names in generated comments, CASEY reports parent/child CWE confusions and information-deficit errors, and A2 documents hallucination-induced execution claims that require validator rejection [2407.14640] [2501.18908] [2508.21579]. Exploit-centric systems report that many failures stem from high-level reasoning gaps rather than low-level execution, including misread vulnerability semantics, wrong attack-surface targeting, and unmet preconditions [2602.14345].

Future directions therefore concentrate on richer context, stronger grounding, and tighter integration. The literature proposes domain-adaptive pretraining on security corpora, ensembles that exploit model orthogonality, graph-based ownership or dependency models, cost-aware orchestration, end-to-end remediation pipelines, better handling of similar classes, multilingual support, and broader industrial datasets [2310.06913] [2407.14640] [2605.01739] [2508.18439]. A plausible implication is that automatic vulnerability triaging is converging toward hybrid systems in which deterministic rules, learned ranking, structured security knowledge, and exploit-backed validation are combined, with human analysts retained as the final authority over high-impact decisions.

Source: https://www.emergentmind.com/topics/automatic-vulnerability-triaging