LLaVul: Multimodal Vulnerability Reasoning

• The paper introduces LLaVul, a multimodal model that fuses code semantics with security-focused questions to generate interpretable vulnerability explanations.
• It employs a two-stage training regime, pretraining a code-to-text projector before fine-tuning on real-world vulnerability QA to maintain full-code context.
• Quantitative evaluations reveal substantial improvements over baselines in both QA metrics and vulnerability classification performance.

Searching arXiv for the main LLaVul paper and closely related papers mentioned in the provided data.

LLaVul, short for “Large Language and Vulnerability assistant,” is a multimodal large-LLM designed for open-ended, interpretable vulnerability reasoning over source code rather than for vulnerability analysis framed only as binary or multiclass classification. It fuses code semantics and security-focused natural-language questions into a shared transformer space, supports fine-grained question answering about real-world software flaws, and is evaluated on both QA and detection tasks. The system is positioned as an extension of the LLaVA multimodal instruction-tuning paradigm in which the image encoder is replaced with a code encoder and downstream tuning is specialized for security QA (Jararweh et al., 22 Sep 2025).

1. Conceptual scope and problem framing

LLaVul is motivated by the claim that many current approaches oversimplify vulnerability analysis by treating it primarily as classification, even though real-world vulnerability reasoning is nuanced and context-dependent. Its central design choice is to cast vulnerability understanding as a QA problem over paired code and natural-language queries, with the objective of producing explanations that are more interpretable and more security-focused than label-only outputs (Jararweh et al., 22 Sep 2025).

A common misconception is that “multimodal” in this setting implies vision-language processing. In LLaVul, the modalities are source code and natural-language security questions. The model therefore uses multimodality to align program text with analyst-style instructions and answers, not to combine code with images or other sensory inputs. This distinction is explicit in the architecture: LLaVul extends LLaVA by replacing the image encoder with a code encoder (Jararweh et al., 22 Sep 2025).

This framing contrasts with work that treats software vulnerability detection as a binary prediction problem. For example, “Evaluating LLaMA 3.2 for Software Vulnerability Detection” fine-tunes a 1.23 B-parameter decoder-only Transformer with LoRA on a balanced subset of 10,000 vulnerable and non-vulnerable C/C++ functions and evaluates binary cross-entropy training with precision, recall, accuracy, and F1-Score, reporting an F1-Score of 66% on its test setting (Gonçalves et al., 10 Mar 2025). This suggests that LLaVul belongs to a different methodological strand: it preserves the detection use case, but prioritizes reasoning and answer generation as first-class outputs.

2. Architectural design

At the architectural level, LLaVul comprises a code encoder, a language encoder, a learnable projection module, and a transformer decoder. Given a code snippet $\mathcal{S}$ of length $T_1$ tokens, the model maps it through the CodeSage-small encoder $\phi_{cs}$ to obtain

$$Z_v = \phi_{cs}(\mathcal{S}) \in \mathbb{R}^{d \times T_1}.$$

In parallel, a security-focused question or instruction $X_q$ of length $T_2$ is encoded using a Vicuna 1.5 (7B) LLM encoder $\phi_{LLM}$ as

$$H_q = \phi_{LLM}(X_q) \in \mathbb{R}^{k \times T_2},$$

with $d$ denoting the code-token embedding dimension and $k$ the language-token embedding dimension (Jararweh et al., 22 Sep 2025).

Because the code and language streams inhabit different representational spaces, LLaVul uses a learnable linear projector $T_1$0 to map code embeddings into the LLM hidden space:

$T_1$1

The projected code tokens are then concatenated with the question tokens along the sequence dimension,

$T_1$2

and the merged representation is fed into the decoder $T_1$3 for autoregressive generation (Jararweh et al., 22 Sep 2025).

The decoder is a standard transformer-autoregressive block based on Vicuna 7B with self-attention and cross-attention to the fused embeddings $T_1$4. At generation step $T_1$5, the model samples from

$T_1$6

The self-attention mechanism inside each transformer layer is written as

$T_1$7

where $T_1$8, $T_1$9, and $\phi_{cs}$0 are linear projections of the decoder hidden states. Architecturally, this makes LLaVul a code-language fusion model in which security reasoning is mediated by the LLM decoder but grounded in a projected code representation (Jararweh et al., 22 Sep 2025).

3. Training objectives and optimization regime

LLaVul is trained in two stages. The first stage pretrains the code-to-text projector on code summarization data derived from CodeXGLUE. During this stage, both $\phi_{cs}$1 and $\phi_{cs}$2 are frozen, and only the projection matrix $\phi_{cs}$3 is trained. Each example consists of a code snippet $\phi_{cs}$4 and a target summary $\phi_{cs}$5, and the loss is the negative log-likelihood of the target summary tokens conditioned on the projected code representation (Jararweh et al., 22 Sep 2025).

The second stage fine-tunes the model on vulnerability QA. Here, the projector $\phi_{cs}$6 and the LLM’s LoRA adapters are unfrozen, while the code encoder $\phi_{cs}$7 remains frozen. Each training instance is a triple $\phi_{cs}$8 consisting of code, question, and answer, and the optimization objective is again autoregressive negative log-likelihood over the answer tokens conditioned on the fused representation $\phi_{cs}$9 (Jararweh et al., 22 Sep 2025).

An optional vulnerability-classification pathway is added for detector-style evaluation. In that setting, the model extracts a pooled output $$Z_v = \phi_{cs}(\mathcal{S}) \in \mathbb{R}^{d \times T_1}.$$0 from $$Z_v = \phi_{cs}(\mathcal{S}) \in \mathbb{R}^{d \times T_1}.$$1 or from the decoder, applies a linear head $$Z_v = \phi_{cs}(\mathcal{S}) \in \mathbb{R}^{d \times T_1}.$$2, and minimizes binary cross-entropy over the two classes. The fine-tuning objective combines $$Z_v = \phi_{cs}(\mathcal{S}) \in \mathbb{R}^{d \times T_1}.$$3 with $$Z_v = \phi_{cs}(\mathcal{S}) \in \mathbb{R}^{d \times T_1}.$$4, where $$Z_v = \phi_{cs}(\mathcal{S}) \in \mathbb{R}^{d \times T_1}.$$5 trades off QA generation and classification. The paper explicitly states that no separate contrastive loss was employed; alignment is driven instead by the generation objective (Jararweh et al., 22 Sep 2025).

This training design is significant because it separates representation alignment from domain specialization. Projector pretraining learns a code-to-language interface, while instruction tuning on vulnerability QA injects the security-specific behavior. The ablation results reported later strongly support that division of labor.

4. Data resources and curation

LLaVul relies on two curated corpora: one for projector pretraining and one for security QA fine-tuning and evaluation. The pretraining corpus is derived from CodeXGLUE and converted into “Summarize this script” QA pairs. The fine-tuning corpus, called LLaVul-QA, is constructed from real-world vulnerabilities drawn from CVEfixes and MSR, spanning C, C++, and JavaScript, and annotated with security-focused questions about vulnerability type, location, mechanism, fix, and severity (Jararweh et al., 22 Sep 2025).

Corpus	Content	Scale
CodeXGLUE pretraining	Code-summary pairs converted into “Summarize this script” QA	549,708 pairs; 78.1 M code tokens
LLaVul-QA fine-tuning	Vulnerable functions with security-focused QA	104,977 functions; 6,760 CVEs; 406,885 QA pairs; 60.3 M code tokens + descriptions

The LLaVul-QA dataset contains 1–11 questions per code sample, with mean 3.87, which operationalizes vulnerability understanding as a multi-question reasoning task rather than a single label lookup. This supports varied query types over the same vulnerable function and makes the benchmark explicitly conversational and interpretability-oriented (Jararweh et al., 22 Sep 2025).

The paper also evaluates vulnerability classification on DiverseVul, using 18,945 vulnerable and 18,945 non-vulnerable functions. That benchmark role is narrower: it tests whether representations learned for QA can transfer to detector-style classification. A plausible implication is that LLaVul is intended not to replace classification benchmarks, but to subsume them within a broader code-understanding framework.

5. Quantitative performance

On the held-out LLaVul-QA test set, LLaVul is compared against general-purpose LLMs and code-specialized LLMs using lexical metrics—BLEU-2, BLEU-4, ROUGE-1, ROUGE-2, ROUGE-L, and METEOR—as well as semantic similarity via BERTScore. The reproduced results report CodeT5-base-multi-sum at BLEU-2 0.043, BLEU-4 0.017, ROUGE-1 0.199, ROUGE-2 0.048, ROUGE-L 0.161, METEOR 0.129, and BERTScore 0.821, whereas LLaVul achieves BLEU-2 0.267, BLEU-4 0.168, ROUGE-1 0.390, ROUGE-2 0.234, ROUGE-L 0.350, METEOR 0.353, and BERTScore 0.910. The paper characterizes this as a more than fivefold gain in BLEU-4 and more than eight points in BERTScore over the strongest baseline (Jararweh et al., 22 Sep 2025).

For vulnerability classification on DiverseVul, the paper reports two LLaVul variants. The generative variant reaches accuracy 0.52, precision 0.55, recall 0.52, and F1 0.48, improving over CodeT5 used generatively at accuracy 0.47, precision 0.48, recall 0.47, and F1 0.36. The encoder-only LLaVul variant reaches accuracy 0.75, precision 0.71, recall 0.84, and F1 0.77, compared with CodeBERT at F1 0.69, GraphCodeBERT at F1 0.74, and CodeBERTa at F1 0.76 (Jararweh et al., 22 Sep 2025).

The ablation study clarifies where these gains arise. Using only the LLM without code yields BLEU-4 0.082, ROUGE-L 0.287, METEOR 0.313, and BERTScore 0.886. Using only projector pretraining yields BLEU-4 0.004, ROUGE-L 0.045, METEOR 0.080, and BERTScore 0.627. Fine-tuning with 100 code tokens yields BLEU-4 0.031, ROUGE-L 0.109, METEOR 0.249, and BERTScore 0.729, whereas fine-tuning with full tokens yields BLEU-4 0.168, ROUGE-L 0.350, METEOR 0.353, and BERTScore 0.910. The paper interprets this as evidence for the necessity of instruction tuning with real QA data, the value of full-code context over truncated context, and the benefit of incorporating code embeddings into the LLM (Jararweh et al., 22 Sep 2025).

6. Interpretability, limitations, and future directions

The qualitative analysis emphasizes that LLaVul is intended to reason about vulnerabilities in a way that remains inspectable. In the example of CVE-2017-12595 involving the QPDF inline-image parser, the question “What is the impact?” is answered by the model with “Denial of Service (buffer under-read and application crash) or …,” which the paper presents as grounding in both code and textual description. For CVE-2022-3138, asked “What is the type of vulnerability?”, the model outputs “Cross-site Scripting (XSS) – Generic …,” again aligning with the ground-truth type label (Jararweh et al., 22 Sep 2025).

The same analysis also identifies failure modes. For CVE-2021-27341, when asked whether a vulnerable snippet is present, the ground truth states that the snippet is not included, but the model answers by asserting that “The provided code is not related to a vulnerability. The vulnerability is in helpController.php …,” which the paper labels hallucination because the filename is not present in the evidence (Jararweh et al., 22 Sep 2025). This is a critical limitation for security workflows, where unsupported specificity can mislead triage or remediation.

The broader impacts section proposes several application settings: IDE integration for on-the-fly vulnerability explanations, automated security review bots that can pinpoint flaw locations and suggest fixes, and human-in-the-loop validation pipelines to catch and correct hallucinations. Future work is described along three axes: reducing hallucinations via two-stage verification or retrieval-augmented generation, extending from single functions to multi-file or whole-project reasoning, and incorporating richer control-flow or data-flow graphs into the encoder (Jararweh et al., 22 Sep 2025).

Within the landscape of vulnerability analysis systems, LLaVul is therefore best understood as a multimodal QA-centered security model whose primary contribution is not only improved task performance, but a redefinition of vulnerability reasoning as conversational, context-sensitive, and partially inspectable. Its results indicate that code-language fusion can substantially improve security-focused QA while remaining competitive with specialized encoders on classification benchmarks (Jararweh et al., 22 Sep 2025).