Vulnerability Detection with Interprocedural Context in Multiple Languages: Assessing Effectiveness and Cost of Modern LLMs

Abstract: LLMs have been a promising way for automated vulnerability detection. However, most prior studies have explored the use of LLMs to detect vulnerabilities only within single functions, disregarding those related to interprocedural dependencies. These studies overlook vulnerabilities that arise from data and control flows that span multiple functions. Thus, leveraging the context provided by callers and callees may help identify vulnerabilities. This study empirically investigates the effectiveness of detection, the inference cost, and the quality of explanations of four modern LLMs (Claude Haiku 4.5, GPT-4.1 Mini, GPT-5 Mini, and Gemini 3 Flash) in detecting vulnerabilities related to interprocedural dependencies. To do that, we conducted an empirical study on 509 vulnerabilities from the ReposVul dataset, systematically varying the level of interprocedural context (target function code-only, target function + callers, and target function + callees) and evaluating the four modern LLMs across C, C++, and Python. The results show that Gemini 3 Flash offers the best cost-effectiveness trade-off for C vulnerabilities, achieving F1 >= 0.978 at an estimated cost of $0.50-$0.58 per configuration, and Claude Haiku 4.5 correctly identified and explained the vulnerability in 93.6% of the evaluated cases. Overall, the findings have direct implications for the design of AI-assisted security analysis tools that can generalize across codebases in multiple programming languages.

• The paper shows that interprocedural context can degrade performance, with GPT models experiencing up to 25 percentage point accuracy losses.
• It employs a rigorous evaluation of four LLMs under three context configurations across C, C++, and Python, highlighting model-specific behavior.
• Findings reveal that a function-level context is often optimal for balancing detection accuracy, cost-efficiency, and explanation quality.

LLMs for Interprocedural Vulnerability Detection Across Multiple Languages

Motivation and Contextual Challenge

Automated vulnerability detection increasingly relies on LLMs due to their semantic reasoning over code. However, most prior research restricts evaluation to intra-function vulnerabilities, neglecting those arising from interprocedural dependencies or cross-function data/control flow [li24]. Such omission is critical for vulnerability classes requiring context propagation, including use-after-free, buffer overflows induced by caller/callee effects, and injection vulnerabilities with argument mediation.

This work empirically evaluates the effect of varying interprocedural context on LLM-based vulnerability detection. Four LLMs—Claude Haiku 4.5, GPT-4.1 Mini, GPT-5 Mini, and Gemini 3 Flash—are scrutinized for detection efficacy, inference cost, and explanation quality across 509 vulnerabilities from the ReposVul dataset [wang2024repository] spanning C, C++, and Python. Context configurations include code-only (CO), code plus callee (CC), and code plus caller (CK).

Study Design and Methodology

The evaluation framework leverages three precise context variants. Each vulnerability instance is presented as either the core function, augmented with callees, or augmented with callers. Prompt templates enforce structured explanations and binary classification.

The selected LLMs ensure long-context window compatibility and represent a spectrum of cost-performance profiles. All models are queried via official APIs at temperature zero, producing consistent output. Performance metrics adhere to recall (accuracy, precision is always 1 by construction), F1-score, and inference cost per token. Statistical significance of context-dependent accuracy is assessed via McNemar's test and effect sizes using the phi coefficient.

Qualitative explanation quality is manually scored along two axes: correctness (type, location, and root cause), and comprehensiveness (with exploitation vector or repair suggestion). Stratified sampling guarantees statistical confidence per language.

Empirical Results

Effectiveness of Interprocedural Context (RQ1)

Inclusion of interprocedural context does not universally improve detection accuracy. For Claude Haiku 4.5 and Gemini 3 Flash, accuracy is stable regardless of context. In contrast, GPT-4.1 Mini and GPT-5 Mini suffer pronounced performance degradation when context is expanded, especially in C. GPT-4.1 Mini loses up to 25 percentage points in accuracy with callee/caller context, while GPT-5 Mini loses up to 11 points. The CO configuration consistently outperforms CC/CK across almost all models, indicating that additional context can introduce irrelevant noise, particularly in smaller LLMs.

Figure 1: Accuracy distribution for Code-only (CO), Code+Callers (CK), and Code+Callees (CC) for C code across models, highlighting context-induced accuracy degradation.

Language-Dependent Effectiveness (RQ2)

Language strongly modulates performance sensitivity. In C, the inclusion of interprocedural context most severely impacts the GPT models, with statistically significant accuracy loss. In Python, the effect is attenuated and not statistically significant. C++ subset statistics are unreliable due to insufficient sample size. This confirms prior findings that vulnerability manifestation and detection complexity are language-dependent, due to syntactic and semantic variance in memory management and abstraction level.

Cost-Performance Trade-off (RQ3)

Token consumption doubles when interprocedural context is added, directly inflating inference cost. Gemini 3 Flash demonstrates optimal cost-performance for C vulnerabilities (F1 ≥ 0.978 at ~$0.50–$0.58 per configuration). Claude Haiku 4.5 achieves slightly higher F1 at more than twice the cost. The GPT models exhibit price-performance asymmetry: lowest cost, but accuracy suffers significantly with context expansion. Across all languages, additional context does not yield proportional accuracy gains, and in many cases, degrades both performance and efficiency.

Explanation Quality (RQ4)

Manual assessment reveals high explanatory quality overall, but with model-dependent granularity. Claude Haiku 4.5 achieves perfect correctness and completeness in 93.6% of evaluated cases. Gemini 3 Flash matches correctness but lags in comprehensiveness. The GPT models demonstrate a higher rate of incorrect or incomplete explanations (zero-score rate five times higher than Haiku/Gemini), reflecting their reduced reasoning and generalization depth. Detection quality and explanation quality are closely correlated: models committing more classification errors also produce less actionable explanations.

Theoretical and Practical Implications

Interprocedural context sensitivity is not a universal property, but is tightly coupled to LLM architecture and code language. Prompt engineering and context augmentation strategies must be tuned per model and per language—blindly increasing context can degrade performance and inflate operational cost. Model selection should reflect application priorities: Claude Haiku 4.5 excels in explainability, while Gemini 3 Flash maximizes cost-accuracy balance. The evaluation methodology presented sets a new baseline for rigorous assessment of LLMs in real-world, multi-language vulnerability detection scenarios.

Empirical evidence suggests that a function-level context (CO) is sufficient for robust detection in the best-performing models. The assumption that “more context improves detection” is invalidated for certain LLMs and languages. This insight informs the design of scalable, cost-efficient AI-powered vulnerability detection pipelines. Integrating LLMs with interprocedural analysis should be approached judiciously, considering model and language-specific behaviors.

Future Directions

Further research should analyze the mechanistic underpinnings of context sensitivity in LLMs, evaluate higher-capacity and code-specialized models (e.g., CodeLlama, CodeBERT), and benchmark detection/explanation quality on functions exhibiting more complex interprocedural coupling. Longitudinal studies on token consumption and cost scaling in production environments will clarify the practical deployment of context-sensitive LLMs. Additionally, incorporation of non-vulnerable functions and balanced datasets is essential for unbiased false positive rate characterization.

Conclusion

This paper demonstrates that interprocedural context does not consistently improve vulnerability detection with LLMs; only certain architectures retain robustness to context expansion. The best-performing models achieve high accuracy and explanation quality with function-level context, while additional context can degrade both performance and cost-efficiency in competitive models. These findings challenge prevailing assumptions and inform the next generation of LLM-assisted security analysis tools for heterogeneous codebases (2604.08417).