- The paper demonstrates limited spontaneous emergence of comprehensive software world models in coding LLMs, with F1 scores below 0.85 and systematic calibration errors.
- It introduces an evaluation framework using a real-world Python bug-fix dataset with detailed observables like execution time, memory, and profiling granularities.
- Empirical results reveal that even top models exhibit low profiler ranking metrics and biased resource predictions, highlighting the need for enhanced dynamic reasoning.
Evaluation of Implicit Software World Models in Coding LLMs
Motivation and Context
This paper addresses a salient gap in the evaluation of code-fluent LLMs: the extent to which these models develop implicit "software world models," beyond merely predicting control flow or program outputs. The work aims to probe LLMs' capacity for practical reasoning about software execution—including resource utilization (e.g., peak memory, wall-clock time), exception prediction, and profiling at method and line granularity—reflecting the multifaceted demands of software engineering. Prior benchmarks have focused predominantly on isolated, often synthetic snippets, emphasizing outputs or code coverage, thereby neglecting holistic, real-world software behaviors contingent on broader context.
Dataset Construction and Experimental Design
The evaluation framework expands upon function-level benchmarks by leveraging SWE-bench Verified, a curated corpus of real-world bug-fix tasks across 12 Python repositories. The authors construct a dataset of 435 examples, each paired with structured ground truth execution observables generated by instrumented runs in a controlled Docker environment. These observables include:
- Test outcomes and exceptions
- Wall-clock execution time
- Peak memory consumption
- Fine-grained (method and line) ranked profiler outputs for both time and memory
Contextual information for each instance is maximally constructed up to 500,000 characters, simulating realistic software comprehension scenarios where critical context is necessary but nontrivial to fit within LLM input windows.
The evaluation task for each model is to generate a structured JSON object containing fields reflecting both direct outcomes and execution resource predictions, coupled with ranking lists for profiling results.
Three principal axes of evaluation are introduced:
- Test Outcome Prediction: Binary classification (pass/fail) with precision, recall, and F1.
- Resource Prediction: Calibration and systematic bias assessment for log-scale wall time and peak memory predictions. Models' outputs are linearly regressed against true log-values to extract slope, intercept, and mean absolute error.
- Profiler Ranking: NDCG@5 and Recall@5 for method/line/time/memory profiles, reflecting models’ ability to localize execution and resource hotspots at multiple granularities.
Empirical Results and Observations
Performance across twelve models (Anthropic Claude-4 variants, OpenAI GPT-5 family, Alibaba Qwen3, and CWM from FAIR) demonstrates consistent brittleness. The paper highlights several critical observations:
- Test Outcome Prediction: Even highest-performing models (e.g., GPT-5.5) achieve F1 scores below 0.85, with recall universally low, indicating a default bias toward reporting passing tests. The best open-weight model underperforms compared to API-hosted proprietary models.
- Resource Prediction: Systematic calibration error is evident in all models—slope values significantly <1 and intercepts >0 (indicating consistent overestimation). Mean absolute errors in log-space remain nontrivial, and predictions cluster around "round" numbers, especially in smaller or open-weight models, reducing utility for real-world resource planning.
- Profiler Ranking: Models rarely identify true hotspots. NDCG@5 and Recall@5 values are low (<0.2), and many predictions are for methods/lines not executed at all, indicating hallucinations of execution scope. Notably, even when execution scope is correct, the ranking order remains largely uninformative, highlighting a fundamental weakness in execution-modeling capabilities.
- Model Specifics: The trace-trained explicit world model CWM does not outperform comparably sized LLMs on these tasks, suggesting current architectures and training regimes do not imbue these models with robust implicit software execution understanding.
Implications and Research Trajectory
The results establish that, even with abundant training data, the spontaneous emergence of comprehensive software world models in code LLMs is limited. Control flow reasoning—the focus of most prior benchmarks—does not subsume the richness of real-world software dynamics, where resource usage, exception contexts, and interaction effects predominate.
The systematic errors in resource predictions and the difficulty of localizing time/memory hotspots illuminate structural deficiencies in models’ "mental execution" of code, pointing to deficits in learned representations related to dynamic semantics and execution context. These insights motivate future research into:
- Augmenting training with explicit dynamic traces or weak supervision from profilers
- Prompt engineering and context curation to better expose execution-relevant detail
- Model architectures that more explicitly represent dynamic state and resource flows
- Expanding evaluation to encompass build/dependency resolution, CI/CD, concurrency, and deployment effects
This direction is crucial for trustworthy agentic coding systems and automated software maintenance tools, where robust prediction of side effects and resource impacts is mission-critical.
Conclusion
The paper provides a methodical, empirically grounded evaluation framework and dataset for implicit software world model assessment, demonstrating clear limitations in frontier and open-weight LLMs' execution reasoning abilities. By making both the dataset and code available, the authors establish a reproducible testbed for future research. Targeted advances in representation, training, and evaluation for dynamic software properties are imperative for meaningful progress toward agentic AI in software engineering contexts.
