Calibration, Not Compilation: Detecting and Repairing Misspecified Probabilistic Programs Written by Language Models

Abstract: LLMs increasingly write probabilistic programs (in NumPyro, Stan, or Pyro), but a program that compiles, runs, and passes every unit test can still be \emph{statistically} wrong -- a Gaussian likelihood for heavy-tailed data, a Poisson for over-dispersed counts, an invalid prior support, or a pathological parameterization. The right verifier is therefore not a test suite but the Bayesian workflow itself: posterior predictive checks, simulation-based calibration, sampler diagnostics ($\hat R$, divergences, ESS), and held-out predictive density. We study this calibration oracle along three axes. \textbf{Detection:} on a benchmark of $14$ misspecification types across $10$ model families ($200$ instances), it flags the bug with AUC $0.97$ ($88\%$ at $2\%$ FPR \emph{when handed the correct reference program, an upper bound}) -- and a fully \emph{reference-free} version that uses no correct program reaches $62$--$78\%$ (the upper figure from a small automated model search), versus $0\%$ for a unit-test oracle. \textbf{Repair:} used as feedback in an LLM repair loop across fifteen models, calibration significantly outperforms unit-test feedback -- which is itself \emph{significantly worse than no feedback at all}, a passing test inducing false confidence that suppresses repair -- and improves over no feedback on strong-but-unsaturated models (GPT-5.1 $33{\to}92\%$, Claude $75{\to}100\%$; paired McNemar, $n{=}228$). \textbf{Reality:} on programs LLMs write from scratch for neutral briefs, $15$--$47\%$ of runnable ones are statistically misspecified (unit tests catch none), and calibration-guided repair significantly beats LLM-as-judge review, a Bayesian-workflow checklist, and data-summary self-debug. Across all three, the lesson is the same: for probabilistic programs, correctness is calibration, not compilation.

• The paper introduces a calibration-as-oracle approach that detects 88% of misspecified probabilistic programs, including 93% of elusive, code-invisible errors.
• It presents a comprehensive benchmark and detailed ablation studies to identify the roles of held-out predictive density and PPC in diagnosing statistical misfits.
• The study demonstrates that calibration feedback significantly improves LLM repair rates compared to unit-test feedback, even reversing harmful effects from the latter.

Calibration-as-Oracle: Detecting and Repairing Misspecified Probabilistic Programs from LLMs

Overview of Objectives and Contributions

This paper presents a rigorous investigation into the limitations of conventional unit-testing approaches for LLM-generated probabilistic programs. It introduces the notion that Bayesian calibration diagnostics—encompassing posterior predictive checks (PPC), simulation-based calibration (SBC), sampler geometry diagnostics, and held-out predictive density—function as a superior oracle for both detecting and repairing statistical misspecification. The work entails the development of a comprehensive misspecification benchmark, systematic detection and repair studies across state-of-the-art LLMs, and a thorough ablation of the calibration oracle's components to characterize its diagnostic efficacy.

Failure of Unit Tests and the Calibration-as-Oracle Paradigm

The central technical claim is that compilation and execution status—or even passing all possible unit tests—are not sufficient for the statistical validity of an LLM-generated probabilistic program. The authors demonstrate that code-invisible misspecifications (e.g., using a Gaussian instead of a Student-$t$ for a heavy-tailed dataset, or Poisson instead of Negative Binomial for overdispersed counts) elude all test-based detection, instead requiring distributional calibration as verified through Bayesian workflow diagnostics.

Figure 1: Pipeline for calibration-as-oracle repair, with the Bayesian workflow providing structured diagnostic feedback to the program-repair loop.

By formalizing the calibration oracle, the paper offers an explicit decision rule aggregating sampler diagnostics, PPCs, SBC checks, and predictive likelihood-based scoring rules. This approach enables automated flagging of statistical misfits, with structured feedback informing program repair.

Empirical Evaluation: Detection Power by Calibration

The detection benchmark includes 200 instances spanning 10 model families and 14 misspecification types, distinguishing code-invisible from code-visible bugs. Results indicate that the calibration oracle detects 88% of misspecified programs (93% within code-invisible classes) at a 2% false-positive rate. In contrast, the unit-test oracle fails to detect any such bugs, conforming to the formal result that such tests are blind to statistical misspecification.

Figure 2: Calibration detection rates by misspecification type; calibration vastly outperforms unit tests, which catch none.

Disaggregated analysis reveals that standard failure modes are reliably detected, with the exception of stochastic volatility and subtle link function errors where chosen PPC statistics are insufficiently sensitive.

Figure 3: ROC curve of the calibration oracle used as a continuous detector across 200 instances (AUC = 0.97), showing strong discrimination between correct and buggy programs.

Reference-Free Calibration and Oracle Component Analysis

Significantly, the calibration oracle can operate in a reference-free mode—using only model-internal diagnostics—performing at 62% detection for code-invisible bugs. Augmenting this with a GLM baseline or a small automated model search yields detection rates up to 78%, all without access to ground-truth programs.

Ablation studies establish that held-out predictive density and PPC are the primary contributors to detection; sampler diagnostics and SBC are less diagnostic for typical statistical misfits, with SBC alone being nearly uninformative except for inference failures.

Repair Loop: Comparative Feedback and Non-Monotone Model Performance

The core repair experiment evaluates feedback mechanisms (no feedback, unit test, calibration diagnostics) across fifteen LLM families. Calibration feedback consistently enables higher fix rates on code-invisible bugs, especially for strong-but-unsaturated models such as GPT-5.1 and Claude Sonnet, elevating fix rates by 59 and 25 percentage points, respectively.

Figure 4: Fix-rate improvement with calibration feedback on code-invisible bugs, relative to the better of “none” or “unit test” feedback, by model.

Notably, unit-test feedback is not merely inert but harmful: a passing verdict suppresses further repairs, yielding lower fix rates than even no feedback across several capable LLMs (e.g., Claude Sonnet: 25% with test feedback vs. 75% with no feedback). The benefit of calibration feedback is largest for models competent enough to interpret the diagnostic but not so powerful as to repair via blind rewriting.

Diagnosis Granularity and Bug Localization

Calibration diagnostics often localize bugs with high specificity—the main confusion arises among error modes that induce similar predictive under-dispersion (over-dispersion, tight priors, prior-data conflict). Otherwise, wrong-likelihood and parameterization errors are cleanly separated by the structured feedback.

Figure 5: Diagnostic-to-bug-type confusion matrix, showing that most misclassifications occur among under-dispersion symptoms.

From Injected Bugs to Real-World LLM-Generated Programs

The study further evaluates programs written from scratch by LLMs in response to neutral task briefs. Of the programs that compile and run (80–100% of cases depending on the LLM), 15–47% are flagged as statistically misspecified by the calibration oracle. Unit tests never catch these errors. Calibration-guided repair robustly outperforms all baselines—including LLM-as-judge code review, Bayesian-workflow checklists, or data-summary-driven self-debugging—by statistically significant margins.

Figure 6: Calibration-based repair success on real LLM-generated misspecified programs, demonstrating superiority over all alternative feedback mechanisms.

Limitations and Failure Modes

Not all misspecifications are detectable by the chosen statistics: some link errors evade detection unless statistics targeting probability calibration are employed. Calibration also cannot distinguish models with equivalent predictive distributions but different causal structures. Weak LLMs remain unresponsive to even well-localized diagnostics, and the approach struggles with high-dimensional models where computationally intensive diagnostics (notably SBC) are prohibitive. Inference failures are sometimes indistinguishable from model misspecification, as evidenced by nonzero false-positive rates especially on hard model classes.

Theoretical and Practical Implications

This research substantiates the argument that calibration-oriented workflows, not syntactic or property-based testing, are essential for trustworthy LLM-assisted statistical modeling. The diagnostic loop proposed here serves as a foundation for agentic modeling systems, where statistical correctness is defined operationally by calibration to data. The non-monotonicity of repair efficacy reinforces the importance of both LLM capability and the informativeness of feedback.

In practice, integrating Bayesian workflow diagnostics as first-class citizens in probabilistic modeling toolchains, and using their outputs to autonomously close repair loops, can substantially elevate the utility of LLM-based code generation in statistics and data science. Theoretically, the findings suggest that code-execution or tests as “pass/fail” oracles are obsolete for model correctness in statistical domains, motivating broader adoption of workflow-based oracles.

Future Directions

Key open directions include extending the calibration-as-oracle framework to higher dimensionality and model complexity (spatiotemporal processes, deep probabilistic models), integrating more powerful and computationally scalable diagnostics, developing better communication modalities between diagnostic oracles and LLMs for repair, and creating human-validated datasets of real LLM-generated probabilistic programs with trusted labels.

Conclusion

The paper delivers a compelling and robust case for Bayesian calibration as the definitive criterion for statistical correctness in LLM-generated probabilistic programs. The calibration-as-oracle paradigm is both theoretically justified and empirically validated as superior for detection and repair of statistical misfits, while also exposing the actual harm of reliance on unit-testing workflows. This methodology establishes a principled foundation for the next generation of LLM-assisted tools in probabilistic programming and statistical modeling.