Programmers Are Poor and Overconfident Judges of LLM-Generated Assertions

Abstract: Code comprehension and code review are already critically important software engineering tasks, and the rising use of AI code generation tools is only increasing that importance. Generative AI has the possibility of supporting these activities, for example by augmenting code with assertions and natural-language explanations describing code behavior. However, little is known about how effective such support may be. We conduct a controlled experiment with 86 Python programmers and a follow-up think-aloud study to examine developers' ability to assess the correctness and completeness of generated assertions of varying quality, and to investigate how natural-language explanations influence these assessments. While programmers can somewhat accurately judge correct assertions (74% accuracy), they perform poorly when shown incorrect assertions (49% accuracy), despite reporting similar levels of confidence in both judgments. This difference in judgment accuracy is statistically significant (p < 0.001): the odds of a developer accurately judging a correct assertion was nearly three times higher than the odds of accurately judging an incorrect assertion (OR = 2.94). Surprisingly, natural-language explanations of assertions provided no overall benefit. Furthermore, low-quality explanations could impair specification assessment accuracy (p = 0.037, OR = 0.58) while simultaneously increasing developer confidence (p = 0.005, 3.99/5 vs. 4.25/5). Our findings suggest that, contrary to common assumptions, AI assistance may not improve the reliability of code comprehension and review. More broadly, our findings highlight the importance of helping developers evaluate machine-generated reliability artifacts, in addition to generating them.

• The paper reveals that programmers are significantly overconfident, with a 73.9% accuracy on valid assertions but only 49.0% on invalid ones.
• It uses a controlled experiment with 86 Python programmers and qualitative think-aloud studies to analyze reasoning patterns and the impact of natural-language explanations.
• The findings imply that current human-in-loop review practices may require enhanced automated verification tools to address misleading low-quality explanations and complex logical errors.

Human Evaluation of LLM-Generated Assertions: Accuracy, Confidence, and the (Non-)Role of Explanations

Introduction

The proliferation of LLMs in code generation and software engineering workflows has heightened the need for reliable mechanisms to assure the correctness of both generated code and the logical assertions (e.g., postconditions) attached to code artifacts. Recent proposals advocate for integrating LLM-generated reliability artifacts such as assertions, specifications, and tests into human-in-the-loop review pipelines, presupposing that human programmers can accurately evaluate these artifacts. The paper "Programmers Are Poor and Overconfident Judges of LLM-Generated Assertions" (2607.08885) provides the first quantitative and qualitative evaluation of this presumption, systematically studying both the performance and reasoning processes of developers assigned to evaluate LLM-generated code assertions.

Experimental Design and Methodology

The core empirical contribution is a controlled experiment with 86 Python programmers presented with Python functions, LLM-generated postcondition assertions, and, variably, natural-language comments explaining these assertions. Stimuli construction leveraged a filtered subset of the HumanEval dataset, ensuring non-trivial functions and postconditions with a clear distinction between correctness conditions. Each participant was asked to assess the correctness and completeness of each assertion, and, for subsets of assertions, whether accompanying comments (of varying quality: exact, over-specified, under-specified, or incorrect) aided their judgment.

A secondary qualitative think-aloud study was conducted with 10 additional programmers to interrogate the cognitive strategies underlying participants' decisions, with detailed qualitative coding of reasoning patterns.

Key Empirical Findings

Asymmetry in Judging Assertion Correctness

A primary quantitative finding is that programmers are much more likely to correctly endorse a valid assertion (73.9% accuracy) than to identify an invalid one (49.0% accuracy, $p < 0.001$, OR = 2.94)—a significant asymmetry. Notably, this difficulty in rejecting incorrect assertions is accompanied by uniformly high confidence ratings (≈4/5 on a Likert scale) irrespective of actual performance, indicating a pronounced overconfidence effect. Time-to-judgment data shows that correct rejection of incorrect assertions took significantly longer than incorrect acceptance, underscoring the additional cognitive load involved in error detection.

Effects of Natural-Language Explanations

Contrary to expectations and pre-registered hypotheses, the presence of natural-language comments had no statistically significant effect on correctness judgment overall ($p > 0.1$). Critically, low-quality explanations—particularly under-specified comments—resulted in a measurable drop in accuracy ($p = 0.037$, OR = 0.58) compared to exact-match comments and even a slight decrease compared to having no comment at all, while simultaneously increasing participants' confidence ($p = 0.005$, 4.25/5 vs. 3.99/5). This suggests a detrimental effect where misleading explanations not only fail to inform but actively impair human validation of generated artifacts.

Assertion Complexity and Structure

Analysis of postcondition complexity indicates that incorrect assertions exhibited somewhat higher cyclomatic complexity than correct ones, but this complexity did not mediate the primary performance asymmetry. Certain logical structures (type checks, arithmetic equalities) correlated with higher judgment accuracy (OR ∼ 2.1), whereas implications and element-property assertions were associated with lower accuracy (OR ∼ 0.7).

Notably, participants' completeness ratings for correct postconditions correlated positively (Spearman’s ρ = 0.18, $p < 0.001$) with independent mutation-based bug detection metrics, demonstrating some intuitive ability to recognize stronger specifications, distinct from their difficulty in logical error detection.

Cognitive Strategies and Reasoning Patterns

The think-aloud study identified five reasoning strategies: direct clause comparison, logic walk-throughs, positive/negative counter-examples, and intuition. Top-down approaches (beginning from the docstring or comment) predominated. Negative examples were predominantly used in attempts to falsify an assertion when a problem was suspected. However, reliance on intuition was strongly associated with errors: when intuition alone was used, incorrect judgments predominated (7 out of 12), highlighting the unreliability of non-systematic reasoning in this domain.

Theoretical and Practical Implications

This work carries significant implications for the integration of LLMs in human-in-the-loop software reliability practices. The findings empirically refute the assumption that current practitioners are reliable filters for machine-generated logical artifacts. Overconfidence is not mitigated by the presence of comments, and in some cases, misleading comments are actively harmful. Cognitive load analysis and strategy taxonomy suggest that evaluation mechanisms relying on direct clause comparison are most tractable for humans, but real-world specifications often evade such tractability.

From a tool-design perspective, the results indicate that next-generation developer assistance must move beyond artifact generation to actively assist in the verification, not merely the explanation, of logical assertions. Flagging assertions with high complexity or with logical constructs shown to induce lower human accuracy may be practical short-term mitigations. In the longer term, approaches that facilitate counterexample construction or interactive code-explanation alignment checks warrant exploration.

Furthermore, these results problematize the growing reliance on human review as a quality gate for LLM outputs, particularly as code and specification complexity escalate in industrial contexts. Reliance on inadequate evaluation mechanisms may systematically introduce undetected logical errors into production systems, even as developer trust remains high.

Future Directions

Potential avenues for future investigation include: (1) augmenting code review pipelines with automated tools that highlight potential specification errors based on human performance bottlenecks identified herein; (2) designing and empirically evaluating interventions—such as structured assertion walkthroughs or counterexample-guided explanations—expressly tailored to human cognitive constraints; (3) extending analysis to multi-function/multi-file software artifacts and to alternative programming languages; and (4) investigating mechanisms to calibrate or attenuate developer confidence in the presence of LLM-generated artifacts.

Conclusion

This study provides robust evidence that programmers systematically overestimate their ability to assess the correctness of LLM-generated logical assertions and that natural-language explanations—especially if of low quality—can exacerbate miscalibration. These findings point to an urgent need for software engineering workflows and AI-assisted tooling that directly address the human limitations in evaluating machine-generated reliability artifacts, lest the promise of greater automation in code reliability be undermined by undetected logical flaws and misplaced human trust.