Predicting LLM Failures from Internal Generation Dynamics

Determine whether large language models can anticipate their own failures by examining their internal generation dynamics, specifically the evolving hidden states and attention-routing patterns during inference, to enable intrinsic self-verification without external judges or multi-sample consistency.

Background

LLMs often produce fluent but incorrect outputs and struggle to recognize their own mistakes. Most existing correctness estimation methods rely on signals external to the model’s internal dynamics, such as external judges, multi-sample agreement, or text-based confidence, which add compute or correlate weakly with true correctness.

This work proposes Gnosis, a lightweight self-awareness mechanism that decodes reliability cues from internal hidden states and attention patterns. The authors explicitly highlight the broader research uncertainty as to whether correctness can be predicted directly from the intrinsic generation process rather than external supervision, motivating their investigation into trajectory-level internal signals.

References

A fundamental open question is whether LLMs can anticipate their own failures by examining the internal dynamics that govern their generation process.

— Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuits  (2512.20578 - Ghasemabadi et al., 23 Dec 2025) in Section 1 (Introduction)

One can ask, for example, whether a recurrent system can learn to detect information represented in its own ongoing state, evaluate an internally represented inconsistency or uncertainty, and selectively modify that state before subsequent inference. Self-correction could then be studied as a causal transformation of an accessible internal state rather than inferred only from improvement in the final output. Whether such dynamics emerges is an empirical question and should not be assumed from recurrence or self-reference alone.

— Beyond Episodic AI: Cognitive Field Networks for Biologically Inspired Persistent Cognition  (2609.16752 - Chae, 15 Sep 2026) in Section V, Discussion

However, two questions are left unresolved: whether a generator's output can be checked reliably by the generator itself, and how often an incorrect acceptance is produced by the external decision mechanism.

— Requirement-Bound Verified Commissioning: A Frozen Four-Billion-Parameter Local Model as a Candidate Generator under an External Acceptance Layer with Verification and Release Authority  (2609.30219 - Iscan, 24 Sep 2026) in Section 1, Introduction

The comparison also leaves open whether compressing the profile to one number discards useful layer-wise information, so the next comparison keeps the entire profile.

— Auditing Routing Entropy as an Uncertainty Signal in Attention-Residual Transformers  (2610.01495 - Liang et al., 1 Oct 2026) in Section 4.2, “Scalar routing and calibration”

We therefore did not evaluate the probe on real correctness labels, and its behaviour there is unknown.

— Auditing Routing Entropy as an Uncertainty Signal in Attention-Residual Transformers  (2610.01495 - Liang et al., 1 Oct 2026) in Section 4.4, “Conditioning on the complete output”

Future work may extend trajectory-based error signals as process-level rewards in agentic settings and verify whether the signature persists at larger model scales.

— Taking the PULSE of Medical Reasoning: Step-Level Retrieval Timing for Knowledge-Grounded Inference  (2610.20292 - Zhao et al., 14 Oct 2026) in Section Conclusion, p. 8