The strength of clinical evidence is recoverable from language model representations but not from their stated grades

Abstract: LLMs increasingly summarize clinical evidence, where a claim's weight depends on how strongly it is supported. Yet these models convey confidence poorly, and properties they never state, such as truth, are often readable from their activations. Whether a clinical model registers evidence strength, distinct from truth, and states it when asked is untested, and any such signal could be lexical. We compiled 45,134 clinical claims from six public sources, harmonized 20,611 into a four-level evidence grade under three independent frameworks, and tested 22 local, open-weight LLMs from several developers (0.6-70 billion parameters; general, medical, and reasoning), with lexical, truth, and cross-framework controls. A linear estimator recovered the grade in every model (median AUROC 71.8), yet decodability did not rise with scale and was weakest in reasoning models. The grade the models stated fell to chance, 25-27 percentage points below the estimator. The recoverable signal was largely lexical and did not transfer across topics or frameworks, yet it was distinct from factual truth and still flagged weakly supported claims (AUROC 69.2). Clinical LLMs thus carry an ordered evidence-strength signal they do not express, so their stated grades fail to convey a claim's support even when it is recoverable from their representations and text.

• The paper demonstrates that clinical LLMs encode graded evidence signals internally—with a median AUROC around 71.8—but fail to state them accurately.
• It uses a comprehensive corpus of 20,611 clinical claims from multiple evidence frameworks to reveal that lexical features drive the decoding of evidence grades.
• The study shows that internal decodability does not equate to actionable output, highlighting the need for external verification in clinical applications.

Internal Decoding and External Expression of Clinical Evidence Strength in LLMs

Motivation and Problem Statement

The evidentiary strength of a clinical claim fundamentally determines the weight and trust clinicians and patients can attribute to LLM-generated informational outputs. In clinical medicine, standard frameworks (e.g., GRADE, USPSTF) formalize this as a four-level graded scale. However, LLMs' inability to express well-calibrated confidence in their outputs has been previously highlighted, but whether these models encode, and can verbalize, the graded strength of clinical evidence has remained unaddressed. This study rigorously interrogates whether clinical LLMs internally represent the evidence grade of a medical claim, if this signal is distinct from truth, how this information manifests across architectures and training specializations, and crucially, if LLMs can reliably state this signal when explicitly prompted.

Corpus Construction and Model Panel

A comprehensive corpus of 45,134 atomic clinical claims from six public biomedical sources was harmonized, yielding 20,611 claims with four-level evidence grades mapped across three independent frameworks, and truth labels assigned orthogonally to evidence strength. The assembly ensures cross-framework, cross-topic, and cross-truth scenarios are analyzable without source confounds.

Figure 1: The study evaluates whether LLMs encode and verbalize graded evidence strength by linear decoding of activations versus prompted grade statements, leveraging a multi-source, multi-framework clinical corpus and a systematic 22-model panel.

The evaluation spans 22 open-weight, instruction-tuned LLMs (0.6–70B parameters): general-domain models, medical domain-adapted variants, reasoning-distilled models, and matched base/derivative pairs, providing controlled axes for family, scale, and training interventions.

Figure 2: Corpus composition and grade/truth distribution across sources, evidencing the grade-truth association and the rare decoupled cells enabling dissociation analyses.

Decodability of Evidence Grade from Internal Representations

A multinomial logistic (linear) probe recovers evidence grade from final-token residual stream activations across all models (median macro AUROC 71.8, range 69.5–75.7). The grade signal is stably decodable across depths but not restricted to a particular layer. Notably, there is no improvement in decodability with scale, and performance often worsens in reasoning-adapted models. The smallest model (Qwen3-0.6B) outperforms larger counterparts (AUROC 75.7), and decodability correlates negatively with parameter count (slope $-0.027$, $p_{\mathrm{FDR}=0.032$ in the scale-controlled Qwen3 ladder). Domain adaptation has no significant main effect; reasoning adaptation reduces decodability.

Figure 3: Linear estimator decodes evidence grade above chance across all 22 models, but decodability shows no improvement, and often declines, with model scale or reasoning adaptation.

Pairwise rank correlation of decoded grade across models (median 0.393) reveals that despite architectural and training differences, models internally rank claims by evidence in a consistent manner. Nonetheless, the practical utility of this internal signal depends on its external expressibility.

Verbalization Gap: Internal vs. Stated Grades

When prompted to select the evidence grade for clinical claims, LLMs perform at chance (median four-way accuracy 22.2–21.8 for zero/few-shot), substantially below what is internally recoverable (median 47.1). The internal-minus-verbalized accuracy gap is 25.4–27.4 percentage points (zero- and few-shot respectively), with every model and prompt regime showing more claims correctly graded by the representation probe than by the model's chosen answer. The stated grade is uncorrelated with the true grade (median Spearman $-0.034$ zero-shot, 0.075 few-shot), and this disconnect is particularly acute at low and very-low evidence levels where clinical safety is most sensitive.

Figure 4: The internal representation robustly encodes graded evidence strength, but LLMs’ explicit grade labels track the true grade only at chance, with a consistent, large gap across model families and grades.

Composition of the Internal Evidence Signal: Lexicality and Generalization

Analysis demonstrates that what is linearly decodable is predominantly a function of the claim's lexical (surface) features. A simple TF-IDF bag-of-words classifier outperforms the hidden-state probe (macro AUROC 78.9 vs. 71.8). Combining hidden and lexical features adds no value and, on matched text pairs, the purported internal signal yields no ordering advantage. The partial residual correlation of the hidden signal after removing the lexical component drops to insignificance (median 0.114). Moreover, grade decodability is not a function of hedging, rarity, or length alone—these are controlled for—but rather, more globally of surface features.

Figure 5: Recovery of evidence grade is largely driven by lexical surface cues, with linear probes encoding little, if any, grade information independent of textual statistics.

Generalization to held-out clinical topics and across evidence frameworks fails: cross-framework transfer accuracy is at, or marginally below, chance (transfer-minus-null $-0.6$ percentage points), with no alignment of grade directions across sources. Topic transfer is highly source-dependent; the estimator does not encode a portable notion of evidentiary strength.

Figure 6: Probes trained on one source or topic do not reliably transfer to others, indicating that decodable evidence strength is not an abstract, generalizable property in these LLMs.

Dissociation of Evidence Strength from Factual Truth

To robustly interrogate whether grade and truth occupy separable representational subspaces, the study constructs balanced analysis grids with decorrelated grade and truth. On these, the principal angle between the “truth” and “grade” directions is near-orthogonal ($90^\circ$), and each direction fails to predict the other (truth prediction from grade direction AUROC 50.4, chance). Crucially, the strong coupling seen in the raw corpus is an artifact of source-level confounding, not shared neural codes.

Figure 7: Grade and factual truth are separable directions in model representation space, only appearing correlated due to dataset-level confounding, not inherent model structure.

Steering and External Activation: Ineffectiveness of Grade Direction Control

Direct intervention experiments—steering the residual stream along the decoded grade direction—do not reliably or significantly affect the model’s stated evidence certainty, performing no better than random directions. Thus, the recoverable linear direction is not causally controlling for evidence certainty expression in these models.

Figure 8: Steering along the evidence-grade vector does not effectively modulate model certainty—grade directions are decodable but not actionable levers for output control.

Practical Deployment: External Flagging of Weak Evidence

An external probe trained to identify weakly supported claims (low/very-low grade) from model activations yields usable accuracy (median AUROC 69.2, AUPRC 83.9). This enables a triage mechanism to flag claims for further review. As with grade, a text-only classifier achieves similar performance, underscoring the lexicality of the signal.

Figure 9: External flaggers can identify weak-evidence claims from internal activations or text—valuable in clinical deployment even though the model itself cannot state this information reliably.

Implications and Future Directions

The central claim of the study—that LLMs encode a robust internal signal of evidence strength, but fail to communicate it when prompted—stands in stark contrast to expectations that such interpretability signals are useful proxies for model knowledge or judgment. Additionally, larger and more sophisticated models (particularly those distilled for reasoning) are not superior at encoding or expressing this signal. This nullifies the assumption that LLM scaling or domain adaptation always enhances internal accessibility of abstract properties such as evidentiary strength.

For practice, reliance on LLM-stated evidence confidence is unsafe. Extraction of the internal signal, or equivalently high-fidelity lexical features, is required for clinical usage to discern claim strength. More broadly, decodability is shown not to imply deep, generalizable, or causal encoding—a conclusion of consequence for interpretability research and safe-AI design. Future research must target interventions at the interface between model knowledge and expression: designing prompts, fine-tuning regimes, or representational interventions that force LLMs to surface internally recoverable graded signals. Richer, adversarially constructed corpora, and alternative probing paradigms (nonlinear, context-conditioned) are warranted to test the robustness and causal salience of decodable signals.

Conclusion

This work rigorously demonstrates that clinical LLMs encode an internally recoverable, but primarily lexical, evidence-strength signal for medical claims—a signal orthogonal to truth and not reflected in their stated grades. Neither scale, domain adaptation, nor reasoning tuning suffices to induce consistent or reliably expressive grades. Until methods exist to both imbue and elicit abstract, non-lexically bound evidence-strength representations, clinical LLM outputs must be accompanied by explicit downstream checks for evidentiary support, eschewing reliance on the model’s self-expressed grades for any safety-critical use.

(2606.29034)