LLMs as Signal Detectors: Sensitivity, Bias, and the Temperature-Criterion Analogy

Abstract: LLMs are evaluated for calibration using metrics such as Expected Calibration Error that conflate two distinct components: the model's ability to discriminate correct from incorrect answers (sensitivity) and its tendency toward confident or cautious responding (bias). Signal Detection Theory (SDT) decomposes these components. While SDT-derived metrics such as AUROC are increasingly used, the full parametric framework - unequal-variance model fitting, criterion estimation, z-ROC analysis - has not been applied to LLMs as signal detectors. In this pre-registered study, we treat three LLMs as observers performing factual discrimination across 168,000 trials and test whether temperature functions as a criterion shift analogous to payoff manipulations in human psychophysics. Critically, this analogy may break down because temperature changes the generated answer itself, not only the confidence assigned to it. Our results confirm the breakdown with temperature simultaneously increasing sensitivity (AUC) and shifting criterion. All models exhibited unequal-variance evidence distributions (z-ROC slopes 0.52-0.84), with instruct models showing more extreme asymmetry (0.52-0.63) than the base model (0.77-0.87) or human recognition memory (~0.80). The SDT decomposition revealed that models occupying distinct positions in sensitivity-bias space could not be distinguished by calibration metrics alone, demonstrating that the full parametric framework provides diagnostic information unavailable from existing metrics.

• The paper demonstrates that applying full SDT to LLMs reveals dissociable shifts in sensitivity (dₐ) and bias (c) as temperature increases.
• It finds that elevated temperature not only boosts AUC and dₐ but also shifts the response criterion, refuting a simple temperature-criterion analogy.
• The analysis highlights limitations of standard calibration metrics and offers new diagnostic tools for targeted model improvements.

LLMs as Signal Detectors: Signal Detection Theory Decomposition, Temperature Effects, and Calibration

Overview

This paper rigorously applies the full Signal Detection Theory (SDT) framework to LLMs, with a particular focus on decomposing their response behavior into sensitivity (discriminability, $d_a$) and bias (criterion, $c$) and investigating how temperature scaling manipulates these parameters. The study systematically evaluates three contemporary LLMs (Llama-3-8B-Instruct, Mistral-7B-Instruct, Llama-3-8B-Base), conducting 168,000 factual question-answering trials and 6,000 forced-choice tasks across two datasets. By leveraging unequal-variance SDT modeling, ROC and z-ROC analyses, and pre-registered hypotheses, the paper surfaces critical dissociations between calibration, discriminability, and model response bias, and directly tests the widely assumed temperature-criterion analogy. Results demonstrate the clear limitations of standard calibration metrics, the theoretical and practical utility of SDT analysis, and provide new diagnostic tools for LLM evaluation, including model selection and targeted intervention strategies.

SDT Decomposition of LLM Behavior

The paper formalizes the analogy of LLMs as signal detectors where, for each question, the model produces an answer (signal or noise), and the log-probability per token (NLP) is interpreted as the evidence variable. Correct answers are mapped to the signal distribution and incorrect answers to the noise distribution. Model confidence intervals are decomposed via ROC construction over 20 NLP bins, and UVSD parameters ($d_a$, $c$, slope $s$) are estimated by maximum likelihood, paralleling psychophysical protocol.

The rationale for evaluating LLMs via SDT rather than aggregate calibration metrics is explicit: sensitivity ($d_a$) is mathematically independent from criterion ($c$), and interventions to improve either require distinct strategies. For example, poor discriminability demands architectural or data improvements, whereas criterion misplacement can be corrected by policy or post-processing.

Temperature as Criterion Manipulation: Empirical Breakdown

A central hypothesis is that temperature scaling in the LLM softmax is formally analogous to criterion shifts induced by payoff manipulations in human SDT experiments. In classical SDT, such manipulations shift the criterion but do not affect sensitivity ($d'$). The empirical results, however, provide a robust rejection of this analogy for LLMs: temperature simultaneously induces monotonic increases in both AUC and $d_a$, as well as criterion shifts from strongly liberal to more conservative stances.

Figure 1: ROC curves for all three LLMs on TriviaQA, across seven temperature points; the outward shift under increasing temperature reflects AUC increase, not mere criterion translation.

Figure 2: Temperature effects; AUC and $d_a$ increase monotonically with $c$0, while $c$1 shifts from highly liberal toward less liberal values.

The underlying mechanistic reason is the generative nature of LLMs—higher temperature not only flattens the output distribution, but also changes the actual sampled answer, altering the evidence variable itself and thereby violating the isolation assumption of SDT criterion manipulations. Increases in temperature spread the NLP distribution, producing greater evidence separation between correct/incorrect responses—but this does not index improved model knowledge, as accuracy decreases with temperature. This illustrates a critical dissociation between discriminability (as measured via ROC/AUC) and raw factual competence.

Unequal Variance and Structural Evidence Distribution Shifts

LLMs globally exhibit unequal-variance evidence distributions—z-ROC slopes are substantially below 1.0, especially for instruction-tuned models (0.52–0.63) compared to base (0.77–0.87). This pattern is more extreme than observed in human recognition memory. This is systematically replicated across models and temperatures.

Figure 3: z-ROC plots at all temperatures, visually confirming slope $c$2 1 for instruct models, diagnostic of unequal evidence variance for correct versus incorrect answers.

Instruction-tuned models reveal more variance asymmetry, a plausible result of RLHF amplifying response confidence variability for correct answers. This structural property is not captured by aggregate AUC or ECE, and its quantification has direct implications for understanding the representational geometry of LLM-generated evidence.

SDT Decomposition Reveals Orthogonality to Calibration

The analysis demonstrates that calibration metrics such as ECE fail to differentiate models with identical calibration but distinct operating characteristics in the ($c$3, $c$4) space. For instance, Mistral-7B-Instruct exhibits higher discriminability but operates with a more liberal criterion than Llama-3-Instruct, a dissociation invisible to ECE.

Figure 4: SDT operating points at $c$5 plotted for all models/datasets; distinct ($c$6, $c$7) combinations occur at similar ECE levels, signaling the limitations of calibration metrics.

Domain-decomposed analyses further establish that differences in sensitivity across domains (e.g., Science {content} Technology is hardest) are not reflected in aggregate calibration errors.

Figure 5: Domain-specific $c$8 at $c$9; Mistral possesses the widest domain range, and certain categories consistently depress sensitivity, representing targets for focused improvement.

Exploratory and Cross-Paradigm Analyses

The forced-choice (4AFC) paradigm yields near-ceiling performance for instruction-tuned models, confirming internal discriminability but limiting the granularity of $d_a$0 estimation. Exploratory analyses show that instruction tuning primarily shifts $d_a$1 rather than $d_a$2, and that prompt manipulations to shift criterion often impair sensitivity, not just bias, possibly due to disrupting processing.

Temperature’s impact on evidence distribution structure is model-dependent: for Llama-3 variants, z-ROC slope increases significantly with temperature, while Mistral is more stable.

Figure 6: z-ROC slopes as a function of temperature; significant monotonic increases for Llama-3 models but not Mistral, emphasizing model-specific evidence structure dynamics.

Replication on the Natural Questions dataset confirms the primary findings.

Figure 7: Natural Questions dataset AUC and $d_a$3 vs. temperature curves; all main results are robust out-of-domain.

Theoretical and Practical Implications

The decomposition of LLM response characteristics into SDT parameters exposes practical avenues for model development and deployment:

• Targeted improvement: For low-sensitivity/high-bias domains, retraining, not recalibration, is indicated.
• Selective prediction/abstention: SDT provides theoretically optimal threshold placement based on domain-specific $d_a$4, far beyond empirical calibration-based thresholds.
• RLHF effects diagnostics: Instruction tuning operates as a criterion modulator, not necessarily improving discriminability—a key insight for RLHF protocol design.
• Evaluation methodology: Standard nonparametric metrics (AUC/ECE/Brier) obscure structural weaknesses; full SDT analysis is necessary for actionable model characterization.

The study also highlights the need for caution regarding evidence variable selection. NLP, while operationally useful, entangles correctness with model-specific fluency, as force-decode experiments reveal: model-generated incorrect (but fluent) responses may have higher NLP than reference correct answers. This fluency–accuracy dissociation must be accounted for in future SDT-based LLM evaluations.

Limitations and Future Directions

Key limitations include evaluation on only 7–8B parameter models, binning scheme sensitivities (especially for Mistral), and incomplete evidence variable analyses (comparison with first-token logit deferred). The unequal-variance Gaussian SDT may inadequately model the full non-Gaussian structure in LLM-derived evidence distributions.

Future work should prioritize:

• Evidence variable robustness—NLP vs. per-token logits, to better isolate truth tracking from fluency.
• Hierarchical Bayesian and continuous MLE fitting to overcome binning-driven instability.
• Application to larger, proprietary, and domain-specific models to test generality.
• Bridging parametric SDT ($d_a$5, $d_a$6) to meta-cognitive SDT (meta-$d_a$7), estimating LLM metacognitive efficiency.
• Mechanistic study of internal representation changes induced by temperature and tuning paradigms.

Conclusion

The formal application of full parametric SDT to LLM response evaluation reveals critical insights into model structure and functioning that are masked by calibration or aggregate accuracy metrics. The study demonstrates that temperature does not simply shift LLM criterion, due to the generative coupling of answer and policy, invalidating the direct analogy to human payoff manipulations. LLMs exhibit pervasive unequal-variance evidence distributions, and model selection, tuning, and intervention strategies must account for dissociable shifts in sensitivity and bias, not merely calibration error. The generalization and extension of SDT-based evaluation represent a crucial methodological advance for the scientific and practical assessment of LLM reliability.