Discordance-Based Hallucination Criterion

• Discordance-based hallucination criterion is a formalism that detects misleading outputs by quantifying structural mismatches between multiple signals.
• It employs techniques such as perturbation, multi-view evidence fusion, and statistical thresholds across modalities including ASR, RAG, LVLMs, and LLMs.
• The approach improves model reliability by flagging fluent yet factually inconsistent outputs using metrics like cosine similarity, WER, and AUROC.

A discordance-based hallucination criterion is a formalism for detecting hallucinations in generative models—outputs that are fluent but factually or semantically unrelated to the true input—by explicitly quantifying and testing for structural mismatches (“discordance”) between multiple signals. This framework has been developed and applied in multiple modalities, including automatic speech recognition (ASR), retrieval-augmented generation (RAG), vision-LLMs (LVLMs), and LLMs. Discordance-based methods typically work by comparing model outputs under nominal versus perturbed conditions, or contrasting predictions from different sources (internal model, retrieval, symbolic chain-of-thought), and applying statistical or information-theoretic criteria to flag outputs as hallucinated when significant disagreement is detected.

1. Formal Foundations of Discordance-Based Hallucination Criteria

The discordance-based approach identifies hallucinations by operationalizing the notion of semantic, evidential, or representational conflict, often using multiple views or perturbations of the generation process. Across domains, hallucinations are defined not merely as incorrect outputs, but as responses that are:

• Fluent and coherent by standard metrics (e.g., low perplexity, naturalness),
• Low in semantic relatedness or factual consistency with the input or reference,
• Often indistinguishable from valid outputs by surface-level measures (e.g., low WER or BLEU).

For example, in neural ASR, hallucinations are transcriptions $\hat{y}$ such that

$$\text{CosineSim}(\phi(\hat{y}),\,\phi(y)) \ll 1 \quad \text{and} \quad \text{PPL}(\hat{y}) \text{ is small}$$

where $y$ is the reference transcription and $\phi$ is a sentence embedding function (Frieske et al., 2024).

In RAG, hallucination risk at query $x$ is measured by the discordance score

$$\mathscr H_{\mathrm{disc}}(q_0; x) = w_{\mathrm{fact}}(x) \left[1 - q_0\left(y_r(x)\mid x\right)\right]$$

where $w_{\mathrm{fact}}(x)$ is a retrieval-trust weight and $q_0$ is the base LM’s predicted mass on the modal label from the nearest neighbor retriever (Biau et al., 20 Jan 2026).

2. Algorithmic Realization Across Modalities

The operational workflow of discordance-based hallucination detection varies by domain but follows the unifying principle of stress-testing or cross-examination via multiple evidence sources:

• ASR (Perturbation-based): Inject local random noise into the input (e.g., 1s WGN preamble), compare transcription $f(x)$ vs $f(x’)$, and assess changes in semantic similarity, fluency, and WER (Frieske et al., 2024). Hallucinations are flagged if the perturbed output is fluent, has low semantic similarity to ground truth, and if WER spikes above threshold only upon perturbation.
• RAG (Statistical proxy): For each query, compare retriever and LM predictions, compute the trust-weighted discordance, and apply an adaptive gate to switch between evidence sources only when retrieval is local and boosts predictive accuracy (Biau et al., 20 Jan 2026).
• LVLMs (Differential Evidence Fusion): Treat each internal feature as a source of DST-formatted evidence. Fuse via Dempster’s rule and extract a conflict coefficient $K$, where a high $K$ indicates strong internal disagreement suggestive of hallucination (Huang et al., 24 Jun 2025).
• LLMs (Consistency- and Reasoning-based): Fuse multi-path internal representations (direct answer, chain-of-thought, reverse inference) and quantify discordance via a segment-aware cross-attention module. A high discordance score indicates misalignment between internal states and symbolic reasoning (Song et al., 13 Oct 2025). Alternatively, measure NLI-derived contradiction and entailment scores between responses and between responses and query, training a classifier to aggregate these into a hallucination score (Urlana et al., 6 Mar 2025).

3. Quantitative Measures of Discordance

Discordance metrics center on quantifying conflict, contradiction, or semantic divergence. The following table summarizes the primary metrics across paradigms:

Domain	Discordance Metric	Decision Thresholds (typical)
ASR	$1-\cos(\phi(y),\phi(\hat y'))$	cos $<0.2$, PPL $<200$, WER $>30\%$
RAG	$w_{\text{fact}}(x)\,\bigl[1-q_0(y_r(x)|x)\bigr]$	Adaptive per-query via gating
LVLM	Dempster–Shafer conflict $K$	$K_{\max}$ over token positions
LLM (CoT)	$p_{\text{hallu}}$ via fusion	$\tau$ tuned for F1/AUROC
LLM (NLI)	avg. NLI-contradiction ($C^{avg}$)	Classifier aggregation

Discordance is not a simple error rate but an explicit measure of internal or inter-source disagreement when other validity cues (fluency, output format) are satisfied.

4. Empirical Insights and Comparative Evaluation

Discordance-based criteria consistently reveal failure modes missed by conventional error metrics:

• In ASR, models with near-identical WER may differ in hallucination susceptibility; the “UU” model (trained with untranscribed utterances) exhibits high hallucination rates detectable only via discordance after noise perturbation (Frieske et al., 2024).
• In RAG, the discordance measure $\mathscr H_{\mathrm{disc}}$ detects cases where the LLM disagrees with reliable, geometrically proximal retrieved evidence, guiding optimal gating strategies and explaining factuality failures (Biau et al., 20 Jan 2026).
• For LVLMs, the DST-based conflict metric $K$ yields 4–10% AUROC gains in hallucination detection compared to log-probability, entropy, and other baselines (Huang et al., 24 Jun 2025).
• In LLMs, segment-aware fusion of internal and external reasoning boosts AUROC by 2–5 points across both fact-based and logic-based tasks, resolving the typical blind spot of single-modality detectors (Song et al., 13 Oct 2025). NLI-based discordance ensembles achieve balanced accuracy $>0.94$ and F1 $>0.80$ on QA domains (Urlana et al., 6 Mar 2025).

5. Design Variants and Application-Specific Considerations

Different instantiations of discordance-based criteria emphasize particular axes:

• Perturbative discordance (ASR): Localized noise uncovers over-memorization, distinguishing hallucinations from phonetic or random errors. Start-of-input noise is more effective than uniform degradation (Frieske et al., 2024).
• Retrieval-trust weighting (RAG): $w_{\mathrm{fact}}(x)$ penalizes reliance on distant or out-of-distribution neighbors, dynamically downweighting unreliable retrieval when dataset or domain shift is present (Biau et al., 20 Jan 2026).
• Evidential conflict (DST) aggregation (LVLM): Simple mass function assignment and summing within feature dimensions sidesteps combinatorial explosion, making the approach computationally efficient for large models (Huang et al., 24 Jun 2025).
• Cross-modality fusion (LLMs): Multi-path reasoning, segment-aware temporal cross-attention, and gating fuse fine-grained signal with logical chain-of-thought, overcoming the representational alignment barrier (Song et al., 13 Oct 2025).
• NLI-based reference-free detection: Response-response contradiction and query-response neutral/entailment scores, combined via a classifier, enable high-confidence hallucination detection in black-box or closed-source LLM settings (Urlana et al., 6 Mar 2025).

6. Illustrative Example: Discordance-Based Detection in ASR

Consider ASR transcribing an audiobook excerpt:

• Reference $y$: “the old oak tree stood on the hill.”
• Clean output $\hat{y}$: identical to $y$; WER=0%, cosine sim=0.94, PPL=35.
• Perturbed output $\hat{y}'$ (after 1s WGN): “there is nothing left to go south by”; WER=78% (>30%), cosine sim=0.09 (<0.2), PPL=48 (<200).
• Outcome: Both fluency and unrelatedness criteria trigger; the system flags $\hat{y}'$ as a hallucination (Frieske et al., 2024).

7. Implications, Limitations, and Outlook

Discordance-based criteria provide a principled mechanism for hallucination detection, unifying perturbation-based stress testing, evidence/feature conflict quantification, and cross-signal fusion. The approach exposes unreliability that is orthogonal to standard accuracy or fluency metrics, enables robust test-time screening without gold references, and adapts naturally to both white-box and black-box deployment contexts.

Limitations arise from dependence on embedding quality, NLI or confidence model calibration, and the potential brittleness of heuristic thresholds. As evidence grows (e.g., (Frieske et al., 2024, Biau et al., 20 Jan 2026, Huang et al., 24 Jun 2025, Song et al., 13 Oct 2025, Urlana et al., 6 Mar 2025)), discordance-based criteria are converging towards a central formal pillar in the measurement and mitigation of hallucinations in generative and retrieval-augmented systems.