Entity Binding Failures in Speech LLM Reasoning: Diagnosis and Chain-of-Thought Intervention

Abstract: Speech LLMs (SLLMs) underperform their text counterparts on complex reasoning. We reveal that this modality gap is not a uniform cognitive deficit. Evaluating three diverse SLLMs, we show speech-to-text (S2T) matches or exceeds text-to-text (T2T) on spatial, syntactic, and factual tasks. However, on logical tasks requiring entity tracking, S2T accuracy collapses to chance. We diagnose this localized degradation as an entity binding failure: continuous speech features cause models to lose precise entity-property associations during implicit reasoning. To resolve this, we propose Entity-Aware Chain-of-Thought (EA-CoT), forcing SLLMs to explicitly enumerate entities and bind them to claims before reasoning. Strikingly, EA-CoT bridges the gap, even when spoken names are misrecognized, yielding up to a 24.4% absolute accuracy improvement. Ablations confirm these gains stem entirely from explicit semantic binding, reframing the gap as a resolvable bottleneck.

• The paper’s main contribution identifies entity binding failures as the cause of degraded logical reasoning in speech LLMs, particularly in tasks like the 'web of lies'.
• It introduces the Entity-Aware Chain-of-Thought (EA-CoT) intervention that explicitly enumerates and links entities, leading to a recovery of up to 24.4 percentage points in speech accuracy.
• Empirical results and ablation studies confirm that structured, explicit entity binding mitigates the modality gap, underscoring the need for architectural improvements in speech processing.

Diagnosis and Remediation of Entity Binding Failures in Speech LLM Reasoning

Modality Gap Analysis and Localization

The paper "Entity Binding Failures in Speech LLM Reasoning: Diagnosis and Chain-of-Thought Intervention" (2606.04474) systematically characterizes the reasoning gap between Speech LLMs (SLLMs) and their text-only counterparts. The authors present evidence that traditional SLLM evaluations have exaggerated the notion of a broad cognitive deficit when using spoken input by failing to examine task-specific modalities. Instead, their paired evaluation across four reasoning categories—hyperbaton, navigation, sports understanding, and web of lies—demonstrates that SLLMs are generally competitive in spatial, syntactic, and factual tasks. However, performance degrades dramatically in logical tasks with a requirement for entity tracking. For instance, in the "web of lies" category, speech-based reasoning accuracy drops to chance levels, while text-based accuracy remains high.

The paper attributes this sharp task-specific gap to entity binding failures induced by the architectural necessities of speech encoding. Temporal pooling and downsampling, optimized for extracting global semantic features from audio, inadvertently blur discrete token and entity boundaries. This undermines the SLLM’s ability to maintain robust entity-property associations during implicit reasoning, despite preserving overall semantic context.

Figure 1: The Entity-Aware Chain-of-Thought (EA-CoT) method enforces explicit entity binding in generated text, counteracting implicit binding loss incurred during speech input processing.

Entity-Aware Chain-of-Thought Intervention

To resolve the identified entity binding bottleneck, the authors propose the Entity-Aware Chain-of-Thought (EA-CoT) prompting strategy. EA-CoT operates as an inference-time intervention, augmenting baseline question-answering protocols by requiring explicit enumeration and binding of entities to properties in the generated text before reasoning execution. The method extends sequence generation budgets to accommodate structured traces—enabling models to circumvent the loss of fine-grained entity information by creating stable textual anchors.

EA-CoT consists of four instructional steps: entity enumeration, claim recording, iterative reasoning, and answer extraction. This explicit structure converts fragile, implicit acoustic entity tracking into robust, text-based binding.

Empirical Results and Performance Decomposition

Experimental results on web of lies demonstrate that EA-CoT achieves a substantial recovery, yielding up to a 24.4 percentage point absolute improvement in speech input accuracy. The recovery magnitude is consistently correlated with the model’s baseline text accuracy, showcasing that the intervention leverages latent text-based reasoning capabilities. The paper further validates that this result is not a consequence of increased token generation space: ablation studies and controlled experiments verify that instruction content, not token budget, drives performance gains. Increasing the token budget without structured instruction yields negligible improvement for speech inputs.

Figure 2: EA-CoT dramatically reduces the speech-text accuracy gap in web of lies, while speech accuracy recovers from chance to near-text baseline.

Figure 3: Structured EA-CoT instruction solely drives speech performance improvement; increased token budget has negligible effect.

Ablation studies are performed to isolate EA-CoT components, showing that explicit entity enumeration accounts for the majority of the observed gains. The intervention remains effective even when the model misrecognizes spoken entity names phonetically, as long as the CoT establishes consistent anchors in the text—demonstrating that the modality gap is primarily a semantic binding failure, not an artifact of speech recognition quality.

Analysis of Semantic Binding and Modality Effects

The distinction between semantic binding and mere acoustic recognition is reinforced by experiments corrupting entity names in text input. Even aggressive entity name corruption in text yields only a minor accuracy reduction, validating that the gap is not due to phonetic mismatch but rather to the loss of association across reasoning steps. Output-level analyses further show that models revert to trivial responses for speech input in entity-centric logical tasks, with full reasoning chains only produced under EA-CoT prompting.

Additionally, the intervention’s specificity is confirmed; EA-CoT yields no improvement in acoustic-centric spoken language understanding (MMSU) scenarios, where entity tracking is not the critical bottleneck. This validates that EA-CoT specifically addresses semantic structural binding losses rather than generic reasoning or acoustic deficits. EA-CoT’s efficacy fundamentally depends on models having robust text-level entity reasoning capabilities—a precondition for externalization to succeed.

Theoretical and Practical Implications

The findings have significant implications for SLLM architecture and deployment. Whereas prior work has focused on cross-modal alignment and representation-level adjustments, the paper highlights that explicit structural interventions can immediately mitigate entity binding failures, even at inference time. Practically, the intervention comes at a latency cost—trading inference speed for logical accuracy. For real-time applications, architectural solutions to stabilize entity binding in continuous representations (e.g., multimodal binding subspaces, latent trace distillation) remain necessary to avoid explicit token expansion.

Theoretically, these results provide direct evidence that the modality gap in speech reasoning is a resolvable structural bottleneck rather than an inherent limitation of SLLMs. The divergence between speech and text arises from the fragility of implicit binding in continuous input, suggesting that future front-end architectures should prioritize cross-modal entity binding regularization. Such efforts may further mitigate multimodal reasoning gaps across other domains, including vision-language and audio-text integration.

Conclusion

This paper diagnoses entity binding failure as the principal cause of the modality-induced reasoning gap observed in Speech LLMs for logical, entity-centric tasks. The EA-CoT intervention—explicitly enumerating and linking entities in generated text—enables SLLMs to recover from chance-level accuracy to near-text baselines, even with imperfect acoustic transcription. This finding reframes the modality gap as a structural binding deficit amenable to prompting repair. Future work should pursue architectural solutions for binding stabilization and efficient reasoning, advancing multimodal LLM robustness and practical deployment.