Tri-Info: Generalizable, Interpretable Failure Prediction for VLA Models via Information Theory

Abstract: Vision-Language-Action (VLA) models are increasingly deployed across diverse tasks, yet they remain black boxes whose physical interactions can cause irreversible harm, making generalizable and interpretable failure detection essential. We observe that successful and failed rollouts carry systematically different information-theoretic signatures. Building on this, we formalize VLA control as a closed-loop information pipeline and derive the Triple Information-theoretic (Tri-Info) signals that capture whether actions remain diverse, temporally consistent, and coupled to state transitions. Across six VLA models and three benchmark environments, Tri-Info matches the strongest baselines in-domain. Moreover, Tri-Info transfers across architectures, environments, and the sim-to-real gap without retraining, reaching 83\% accuracy on real-world tasks where prior detectors collapse to chance. This establishes Tri-Info as a simple yet powerful method that not only detects failures with strong cross-domain generalization, but also delivers interpretable diagnostics of the underlying failure modes.

• The paper introduces Tri-Info, a novel framework that formalizes VLA control with information theory to predict failures with high AUC and early warning capabilities.
• It reduces eight metrics to three key ones—action entropy, temporal consistency, and state-action coupling—providing clear, interpretable diagnostics of failure modes.
• Empirical results demonstrate superior sim-to-real transfer and generalization across multiple VLA architectures, enabling timely interventions in dynamic environments.

Formal Information-Theoretic Failure Prediction for Vision-Language-Action Models

Motivation and Problem Statement

Vision-Language-Action (VLA) models are increasingly adopted in embodied AI for direct mapping from multimodal inputs to low-level robotic controls. Although state-of-the-art VLA architectures demonstrate high efficacy on in-distribution tasks, they fail silently and instantaneously under distributional shifts—a critical challenge in physical deployment where failures are often irreversible and dangerous. Conventional failure detectors in robotic control either exhibit strong in-domain performance but lack generalizability, or generate scores that are challenging to interpret for actionable diagnostics.

Information-Theoretic Framework and Tri-Info Metrics

The paper proposes a systematic formalization of VLA control as a closed-loop information pipeline, extracting stationary and temporal relationships between actions and states. Eight information-theoretic metrics are derived, spanning marginal statistics (action/state entropy), policy coupling (state-action mutual information), dynamics (action influence on next state and predictability), and temporal coherence (consistency across consecutive steps).

A pooled correlation analysis reveals strong redundancy among several metrics. The reduction process yields a minimal and complementary triplet:

• Action Entropy ($H(A_t)$): Quantifies diversity; surges indicate drift failures, collapse points to freeze.
• Action Temporal Consistency ($I(A_t; A_{t+1})$): Captures whether action sequences maintain coherence over timesteps.
• Action-State Coupling ($I(S_t, S_{t+1}; A_t)$): Measures explanation of actions through state transitions; drops signal phantom grasps or observation-action decoupling.

This reduction independently emerges as the maximally discriminative subset through exhaustive search across all combinations, with larger subsets suffering from redundancy and degraded transfer performance.

Detection Architecture and Online Deployment

Each Tri-Info metric feeds into a dedicated GRU temporal detector with a per-timestep binary label supervised by cross-entropy. The fused posterior across all three detectors is normalized and thresholded online via Functional Conformal Prediction, dynamically adapting confidence to the trajectory progress and capping false positives at deployment.

The detection pipeline is agnostic to architecture and environment, as entropy and mutual information operate on the distributional properties of embedding statistics rather than absolute embedding geometry. All metrics are estimated non-parametrically using k-NN estimators on low-dimensional VAE-compressed latents to mitigate high-dimensional bias and maintain computational tractability.

Empirical Results

Predictive Power and Temporal Dynamics

Every information-theoretic metric independently achieves strong predictive signal on in-domain data (AUC $\geq 0.70$ for logistic regression), with $I(A_t; A_{t+1})$ peaking at $0.90$. Incorporating temporal modeling via GRU saturates in-domain detection ($\sim 0.97-0.98$ AUC). The early warning capability and sensitivity to failure onset are substantially improved with temporal modeling versus instantaneous scoring.

Generalization and Sim-to-Real Transfer

Tri-Info demonstrates substantial generalization across six VLA models and three environments (including LIBERO, CALVIN, ALOHA), maintaining $0.83$ balanced accuracy under real-world sim-to-real transfer where embedding- and score-based baselines collapse to chance. Tri-Info detector flags failures substantially earlier than baselines, enabling timely intervention. In out-of-distribution (OOD) scenarios (cross-architecture or sim-to-real), performance plateaus remain high for Tri-Info (e.g., $0.92$ balanced accuracy on PI0-LIBERO $\rightarrow$ PI0.5-LIBERO), while baselines either degenerate or anti-correlate with failures.

Ablation and Alternatives

Ablation studies confirm the triplet as the optimal subset: single-metric and two-metric detectors cannot simultaneously match in-domain and transfer accuracy; expansion to all eight metrics introduces overfitting and degrades OOD robustness (balanced accuracy drops to $I(A_t; A_{t+1})$0). Action-centric alternatives (variance, KNN, cosine dissimilarity, temporal gradient) evaluated in the same pipeline fail to transfer under distribution shift. FIPER-ACE and STAC, the closest baselines, plateau below Tri-Info and collapse on OOD settings.

Computational Efficiency

Detector latency remains negligible relative to policy inference ($I(A_t; A_{t+1})$1 ms per window on GPU), ensuring practical deployment compatibility.

Interpretability and Diagnostic Implications

Tri-Info metrics offer direct interpretability: each metric aligns with visually and semantically distinct failure modes. The failure-mode dashboard produced by Tri-Info can drive actionable interventions (e.g., exploration re-injection, rollback, or re-grounding), although the current work focuses solely on detection. This supports transparency and risk management in robotic deployments, enabling practitioners to rapidly diagnose failure types and tune sensitivity versus timeliness through threshold calibration.

Theoretical and Practical Implications

The information-theoretic approach substantiates architecture-agnostic, interpretable failure detection for VLA systems, formally bridging classic robotics and modern multimodal policy learning. The substrate-independence of entropy and mutual information enables transferability across models, tasks, and modalities. This opens avenues for safe monitoring in heterogeneous deployments, facilitates deployment-time risk management, and provides a foundation for actionable diagnostics.

Future research may extend Tri-Info to adaptive recovery mechanisms, integrate failure mode-specific interventions, and explore further reduction or expansion of the information-theoretic taxonomy in richer multimodal settings.

Conclusion

This paper establishes an information-theoretic framework for generalizable, interpretable failure detection in VLA models, reducing eight metrics to the Tri-Info triplet that captures action diversity, temporal consistency, and state-action coupling. The Tri-Info detector exhibits strong accuracy, early warning, and transferability, outperforming baselines in in-domain and cross-domain settings. Diagnostics are interpretable and align with underlying failure modes. The approach sets a foundation for robust deployment-time risk management in embodied AI and invites further research in closing the loop from detection to intervention (2606.19998).