Context as Prior: Bayesian-Inspired Intent Inference for Non-Speaking Agents with a Household Cat Testbed

Abstract: Many agents in real-world environments cannot reliably communicate their goals through language, including household pets, pre-verbal infants, and other non-speaking embodied agents. In such settings, intent must be inferred from incomplete behavioral observations in context-rich environments. This creates a core ambiguity: observable behavior is often noisy or underspecified, while context provides strong prior information but can also induce brittle shortcut predictions if used naively. We present CatSignal, a Bayesian-inspired probabilistic framework for multimodal intent inference that models spatial context as a prior-like constraint and behavioral observations as evidence. Rather than treating context as an ordinary input feature, our method uses a context-gated Product-of-Experts formulation to compute posterior-like intent distributions from context, pose dynamics, and acoustic cues. We instantiate this formulation in a household cat setting as a focused proof-of-concept for intent inference in non-speaking agents. Under Leave-One-Video-Out evaluation on a multimodal domestic cat dataset, the proposed prior-guided fusion achieves the best overall accuracy of 77.72%, outperforming feature concatenation (71.83%) and stronger late-fusion baselines. More importantly, it substantially reduces context-driven shortcut failures in ambiguous cases. While simpler fusion strategies remain competitive in Macro-F1 and selective prediction, the proposed model provides the strongest overall accuracy and the best suppression of context-based shortcut collapse.

• The paper introduces CatSignal, a framework that uses a context-gated Product-of-Experts to integrate spatial context, pose, and audio cues for robust intent inference.
• Leveraging a curated household cat dataset with three intent classes, the model achieved 77.72% accuracy and reduced shortcut failures from 100% to 3.7%.
• The study demonstrates that treating spatial context as a prior constraint significantly enhances inference reliability in ambiguous non-verbal behavior scenarios.

Bayesian-Inspired Multimodal Intent Inference for Non-Speaking Agents

Motivation and Problem Formulation

The paper addresses the challenge of intent inference in non-speaking agents—such as household pets and pre-verbal infants—operating in context-rich environments where explicit goal communication via language is impossible. The inherent ambiguity arises from noisy or underspecified behavioral cues, while spatial and environmental context delivers strong but potentially brittle prior information. The central issue is the tendency for discriminative models to develop context-driven shortcut predictions, conflating contextually possible but idle states with genuine goal-directed behavior.

The authors propose CatSignal, a Bayesian-inspired probabilistic framework utilizing spatial context as a prior constraint, and pose dynamics and acoustic cues as evidence streams. Unlike standard multimodal approaches that aggregate features indiscriminately, this framework leverages a context-gated Product-of-Experts (PoE) formulation to integrate modalities, thus yielding posterior-like intent distributions.

Dataset and Experimental Setup

The study leverages a curated household cat testbed—34 videos filtered on pose-validity using DeepLabCut, resulting in 212 three-second clips. Each clip contains synchronized pose-derived and acoustic features, aligned to provide reliable multimodal evidence. The three intent classes—EXIT, FOOD, and IDLE—are strictly constrained by spatial context: EXIT and IDLE near doors, FOOD and IDLE near bowls, and only IDLE in neutral regions. This asymmetry places acute importance on context for narrowing intent space but renders it insufficient for complete disambiguation.

Evaluation is conducted via Leave-One-Video-Out (LOVO) cross-validation to minimize session- and temporal-specific overfitting.

Methodology

CatSignal decomposes the inference process as follows:

• Context Expert: Spatial context (near bowl, near door, neutral) forms prior P(y | c) over feasible intents.
• Pose Expert: Markerless pose analytics via DeepLabCut extract kinematic descriptors (body speed, tail speed, stretch statistics), yielding P(y | x_pose).
• Audio Expert: Synchronized audio yields MFCC and spectral statistics for P(y | x_audio).
• Product-of-Experts Fusion: The final posterior-like distribution is formed by modulating the context prior by pose and audio evidence:

$$P(y | X_{all}) \propto P(y | c)^\alpha \cdot P(y | x_{pose}) \cdot P(y | x_{audio})$$

where $\alpha$ is a tuneable parameter controlling prior strength. This architecture ensures context acts as a constraints filter, corrected or overridden by behavioral evidence.

Empirical Results

The prior-guided PoE model achieves 77.72% overall accuracy, outperforming feature concatenation (71.83%), late-fusion, and partial expert baselines. Both pose-only and audio-only are inadequate, and context-only models collapse to deterministic shortcuts. Ablation shows that removing context (α=0) devastates performance (45.60%), while moderate to full prior strength (α ∈ [0.8, 1.0]) is optimal.

Shortcut failure rates in ambiguous contexts are significantly reduced: context-only models exhibit 100% failure, late-fusion models reduce this to 18.5% (near bowl), but prior-guided PoE suppresses it further to 3.7%. This demonstrates the model's robustness against shortcut collapse, especially disambiguating "idle" states when context is ambivalent.

Accuracy-coverage analysis also confirms superior calibration under ambiguous conditions: PoE attains best overall accuracy and competitive performance in selective prediction, maintaining reliability under uncertainty.

Analysis and Limitations

The study prioritizes not just aggregate accuracy but reliability under behavioral ambiguity and the minimization of context-driven shortcut failures. CatSignal’s context-as-prior strategy proves especially effective where context sharply constrains feasible goals but does not resolve ambiguity between active and idle states.

However, several constraints limit generalizability:

• The dataset is relatively small and contextually simple.
• Vocalization cues, though discriminative, are sparse; pose descriptors are limited by tracking stability.
• Informative, motion-heavy clips may be filtered out during pose-validity preprocessing.
• The current label space is intentionally minimal (EXIT, FOOD, IDLE), restricting broader behavioral analysis.

Future work could extend the label space, investigate multi-home or cross-species datasets, and adapt the PoE approach to richer environmental priors and agent behaviors.

Implications and Future Directions

CatSignal’s probabilistic approach—modeling context as prior and behavioral cues as evidence—suggests a principled alternative to conventional feature fusion for intent recognition in non-speaking agents. Practically, the framework may enhance the reliability of domestic robots, smart monitoring systems, and animal-centered AI deployments, where reducing shortcut failures is critical for safe and interpretable behavior understanding.

Theoretically, this method draws from machine theory of mind, inverse planning, and Bayesian latent goal inference, advocating for a shift in animal and embodied agent behavior modeling toward outcome-sensitive probabilistic reasoning. Broad adoption could foster advances in ecological behavior analysis, interactive robotics, and mental-state attribution in agents lacking explicit communication capabilities.

Conclusion

CatSignal demonstrates that Bayesian-inspired, context-gated PoE fusion significantly improves intent inference for non-speaking agents in context-rich but behaviorally ambiguous environments. By treating spatial context as a prior constraint rather than an ordinary feature, the framework delivers higher accuracy and robust suppression of shortcut failures in ambiguous cases. The study advocates for intent inference as a probabilistic reasoning problem, laying foundational insights for multimodal behavior understanding in animal-centric and non-verbal AI systems (2604.27445).