Meow-Omni 1: A Multimodal Large Language Model for Feline Ethology

Abstract: Deciphering animal intent is a fundamental challenge in computational ethology, largely because of semantic aliasing, the phenomenon where identical external signals (e.g., a cat's purr) correspond to radically different internal states depending on physiological context. Existing Multimodal LLMs (MLLMs) are blind to high-frequency biological time-series data, restricting them to superficial behavioural pattern matching rather than genuine latent-state reasoning. To bridge this gap, we introduce Meow-Omni 1, the first open-source, quad-modal MLLM purpose-built for computational ethology. It natively fuses video, audio, and physiological time-series streams with textual reasoning. Through targeted architectural adaptation, we integrate specialized scientific encoders into a unified backbone and formalize intent inference via physiologically grounded cross-modal alignment. Evaluated on MeowBench, a novel, expert-verified quad-modal benchmark, Meow-Omni 1 achieves state-of-the-art intent-recognition accuracy (71.16%), substantially outperforming leading vision-language and omni-modal baselines. We release the complete open-source pipeline including model weights, training framework, and the Meow-10K dataset, to establish a scalable paradigm for inter-species intent understanding and to advance foundation models toward real-world veterinary diagnostics and wildlife conservation.

• The paper presents a quad-modal LLM that integrates video, audio, and high-frequency physiological data to overcome semantic aliasing in feline behavior analysis.
• It employs specialized TS encoders and formal causal intent inference based on structural causal models for context-sensitive decision making.
• Experimental results on the MeowBench dataset demonstrate a 71.16% Top-1 accuracy, outperforming unimodal and lower-modality baselines by up to 4.3%.

Meow-Omni 1: A Multimodal LLM for Feline Ethology

Introduction

"Meow-Omni 1: A Multimodal LLM for Feline Ethology" (2605.09152) presents a domain-specific MLLM designed to resolve intent ambiguity in feline behavioral interpretation, particularly where identical external signals correspond to distinct latent states—a problem termed semantic aliasing. Current MLLMs are predominantly limited to processing human-centric modalities and are notably deficient in their treatment of high-frequency numerical biological data. Meow-Omni 1 confronts this by architecturally fusing video, audio, and physiological time-series with language, thus enabling context-sensitive, physiologically grounded reasoning regarding feline intent.

Technical Innovations

Quad-modal MLLM Architecture and Grounding

Meow-Omni 1 builds on the MiniCPM-o 4.5 multi-modal backbone, extending it with highly specialized TS encoders from Intern-S1-Pro. Architectural modifications encompass tokenizer extension (with <|ts_start|>, <|ts_unit|>, <|ts_end|> tokens), explicit model layer transplantation for biometric data, and a custom projection layer that unifies vision, audio, and TS modalities in the LLM embedding space. The model processes any subset of these modalities and interleaves their embeddings with language, facilitating joint cross-modal context for intent inference.

Crucially, the physiological time-series modality is treated as a first-class citizen: no flattening or textualization is applied, enabling the causal transformer to directly align high-frequency physiological signals with behavioral displays. This strategic design closes the symbol grounding gap endemic to generalist MLLMs.

Formalization of Intent Inference

Meow-Omni 1 departs from classical heuristic behavior classification by formalizing intention inference via structural causal models. Intention ($Z$) is treated as the latent variable that would maximize the probability of an action under environmental intervention (modeled with Pearl’s do-operator). The objective is thus to estimate $P(Z \mid \mathrm{Encoder}(H_t))$, where $H_t$ aggregates Video, Audio, and Biometrics, and align external behaviors with internal physiological context.

Data and Benchmark Contributions

To operationalize this framework, the authors introduce Meow-10K, a 10,831-sample dataset with joint video, audio, and biometric sequences, as well as MeowBench, an expert-verified quad-modal evaluation suite with multiple-choice intent alignment queries. MeowBench enforces rigorous cross-modal synthesis, verified for physiological plausibility by professional feline ethologists, enabling robust standardized assessment absent in existing ethological ML benchmarks.

Experimental Results

Benchmark Performance

Meow-Omni 1 achieves a notable Top-1 intent recognition accuracy of 71.16% on MeowBench, outperforming all unimodal (Qwen3.5-122B visual baseline: 61.95%; SOTA audio: 36.86%; SOTA TS: 48.98%) and bimodal or tri-modal baselines (Qwen3.5-Omni-Plus V+A+TS: 66.89%). The model’s 4.3-point gain over the best competing omni-modal approach underscores the importance of native physiological signal integration, rather than relying on indirect summaries or late fusion.

Modal Contribution and Ablation

An extensive modality masking ablation quantifies the unique contributions:

• Vision-only: 69.97%
• Audio-only: 51.88%
• TS-only: 55.63%
• V+TS: 70.82%
• V+A: 68.43%
• Full quad-modal (V+A+TS): 71.16%

The sharp performance gradient from unimodal to quad-modal integration evidences the necessity of physiological grounding for resolving visual/auditory ambiguity. Notably, the TS modality adds discriminative value even when vision is available, directly addressing semantic aliasing.

Uncertainty Quantification

The authors implement a predictive entropy framework (N=10, T=0.7):

• Congruent data: Avg. entropy $H = 1.28$ bits (high model consensus).
• Conflict data (TS discordant with video/audio): $H = 3.15$ bits (significant model uncertainty).

This result confirms that Meow-Omni 1 dynamically reflects uncertainty when modalities provide contradictory cues, rather than over-relying on a single (e.g., visual) modality—a requisite trait for clinical deployment.

Implications

Theoretical Implications

The combination of native quad-modal representation and formal causal intent modeling sets a new standard for computational ethology—eschewing naive behavior classification in favor of inferring hidden drives under environmental constraints. Furthermore, this work demonstrates that foundation models can be successfully adapted for non-human context reasoning via architectural and data-centric specialization.

Practical Implications and Applications

The released Meow-Omni 1 pipeline and Meow-10K dataset constitute a scalable blueprint for real-world deployment in veterinary diagnostics, ethological research, and wildlife conservation. By integrating time-series biosignals at the token level, interpretability and safety are enhanced, as evidenced by robust uncertainty signaling in ambiguous cases—an essential requirement for AI in high-stakes settings.

Additionally, the alignment-specialization training schedule (frozen encoder, then LLM fine-tuning) offers a reproducible paradigm for future non-human multimodal LLM engineering.

Limitations and Future Directions

• Temporal and contextual scope: The current NBP paradigm is limited in capturing long-term or hierarchically staged intentions.
• Synthetic evaluation constraints: The MeowBench conflict dataset, while expert-validated, remains partly synthetic; large, naturally co-registered quad-modal animal datasets remain an open desideratum.
• Real-time interactivity: Presently a passive inference engine, Meow-Omni 1 could be extended to active feedback regimes, e.g., duplex Text-to-Speech for responsive wildlife auditing.
• Cross-species generalization: Generalization beyond domestic cats will require zero/few-shot transfer studies leveraging the physiological-behavioral alignments established herein.

Conclusion

Meow-Omni 1 concretely demonstrates how direct integration of physiological time-series signals with audio-visual data in a multimodal LLM architecture improves the disambiguation of animal intent. The presented results substantiate the critical role of native, high-frequency biological grounding in overcoming the inherent ambiguity of non-verbal animal communication. This technical framework lays the foundation for future AI models capable of nuanced, safety-critical behavioral reasoning across non-human species, with significant applicability in veterinary medicine and wildlife conservation.

Reference:

"Meow-Omni 1: A Multimodal LLM for Feline Ethology" (2605.09152)