Group Cognition Learning: Making Everything Better Through Governed Two-Stage Agents Collaboration

Abstract: Centralized multimodal learning commonly compresses language, acoustic, and visual signals into a single fused representation for prediction. While effective, this paradigm suffers from two limitations: modality dominance, where optimization gravitates towards the path of least resistance, ignoring weaker but informative modalities, and spurious modality coupling, where models overfit to incidental cross-modal correlations. To address these, we propose Group Cognition Learning (GCL), a governed collaboration paradigm that applies a two-stage protocol after modality-specific encoding. In Stage 1 (Selective Interaction), a Routing Agent proposes directed interaction routes, and an Auditing Agent assigns sample-wise gates to emphasize exchanges that yield positive marginal predictive gain while suppressing redundant coupling. In Stage 2 (Consensus Formation), a Public-Factor Agent maintains an explicit shared factor, and an Aggregation Agent produces the final prediction through contribution-aware weighting while keeping each modality representation as a specialization channel. Extensive experiments on CMU-MOSI, CMU-MOSEI, and MIntRec demonstrate that GCL mitigates dominance and coupling, establishing state-of-the-art results across both regression and classification benchmarks. Analysis experiments further demonstrate the effectiveness of the design.

• The paper proposes a two-stage collaboration framework integrating Routing and Auditing agents to selectively regulate multimodal information exchange.
• It employs consensus formation via Public-Factor and Aggregation agents to maintain modality-specific features and mitigate dominance.
• Empirical results demonstrate state-of-the-art performance, enhanced noise robustness, and significant efficiency gains on benchmark datasets.

Group Cognition Learning: Protocol-Governed Collaboration for Multimodal Robustness

Motivation and Problem Definition

Multimodal learning demands the integration of heterogeneous cues—language, audio, and vision—while respecting the specialized signal and complementary evidence each modality provides. Existing centralized fusion models, including Tensor Fusion Network (TFN) and transformer-based architectures, compress all modalities into a unified representation. Such paradigms repeatedly exhibit modality dominance (overfitting to the most predictive input, under-utilizing others) and spurious modality coupling (exploiting incidental cross-modal correlations, compromising robustness). This paper proposes Group Cognition Learning (GCL), a protocol-governed, two-stage agent collaboration framework explicitly designed to mitigate these pathologies by regulating information flow at the interaction process level rather than leaving fusion implicit.

Figure 1: Overview of the GCL architecture, illustrating the selective interaction by Routing and Auditing agents and consensus formation by Public-Factor and Aggregation agents.

Protocol-Governed Two-Stage Collaboration

Stage 1: Selective Interaction

The first stage redefines cross-modal fusion as a dynamic, audit-compliant process. The Routing Agent proposes directed exchange routes, evaluating topological feasibility and packaging content. The Auditing Agent then applies rigorous admission control via marginal predictive gain: routes are gated at the sample level only if expected to reduce task loss, with teacher gain used as a supervision signal for gate learning.

Mathematically, for each directed pair ($m \to n$), the Routing Agent computes a routing logit and message vector, while the Auditing Agent estimates gain $\Delta^{m \to n}$ by comparing the loss before/after tentative integration. GCL only admits exchanges with positive predicted gain, and imposes InfoNCE-based redundancy penalties to ensure updated modality representations remain orthogonal. Gain alignment loss further ensures gating decisions track actual utility, avoiding trivial co-adaptation.

Stage 2: Consensus Formation

Following the refined, selectively audited interaction, Stage 2 synthesizes the final prediction. The Public-Factor Agent distills an explicit semantic commonality via permutation-invariant global pooling or attention, enforced with auxiliary supervision for structural validity. The Aggregation Agent applies adaptive weighting conditioned on the public factor—ensuring contribution reflects genuine marginal utility rather than naive dominance. This architecture preserves modality-specific discriminative features until final consensus, preventing feature collapse.

Empirical Results and Robustness

GCL establishes state-of-the-art performance across CMU-MOSI, CMU-MOSEI, and MIntRec benchmarks for both regression and classification, outperforming EMOE, TSDA, MISA, and other strong baselines. On CMU-MOSI, GCL achieves MAE of 0.685 and Acc-2 of 86.79%, substantially reducing error over competitive alternatives.

Figure 2: GCL demonstrates remarkable robustness to Gaussian noise injected into modalities, significantly outperforming baseline models across all noise levels on CMU-MOSI.

Figure 3: Audited Selectivity analysis shows GCL maximizing Positive-Gain Ratio (PGR) while maintaining moderate Activation Rate (AR), affirming efficient filtering of beneficial exchanges.

Ablation studies confirm the necessity of each architectural component, with indiscriminate full exchange, absence of Routing/Auditing Agents, or removal of Public-Factor/aggregation adaptivity yielding consistent performance deterioration. In particular, the redundancy control module is critical for suppressing spurious coupling—a failure in this regard causes catastrophic collapse under message permutation stress tests.

Figure 4: GCL reliability against spurious coupling tested via message permutation, visualized using HSIC/CKA diagnostics; redundancy-governed GCL maintains high accuracy and low dependence.

Adaptivity and consensus analysis unequivocally show that aggregation via public-factor-guided weights prevents dominance collapse and aligns fusion weights with true marginal utility.

Figure 5: Consensus landscape reveals GCL achieves moderated dominance and high alignment correlation, maximizing evidence quality without favoring any single modality.

Efficiency and Practical Implications

GCL's lightweight agent-based protocol achieves superior Pareto efficiency, reducing parameters and computational latency compared to high-capacity contrastive, MoE, and disentanglement baselines. Computational efficiency analyses demonstrate 54% parameter reduction relative to ConFEDE and 25% training time improvement over EMOE.

The explicit governance protocol translates into enhanced robustness to distribution shifts and signal corruption—the architecture actively suppresses the propagation of noise and prevents brittle dependencies on incidental correlations. This practical advantage is critical for real-world deployment in affective computing, intent recognition, and multimodal human-machine interaction.

Theoretical Implications and Future Directions

GCL advances the foundational understanding of multimodal integration: effective learning is not merely a matter of maximizing connectivity or fusing representations, but depends on transparent governance of information flow and semantic consensus. The protocol distinguishes causality from correlation, maximizing label-relevant utility while maintaining interpretability. The separation of specialization channels and a public semantic anchor provides a framework for further exploration of dynamic collaboration under adversarial conditions, missing modalities, or causal intervention scenarios.

Future work may involve extending the governance protocol to larger-scale modality sets, hierarchical aggregator cascades, federated/multi-agent settings, and foundation models. The marginal-gain auditing paradigm could be generalized to other domains where tacit gradient dynamics introduce optimization shortcuts or spurious alignments.

Conclusion

Group Cognition Learning presents a principled, agent-based multimodal learning protocol that mitigates modality dominance and spurious coupling via selective interaction and consensus formation. The architecture transforms opaque fusion into transparent, audit-driven governance, yielding robust, state-of-the-art multimodal predictions, enhanced efficiency, and improved interpretability. GCL thus represents a rigorous framework for future multimodal AI systems requiring both reliability and theoretical soundness (2605.00370).