Generalization beyond English humor and sarcasm benchmarks

Determine whether the modality hierarchies observed in the MultiHuSE, UR-FUNNY, and MUStARD benchmarks and the performance benefits of Inverted Asymmetric Fusion generalize to other languages and to tasks such as emotion recognition, visual question answering, and action recognition.

Background

The evaluation is restricted to English-language datasets focused on humor and sarcasm detection. Although the use of the multilingual E5 encoder suggests possible cross-lingual applicability, the paper does not evaluate Inverted Asymmetric Fusion on non-English data or on other multimodal tasks. Consequently, whether the reported modality hierarchies and architectural benefits transfer to different languages and task domains remains unresolved.

References

It therefore remains unclear whether the modality hierarchies observed here, and the benefits of IAF, generalise to other languages or to tasks such as emotion recognition, visual question answering, or action recognition.

— Mitigating Strong-Modality Collapse in Multimodal Learning via Inverted Asymmetric Fusion  (2608.26879 - Kenneth et al., 27 Aug 2026) in Section “Limitations”

Second, identifying which visual cues provide reliable complementary information to audio remains an open problem.

— Multimodal Temporal Modeling for Continuous Group Emotion Recognition in Multi-party Dialogues  (2609.11164 - Iwata et al., 10 Sep 2026) in Conclusion, final paragraph

While our work advances multimodal emotion understanding from surface-level affective cue recognition toward cognition-grounded appraisal reasoning, several directions remain open for future exploration:

First, CogEmo-40K adopts six complementary appraisal dimensions inspired by established cognitive appraisal theories. These dimensions capture important aspects of emotion formation but are not intended to exhaust the full space of cognitive emotion processes. Future work may explore richer appraisal dimensions and alternative theoretical formulations.

Second, CogEmo-40K is constructed from existing multimodal video data and therefore remains limited by its current domain, language, cultural, and emotion-space coverage. Extending cognition-grounded emotion understanding to more diverse multilingual, cross-cultural, and open-world settings is an important future direction.

— From Surface to Depth: Towards Cognitive Appraisal Reasoning in Multimodal Emotion Understanding  (2610.11918 - Li et al., 8 Oct 2026) in Appendix A, Section “LIMITATIONS AND SOCIAL IMPACT”