OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe (2511.16334v1)

Abstract: Recent advancements in large reasoning models have fueled growing interest in extending such capabilities to multimodal domains. However, despite notable progress in visual reasoning, the lack of transparent and reproducible data curation and training strategies remains a major barrier to scalable research. In this work, we introduce OpenMMReasoner, a fully transparent two-stage recipe for multimodal reasoning spanning supervised fine-tuning (SFT) and reinforcement learning (RL). In the SFT stage, we construct an 874K-sample cold-start dataset with rigorous step-by-step validation, providing a strong foundation for reasoning capabilities. The subsequent RL stage leverages a 74K-sample dataset across diverse domains to further sharpen and stabilize these abilities, resulting in a more robust and efficient learning process. Extensive evaluations demonstrate that our training recipe not only surpasses strong baselines but also highlights the critical role of data quality and training design in shaping multimodal reasoning performance. Notably, our method achieves a 11.6% improvement over the Qwen2.5-VL-7B-Instruct baseline across nine multimodal reasoning benchmarks, establishing a solid empirical foundation for future large-scale multimodal reasoning research. We open-sourced all our codes, pipeline, and data at https://github.com/EvolvingLMMs-Lab/OpenMMReasoner.

• The paper’s main contribution is a two-stage training framework that integrates supervised fine-tuning with reinforcement learning to enhance multimodal reasoning performance.
• It demonstrates that increased answer diversity and cross-domain data curation yield significant improvements in benchmarks and token efficiency.
• The work provides open-source code, data, and empirical guidelines to promote reproducibility and scalable deployment of reasoning models.

OpenMMReasoner: A Unified Recipe for Scalable Multimodal Reasoning

Introduction

The "OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe" (2511.16334) paper addresses the limitations in transparency and scalability that hamper reproducible research in large multimodal reasoning models (LMRMs). The work presents OpenMMReasoner, a fully open-source two-stage training framework consisting of a systematic supervised fine-tuning (SFT) phase, followed by reinforcement learning (RL) for reasoning enhancement. By rigorously dissecting data curation, sampling strategies, and optimization techniques, the authors establish practical guidelines for constructing scalable and robust multimodal reasoning systems. The entire pipeline—including training data, model weights, and code—is made freely available to facilitate reproducibility and empirical validation.

Supervised Fine-tuning Data Curation

The SFT stage uses a cold-start dataset comprising 874,000 samples sourced from diverse public benchmarks, covering both general vision-language reasoning (e.g., LLaVA-CoT, OpenVLThinker) and specialized mathematical reasoning tasks (MMR1, MiroMind-M1). The authors enforce strict stepwise validation procedures such as answer format normalization, rule-based filtering, and LLM-as-judge verification. Teacher model selection is shown to be critical—distilling from stronger models (Qwen3-VL-235B-Instruct) yields a 4.5-point improvement in reasoning benchmarks compared to the baseline (Qwen2.5-VL-7B-Instruct).

A key empirical finding is that answer diversity across repeated teacher sampling is more significant for generalization than mere question diversity. Increasing the number of verified answer traces per question from 1x to 8x elevates MathVision accuracy from 29.3 to 34.6 and MathVerse from 52.9 to 57.1, showing that fine-grained reasoning exposure is essential. Importantly, the study finds that excessive filtering (based on answer length or question difficulty) degrades overall performance, undermining the benefits of dataset heterogeneity.

Cross-domain mixing—incorporating both image-based and text-based mathematical supervision—further boosts transfer learning capabilities, with mixed datasets increasing MathVision and MathVerse scores compared to unimodal sources. This supports the assertion that grounding in varied modalities is beneficial for task generalization.

Reinforcement Learning for Reasoning Enhancement

Following SFT, the RL phase leverages 74k samples spanning science, mathematics, chart understanding, and puzzle domains. The paper rigorously compares recent RL algorithms tailored for LLMs/LMMs:

• GSPO (Group Sequence Policy Optimization), which uses sequence-level importance ratios, is empirically superior to DAPO and GRPO in terms of training stability and convergence rate.
• GSPO with 16-way rollouts at moderate temperature (1.0) achieves the highest reward stability, minimizing entropy collapse observed in DAPO under limited rollouts or aggressive exploration.

A composite reward function balances task accuracy and output format, with ablation showing that low format weighting ($\lambda_{\text{fmt}}=0.1$) maximizes downstream reasoning performance. The approach discourages unnecessarily verbose responses, preserving token efficiency—a direct response to concerns over unbounded chain-of-thought expansions in prior art (e.g., OpenVisionReasoner).

Difficulty-aware curriculum strategies were tested; however, mixed sampling delivered better results, indicating that uniform exposure to problem diversity is preferable for scalable RL in LMRMs.

Numerical Results and Evaluation

OpenMMReasoner achieves an average 11.6% improvement over Qwen2.5-VL-7B-Instruct across nine multimodal reasoning benchmarks (MathVision, MathVerse, MathVista, MMMU, and others), outperforming prior RL-enhanced methods such as ThinkLite-VL, MM-Eureka, and OpenVisionReasoner. In the MathVista testmini split, the model reaches 79.5, compared to 72.1 for OVR. For We-Math and DynaMath, it achieves 79.0 and 34.9 respectively.

Token efficiency analyses reveal that OpenMMReasoner delivers superior accuracy with significantly shorter reasoning traces compared to OVR. The model maintains computational tractability—a key practical consideration for real-world deployment.

Moreover, the RL phase leads to concurrent improvements in textual reasoning tasks (AIME24, AIME25), demonstrably transferring multimodal reasoning skills to purely linguistic domains. This corroborates findings from recent cross-domain transferability studies [huan2025doesmathreasoningimprove].

Implications and Future Directions

From a practical standpoint, OpenMMReasoner establishes a reproducible and open blueprint for building LMRMs, emphasizing data, pipeline, and model transparency. The systematic ablation studies on data sampling, teacher distillation, and RL algorithms provide concrete empirical guidelines for both academic and industrial practitioners seeking to scale multimodal reasoning.

Theoretically, the paper demonstrates that answer diversity, algorithm selection (GSPO), and moderate curriculum interventions represent critical axes for scaling multimodal reasoning models effectively. The observed cross-domain transfer of reasoning capabilities substantiates the hypothesis that LMRMs and LLMs share an underlying cognitive substrate that can be strengthened through multimodal RL.

For future research, the authors suggest extending their unified recipe to other modalities (video, audio, temporal streams) and architectures, exploring even larger-scale data regime scaling. Broader evaluations and algorithmic innovations in curriculum learning, reward shaping, and token efficiency tradeoffs are promising directions for the next generation of agentic multimodal AI systems.

Conclusion

OpenMMReasoner provides a transparent, scalable framework for multimodal reasoning in LMRMs, validated through strong empirical results and extensive ablations. By harmonizing rigorous SFT data curation and RL-based refinement, it enables both reproducibility and scalable performance, setting new standards for open research in the multimodal reasoning community. The findings highlight answer diversity and algorithmic robustness as key levers for achieving generalizable and efficient multimodal reasoning, with significant implications for the future development of agentic, cross-domain AI systems.