Mach-Mind-4-Flash Technical Report

Abstract: We present Mach-Mind-4-Flash, a 35B-parameter Mixture-of-Experts (MoE) agentic model with 3B activated parameters. Through post-training optimization alone without scaling pre-training compute, the model achieves performance on par with or surpassing that of 100B-parameter-class models. By introducing scalable agentic interaction environments for large-scale reinforcement learning, the model attains significant performance gains on real-world application tasks. Our pipeline comprises three stages: (1) a unified RL/OPD training infrastructure with dynamic multi-teacher scheduling and operator-level acceleration, delivering 17\% end-to-end training speedup; (2) multiple domain-specific RL experts trained in parallel across Reasoning, General, and Agent tracks, then fused into a single generalist via Multi-Teacher On-Policy Distillation (MOPD) -- a routed reverse-KL objective that eliminates the see-saw degradation of mixed-reward RL; (3) Hybrid Median-length Policy Optimization (HMPO), a single-stage token-efficiency method that compresses reasoning chains by 19--46\% with $\le$0.7 percentage-point accuracy loss. Mach-Mind-4-Flash scores 92.70 on AIME'26, 82.82 on IFBench, 80.74 on Behavioral-SafetyBench, 75.80 on BFCL-v4, 72.31 on BrowseComp-zh, and 84.20 on ClawBench -- leading or matching models with 10--30$\times$ its activated size at a fraction of the inference cost.

• The paper presents a scalable post-training pipeline that specializes domain expertise to match or exceed 100B-class models using a 35B-parameter MoE with only 3B activated parameters.
• It integrates multi-track expert RL, operator-level optimizations, and multi-teacher on-policy distillation to deliver efficient, robust agentic performance.
• Empirical results demonstrate state-of-the-art benchmarks in reasoning, safety, and tool-use, while significantly reducing computational costs and inference latency.

Mach-Mind-4-Flash: Scaling Agentic Expertise via Specialization-then-Integration Post-Training

Overview and Motivation

Mach-Mind-4-Flash introduces a 35B-parameter Mixture-of-Experts (MoE) LLM operating at 3B activated parameters, specifically engineered to attain parity with or surpass 100B-class models across diverse agentic competencies. The central claim is that, through a sophisticated post-training stack—eschewing additional pre-training compute in favor of coordinated domain specialization and efficient fusion—it is possible to realize performance competitive with the contemporary frontier at a fraction of the inference and deployment costs (Figure 1).

Figure 1: Mach-Mind-4-Flash outperforms or matches models with much larger activated parameter counts across multiple capability dimensions.

This report delineates the architecture, the multi-stage post-training pipeline, the supporting infrastructure optimizations, and strong empirical results, while also identifying limitations and indicating future research directions.

Unified Infrastructure and Operator-Level Acceleration

A cornerstone of Mach-Mind-4-Flash is a unified RL/OPD training infrastructure that simultaneously supports RL, on-policy knowledge distillation, and hybrid training. The operator-level optimizations comprise the deep integration of SonicMoE—leveraging indexed and grouped GEMMs and segmented expert fusion—yielding significant system-level speedups.

Figure 2: Unified RL/OPD training framework underpins all post-training stages, providing dynamic phase-switching and cost-efficient infrastructure.

Figure 3: Operator-level design enables communication-computation overlap and scalable resource utilization in MoE architectures.

These enhancements enable dynamic multi-teacher scheduling at scale, elastic resource orchestration, and practical expert parallelism. The pipeline achieves up to 17% improved training throughput compared to prior baselines, critically expanding the feasible envelope for specialist RL.

Specialization-then-Integration: Multi-Track Expert RL and Distillation

The post-training approach organizes expertise development into three orthogonal tracks: Reasoning (Math, Code, STEM), General (Instruction Following, Writing, Safety), and Agent (Tool-Use, DeepSearch, Code Agent, Claw Agent). Each domain is trained with tailored RL objectives, verifiable reward definitions, and high-quality, automatically curated data with difficulty pruning.

Figure 4: Post-training pipeline: SFT initialization, parallel domain-specific RL, fusion via MOPD, and token efficiency optimization consolidate domain expertise.

Figure 5: SFT corpus allocation: code and math are token-dominant reflecting the high complexity and long-chain reasoning required for agentic and coding domains.

Distinct reward design paradigms—such as two-stage curricula for sparse-outcome domains and rubric-enhanced automated evaluation for writing—combat reward sparsity and hacking, and improve sample efficiency (Figure 6).

Figure 6: Two-stage curriculum overcomes reward sparsity and reward hacking, yielding stable early learning followed by outcome-aligned improvement in structured reasoning tasks.

For long-horizon agent tasks (tool use, code agent, general autonomous agents), the environment pool is scaled not by increasing trace count but by diversifying verifiable interactive environments, allowing for robust, realistic trajectory optimization (Figure 7).

Figure 7: EnvScaling: file-system, verifiable, and model-simulated environments enable synthesis and evaluation of complex agentic multi-turn tasks.

Code agent and DeepSearch RL components further combine end-to-end execution environments, curriculum synthesis, and enhanced context persistence with error masking, trajectory distillation, and functional rewriting (Figures 8 and 9).

Figure 8: DeepSearch synthesis pipeline generates high-difficulty, multi-hop QA requiring cross-domain retrieval and reasoning.

Figure 9: Multi-stage code agent data curation, distillation, and augmentation produce robust agentic trajectories.

The Claw Agent loop (Figure 10) generalizes to heterogeneous, sandboxed tool-execution tasks, tracking trajectories and rewards for dozens of turns and enabling credit reweighting.

Figure 10: Claw Agent framework supports multi-turn, tool-rich, sandboxed tasks with modular resource provisioning and distributed asynchronous rollouts.

Multi-Teacher On-Policy Distillation: Fusing Heterogeneous Experts

Multi-Teacher On-Policy Distillation (MOPD) is the core mechanism for fusing diverse expert models into a unified, generalist agent. Each data sample is routed to its respective frozen expert; the student model is optimized directly on its own generation through a reverse-KL objective, circumventing capability degradation and minimizing destructive interference endemic to mixed-reward RL.

Figure 11: MOPD top-$K$ overlap: distillation yields monotonic alignment between the student and the most relevant expert for each domain sample.

MOPD demonstrates stable convergence across all expertise tracks and achieves high teacher-student overlap rates without need for additional off-policy corrections. Notably, expert composition studies reveal that explicit inclusion of code experts elevates reasoning and safety capabilities via cross-domain transfer, and that capabilities absent from the teacher ensemble tend to regress in the fused model.

Token Efficiency Optimization using HMPO

To address reasoning overthinking and excessive verbosity, Hybrid Median-length Policy Optimization (HMPO) applies a group-adaptive, correctness-first, length-second reward via a single-stage RL schedule. The median length of correct rollouts is utilized as a dynamic budget, with cosine-decay token rewards enforcing efficient, concise solutions without accuracy loss.

Figure 12: HMPO: Rollouts with lengths above the adaptive median of correct samples are penalized, ensuring only concise and fully correct generations receive reward.

Empirical results show 19–46% reduction in output chain lengths at a maximum of 0.7 pp accuracy loss, reshaping efficiency-accuracy Pareto curves and enabling the compact model to vastly outperform much larger models on both cost and capability axes (Figure 13).

Figure 13: HMPO delivers performance competitive with massive models, but with sharply reduced average token consumption per answer.

Empirical Results and Capability Analysis

Mach-Mind-4-Flash achieves leading or highly competitive scores against state-of-the-art open and closed LLMs (including 100B and trillion-class activated-parameter models) across a battery of challenging agentic, reasoning, instruction following, safety, tool-use, and autonomous benchmarks. Highlights include:

• AIME'26 (competition math): 92.70, on par with Qwen3.5-122B (91.67) and within 1 point of Kimi-K2.5-1T (93.30) despite a $\sim$30$\times$ activated parameter disparity.
• Behavioral-SafetyBench: 80.74, >13 points ahead of the nearest non-proprietary baseline, indicating strong behavioral safety in agentic, tool-rich deployments.
• IFBench (instruction following): 82.82, top performance among all reported models, verifying non-reliance on template memorization.
• BrowseComp-zh (web agentic QA): 72.31, matching or exceeding multimodal 122B and 1T models on complex multi-hop agentic tasks.

Agentic coding, reasoning, and multi-turn tool-use scores closely trail specialist models specifically optimized for those domains, validating the sufficiency of consolidation and distillation methods.

Practical and Theoretical Implications

This work demonstrates that meticulous post-training optimization—centered on scalable specialization, data-centric curation, flexible infrastructure, and judicious distillation—can deliver MoE agents with capabilities rivaling models an order of magnitude larger, thus challenging the prevailing scaling-centric paradigm. The practical implications are immediate for deployment in cost-sensitive or latency-bound scenarios, where inference cost and hardware constraints preclude the use of giant activated parameter budgets.

Theoretical contributions include robust multi-domain specialist fusion via routed reverse-KL objectives, showing mitigation of destructive interference, and adaptive RL reward shaping for token-efficient reasoning without auxiliary pruning stages.

Limitations and Future Directions

The paper identifies persistent challenges:

• Long-horizon, repository-level tasks remain vulnerable to partial expert knowledge smoothing during MOPD.
• Extension of HMPO to multi-turn, agentic trajectories and reasoning beyond the single-turn remains unaddressed.
• Long-context retention and coherent, persistent web browsing remain the weakest axes for the compact model, corroborating the insufficiency of horizon scaling in isolation.

Anticipated research directions include domain- and turn-aware token efficiency, advanced distillation preserving rare, high-difficulty expert behaviors, and multimodal agentic fusion including GUI and visual modalities.

Conclusion

Mach-Mind-4-Flash establishes a scalable blueprint for building compact, high-capacity agentic LLMs through specialization-then-integration post-training, supported by unified infrastructure and adaptive objective design. Its performance across a wide array of benchmarks demonstrates that post-training RL, targeted distillation, and token efficiency optimization can enable compact MoE models to rival or surpass contemporary frontier models while minimizing computational and operational burdens (2607.09375).