NVIDIA Nemotron 3: Efficient and Open Intelligence (2512.20856v1)

Abstract: We introduce the Nemotron 3 family of models - Nano, Super, and Ultra. These models deliver strong agentic, reasoning, and conversational capabilities. The Nemotron 3 family uses a Mixture-of-Experts hybrid Mamba-Transformer architecture to provide best-in-class throughput and context lengths of up to 1M tokens. Super and Ultra models are trained with NVFP4 and incorporate LatentMoE, a novel approach that improves model quality. The two larger models also include MTP layers for faster text generation. All Nemotron 3 models are post-trained using multi-environment reinforcement learning enabling reasoning, multi-step tool use, and support granular reasoning budget control. Nano, the smallest model, outperforms comparable models in accuracy while remaining extremely cost-efficient for inference. Super is optimized for collaborative agents and high-volume workloads such as IT ticket automation. Ultra, the largest model, provides state-of-the-art accuracy and reasoning performance. Nano is released together with its technical report and this white paper, while Super and Ultra will follow in the coming months. We will openly release the model weights, pre- and post-training software, recipes, and all data for which we hold redistribution rights.

• The paper introduces a hybrid architecture integrating Mamba-2 and sparse MoE layers with selective self-attention to achieve 3.3× throughput gains and superior long-context reasoning.
• It employs LatentMoE to project token representations into a lower-dimensional space, reducing parameter and communication costs while enhancing downstream accuracy.
• Multi-token prediction, NVFP4 quantization, and multi-environment RL are combined to improve training sample efficiency and enable effective agentic operation across diverse tasks.

NVIDIA Nemotron 3: Efficient Architectures and Open Agentic AI

Hybrid Mamba-Transformer Mixture-of-Experts Architecture

Nemotron 3 introduces a Mixture-of-Experts (MoE) hybrid architecture combining Mamba-2 sequence modeling layers, sparse MoE layers, and limited self-attention. The design maximizes throughput for reasoning-centric and agentic workloads requiring both compute-efficient inference and extended context capabilities. Unlike conventional Transformer MoEs that often interleave costly attention layers, Nemotron 3 predominantly integrates Mamba-2 and MoE blocks, relegating self-attention to select positions to preserve global information routing.

Figure 1: Nemotron 3 leverages a hybrid Mamba-Transformer MoE stack, interleaving Mamba-2 and MoE layers with sparse inclusion of attention components.

Extensive benchmarks demonstrate the hybrid architecture's state-of-the-art accuracy across multi-step reasoning, coding, mathematical, and long-context retrieval tasks. For instance, Nemotron 3 Nano achieves 3.3$\times$ greater inference throughput than a size-matched Transformer MoE variant at typical reasoning sequence lengths, and superior accuracy on sequence extrapolation tasks up to 1M tokens.

Figure 2: The hybrid design delivers both leading accuracy and throughput on challenging reasoning and ultra-long-context tasks compared to Transformer MoEs.

This architecture exploits sparse expert routing and constant state Mamba layers, yielding superior accuracy-to-compute scaling. The inclusion of MoE and attention provides adaptability for agentic environments and multi-agent orchestration.

LatentMoE: Hardware-Efficient, Expressive Sparsity

The LatentMoE innovation in Super and Ultra models targets inference bottlenecks characteristic of large-scale sparse architectures. Unlike a standard MoE, LatentMoE projects token representations from the full hidden dimension $d$ to a reduced latent space $\ell$, routing and computing expertise exclusively in this lower-dimensional manifold. This architectural choice dramatically reduces routed parameter and communication bandwidth by a factor of $d/\ell$; e.g., typically 4$\times$.

Figure 3: Standard MoE architecture routes full hidden dimension through experts, incurring substantial parameter and communication cost.

LatentMoE enables the simultaneous scaling of expert diversity ($N$) and active expert selection per token ($K$) without increasing inference latency, thus improving nonlinear modeling capacity at fixed resource budgets.

Figure 4: Relative loss differences between NVFP4 and BF16 across model scales reveal diminishing quantization-induced gaps with larger parameterization; keeping sensitive projection layers at high precision is critical.

Empirical results show uniform downstream accuracy improvements (e.g., MMLU, code, math) versus conventional MoE baselines, validating LatentMoE's accuracy-per-byte advantage.

Multi-Token Prediction for Training and Speculative Decoding

Multi-Token Prediction (MTP) layers augment the two larger Nemotron models, simultaneously improving training sample efficiency and facilitating speculative decoding for accelerated inference. By predicting multiple future tokens per step, MTP increases supervisory density, enhances multi-step planning capability, and provides high-fidelity "draft" tokens.

Quantitative downstream ablations report 2.4\% mean accuracy improvement across general knowledge, code synthesis, commonsense, reading comprehension, and math, confirming MTP's utility for both data efficiency and reasoning quality.

NVFP4 Quantized Training and Efficient Inference

Nemotron 3 advances model quantization via full utilization of the NVFP4 format for weight, activation, and gradient tensors, leveraging native NVFP4 GEMMs across the training pipeline. This enables up to 3$\times$ higher training throughput compared to FP8 on Blackwell GPUs and aligns with emerging scaling laws for quantization-aware training, providing near-parity performance with BF16 at scale.

Post-ablation, critical layers such as the Mamba output projection and attention QKV mapping are retained in higher precision to avoid excessive information nullification—empirical validation finds <1\% relative train/val loss gaps when these layers are appropriately managed, and downstream evaluation accuracy is unaffected.

Figure 5: Downstream accuracy parity between NVFP4 and BF16 trajectories for an 8B active parameter MoE model, underscoring the stability of FP4 training at scale.

Extended Context: Robustness and Predictive Utilization

Nemotron 3 enables context lengths up to 1M tokens, essential for multi-turn agentic reasoning and retrieval-augmented pipelines. Removal of RoPE in attention layers circumvents positional extrapolation instability, leveraging Mamba's implicit positional encoding. Continued pre-training and SFT at 512k/256k context lengths, reinforced by RL-based long-context environments, yield strong context adaptation.

NLL analysis on million-token code sequences demonstrates effective utilization of extended context, with NLL monotonically decreasing with token position, indicative of robust long-range dependencies modeling.

Figure 6: Code NLL as a function of position for Nemotron Nano 3, showing improved prediction up to 1M tokens.

Multi-Environment RL and Reasoning Budget Control

Final Nemotron 3 capabilities owe to multi-environment RL post-training, in which agentic benchmarks (coding, math, tool use, search, chat, software engineering) are simultaneously optimized. The RL stack incorporates asynchronous rollout generation and policy-tuned GRPO algorithms, with MTP accelerating inference. This paradigm avoids task-specific catastrophic forgetting and reward hacking common in staged RL approaches.

Figure 7: Simultaneous RL optimization across diverse agentic environments, promoting uniformly increasing benchmarks during training.

Models support granular reasoning budget control at inference: users can cap internal "thinking" token traces, trading off efficiency and output quality for real-world deployment flexibility.

Figure 8: Reasoning budget control curve quantifies accuracy versus computational efficiency, enabling user-directed trade-offs.

Implications and Future Outlook

Nemotron 3's open, transparent release strategy—encompassing model weights, rich datasets (10T+ tokens), and pre-/post-training recipes—positions it for broad adoption in agentic, multi-modal, and long-context AI systems. The demonstrated scaling robustness of FP4, hardware-aware efficient MoE, adaptive RL optimization, and pragmatic inference controls answer key limitations for practical LLM deployment in latency-critical and throughput-intensive settings.

Future research directions include further expert design for hardware specialization, integration of multimodal environments, more efficient RL paradigms exploiting off-policy and adaptive curriculum learning, and compositional reasoning pipelines. The model's open-source RL and data generation stacks could democratize advanced agent benchmarking, facilitate benchmarking on evolving domains (e.g., SWE, coding, interactive multi-agent), and encourage downstream safety/alignment contributions.

Conclusion

Nemotron 3 substantiates efficient, open agentic architectures for long-context reasoning, combining hybrid Mamba-Transformer MoE, LatentMoE, MTP, NVFP4 quantization, and multi-environment RL post-training. Its accuracy, throughput, adaptability, and deployment flexibility set a new standard for practical and scalable LLM design. Continuing efforts in integrating new modalities, data-centric scaling, and efficient agentic training are expected to further advance the performance and democratization of open frontier models.

(2512.20856)