---
title: 'Nanbeige4.1-3B: Versatile 3B LLM for Reasoning & Coding'
url: https://www.emergentmind.com/topics/nanbeige4-1-3b
type: topic
---

# Nanbeige4.1-3B: Versatile 3B LLM for Reasoning & Coding

Nanbeige4.1-3B is a 3B-parameter open-source generalist language model from the Nanbeige LLM Lab (Boss Zhipin), presented as a compact successor within the Nanbeige 3B line and optimized to unify general reasoning, preference-aligned dialogue, code generation, tool use, and deep-search behavior in a single dense model. The designation “reasons, aligns, and acts” summarizes its intended scope: stronger math and science problem solving, better human preference fit and conversational quality, and sustained multi-step interaction with tools and retrieval systems. The released checkpoint is hosted at `https://huggingface.co/Nanbeige/Nanbeige4.1-3B` [2602.13367].

## 1. Lineage and conceptual scope

Nanbeige4.1-3B is introduced as a unified generalist successor to prior Nanbeige 3B models. It is initialized from **Nanbeige4-3B-Base** and post-trained with a broader recipe than **Nanbeige4-3B-2511**, with explicit emphasis on three capability clusters: **general reasoning**, **code generation**, and **long-horizon agentic / deep-search behavior**. Its intended use cases include general reasoning, mathematical reasoning, scientific reasoning, coding and competitive programming, instruction following, dialogue, preference-aligned responses, tool use and function calling, deep search and browsing, research-agent-style tasks, and multi-turn long-horizon interaction [2602.13367].

This positioning is best understood against the earlier **Nanbeige4-3B** family, which the preceding technical report describes as a family of small-scale autoregressive language models with explicit **Base** and **Thinking** variants. That family was framed around **23T high-quality pretraining tokens**, **over 30 million diverse instructions**, **64K context**, **Fine-Grained Warmup-Stable-Decay (FG-WSD)** scheduling, **Solution Refinement**, **CoT Reconstruction**, **Dual-Level Preference Distillation**, and multi-stage reinforcement learning [2512.06266]. Nanbeige4.1-3B inherits the base-model substrate from that line, but its defining identity is not merely as another reasoning-specialized checkpoint; it is presented as a single 3B model intended to combine reasoning, alignment, coding, and agentic search within one deployment target [2602.13367].

The authors further characterize Nanbeige4.1-3B, to the best of their knowledge, as the first open-source small language model to achieve this level of versatility in a single model. That formulation is best treated as a positioning claim rather than a formally provable property, although the benchmark suite does support unusually broad capability concentration at 3B scale [2602.13367].

## 2. Model specification and the boundary of disclosure

The explicit model specification is sparse. The report gives a **parameter count of 3B**, states that the system is a **single dense 3B model**, identifies **Nanbeige4-3B-Base** as the initialization point, and states that the **context length after post-training reaches 256K**. It also makes clear that the model is open-source [2602.13367].

By contrast, most conventional architectural descriptors are omitted. The report does **not** specify the backbone family, number of layers, hidden size, attention head count, tokenizer or vocabulary, positional encoding type, grouped-query or multi-query attention, RoPE or ALiBi usage, MLP variant, activation function, normalization type, KV-cache design, quantization guidance, serving throughput, or memory footprint. It also provides **no full architectural equations** for the transformer itself [2602.13367].

This omission matters because Nanbeige4.1-3B is presented as a strong empirical system rather than as an architecture paper. A plausible implication is that the paper’s main scientific contribution lies in post-training design, reward construction, and agentic data synthesis, not in a newly disclosed backbone. For readers interested in the inherited training philosophy beneath the base checkpoint, the earlier Nanbeige4 report is materially more informative, especially on the prior 23T-token pretraining recipe and the preceding 64K long-context setup [2512.06266].

## 3. Supervised post-training and the 256K curriculum

The post-training recipe begins from **Nanbeige4-3B-Base** and first applies an expanded **supervised fine-tuning** stage. Relative to the earlier **Nanbeige4-3B-2511** line, the changes are explicit: the SFT mixture is rebalanced toward **more code**, **more challenging math**, and **more hard general-domain problems**; the context curriculum is extended from **32K → 64K** to **32K → 64K → 256K**; and the synthetic reasoning pipeline is strengthened through **scaled-up Solution Refinement iterations** and a **stronger CoT Reconstruction model** for cleaner reasoning traces [2602.13367].

The final **256K-stage** SFT mixture is deliberately multi-domain: **27% Code, 26% Deep-Search, 23% STEM, 13% Tool-use, and 10% General**. This distribution is important because it shows that long-context post-training is not reserved for only one capability family; the model is shaped as a unified generalist even at the longest context stage [2602.13367].

The terms **Solution Refinement** and **CoT Reconstruction** are inherited from the Nanbeige4 framework. In the earlier report, Solution Refinement is described as a checklist-conditioned process in which a teacher-selection mechanism chooses the best teacher for a task, candidate answers are co-generated, and an evaluator performs cross-evaluation against criteria such as correctness, completeness, consistency, executability, and safety, followed by iterative generate-review-revise cycles. CoT Reconstruction then uses a chain-completion model to generate a brief summary chain of thought and a fuller explicit chain of thought consistent with the refined final solution [2512.06266]. Nanbeige4.1-3B does not redefine these ideas; rather, it strengthens their use in synthetic reasoning data generation [2602.13367].

The SFT ablation confirms that these adjustments are consequential. Against the previous Nanbeige SFT baseline, **LCB V6** rises from **45.5** to **62.0**, **LCB Pro Medium** from **1.8** to **22.8**, **HMMT Nov** from **60.7** to **74.3**, **IMO-Answer-Bench** from **34.8** to **48.9**, **Arena-Hard V2** from **45.5** to **60.2**, and **Multi-Challenge** from **42.6** to **44.4** [2602.13367].

## 4. Reward modeling, alignment, and coding optimization

After SFT, the training pipeline moves into general reinforcement learning because the authors observe **repetition**, **redundant thinking**, and **formatting issues**. The first RL stage uses **GRPO**, with **8 rollouts per prompt**, and a **point-wise general reward model** trained on curated large-scale human preference data. Because the point-wise reward model scores each sampled response independently, it is intended to suppress repetitive outputs, low-readability outputs, and over-redundant reasoning [2602.13367].

A second general RL stage then introduces **pair-wise reward modeling**. The pair-wise reward model is trained on paired comparison data from **code generation** and **LMArena-style conversations**, both **single-turn** and **multi-turn**. Response pairs are formed from a **strong model** and a **weak model**, then filtered with the same checklist strategy used in Nanbeige4; for multi-turn data, the **full dialogue history is concatenated** into the reward-model input. The report also mentions a **swap-consistency regularizer** to mitigate position bias, defined as an MSE-style consistency constraint under response-order swapping, though the exact formula is not provided [2602.13367].

The pairing of these RL stages is sequential rather than collapsed into one objective: **SFT → point-wise RL → pair-wise RL**. Their empirical roles are differentiated. On **Arena-Hard V2**, scores increase from **60.2** at SFT to **66.6** after point-wise RL and **73.8** after pair-wise RL; on **Multi-Challenge**, the sequence is **44.4 → 47.7 → 55.1**. On **LCB V6**, the corresponding progression is **62.0 → 66.0 → 65.6**, which suggests that pair-wise RL contributes most clearly to alignment-style benchmarks rather than to coding performance [2602.13367].

Coding receives its own dedicated RL pipeline. The system uses a **unified judge system** with two components: an **execution-based sandbox** for correctness and an **instruct judge model** for time-complexity comparison. In the first code-RL stage, reward is based on **pass rate**, and on-policy filtering keeps only problems of moderate difficulty: from **8 rollouts**, a problem is retained when the number of successful solves satisfies **\(k \in [1,5]\)**. In the second stage, the reward becomes **gated by correctness**: if a solution is not fully correct, the objective optimizes **formatting + correctness** only; if it **passes all tests**, a **time-complexity reward** is added. Each sample includes the problem statement, test cases, a time-complexity-optimal solution, and an optimal complexity label [2602.13367].

This correctness-first, efficiency-second structure is meant to avoid a common coding-RL failure mode in which a model passes tests through brute force or suboptimal algorithms. The appendix examples support that framing qualitatively, showing improvements such as **\(O(N \log N)\) to \(O(N)\)**, **\(O(N + M\log N)\) to \(O(N)\)**, and **\(O(N^2 \log N)\) to \(O(N \log N)\)**, although the report does not provide a final benchmark table isolating the marginal effect of the time-complexity reward alone [2602.13367].

## 5. Agentic behavior, deep search, and long-horizon tool use

In this report, **deep search** denotes retrieval-centric tasks involving **complex multi-hop reasoning**, **extremely long-context settings**, and **iterative interaction with the environment** to gather information. The model is trained for this setting with a synthetic data pipeline built around a **Wikipedia entity-relation graph**. Head entities are chosen from those updated in the past **six months**; the system performs **conditional random walks** over the graph; and the resulting relational chains, augmented with temporal context, are fed to a strong LLM to synthesize hard questions [2602.13367].

Those questions are then solved using **multiple agent frameworks**, which generate diverse reasoning paths and tool trajectories. The resulting traces are mapped into **unified multi-turn tool-invocation sequences**, after which **turn-level judgment** is applied. Each turn is evaluated for **logical soundness**, **tool-call accuracy**, and **informational gain**. Failed turns are either **excluded from training** or assigned **negative reward** [2602.13367].

The paper argues that this turn-level supervision improves long-horizon stability by sharpening credit assignment: bad intermediate steps can be penalized without waiting for final task completion, and good intermediate steps can be reinforced before the final answer is reached. There is no direct ablation isolating turn-level supervision alone, so that specific causal claim remains less directly demonstrated than the effect of synthetic search data itself [2602.13367].

The evaluation environment is also explicitly specified. Search and tool-use experiments are run in the **Mindflow** framework with **Serper** for search, **Jina** for webpage content extraction, and **E2B Sandbox** as the execution environment; **HuggingFace** is explicitly disabled during search evaluation. The report further claims that Nanbeige4.1-3B can **reliably execute up to 600 tool-call turns** for complex problem solving. That claim is important, but it is only partially substantiated: the paper does not provide a benchmark table by turn count, a success-versus-depth curve, or trace-level statistics for very long trajectories [2602.13367].

## 6. Empirical profile, comparative standing, and caveats

The evaluation suite spans code, math, science, alignment, tool use, and deep-search benchmarks. Relative to its predecessor and to several Qwen baselines, Nanbeige4.1-3B shows broad gains, with especially large improvements in coding and agentic search [2602.13367].

| Benchmark | Nanbeige4-3B-2511 | Nanbeige4.1-3B |
|---|---:|---:|
| LCB-V6 | 46.0 | **76.9** |
| AIME 2026 I | 84.10 | **87.40** |
| GPQA | 82.2 | **83.8** |
| Arena-Hard-V2 | 60.0 | **73.2** |
| BFCL-V4 | 53.80 | **56.50** |
| GAIA (text-only) | 19.42 | **69.90** |
| xBench-DeepSearch-10 | 11.00 | **39.00** |

The coding results are among the clearest. Nanbeige4.1-3B scores **76.9** on **LCB-V6**, **81.4** on **LCB-Pro-Easy**, and **28.1** on **LCB-Pro-Medium**, beating all listed Qwen baselines in that table, including much larger models. On a real-world stress test covering **LeetCode Weekly Contests 484–488**, it achieves an **85.0%** pass rate (**17/20**), compared with **65.0%** for **Qwen3-30B-A3B-2507**, **55.0%** for **Qwen3-4B-2507**, and **50.0%** for **Qwen3-32B** [2602.13367].

Its reasoning profile is strong but not uniformly dominant. In mathematics, it reaches **87.40** on **AIME 2026 I** and **77.92** on **HMMT Nov**, beating **Qwen3-4B-2507**, **Qwen3-32B**, and **Qwen3-30B-A3B-2507** on those two benchmarks, while on **IMO-Answer-Bench** it scores **53.38**, slightly below **Qwen3-30B-A3B-2507** at **54.34** and below **Qwen3-Next-80B-A3B** at **58.00**. In science, it posts **83.8** on **GPQA** and **12.60** on **HLE (Text-only)**, improving over small-model baselines and the predecessor but still trailing the 80B model on HLE [2602.13367].

Alignment and tool use are also materially improved. On **Arena-Hard-V2**, Nanbeige4.1-3B reaches **73.2**, above **Qwen3-30B-A3B-2507** at **60.2**, **Qwen3-32B** at **56.0**, and **Nanbeige4-3B-2511** at **60.0**. On **Multi-Challenge**, it scores **52.21**, above **Qwen3-30B-A3B-2507** at **49.40** but below **Qwen3-Next-80B-A3B** at **56.52**. For tool use, it attains **56.50** on **BFCL-V4**, the best score in the reported table, and **48.57** on **Tau2-Bench**, which is competitive but below **Qwen3-Next-80B-A3B** at **57.40** [2602.13367].

The most striking gains appear in deep-search and agentic evaluation. Against the reported baselines, Nanbeige4.1-3B reaches **69.90** on **GAIA (text-only)**, **19.12** on **BrowseComp**, **31.83** on **BrowseComp-ZH**, **22.29** on **HLE (text-only)**, **41.44** on **SEAL-0**, **75.00** on **xBench-DeepSearch-05**, and **39.00** on **xBench-DeepSearch-10**. The synthetic deep-search data ablation is especially informative: starting from **Nanbeige4-3B-2511**, adding **Synthetic QA** raises **GAIA** from **19.4** to **58.3**, **BrowseComp** from **0.8** to **14.4**, **BrowseComp-ZH** from **3.1** to **30.1**, **HLE (text-only)** from **13.9** to **22.4**, **SEAL-0** from **12.6** to **36.0**, **xBench-DeepSearch-05** from **33.0** to **76.0**, and **xBench-DeepSearch-10** from **11.0** to **30.0**. This strongly suggests that synthetic search-data construction is one of the principal drivers of the model’s agentic performance [2602.13367].

The paper’s caveats are substantial. It is much stronger on capability claims than on safety analysis. There is no dedicated section on harmful content refusal, jailbreak robustness, tool misuse prevention, hallucination under search, benchmark contamination, or privacy and security risks in agent settings. Architectural and compute transparency are limited: the report does not provide optimizer settings, batch sizes, hardware, total compute, or training duration. The 600-turn tool-call claim remains only partially evidenced. Finally, several search results depend on a specific tool stack—**Mindflow**, **Serper**, **Jina**, and **E2B**—and some evaluations, notably **GAIA** and **HLE**, are reported only on **text-only subsets** [2602.13367].

Taken together, Nanbeige4.1-3B is best understood as a 3B-scale demonstration that broad competence and specialization need not be separated into different small models. Its distinctive contribution is not a newly disclosed backbone, but a stacked post-training recipe that combines long-context SFT, sequential point-wise and pair-wise reward modeling, correctness-first and complexity-aware code RL, and synthetic deep-search trajectories with turn-level supervision. Within the evidence reported, that recipe yields an unusually broad empirical profile for a compact open model [2602.13367].

Source: https://www.emergentmind.com/topics/nanbeige4-1-3b