Qwen3-4B-Instruct Transformer

• Qwen3-4B-Instruct is a dense autoregressive transformer with 4 billion parameters that integrates dynamic mode switching and chain-of-thought reasoning for enhanced instruction following.
• It employs a three-stage curriculum pretraining and UniAPL tuning, combining advanced methods like behavior calibration and hallucination mitigation to boost performance.
• The model enables configurable thinking budgets and unified dynamic modes, making it a robust open-source option for tasks in code, math, reasoning, and multilingual applications.

Qwen3-4B-Instruct is a 4-billion-parameter dense autoregressive transformer from the Qwen3 LLM family, optimized for instruction-following both with and without explicit chain-of-thought reasoning. This model integrates recent advancements in architecture, training methods, alignment algorithms, and behavior calibration, making it a state-of-the-art open-source option for code, math, reasoning, and agentic tasks. It introduces dynamic mode switching, a configurable "thinking budget," and multi-stage curriculum and distillation workflows, alongside robust alignment and hallucination mitigation techniques. Below, key technical and methodological aspects are detailed for research-oriented analysis.

1. Model Architecture and Technical Specifications

Qwen3-4B-Instruct adopts a dense decoder-only transformer architecture with 36 pre-norm layers, each featuring Grouped Query Attention (GQA) (32 query heads / 8 key-value heads per layer), SwiGLU feed-forward sublayers, RoPE positional embeddings, and RMSNorm normalization. Both input and output embeddings are tied ("tie embedding = Yes"). The default context window is 128,000 tokens, enabled via YARN and Dual Chunk Attention (DCA), with ABF-adjusted RoPE for long-context support. Unlike MoE variants in the Qwen3 family, such as Qwen3-30B-A3B and Qwen3-235B-A22B, Qwen3-4B-Instruct is purely dense: no expert gating or load-balancing loss is present, so there is no router $z_n = \mathrm{softmax}(W_g h_n)$ active in forward passes (Yang et al., 14 May 2025).

Pretraining follows a three-stage curriculum:

• Stage 1 ("General"): ~30 trillion tokens, sequence length 4,096, trained by autoregressive cross-entropy.
• Stage 2 ("Reasoning"): ~5 trillion tokens, focused on STEM, code, and synthetic CoT data, with accelerated learning-rate decay.
• Stage 3 ("Long Context"): hundreds of billions of tokens, sequences up to 32,768; incorporates context extension methods.

Following base pretraining, instruction fine-tuning endows Qwen3-4B-Instruct with both "thinking" (multi-step reasoning) and "non-thinking" (rapid completion) capabilities. The training process leverages Strong-to-Weak Distillation: off-policy distillation on teacher logits, followed by on-policy distillation aligning student outputs with high-capability teacher models (Qwen3-32B / Qwen3-235B-A22B). Compared to full RL-HF workflows, this two-step distillation matches or exceeds performance at approximately 10% of GPU-hours (Yang et al., 14 May 2025).

2. Dynamic Thinking Modes and Thinking Budget

A core innovation is the unified dynamic mode switching: a single checkpoint supports both rapid-response and complex-reasoning modes. Mode is selected via prompt flags ("/think" or "/no_think"), with user-provided "thinking budget" $B$ (in tokens) setting an upper bound on chain-of-thought reasoning within the > … block. The system forcibly summarizes after $B$ tokens, allowing task-dependent allocation of computation and latency. At inference, <think> remains empty for "/no_think"; otherwise, the model privately reasons until the token budget is depleted.

Recommended sampling hyperparameters are mode-dependent: "thinking"—temperature 0.6, top-p 0.95, top-k 20; "non-thinking"—temperature 0.7–0.8, top-p unchanged, top-k 20. $B$ may be dynamically adjusted by prompt length or fixed per deployment.

Typical prompt structure for production use:

<|im_start|>user: {User question} [/think|/no_think]<|im_end|> <|im_start|>assistant: <think>…</think>{Final answer}<|im_end|>

You may suppress the <think> block by passing "enable_thinking=False" (Yang et al., 14 May 2025).

3. Instruction Tuning and UniAPL Alignment

Instruction tuning is conducted both by standard SFT and the Unified Adversarial Preference Learning (UniAPL) framework (Qian et al., 29 Sep 2025). UniAPL views post-training alignment as a constrained optimization:

$$\pi_{\mathrm{aligned}} = \arg\max_{\pi_\theta} \mathbb{E}_{x, y}[R_\psi(y | x)] \quad \text{subject to} \quad D(\pi_\theta \| \pi^*) \leq \epsilon$$

where $R_\psi$ is the preference-based reward model, $\pi^*$ is the teacher policy, and $D$ is KL divergence.

The unified single-stage objective mixes adversarially-regularized SFT (A-SFT) and preference-based RL (A-GRPO) losses via a discriminator $D_\phi$ (POLAR backbone):

$$\mathcal{L}_{\mathrm{Unified}} = \alpha \, \mathcal{L}_{\mathrm{A{-}SFT}} + (1{-}\alpha) \, \mathcal{L}_{\mathrm{A{-}GRPO}}$$

Each training step samples a mini-batch combining SFT and preference data, updates the policy via gradient signals from both, and includes adversarial regularization. No architectural changes are made to the student besides the added loss term; the POLAR discriminator is pre-trained and frozen for stability.

UniAPL demonstrably resolves the standard SFT$\rightarrow$RL distributional mismatch, leading to higher instruction-following performance, more grounded outputs, and a response length/log-probability profile closer to expert teachers. On the IFBench-38K benchmark, UniAPL yields an average I-Following score of 90.65 (+3.75 vs. GRPO baseline), confirmed by paired bootstrap significance (Qian et al., 29 Sep 2025).

4. Hallucination Mitigation and Behavioral Calibration

Qwen3-4B-Instruct addresses hallucination via behaviorally calibrated reinforcement learning (Wu et al., 22 Dec 2025). Standard binary-reward RL (e.g., PPO with $R_{\mathrm{binary}}$) incentivizes models to "guess" rather than refuse, leading to uncalibrated, hallucinated responses. By contrast, strictly proper scoring-rule rewards—Brier score ($R_{\mathrm{brier}}(p, y) = y - (p-y)^2$) and log-score ($R_{\mathrm{CE}}$ as truncated cross-entropy)—align model's predicted confidence $p$ with real accuracy. The model outputs a calibrated probability of correctness and may abstain when uncertain (threshold $t$); at test, answers are given only if $p \geq t$.

RL algorithms for behavior calibration include:

• Verbalized confidence (model emits both answer and scalar $p$)
• Critic value as confidence (use $V_\phi$ directly as $p$)
• Individual-claim calibration (for CoT outputs, each claim $y_i$ paired with $p_i$ and a rationale; aggregate via product or min)

Empirically, Qwen3-4B-Instruct trained under this regime achieves superior Accuracy-to-Hallucination Ratio (SNR-Gain), confidence AUC, and calibration error (smECE) compared to frontier models. For example, on the BeyondAIME math benchmark, log-SNR-Gain for Conf-Brier and Conf-Prod variants is $\approx$0.80, exceeding GPT-5's 0.21; PPO-Value baseline exceeds 1.2 (Wu et al., 22 Dec 2025). On SimpleQA cross-domain factual QA, the math-trained model matches frontier models in calibration metrics, demonstrating transferability of calibration as a meta-skill independent of model scale.

5. Multilingual Support and Benchmark Performance

Qwen3-4B-Instruct extends full context-window and instruction-following functionality to 119 languages and dialects, enhancing performance for multilingual users. Benchmark scores (thinking mode) include: 83.7 on MMLU-Redux (4-shot CoT), 87.8 on GSM8K, 54.1 on MATH, and 72.05 on EvalPlus (average HumanEval, MBPP). Non-thinking mode results are competitive across all tasks.

Relative to prior models, Qwen3-4B-Instruct consistently outperforms Qwen2.5-3B and rivals models of larger scale (7B–12B) such as Qwen2.5-7B and Gemma-3-12B (Yang et al., 14 May 2025). These results corroborate the effect of knowledge distillation from flagship models and the alignment benefits of UniAPL.

Benchmark	4B-Instruct (think)	4B-Instruct (no-think)	Qwen2.5-3B	Qwen2.5-7B/Gemma-12B
MMLU-Redux	83.7	77.3	lower	comparable
GSM8K	87.8	–	lower	–
MATH	54.1	–	lower	–
EvalPlus (avg)	72.05	63.53	lower	comparable
HumanEval	–	67.00	lower	–

6. Qualitative Output and Practical Deployment

UniAPL tuning yields outputs with length, stylistic, and log-probability characteristics closely matching expert demonstrations; qualitative inspection confirms better usage of keywords and more detailed, instructive explanations compared to preference-only RL. Calibrated RL interventions enable the model to abstain and label individual claims for uncertainty, improving reliability in critical domains. For math CoT solutions, claims may be flagged with low confidence scores and rationale, supporting introspection and error signaling.

Limitations include difficulty in localizing uncertainty at the claim level via PPO-Value methods, conservative thresholds yielding high abstention rates, and nontrivial aggregation of claim-level confidences. Nevertheless, the model demonstrates high transferability of calibration skills across domains, even when trained solely on math reasoning (Wu et al., 22 Dec 2025).

7. Availability and Reproducibility

Qwen3-4B-Instruct and related models in the Qwen3 series are released under the Apache 2.0 license, with public checkpoints and reproducibility tools. This supports robust community-driven research in scaling, alignment, long-context reasoning, and factuality calibration (Yang et al., 14 May 2025).