GPT-OSS-120B: Open-Weight Reasoning Model

• GPT-OSS-120B is an open-weight MoE reasoning model defined by its Mixture-of-Experts architecture, configurable chain-of-thought, and role-based Harmony chat interface.
• It employs a scalable, autoregressive transformer design and advanced retrieval techniques for high-capability reasoning, coding, and multilingual evaluations.
• The model serves both as a released artifact for open research and as a platform for developing agentic, self-improving systems in sensitive deployment contexts.

Searching arXiv for papers on GPT-OSS-120B and related evaluations. Search query: "GPT-OSS-120B arXiv" GPT-OSS-120B is an open-weight reasoning model in OpenAI’s GPT-OSS family, released alongside GPT-OSS-20B as a text-only model under the Apache 2.0 license and the gpt-oss usage policy. It is presented as a locally deployable, agentic, tool-capable system that uses a rendered Harmony chat format with explicit role delineation and configurable reasoning effort, and it is accompanied by released weights, inference implementations, tool environments, and tokenizers (OpenAI et al., 8 Aug 2025). Within subsequent literature, GPT-OSS-120B has been treated both as a high-capability open model and as a research object for work on reasoning traces, prompt saturation, safety auditing, deployment optimization, multilingual evaluation, and self-improving agents (OpenAI et al., 8 Aug 2025).

1. Release context and model identity

GPT-OSS-120B is the larger member of the two-model GPT-OSS family. The model card describes both GPT-OSS models as open-weight reasoning models optimized for strong agentic capabilities, including deep research browsing, Python tool use, and support for developer-provided functions, while using the Harmony rendered chat format for instruction following and role delineation (OpenAI et al., 8 Aug 2025).

The model is explicitly positioned for applications in which openness, customizability, and local deployment matter. In downstream studies, this local-deployment property is treated as operationally important in settings such as digital forensics and clinical decision support, where privacy, auditability, or infrastructure constraints disfavor cloud-only deployment. A plausible implication is that GPT-OSS-120B occupies a hybrid role: it is simultaneously a released model artifact, a benchmarked reasoning model, and a platform for downstream system-building (OpenAI et al., 8 Aug 2025).

The release materials distinguish between total and active parameters, reflecting the model’s Mixture-of-Experts design. The model is referred to as “120B” for simplicity, although the exact parameter count is 116.8B. Its published knowledge cutoff is June 2024, and the release emphasizes compatibility with OpenAI’s Responses API, structured outputs support, and full chain-of-thought availability, while also warning that unrestricted CoT should not be shown directly to end users without filtering, moderation, or summarization (OpenAI et al., 8 Aug 2025).

2. Architecture, tokenizer, and interface conventions

GPT-OSS-120B is an autoregressive Mixture-of-Experts transformer built on GPT-2/GPT-3-style foundations. The architecture and deployment interface reported in the model card are summarized below (OpenAI et al., 8 Aug 2025).

Attribute	Value	Note
Layers	36	Autoregressive transformer
Total parameters	116.83B	Referred to as “120B”
Active parameters	5.13B	Per token per forward pass
Checkpoint size	60.8GiB	Post-training release artifact
Experts per MoE block	128	Top-4 selected per token
Context length	131,072 tokens	Dense layers extended via YaRN
Tokenizer size	201,088 tokens	o200k_harmony

The residual stream dimension is 2880. RMSNorm is applied before each attention and MoE block in a Pre-LN placement. Each MoE block contains 128 experts, and for each token the router selects the top-4 experts, weighting their outputs by the softmax over only those selected experts. The MoE blocks use gated SwiGLU, and the model card notes that the SwiGLU implementation is unconventional, including clamping and a residual connection (OpenAI et al., 8 Aug 2025).

The attention stack follows a GPT-3-like alternating sparse/dense pattern in which banded window and fully dense attention layers alternate, with a bandwidth of 128 tokens. Each layer has 64 query heads of dimension 64 and uses Grouped Query Attention with 8 key-value heads. Rotary position embeddings are used, and dense layers are extended to 131,072 tokens via YaRN. The paper also reports a learned bias in each attention head’s softmax denominator, “similar to off-by-one attention and attention sinks,” allowing the mechanism to pay no attention to any tokens (OpenAI et al., 8 Aug 2025).

The tokenizer is the open-sourced o200k_harmony tokenizer, a BPE tokenizer extended with special tokens for the Harmony chat format. Harmony is not a superficial wrapper but part of the model’s intended interface: messages are structured with roles such as System, Developer, User, Assistant, and Tool, with a role hierarchy of System > Developer > User > Assistant > Tool, and channels such as analysis, commentary, and final. This interface is central to GPT-OSS-120B’s tool use, role separation, and reasoning-trace behavior (OpenAI et al., 8 Aug 2025).

3. Training regime, reasoning modes, and trajectory structure

Training began with large-scale pretraining on trillions of text tokens focused on STEM, coding, and general knowledge. Harmful content was filtered during pretraining, especially CBRN-related material, using GPT-4o’s CBRN pre-training filters. Post-training then used “similar CoT RL techniques as OpenAI o3,” with the explicit objective of teaching the model to reason, solve problems using chain-of-thought, and use tools. The model card also states that OpenAI did not apply direct optimization pressure to the chain-of-thought itself, leaving CoT unrestricted to preserve monitorability (OpenAI et al., 8 Aug 2025).

GPT-OSS-120B supports three reasoning levels—low, medium, and high—set through the system prompt. Higher reasoning effort increases average chain-of-thought length, and the model card reports smooth test-time scaling in which accuracy rises roughly log-linearly as CoT length increases, at the cost of latency and compute. This makes “reasoning effort” a deployment-time control variable rather than a fixed architectural constant (OpenAI et al., 8 Aug 2025).

A separate line of work treats GPT-OSS-120B not merely as a student or evaluator, but as a source of verified reasoning trajectories. In a controlled SFT comparison using identical problem sets, verified GPT-OSS-120B CoT traces generalized better than verified DeepSeek-R1-0528 traces, even though the latter produced lower SFT loss. The reported explanation is structural: GPT-OSS-120B trajectories were “highly convergent and deductive,” whereas DeepSeek-R1-0528 trajectories were more “divergent and branch-heavy.” Students trained on GPT-OSS-120B data achieved higher average accuracy across five reasoning benchmarks, including gains of +5.1% for Qwen2.5-7B, +4.8% for Qwen2.5-32B, +21.0% for Llama3.1-8B, and +5.3% for Qwen3-8B relative to students trained on the alternative teacher traces (Li et al., 2 Apr 2026).

This teacher-trace result is significant because it separates answer correctness from reasoning-pattern transfer. The paper’s interpretation is not that branching is always harmful, but that frequent shallow branching induces redundant exploratory behavior in students, whereas GPT-OSS-120B-style trajectories more often teach “reason forward” behavior. That distinction places GPT-OSS-120B within research on long-CoT data curation as well as within model evaluation proper (Li et al., 2 Apr 2026).

4. Capability profile and evaluation variance

The model card reports strong benchmark performance across mathematics, coding, tool use, multilingual evaluation, and health-related tasks, with results stratified by reasoning effort and tool availability. On AIME 2025, GPT-OSS-120B is reported at 50.4 / 80.0 / 92.5 without tools and 72.9 / 91.6 / 97.9 with tools across low / medium / high reasoning. On SWE-Bench Verified it is reported at 47.9 / 52.6 / 62.4, on MMLU at 85.9 / 88.0 / 90.0, on MMMLU average at 74.1 / 79.3 / 81.3, and on Codeforces Elo at 1595 / 2205 / 2463 without tools and 1653 / 2365 / 2622 with tools. The model card characterizes GPT-OSS-120B as particularly strong on coding and tool-use tasks, while noting that knowledge-heavy tasks such as GPQA remain limited by model size (OpenAI et al., 8 Aug 2025).

At the same time, independent evaluations report a more mixed standing. In a standardized ten-benchmark comparison against seven other open-source models, GPT-OSS-120B scored 66 on MMLU, 75 on GSM8K, 71 on HumanEval, 65 on FinQA, 71 on PIQA, 72 on SciQ, 59 on MedQA, 62 on LegalQA, 65 on DialogSum, and 42 on C-Eval, for an average of 64.8. In that study, GPT-OSS-20B scored 67.7 on average and exceeded GPT-OSS-120B on every listed benchmark, leading the authors to argue that larger sparse capacity did not translate into better performance under their evaluation conditions (Bi et al., 17 Aug 2025).

Research using specialized inference pipelines often reports substantially higher task-specific results. On IOI 2025, the GenCluster framework combined large-scale generation, behavioral clustering, tournament-based ranking, and round-robin submission to push GPT-OSS-120B to a final submitted score of 446.75 at $K=5000$, which the paper states is enough for gold at IOI 2025. In the same study, GPT-OSS-120B was the best open-weight model under increasing test-time compute budgets and was reported to scale better than the compared open alternatives (Samadi et al., 16 Oct 2025).

Other task-specific studies emphasize both strength and boundedness. In the SAIR Equational Theories Stage 1 competition, GPT-OSS-120B was the primary competition model and the main subject of the study, but prompt engineering exhibited a “single-prompt ceiling.” The no-cheatsheet baseline on hard3 was 59.75% accuracy with 82.6% True recall and 38.0% False recall, while the best prompt, AN45c, reached 79.25% accuracy with 95.9% True recall and 63.4% False recall. The authors report an empirical saturation region in the 71–79% balanced-hard-accuracy range and attribute it to undecidability of the True case, finite-search asymmetry for False, and fragile ordering effects (Cazares, 20 Apr 2026).

Multilingual and domain-specific evaluations similarly show a mixed but high-variance profile. On GSM-Plus-BN, GPT-OSS-120B achieved 88.03% seed accuracy under Standard Prompting and 92.70% under CoT Prompting, with especially strong CoT results on Problem Understanding at 91.82% and weaker performance on Critical Thinking at 31.80%. In Bangla biomedical QA, the best reported configuration for GPT-OSS-120B was Agentic RAG at 89.54% accuracy on BanglaMedQA. In an on-device clinical benchmarking study, GPT-OSS-120B achieved 84.1% overall average accuracy in generalist radiology at low reasoning, 77.7% overall on ophthalmology at medium reasoning, and a diagnostic-judge median error of 0.00 $(-0.50\text{–}0.25)$, indicating strong alignment with human expert diagnosis scoring in that specific evaluation setup (Paul et al., 14 Jul 2026, Sultana et al., 6 Nov 2025, Munim et al., 18 Dec 2025).

Taken together, these results suggest not a single stable ranking but a model whose observed standing is highly sensitive to harness design, test-time compute, prompting regime, retrieval strategy, and task structure. A plausible implication is that GPT-OSS-120B is better understood as a platform with strong upside under matched protocols than as a uniformly dominant checkpoint.

5. Safety, jailbreaks, and internal auditing

Safety evaluation in the model card is framed through both alignment metrics and OpenAI’s Preparedness Framework. The release states that GPT-OSS-120B does not reach the indicative High threshold in Biological and Chemical capability, Cyber capability, or AI Self-Improvement. On StrongReject, it is reported at 0.979 on illicit/non-violent crime prompts, 0.983 on violence prompts, 0.993 on abuse/disinformation/hate prompts, and 0.989 on sexual-content prompts. The same model card, however, explicitly notes weaker instruction-hierarchy performance than o4-mini, including 0.832 on system prompt extraction and 0.780 on prompt injection hijacking in system-vs-user conflict tests (OpenAI et al., 8 Aug 2025).

Subsequent auditing papers show that “robustness” is method-dependent. In an interpretability-based jailbreak audit using Universal Steering and Representation Engineering, GPT-OSS-120B was the only reported model with 0% jailbreak success under both methods, with no valid steering coefficient found. The paper states that the model mostly refused at coefficients $\le 75$ and produced gibberish at coefficients $\ge 100$, never entering a stable compliance region under the search procedure (Agarwal et al., 22 Apr 2026). By contrast, ContextualJailbreak, a black-box evolutionary method over simulated multi-turn priming dialogues, achieved 90% ASR@4 and 72% ASR@5 on GPT-OSS-120B, even though the same paper reports 0% ASR@4 on DirectRequest. The result positions GPT-OSS-120B as harder than several other open-source targets, but still substantially vulnerable to contextual priming (Béjar et al., 4 May 2026).

Internal auditing has also been pursued through weight-only analysis. An lm_head SVD study argues that GPT-OSS-120B has a gentle, monotone singular-value decay and a “graduated hierarchy” of functionally differentiated vocabulary subspaces. Reported GPT-OSS-120B statistics include mean VCS $= 0.14$, max VCS $= 0.30$, $S[0] = 21.76$, $S[1] = 11.44$, and $S[0]/S[1] = 1.90$. The same paper defines the Weighted Projection Score as a static glitch-token detector and reports $\mu = 9.95$, $(-0.50\text{–}0.25)$0, a threshold of 4.07, and 9,819 tokens (4.88%) below threshold. Using WPS alone, it recovers the known glitch token shokubutsu-hyakka-tsu (token ID 137606, WPS $(-0.50\text{–}0.25)$1) without any model inference, and interprets this as evidence that problematic vocabulary content can already be latent in the weight matrix (Miyashita, 21 May 2026).

A common misconception is that a favorable safety result in one audit implies general immunity. The literature on GPT-OSS-120B does not support that view. Instead, it presents a layered picture: strong refusal on some benchmarks, failure to yield compliant outputs under certain internal steering attacks, measurable weakness under contextual multi-turn priming, and auditable lexical pathologies detectable directly from weights.

6. Derivatives, agentic optimization, and specialized deployments

GPT-OSS-120B has served as the parent model for deployment-oriented derivatives. The most explicit example is gpt-oss-puzzle-88B, produced by extending the Puzzle post-training decomposed NAS framework to MoE reasoning models. The resulting derivative uses heterogeneous MoE expert pruning, selective replacement of full-context attention with window attention, FP8 KV-cache quantization with calibrated scales, and post-training RL. On an 8×H100 node, the paper reports 1.22× throughput in 4K/4K and 1.63× throughput in 64K/64K; on a single H100 GPU it reports 2.44× and 2.82× throughput, respectively. Across the accuracy–speed frontier, gpt-oss-puzzle-88B is reported to improve over the parent along the entire frontier, reaching up to 1.29× higher request-level efficiency while retaining 100.8% of parent accuracy at high effort, 103.9% at medium effort, and 108.2% at low effort (Bercovich et al., 12 Feb 2026).

The model has also been used as a base executor in self-improving agent research. GRASP, a bounded-skill-library method with a balanced held-out probe and a hard regression budget, improved GPT-OSS-120B on MedAgentBench from 40.6% to 88.8% test accuracy and on MedAgentBench-v2 from 61.1% to 76.9%. The paper’s ablations are especially strong: removing the acceptance gate with $(-0.50\text{–}0.25)$2 reduced performance to 40.1%, essentially matching the no-skills baseline, while removing the regression budget reduced performance from 88.8% to 81.8%. This suggests that for GPT-OSS-120B, validated skill editing—not ungated accumulation of natural-language guidance—is the main source of improvement (Moll et al., 28 May 2026).

Specialized application studies further show that GPT-OSS-120B is often most effective when embedded in system-level scaffolds rather than used as a bare prompt-response model. In Bangla biomedical multiple-choice QA, Agentic RAG (Local + Web + Zero-Shot Fallback) produced the best reported GPT-OSS-120B result, 89.54% accuracy on BanglaMedQA, outperforming the same model under Traditional RAG, Zero-Shot Fallback, Iterative Feedback RAG, and Aggregate RAG. In the clinical decision-support appendix, GPT-OSS-120B is consistently evaluated through structured prompting and majority-vote self-consistency rather than a single sample. These studies suggest that the model’s observed utility depends heavily on retrieval routing, voting, or external control structure, not solely on its base checkpoint (Sultana et al., 6 Nov 2025, Munim et al., 18 Dec 2025).

As a research object, GPT-OSS-120B therefore spans several layers of analysis: a released open-weight reasoning model; a specific MoE architecture with Harmony-native interaction conventions; a source of comparatively convergent long-CoT supervision; a benchmark subject with high protocol sensitivity; a model whose safety profile changes substantially across attack families; and a parent checkpoint for acceleration, agentic self-improvement, multilingual evaluation, and domain-specific deployment.