OLMo-2-1124-7B-Instruct Overview

• The paper introduces a 7B-parameter decoder-only transformer enhanced with stability innovations like RMSNorm and QK-Norm to reduce training divergence.
• It employs a staged pretraining on Dolmino Mix 1124 and a multi-stage instruction tuning pipeline using supervised finetuning, DPO, and RLVR for optimized task performance.
• The model achieves competitive benchmark scores while offering full open-source transparency with accessible code, data mixes, and intermediate checkpoints.

OLMo-2-1124-7B-Instruct denotes the 7-billion parameter instruction-tuned variant in the OLMo-2 LLM family, developed with an open training, code, and evaluation framework for high transparency and reproducibility. Incorporating advanced pretraining mixtures, stability-oriented architectural modifications, and a staged supervised–reinforcement pipeline, OLMo-2-1124-7B-Instruct aims for Pareto-optimal performance per FLOP among fully open models, targeting broad downstream usability—from structured dialogue agents to compositional reasoning—while providing all intermediate recipes and checkpoints to the research community (OLMo et al., 2024).

1. Model Architecture and Stability-Driven Innovations

OLMo-2-1124-7B-Instruct is a decoder-only transformer with 32 layers, hidden size $d_\text{model} = 4096$, approximately $11008$ hidden units in the feedforward sublayers (rounded from $8/3 \, d_\text{model}$), 32 attention heads, and a context window up to 4096 tokens. It employs rotary positional embeddings (RoPE) with a base frequency parameter of $\theta = 5\times 10^5$, and SwiGLU nonlinearity in the MLP blocks:

$$\mathrm{MLP}(x) = W_2 \cdot (\mathrm{SWIGLU}(W_1 x)),\quad \mathrm{SWIGLU}(u,v) = u \cdot \mathrm{SiLU}(v)$$

Key stability modifications introduced in OLMo-2 include:

• RMSNorm, applied after each sublayer rather than the nonparametric LayerNorm used in previous models.
• QK-Norm: RMSNorm is applied to both queries and keys before the scaled dot-product in attention.
• Z-loss regularization: a quadratic penalty $\lambda \log^2 Z$ (with $\lambda=10^{-5}$ and $Z$ being the softmax normalizer) added to the cross-entropy, shown to improve training stability.
• Removal of bias terms, and disabling weight decay on embedding layers.

These changes, validated empirically and via activation/gradient stability metrics, substantially reduce gradient spikes and training divergence over previous OLMo releases (OLMo et al., 2024).

2. Pretraining and Curriculum: Dolmino Mix 1124

The OLMo-2 training recipe is staged. The base model is pretrained on the OLMo 2 1124 Mix (approximately $3.9$T tokens), composed primarily of high-quality web data (95.2%), permissive code, academic/STEM literature, and encyclopedic corpora.

This is followed by a curriculum or mid-training stage (“Dolmino Mix 1124,” 0.83T tokens), a mixture targeting domain richness and coverage:

• ~50% filtered web (FineWeb, FastText)
• ~12–20% mid-train synthetic/math (Dolmino Math)
• ~11–17% FLAN-style instructions (decontaminated)
• ~2–9% academic and Stack Exchange
• ~4–7% encyclopedic content

The curriculum annealing schedule linearly decays the learning rate from the maximum to zero over the mid-training tokens, and each data blend is shuffled in multiple randomized “soups” with final weights averaged to improve checkpoint selection (OLMo et al., 2024).

3. Multi-Stage Instruction Tuning and RLVR

Instruction tuning follows a staged protocol:

1. Supervised Finetuning (SFT): Using approximately $0.94$M permissively licensed and synthetic instructional samples, following the Tulu 3 SFT recipe.
2. Direct Preference Optimization (DPO): Models are updated with both on-policy (self-generated) and off-policy (responses from 20 open competitors) preference data. GPT-4o acts as the judge to produce binarized preferences. The DPO objective is:

$11008$0

1. Reinforcement Learning with Verifiable Rewards (RLVR): Rewards are assigned based on exact correctness on benchmarks such as GSM8K and MATH (i.e., $11008$1 iff correct, 0 otherwise), using a PPO-style objective with a KL penalty against a reference policy.

This staged approach results in OLMo-2-1124-7B-Instruct and 13B-Instruct models competitive with contemporaneous open models in both instruction following and safety (OLMo et al., 2024).

4. Instruction-Following Quality and Early Stopping: IFS and ObjecQA

Instruction-following capability is quantified using the Instruction Following Score (IFS):

$11008$2

where $11008$3 denotes if the model’s response is classified (by a pre-trained “tone” classifier) as answer-like (1) or continuation-like (0) for held-out instruction prompts. IFS is further split into $11008$4 (fragmented instructions) and $11008$5 (well-formed instructions).

OLMo-2-1124-7B-Instruct’s IFS rapidly approaches 0.9 after ~8,000 examples during SFT, then plateaus in the 0.9–0.95 range. The ObjecQA metric—fraction of “objective” answers on a suite of preference queries—remains stable up to the IFS plateau, but quickly changes if SFT continues, indicating semantic drift. The resulting protocol is to halt finetuning as soon as IFS ceases to improve by more than 0.01 over two successive checkpoints, minimizing unwarranted semantic changes (AlShikh et al., 2023).

5. Empirical Evaluation and Benchmarks

OLMo-2-1124-7B-Instruct achieves robust performance across core benchmarks:

• OLMo-2-7B Avg. (9 dev tasks): 61.2% (base)
• OLMo-2-7B-Instruct Avg.: 55.2%; GSM8K: 85.1; BBH: 51.4; MMLU: 61.3; Safety: 80.6

These figures place OLMo-2-1124-7B-Instruct on par with Qwen 2.5 7B Instruct (Avg: 61.6; GSM8K: 83.8) and ahead of Llama 3.1 8B Instruct (Avg: 59.1; GSM8K: 83.4) when normalized for compute (OLMo et al., 2024).

In dialogue state tracking via in-context learning, OLMo-2-1124-7B-Instruct achieves slot-level precision/recall competitive with models like Mistral-7B-Instruct-v0.3 and Llama3.2-3B-Instruct. Using LaBSE-based demonstration retrieval and 3–5 context examples, slot precision reaches ~71.6% and recall ~80%, with minimal architectural modifications. Prompt length limitations (≈2048 tokens) restrict gains at high $11008$6. Zero-shot ability is weak, underscoring the criticality of prompt demonstrations (Hegde et al., 10 Jun 2025).

6. Robustness, Syntactic Bias, and Safety

Recent analysis highlights that OLMo-2-1124-7B-Instruct, like its peers, can learn spurious syntactic-domain correlations during instruction tuning, wherein prompt structure (as defined by PoS-tag templates) becomes spuriously predictive of content domain. Controlled studies demonstrate:

• In-domain template performance: Exact (0.92), Synonym (0.91), Antonym (0.90)
• Cross-domain drops: Exact (0.51), Synonym (0.43), Antonym (0.49)
• Mean drop (Δ): ~0.43 ± 0.04 across semantic-preserving/-breaking templates

These correlations can be adversarially exploited to induce refusal bypasses: prefixing or suffixing “cross-domain” syntactic templates with harmful prompts drives refusals down from ~40% to as low as 2.5%. Addressing this, the literature recommends explicit in- versus cross-domain template benchmarking and training-time decorrelation (e.g., via oversampling/undersampling, regularization on $11008$7, or minimizing mutual information between syntax and domain) (Shaib et al., 25 Sep 2025).

7. Reproducibility and Resource Availability

All OLMo-2-1124-7B-Instruct artifacts—including code, data mixes (OLMo-mix-1124, Dolmino-mix-1124), tuning recipes, logs, and thousands of intermediate checkpoints—are fully open to the research public under permissive licenses. GitHub repositories cover the entire workflow from pretraining to posttraining, and models are hosted on HuggingFace with evaluation harnesses (OLMES, dolma). Hyperparameters, curation code, and mid-train evaluation splits are disclosed to support transparent and principled benchmarking (OLMo et al., 2024).

8. Usage Protocols and Best Practices

Deployment and tuning of OLMo-2-1124-7B-Instruct should:

• Employ minimal SFT (≈8,000–10,000 curated instruction–response pairs) for domain adaptation, monitoring IFS for early stopping.
• Validate both IFS_partial and IFS_full (≥0.90) and constrain semantic drift via ObjecQA (+5 points threshold).
• In in-context learning scenarios (e.g., DST), employ LaBSE-based retrieval on user messages, maintain concise JSON-structured prompts, and use 3–5 demonstrations maximizing slot coverage.
• Audit safety via template perturbation protocols to quantify and mitigate syntactic-domain reliance before deployment (AlShikh et al., 2023, Hegde et al., 10 Jun 2025, Shaib et al., 25 Sep 2025, OLMo et al., 2024).