---
title: 'SmolTulu-1.7b-Instruct: Optimized Small LLM'
url: https://www.emergentmind.com/topics/smoltulu-1-7b-instruct
type: topic
---

# SmolTulu-1.7b-Instruct: Optimized Small LLM

SmolTulu-1.7b-Instruct, also referenced as SmolTulu-DPO-1130, is an instruction-tuned language model that adapts AllenAI’s Tulu 3 post-training pipeline to Huggingface’s SmolLM2-1.7B base model. Its design leverages empirical findings on the critical role of the learning-rate-to-batch-size ratio in optimizing reasoning and pattern recognition performance in small language models (SLMs). SmolTulu-1.7b-Instruct achieves state-of-the-art results among sub-2B parameter models on instruction following and mathematical reasoning tasks by meticulous adaptation of post-training procedures and optimization dynamics, without modifying the underlying model architecture [2412.08347].

## 1. Model Foundation and Architectural Characteristics

SmolTulu-1.7b-Instruct is built upon the SmolLM2-1.7B decoder-only Transformer architecture described by Allal et al. (2024). The key configuration parameters are:

- Number of Transformer layers: 32
- Model (hidden) dimensionality: 2,048
- Feed-forward inner dimensionality: 8,192
- Number of self-attention heads: 16
- Total parameter count: approximately 1.7 billion

No architectural modifications are introduced by SmolTulu; all improvements are attributable to post-training and alignment strategies. The model’s enhancements derive exclusively from procedure-level adaptations, encompassing instruction tuning and preference modeling, rather than network structural changes.

## 2. Instruction-Tuning and Alignment Pipeline

SmolTulu-1.7b-Instruct employs a post-training pipeline closely following the Tulu 3 recipe but calibrated for 1.7B parameters, comprising two principal stages:

**A. Supervised Finetuning (SFT):**
- Data: allenai/tulu-3-sft-mixture (includes sources such as alpaca, BBH, GSM8K, IFEval).
- Max sequence length: 2,048 tokens.
- Training: Single epoch over the dataset.
- Learning rate schedule: Linear decay with 10% warm-up.

**B. Direct Preference Optimization (DPO):**
- Data: allenai/llama-3.1-tulu-3-8b-preference-mixture (UltraFeedback + Tulu3 synthetic preferences).
- Loss function: Length-normalized DPO loss with KL-penalty $\beta=5$,
  $$
  \mathcal{L}_{\mathrm{DPO}} = -\mathbb{E}_{(x,y_c,y_r)\sim\mathcal{D}}
  \log
  \sigma\biggl(
    \beta \Bigl[\log \frac{\pi_\theta(y_c|x)}{\pi_{\rm ref}(y_c|x)}\Bigr]
    -\beta \Bigl[\log \frac{\pi_\theta(y_r|x)}{\pi_{\rm ref}(y_r|x)}\Bigr]
  \biggr)
  $$
  where $\pi_{\rm ref}$ refers to the SFT model.
- Max sequence length: 2,048 tokens.
- Training: One epoch, linear learning rate decay with 10% warm-up.

These procedures allow transfer of alignment strategies designed for larger language models to computationally efficient SLM baselines, preserving key alignment properties post-tuning.

## 3. Optimization Dynamics and Task-Dependent Hyperparameter Effects

A central finding is the strong impact of the learning-rate-to-batch-size ratio $R = \eta / B$ (where $\eta$ is peak learning rate and $B$ is batch size) on downstream model performance. Empirical exploration of several $(\eta, B)$ configurations elucidated that optimal $R$ is task dependent:

**SFT Hyperparameters:**

| Variant                     | $\eta$           | $B$  | $R = \eta/B$         |
|-----------------------------|------------------|------|----------------------|
| SmolTulu SFT-1130           | $9.0 \times 10^{-5}$  | 8    | $1.125 \times 10^{-5}$ |
| SmolTulu SFT-1207           | $3.1 \times 10^{-6}$  | 32   | $9.69 \times 10^{-8}$  |
| Tulu 3 (8B)                 | $5.0 \times 10^{-6}$  | 128  | $3.91 \times 10^{-8}$  |
| Tulu 3 (70B)                | $2.0 \times 10^{-6}$  | 128  | $1.56 \times 10^{-8}$  |

**DPO Hyperparameters:**

| Variant                   | $\eta$           | $B$  | $R = \eta/B$         |
|---------------------------|------------------|------|----------------------|
| SmolTulu DPO-1130         | $8.0 \times 10^{-7}$   | 12   | $6.67 \times 10^{-8}$ |
| SmolTulu DPO-1207         | $5.0 \times 10^{-7}$   | 32   | $1.56 \times 10^{-8}$ |
| Tulu 3 DPO 8B             | $5.0 \times 10^{-7}$   | 128  | $3.91 \times 10^{-9}$ |
| Tulu 3 DPO 70B            | $2.0 \times 10^{-7}$   | 128  | $1.56 \times 10^{-9}$ |

In all cases, reasoning-centric benchmarks (e.g., ARC, GSM8K) benefit from higher $R$, while pattern-recognition benchmarks (e.g., HellaSwag, IFEval) favor lower $R$. The model capacity at 1.7B parameters generates a regime in which a single $R$ cannot optimize all metrics uniformly, necessitating task-specific tuning.

## 4. Empirical Performance Evaluation

Post SFT and DPO, SmolTulu-1.7b-Instruct (DPO-1130) demonstrates substantial improvements on key tasks:

| Task             | SmolLM2-1.7B | SmolTulu DPO-1130 | $\Delta$  |
|------------------|--------------|-------------------|-----------|
| GSM8K (5-shot)   | 48.2%        | 51.6%             | +3.4%     |
| ARC (avg)        | 51.7%        | 51.5%*            | –0.2%*    |
| ARC (alt config) | —            | 57.1%             | +5.4%     |
| HellaSwag        | 66.1%        | 61.1%*            | –5.0%*    |
| IFEval (avg)     | 56.7%        | 67.7%             | +11.0%    |

*DPO-1130 yields strong reasoning gains on GSM8K and IFEval, with alternate (lower $R$) DPO runs restoring ARC accuracy substantially. The improvements on GSM8K and IFEval are substantially above typical benchmark noise (±1–2%). No formal statistical confidence intervals were reported.

## 5. Ablation Analyses and Theoretical Context

Comprehensive ablation studies on a 135M-parameter SmolLM2 model reveal near-monotonic improvement in ARC and GSM8K accuracy as $R$ increases, whereas HellaSwag and IFEval show non-monotonic behavior, peaking at intermediate $R$ values ($\approx 2\times10^{-6}$–$5\times10^{-6}$). At 1.7B parameters, the model reveals capacity-limited divergence: high $R$ benefits reasoning, while low $R$ benefits pattern tasks, limiting universal gains from a single configuration.

Varying training epochs (from 1 to 2) shifts absolute scores by approximately 1–2% without altering $R$–performance curve shapes. These trends substantiate the theoretical perspective articulated by Keskar et al. (2017) and Masters & Luschi (2018), suggesting that high-$R$ (small batch, higher LR) training encourages flatter minima conducive to reasoning, whereas low-$R$ (large batch) training supports pattern matching.

A plausible implication is that in small model regimes, optimization hyperparameters deserve particular scrutiny, specifically with respect to the end-task distribution of interest.

## 6. Training Recipe, Best Practice, and Generalization Guidelines

The canonical SmolTulu-1.7b-Instruct (DPO-1130) was produced with:

- **SFT**: LR $=9\times10^{-5}$, BS $=8$ ($R=1.125\times10^{-5}$), 1 epoch, max seq. length $=2048$, 10% warm-up.
- **DPO**: LR $=8\times10^{-7}$, BS $=12$ ($R=6.67\times10^{-8}$), 1 epoch, max seq. length $=2048$, 10% warm-up, KL penalty $\beta=5$.

Best practice guidelines for sub-2B parameter LMs are documented as:

1. **For SFT**, use $R\approx10^{-5}$ if $B\leq16$.
2. **For DPO**, employ $R$ values 5–10× larger than those for 8B+ models ($R\approx10^{-7}$–$10^{-8}$).
3. **Favor higher $R$** for math and logic reasoning tasks.
4. **Favor lower $R$** for pattern recognition/commonsense tasks.
5. **Only adjust $\eta$ and $B$**—keep all other recipe elements (sequence length, warmup, epochs) consistent with Tulu 3.

Careful calibration of $R$ enables SLMs to bridge much of the capability gap with multi-billion-parameter models on both instruction-following and reasoning benchmarks [2412.08347].

Source: https://www.emergentmind.com/topics/smoltulu-1-7b-instruct