Chinchilla-Style Scaling Laws

• The paper defines Chinchilla-style scaling laws as empirical power-law models that decompose pretraining loss into capacity (model size) and data-scarcity terms using fitted exponents.
• It prescribes compute-optimal allocation by balancing total parameters and training tokens, yielding near-equal exponents (around 0.50) under fixed compute budgets.
• Extensions incorporate optimizer choice, repeated-data regimes, data quality, sparsity, and architectural trade-offs to refine loss predictions and performance scaling.

Chinchilla-style scaling laws are empirical power-law models for pretraining loss that relate model size, data size, and often compute through a small set of fitted coefficients. In their canonical form, they model held-out or validation loss as an irreducible floor plus separate finite-parameter and finite-data terms, and they become prescriptive when combined with the transformer-compute approximation $C \simeq 6ND$. Subsequent work has treated this form not as a single immutable law but as a family of related laws whose appropriate parameterization depends on whether tokens are unique or repeated, whether optimizer or architecture vary, whether deployment or wall-clock time is the binding budget, and whether further axes such as data quality, sparsity, modality, and full hyperparameter configuration must be made explicit (Lovelace et al., 2 May 2026, Pearce et al., 2024, Caballero et al., 25 May 2026).

1. Canonical formulation and compute-optimal allocation

The baseline Chinchilla law writes language-model pretraining loss as

$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$

where $N$ is the number of model parameters, $D$ is the total number of training tokens, $E$ is the irreducible loss floor, and $A,B,\alpha,\beta$ are fitted constants. In this decomposition, $A/N^\alpha$ is the capacity or under-parameterization term and $B/D^\beta$ is the data-scarcity term. This formulation is treated as the standard reference point in later work on optimizer choice, data repetition, architecture, and deployment-aware scaling (Lovelace et al., 2 May 2026).

When total training compute is approximated by $C \simeq 6ND$, minimizing the loss under fixed $C$ yields

$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$0

Pearce and Song emphasize that this is the asymptotic Chinchilla prescription once one counts total parameters and total compute rather than non-embedding parameters alone. Using the original Hoffmann et al. fit $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$1 and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$2 gives exponents $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$3 and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$4, whereas the Epoch AI re-analysis $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$5 and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$6 gives $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$7 and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$8. They therefore recommend the rule of thumb $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$9 and $N$0, together with the use of total parameters $N$1 and total compute $N$2 in future scaling studies (Pearce et al., 2024).

A separate replication of Chinchilla’s third fitting procedure reaches a closely related conclusion. Refitting the five-parameter surface gave $N$3, which implies

$N$4

Because these exponents are nearly equal, the optimal tokens-per-parameter ratio remains near a constant of about $N$5 for practical budgets; in that replication, the corrected parametric fit is presented as restoring consistency with the empirically validated $N$6 tokens-per-parameter rule (Besiroglu et al., 2024).

2. Parameter counting, fitting procedures, and methodological controversies

A central methodological controversy concerns why Kaplan et al. reported $N$7 while Chinchilla-style analyses favor approximately $N$8. Pearce and Song attribute much of this discrepancy to two factors: Kaplan counted only non-embedding parameters $N$9, and the analysis was performed at small scale. Rewriting the Chinchilla loss in terms of non-embedding parameters introduces curvature in the compute-efficient frontier; at very small $D$0, the effective exponent approaches $D$1, while at large scale the embedding fraction vanishes and one recovers the Chinchilla exponent $D$2. In their synthetic reconstruction of Kaplan’s regime, local fits produce exponents $D$3 and $D$4, close to Kaplan’s original value (Pearce et al., 2024).

The reliability of particular fitting procedures has also been contested. The replication study of Chinchilla Approach 3 argues that the originally reported confidence intervals were implausibly narrow, noting that intervals of that width would require roughly $D$5 independent runs, whereas the original study likely used fewer than $D$6. It attributes the discrepancy to early stopping of L-BFGS caused by averaging rather than summing Huber losses in both the main fit and the bootstrap procedure. In that account, the rederived Approach 3 estimates become compatible with Chinchilla’s first two estimation procedures once the optimization is corrected (Besiroglu et al., 2024).

A more recent critique targets Chinchilla Approach 2, the IsoFLOP parabola-fit method. Czech et al. identify three systematic bias sources: loss-surface asymmetry $D$7, uncentered IsoFLOP sampling, and sampling-grid width. On digitized Llama 3 IsoFLOP data, these biases imply a parameter underallocation corresponding to $D$8 of the $D$9 FLOP training budget and $E$0K–$E$1M. The same work argues that direct surface fitting, especially via Variable Projection, largely eliminates these biases and converts the five-parameter inference problem into a well-conditioned two-dimensional optimization over $E$2 (Czech et al., 21 Mar 2026).

Methodological criticism has also come from work on the origin of neural scaling laws. There the authors report that the two-dimensional Chinchilla fit $E$3 often gives worse fits than learned surrogate surfaces based on a small three-layer fully connected network or a kernel model; they use those surrogates to obtain compute-optimal curves by grid search rather than by enforcing the analytic Chinchilla surface directly (Barkeshli et al., 15 Jan 2026).

3. Theoretical interpretations and origins of the power-law form

One line of theory derives Chinchilla-style structure from error decomposition. In the optimizer-scaling study, the proxy problem is a convex quadratic objective

$E$4

optimized by gradient descent on the first $E$5 eigenmodes of $E$6. The resulting excess loss decomposes into approximation error and optimization error. If the spectrum has spectral dimension $E$7, then in the power-law regime $E$8,

$E$9

while for $A,B,\alpha,\beta$0 the dynamics exhibit exponential saturation $A,B,\alpha,\beta$1. Identifying $A,B,\alpha,\beta$2, $A,B,\alpha,\beta$3, and $A,B,\alpha,\beta$4 recovers the two-term Chinchilla form $A,B,\alpha,\beta$5; the paper treats empirical fits with $A,B,\alpha,\beta$6 as accommodating nonideal spectra rather than overturning the underlying decomposition (Volkova et al., 7 Feb 2026).

A second line of work investigates whether power laws require power-law structure in the data. Training transformers to predict random walks on graphs, including Erdős–Rényi and Barabási–Albert ensembles, already produces neural scaling laws in settings that do not contain power-law data correlations. The same study systematically simplifies natural language by sampling sequences from increasingly simple generative models, from four-layer and two-layer transformer LLMs down to bigrams, and reports a monotonic evolution of the scaling exponents across that ladder of complexity. In its summary table, the model-size exponent $A,B,\alpha,\beta$7 falls from values near $A,B,\alpha,\beta$8 on simple graph or bigram data to about $A,B,\alpha,\beta$9 on natural text, whereas the data exponent $A/N^\alpha$0 stays near $A/N^\alpha$1 in many settings (Barkeshli et al., 15 Jan 2026).

That work also revisits parameterization. Under standard parameterization, two-layer natural-language experiments with embeddings included reproduce Chinchilla-like behavior with an approximately constant tokens-per-parameter ratio. Under maximal-update parameterization ($A/N^\alpha$2), the reported $A/N^\alpha$3 values rise to about $A/N^\alpha$4, and compute-optimal regressions shift toward $A/N^\alpha$5, $A/N^\alpha$6, which the authors interpret as greater parameter efficiency and a stronger case for allocating compute to data (Barkeshli et al., 15 Jan 2026).

4. Optimizer- and configuration-aware generalizations

The canonical Chinchilla law usually fixes the optimizer, typically AdamW. “Towards Robust Scaling Laws for Optimizers” argues that this is insufficient once newer optimizers such as Muon, Shampoo, Scion, and SOAP are considered. Separate Chinchilla-style fits for each optimizer are reported to be ill-conditioned, with extremely high covariance between $A/N^\alpha$7 and between $A/N^\alpha$8, wide error bars on the exponents, and little power to distinguish whether one optimizer truly has a larger $A/N^\alpha$9 or $B/D^\beta$0. The proposed remedy is a shared-exponent law in which $B/D^\beta$1 are shared across optimizers and each optimizer contributes only efficiency rescalings $B/D^\beta$2, $B/D^\beta$3, or $B/D^\beta$4. AdamW is taken as the reference with $B/D^\beta$5 (Volkova et al., 7 Feb 2026).

Empirically, that study first fits $B/D^\beta$6 on AdamW alone and then freezes them while fitting $B/D^\beta$7 for the other optimizers. On OLMo-family experiments, this shared-exponent method yields a $B/D^\beta$8 reduction in MSE of held-out loss predictions, $B/D^\beta$9–$C \simeq 6ND$0 leave-one-out error bars on $C \simeq 6ND$1, and a consistent ordering of data-efficiency $C \simeq 6ND$2 across architectures and token budgets. A representative result gives $C \simeq 6ND$3 and

$C \simeq 6ND$4

for $C \simeq 6ND$5, which the authors summarize by saying that Shampoo, Scion, Muon, and SOAP behave as if training a slightly smaller model but with $C \simeq 6ND$6–$C \simeq 6ND$7 more tokens. The same parameterization supports direct Pareto comparisons: token-limited settings favor the optimizer with the largest $C \simeq 6ND$8, parameter-limited settings favor the largest $C \simeq 6ND$9, and FLOP- or wall-time-limited settings favor the largest $C$0 (Volkova et al., 7 Feb 2026).

A more radical extension treats Chinchilla as only a baseline and learns the residual dependence on the full training configuration. The Configuration-to-Performance Scaling Law (CPL) maps

$C$1

and its neural instantiation, NCPL, fine-tunes Qwen3-1.7B as a regressor that predicts residual loss relative to a Chinchilla baseline. Using $C$2 runs from Marin and StepLaw, NCPL is reported to achieve $C$3–$C$4 lower prediction error than the configuration-agnostic Chinchilla law and to generalize to runs using up to $C$5 more compute than any run in the training set. It also supports joint hyperparameter tuning and extension to loss-curve prediction (Zhang et al., 10 Feb 2026).

5. Data-constrained, repeated-data, and quality-aware laws

Many extensions begin from the observation that the classical Chinchilla form was calibrated for data-rich, single-epoch training, whereas contemporary pretraining increasingly revisits finite corpora for multiple epochs. In this regime, later work distinguishes unique data from repeated exposures, models explicit overfitting penalties, and sometimes replaces the additive separation of model and data terms with coupled bottlenecks.

Variant	Core modification	Target regime
Prescriptive $C$6 law	$C$7	repeated data with fixed unique-token budget
Practical Scaling Laws	$C$8, $C$9	finite baseline, overfitting, multi-epoch training
SoftQ	$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$00	coupled model–data bottleneck
Quality-aware law	$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$01 in the data term	noisy, redundant, or deficient corpora

In “Prescriptive Scaling Laws for Data-Constrained Training,” the overfitting penalty is isolated by first fitting the Chinchilla constants $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$02 on single-epoch runs and then modeling multi-epoch excess loss with a one-parameter additive term: $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$03 Here $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$04 is the number of unique tokens, $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$05 is the number of extra epochs beyond the first, and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$06 is the single fitted overfitting coefficient. The paper reports that the compute-optimal frontier eventually “turns back”: beyond some point, additional repetition is counterproductive and compute is better spent on model capacity. At $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$07 M and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$08 FLOPs, the Chinchilla law suggests $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$09 M and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$10 epochs, the Muennighoff-style effective-data law suggests $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$11 M and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$12 epochs, and the one-parameter law suggests $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$13 B and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$14 epochs. On a held-out prescription at $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$15 M and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$16 FLOPs, the resulting configuration achieves $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$17, compared with $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$18 for Chinchilla and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$19 for the effective-parameter baseline. As a regularization case study, strong weight decay $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$20 reduces $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$21 by approximately $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$22, from $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$23 to $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$24, and produces a crossover at $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$25 FLOPs (Lovelace et al., 2 May 2026).

“Practical Scaling Laws: Converting Compute into Performance in a Data-Constrained World” replaces Chinchilla’s unbounded additive law by

$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$26

where $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$27 is the count of unique training examples and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$28 is the total number seen with repetition. This form is designed to correct three structural failures of the classical law outside the single-epoch regime: lack of saturation at an uninformed baseline $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$29, inability to represent overfitting, and conflation of unique and repeated data. In the limit of small $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$30, negligible overfitting, and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$31, it reduces back to Chinchilla. Empirically, the eight-parameter form achieved state-of-the-art held-out RMSE in $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$32 cells of constructed multi-epoch experiments and won all $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$33 external-grid columns when refit to five published LLM scaling grids; the summary reports average RMSE $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$34 lower than the best Chinchilla-style competitor and roughly $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$35 lower on average than the second best in the external-grid comparison (Bryant et al., 9 May 2026).

SoftQ addresses the same regime through a coupled rather than additive law,

$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$36

where $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$37 is unique-data size. Its motivation is that, under repeated-data training, the loss gap between small and large unique-data budgets grows with model size, contradicting the additive Chinchilla prediction that the gap should be independent of $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$38. On a $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$39 grid of strongly regularized autoregressive transformers with model sizes from $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$40 M to $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$41 B and unique-data budgets from $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$42 M to $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$43 M tokens, SoftQ attained RMSE $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$44 on the full fit versus $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$45 for Chinchilla, held-out RMSE $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$46 versus $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$47, and RMSE $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$48 versus $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$49 on an independent Kim et al. grid. In the same framework, Masked-Input Regularization (MIR) is estimated as worth roughly $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$50 as much unique training data, with equivalent-data ratios around $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$51–$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$52 at $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$53 M–$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$54 M tokens (Xu et al., 5 Jun 2026).

A different axis of generalization introduces explicit data quality. The quality-aware Chinchilla law

$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$55

adds a dimensionless quality parameter $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$56, where $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$57 denotes ideal data and smaller values represent corruption or deficiency. Two practical estimators are proposed: a corruption-rate proxy $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$58 and a deficiency-based map $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$59. In synthetic NMT and causal language-modeling experiments, the fitted quality exponent satisfies $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$60 in both tasks, which the authors interpret as sublinear decay of effective data with quality and relative robustness to moderate corruption. Their summary gives a rule of thumb that increasing $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$61 from $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$62 to $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$63 can reduce required $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$64 by about $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$65 at fixed $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$66 and target loss (Subramanyam et al., 30 Sep 2025).

6. Architectural, sparsity, modality, and domain-specific variants

Not all Chinchilla-style extensions change the data regime; many instead change what counts as “size.” In sparse pretraining, “The Journey Matters” replaces the final parameter count by the average active parameter count over the training trajectory,

$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$67

Here $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$68 is the average number of non-pruned parameters during pretraining. On $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$69 sparse-and-dense experimental points spanning three model sizes, two durations, and five sparsity levels, this unified law achieved mean absolute error $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$70 nats and is presented as fitting both dense and sparse runs without any explicit sparsity term. The same work reports that beginning pruning at $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$71 of total training compute and concluding at $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$72 achieves near-optimal final evaluation loss (Jin et al., 21 Jan 2025).

Architecture-aware variants target inference efficiency. One such law augments Chinchilla with an aspect-ratio term,

$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$73

where $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$74. The motivating empirical fact is that models of the same size can have up to $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$75 different inference latency. Fitted on $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$76 models spanning $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$77 M to $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$78 B parameters, the law is used in a predict–rank–train loop for architecture search. The resulting Morph-1B model is reported to achieve $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$79 lower latency while maintaining accuracy on downstream tasks, with the final released model running in $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$80 s at average zero-shot accuracy $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$81 (Bian et al., 30 Jan 2025).

A closely related conditional law separates two architectural calibration factors: hidden size $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$82 and the MLP-to-attention parameter ratio $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$83. Its multiplicative form predicts a U-shaped loss dependence on both variables at fixed $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$84 and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$85, while grouped-query attention is handled in a downstream throughput search. Fitted on more than $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$86 models from $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$87 M to $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$88 B parameters and up to $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$89 B tokens, the law yields an optimum near $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$90 and $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$91 for a $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$92 B-parameter, $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$93 B-token setting. The corresponding Panda-1B model is reported to outperform LLaMA-3.2-1B by $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$94 average accuracy, and the Surefire family achieves up to $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$95 inference throughput under the same loss constraint (Bian et al., 21 Oct 2025).

Domain-specific scaling can shift the exponents themselves. For code-only pretraining, a Chinchilla fit over $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$96 runs with model sizes $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$97 B–$L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$98 B and training tokens $L(N,D)=E+\frac{A}{N^\alpha}+\frac{B}{D^\beta},$99 B–$N$00 B produced

$N$01

Under the compute-optimal derivation, these values imply a data-to-parameter ratio that grows with scale and reaches approximately $N$02 at $N$03 FLOPs, compared with the natural-language reference value of about $N$04. The authors describe code as a more data-hungry regime and report that a more expressive Farseer law fits about $N$05 better in mean relative error than the Chinchilla form (Luo et al., 9 Oct 2025).

Mixed-modal generative LLMs require yet another extension. For two modalities $N$06 and $N$07, the proposed joint loss combines the corresponding unimodal laws with an explicit interaction term that includes a maximal synergy or competition constant $N$08, plus modality-pair-specific parameter and data terms. The fitted unimodal exponents vary widely across modalities; in the bimodal setting, the interaction exponents $N$09 range roughly from $N$10 to $N$11, $N$12 spans $N$13–$N$14, and $N$15 can reach about $N$16 nats of asymptotic improvement. A $N$17 B speech–text run is reported to validate the predicted emergence of synergy by achieving joint loss below the average of the separate speech-only and text-only models (Aghajanyan et al., 2023).

7. Inference, wall-clock time, and fully unified multi-axis laws

Classical Chinchilla laws optimize pretraining alone; several later works instead optimize end-to-end usage. “Beyond Chinchilla-Optimal: Accounting for Inference in LLM Scaling Laws” adds lifetime inference demand $N$18 to the compute or dollar-cost objective. Training cost scales as $N$19, inference cost as $N$20, and the resulting constrained optimization typically favors smaller models trained on more data when inference demand is large. In the authors’ summary, researchers expecting reasonably large inference demand of about $N$21B requests should train models smaller and longer than Chinchilla-optimal. Their empirical validation spans $N$22 models and token-to-parameter ratios up to $N$23, and an ablation shows that fitting only on moderate $N$24 regimes overestimates the benefit of additional tokens at these extreme ratios (Sardana et al., 2023).

Train-to-Test ($N$25) scaling laws push this logic further by modeling repeated sampling at inference through an explicit $N$26 term: $N$27 With training FLOPs $N$28 and inference FLOPs $N$29, joint optimization yields closed-form optima for $N$30. Across eight downstream tasks, the reported optimum shifts strongly into the overtraining regime: smaller $N$31, much larger $N$32, and $N$33. The study validates this prediction by pretraining $N$34 heavily overtrained models in the forecast region and finds that the best overtrained checkpoints substantially outperform Chinchilla-optimal checkpoints under the same train/test budgets (Roberts et al., 1 Apr 2026).

Other work replaces FLOPs by wall-clock time as the principal constraint. “Time Matters: Scaling Laws for Any Budget” builds a step-time proxy from memory copies and FLOPs, reporting $N$35 for the time estimator across thousands of models. Combined with a Chinchilla-style loss law fitted over $N$36 transformer runs,

$N$37

this yields a closed-form predictor $N$38 from hyperparameters and wall-clock budget. In that framework, the gradient field on constant-parameter contours always points toward increasing width and decreasing depth, leading to the conclusion that under a fixed time budget models should be wider rather than deeper (Inbar et al., 2024).

At the broadest end of the spectrum, the Unified Neural Scaling Law (UNSL) treats model parameters, dataset size, training steps, inference steps, compute, and various hyperparameters as simultaneous inputs to a broken-power-law family with explicit overfitting and hyperparameter nonmonotonicities. In the two-resource regime with other hyperparameters fixed, UNSL reduces to

$N$39

so the Chinchilla-style compute-optimal split reappears as

$N$40

In this sense, Chinchilla-style laws appear in UNSL as a low-dimensional special case of a broader family of multivariate neural scaling models rather than as an isolated empirical regularity (Caballero et al., 25 May 2026).

Chinchilla-style scaling laws thus designate a modeling tradition rather than a single formula. Its canonical two-term surface remains the reference model for data-rich, single-epoch pretraining, but later work shows that optimizer choice, repeated-data overfitting, data quality, sparsity trajectories, architecture, modality mixture, inference demand, and wall-clock budget can each alter either the functional form or the interpretation of its coefficients. A plausible implication is that “Chinchilla-optimal” is best understood as a regime-specific optimum inside a larger hierarchy of scaling laws, all of which retain the same central ambition: converting limited experimental grids into quantitative prescriptions for how to allocate parameters, data, compute, and deployment cost.