Deployment-Aware Chinchilla Law

• Deployment-Aware Chinchilla Law is a framework for LLMs that integrates inference costs and deployment constraints into traditional scaling laws.
• It jointly optimizes training compute and test-time resources, leading to smaller models with higher tokens-per-parameter ratios under high inference demand.
• Empirical results using T² scaling laws validate that overtraining smaller models for intensive deployment can outperform larger, conventional architectures.

Deployment-aware Chinchilla law refers to a family of scaling laws for LLMs that extend the original Chinchilla optimality principle by factoring in both training and inference cost, as well as deployment characteristics such as inference demand and repeated sampling. Classical scaling laws, typified by the Chinchilla regime, prescribe optimal tradeoffs between parameter count and training tokens for fixed pretraining compute. Deployment-aware modifications rigorously incorporate inference (test-time) budget constraints, resulting in different prescriptions for pretraining and model sizing. This paradigm is formalized in the “Beyond Chinchilla-Optimal” framework by Sardana & Frankle (Sardana et al., 2023) and further generalized by the “Train-to-Test (T²) scaling laws” of Chen et al. (Roberts et al., 1 Apr 2026). These developments have significantly revised practical guidelines for resource allocation during pretraining and real-world model deployment.

1. Classical Chinchilla Law and Its Limitations

The Chinchilla law models LLM loss as a sum of irreducible and two reducible error terms that decay as power laws in model size $N$ and training tokens $D$: $$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$ where $E$ is an empirical lower bound, and $A$, $B$, $\alpha$, $\beta$ are fitted on experimental data (Chinchilla: $\alpha \approx 0.34$, $\beta \approx 0.28$). Training compute is $D$0, with $D$1 FLOPs-per-param-per-token.

Under a fixed training compute budget $D$2, the optimal $D$3 allocation is the solution to: $D$4 with the analytic solution yielding

$D$5

Numerically, Chinchilla optimal models use $D$6 tokens per parameter. However, this regime neglects inference cost and does not address decisions at deployment, such as selecting $D$7 output samples per query or high-volume inference workloads (Sardana et al., 2023).

2. Incorporating Inference Demand into Scaling Laws

Deployment-aware Chinchilla law modifies the objective to include both pretraining and (aggregate or per-query) inference cost. For expected inference demand $D$8 (tokens served post-training) and with cost per FLOP $D$9 (training) and $$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$0 (inference), total cost reads: $$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$1 with $$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$2 FLOPs-per-param-per-inference-token. The deployment-aware optimization seeks

$$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$3

for some target loss $$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$4.

Stationarity conditions from the Lagrangian yield a transcendental but numerically tractable system whose solution $$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$5 directly depends on $$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$6, leading to new optimal ratios $$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$7 that increase with inference demand. Empirically, for realistic $$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$8 tokens, optimal models are substantially smaller and are trained for much longer (tokens-per-param $$L(N, D) = E + \frac{A}{N^\alpha} + \frac{B}{D^\beta}$$9), with Sardana & Frankle verifying loss-power-law consistency up to $E$0 (Sardana et al., 2023).

3. Joint Optimization of Pretraining and Test-time Decisions

Modern deployments often require repeated test-time sampling (e.g., pass@$E$1 metrics), making inference cost proportional to both model size $E$2 and number of samples $E$3. The T² scaling laws (Roberts et al., 1 Apr 2026) formalize the joint end-to-end allocation over $E$4 under global compute: $E$5 via the objective: $E$6 with $E$7 modeled either as a power law extension of Chinchilla incorporating $E$8 (e.g., $E$9) or directly via pass@$A$0 accuracy.

This results in an optimal triple $A$1 whose allocation exponents obey: $A$2 providing a principled foundation for deciding model size, tokens, and per-query inference under fixed resources (Roberts et al., 1 Apr 2026).

4. Empirical Regimes: Overtraining and the Deployment-Optimal Frontier

Extensive experiments demonstrate that deployment-aware law predicts an "overtraining" regime where models are orders of magnitude smaller than Chinchilla-optimal, yet trained on vastly more tokens per parameter. When evaluated under realistic inference constraints, such as fixed FLOPs per query or total reused inference budget, deployment-optimal models consistently outperform larger Chinchilla-optimal counterparts on both downstream benchmarks and synthetic reasoning tasks.

Benchmarks validate that, with fixed inference resources, the T²-predicted optimum shifts deep into the overtraining region ($A$3 from $A$4 to $A$5), and actual pretrained models at T² optima dominate Chinchilla-models in pass@$A$6 and NLL—effects that survive both standard and supervised post-training (Roberts et al., 1 Apr 2026).

5. Statistical Modeling Approaches: Loss and pass@$A$7

Deployment-aware scaling frameworks incorporate two main evaluation methodologies:

• Loss-based: Extends NLL Chinchilla law to include $A$8 and optimizes $A$9 under compute constraints.
• Pass@$B$0 modeling: Maps $B$1 to single-pass accuracy via a sigmoid, then models pass@$B$2 using Beta-distributed per-example probabilities, yielding

$B$3

where $B$4 are tuned to $B$5 (Roberts et al., 1 Apr 2026).

Optimization of pass@$B$6 accuracy delivers the same qualitative regime shift: smaller models, far greater training depth, and deliberate exploitation of repeated sampling. Both approaches are in strong empirical agreement and robust across tasks and post-training variations.

6. Practical Guidance and Implications

The deployment-aware Chinchilla law furnishes a concrete blueprint for resource allocation:

• For large-scale or high-repetition inference, allocate pretraining and inference compute jointly, not sequentially.
• For a given inference budget per query, train a smaller model for much longer, and utilize as many output samples as the query budget permits.
• Coefficient fits for scaling laws must account for data outside the standard $B$7 range; restricting to Chinchilla-regime data misestimates the value of larger $B$8, especially at extreme $B$9 (Sardana et al., 2023).
• Post-training (fine-tuning) does not erase the deployment-aware overtraining benefit, cementing the revised optimum's relevance for modern practice (Roberts et al., 1 Apr 2026).

7. Table: Comparison of Scaling Regimes

Criterion	Chinchilla Law	Deployment-Aware / T² Law
Optimization variables	$\alpha$0, $\alpha$1	$\alpha$2, $\alpha$3, $\alpha$4, $\alpha$5
Train tokens per param	$\alpha$6	$\alpha$7 (depending on $\alpha$8, $\alpha$9)
Model size $\beta$0 at optimum	Larger	Significantly smaller
Inference objective	Not included	Explicitly budgeted
Empirical region of validity	Standard suite	Verified up to $\beta$1

This deployment-aware perspective reorients model design towards maximizing utility under real-world constraints, unifying pretraining and deployment strategies, and ensuring reliable performance forecasts for production-scale LLMs.