Skaling: Chinchilla's Exponents Meet Kaplan's Coupling

Abstract: Neural scaling laws are foundational for LLM development, yet standard formulations systematically under- and overestimate loss at data-scarce and overtraining extremes. This failure originates in the underlying assumption that model size and training data impact the loss independently. To address this, we introduce the Skaling law, a generalized functional form that couples model capacity and data through a single interaction exponent. This simple extension reduces the Mean Absolute Percentage Error (MAPE) by 1.5-3x across both interpolation and extrapolation regimes. When paired with a sparse grid strategy restricted to low-compute regimes, the Skaling law achieves accurate full-grid extrapolation using approximately 10x less compute than uniform sweeps. By enabling reliable performance prediction from small-scale experiments, the Skaling law provides a more robust and resource-efficient framework for allocating compute budgets in next-generation model training.

• The paper introduces Skaling, a six-parameter scaling law with an outer coupling exponent that captures the nonzero, predominantly negative interaction between model size and training data.
• Skaling typically achieves 1.5–3× lower MAPE than Chinchilla, including a reduction from 14.63% to 1.15% under sparse-grid far extrapolation on SK-Grid, and outperforms Chinchilla on 76% of evaluated configurations.
• The paper shows that L-shaped profiling can reduce fitting compute by roughly 10× while preserving predictive accuracy, enabling more reliable extrapolation and compute-allocation decisions from low-cost boundary measurements.

Skaling: Chinchilla’s Exponents Meet Kaplan’s Coupling

Central thesis

“Skaling: Chinchilla’s Exponents Meet Kaplan’s Coupling” argues that the additive Chinchilla scaling law fails for a structural reason: it assumes that model size $N$ and training data $D$ affect loss independently. This assumption is encoded mathematically by the zero mixed derivative $\partial^2 L/\partial N\partial D=0$. Although the additive form can fit the interior of an experimental grid with high accuracy, it systematically mispredicts the loss at data-scarce, overtrained, and jointly extrapolated boundaries, precisely where scaling-law predictions are most consequential.

The paper introduces Skaling, a six-parameter generalization that preserves Chinchilla’s independent inner exponents while adding a single outer coupling exponent:

$$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$

The additive Chinchilla law is recovered when $k=1$. For $k\neq 1$, model size and data interact through the nonlinear aggregation of the two reducible-loss terms. The empirical fits consistently favor a sub-unit coupling exponent, typically $k\approx0.31$--$0.45$ on the principal datasets. The authors report reductions in MAPE of approximately $1.5$--$3\times$ over Chinchilla across interpolation, single-axis extrapolation, and far-extrapolation regimes, while an L-shaped sparse profiling design achieves comparable predictive accuracy with approximately $D$0 less fitting compute (2608.07222).

Failure mode of additive scaling laws

The paper’s argument begins with a distinction between interpolation quality and structural validity. Chinchilla often achieves excellent interpolation metrics: on the Farseer and SK-Grid datasets, its full-grid interpolation $D$1 values are $D$2 and $D$3, respectively. These values might suggest that the law is an adequate representation of the loss surface. However, interpolation does not test how the fitted surface bends outside the observed interior.

The residual pattern is instead saddle-shaped. Chinchilla overestimates loss in some boundary regions and underestimates it in others, with errors growing as $D$4 and $D$5 become increasingly imbalanced. This behavior follows directly from its separable form,

$D$6

because no choice of the individual power-law exponents can produce a nonzero cross-derivative.

Figure 1: Skaling removes the boundary-concentrated residual bias of the additive Chinchilla law and is more accurate on 76% of evaluated configurations.

The contrast between the two functional forms is especially important for experimental design. A law may be highly accurate at densely sampled interior points while remaining unreliable for predicting larger models, longer training runs, or configurations in which one resource scales substantially faster than the other. The paper therefore treats boundary extrapolation—not merely aggregate fit quality—as the principal diagnostic.

Derivative evidence for $D$7--$D$8 interaction

To test whether the observed coupling is an artifact of a particular parameterization, the authors analyze the empirical loss surface directly. They estimate derivatives in logarithmic coordinates using two mesh-free approaches: local moving least squares (MLS) and a global Gaussian-process (GP) surrogate.

The first-order derivatives exhibit approximate power-law behavior. In particular, the same-variable slopes are close to linear on log-log axes, with estimates around

$D$9

for the real-space derivative diagnostics. The cross-axis first-order slopes are comparatively small, approximately $\partial^2 L/\partial N\partial D=0$0 and $\partial^2 L/\partial N\partial D=0$1. This near-separability at first order could be interpreted as evidence for the Chinchilla assumption, but the paper correctly emphasizes that first-order projections are insufficient to identify interaction.

Figure 2: First-order derivative projections show dominant same-variable power-law decay and weaker cross-variable dependence.

The decisive observation is that the empirical mixed derivative is nonzero throughout the grid. Its magnitude follows an approximate power law in both axes, with fitted exponents near $\partial^2 L/\partial N\partial D=0$2, and its sign is predominantly negative. A negative mixed derivative means that the marginal benefit of increasing one resource becomes larger when the other resource is also increased. Jointly scaling model size and data therefore lowers loss more than an additive model predicts.

Figure 3: The empirically nonzero, predominantly negative mixed derivative contradicts the exact separability imposed by Chinchilla scaling.

This derivative evidence gives the paper’s main theoretical claim a direct empirical basis: the loss surface can appear nearly separable in first-order marginal trends while remaining materially nonseparable in second-order interaction structure. The distinction is subtle but operationally important. Small cross-slopes do not imply a negligible cross-derivative, particularly when predictions are extrapolated over multiple orders of magnitude.

The Skaling parameterization

Skaling introduces coupling without abandoning the interpretability of Chinchilla’s two source terms. The exponent $\partial^2 L/\partial N\partial D=0$3 governs how the size-dependent and data-dependent contributions aggregate. Its derivative structure is

$\partial^2 L/\partial N\partial D=0$4

with an analogous expression for $\partial^2 L/\partial N\partial D=0$5. For $\partial^2 L/\partial N\partial D=0$6, both derivatives remain negative, so the model is monotone in model size and data. For $\partial^2 L/\partial N\partial D=0$7, the mixed derivative is negative:

$\partial^2 L/\partial N\partial D=0$8

This permits the model to represent synergy while preserving physically sensible monotonicity.

The authors contrast this construction with adding a signed product term to the Chinchilla law. A term such as $\partial^2 L/\partial N\partial D=0$9 can create a nonzero mixed derivative, but matching the empirically negative cross-derivative requires $$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$0. That sign can make the model-size derivative positive in parts of the domain, implying that increasing model size increases loss. Choosing $$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$1 preserves monotonicity but produces the wrong interaction sign. Skaling avoids this conflict because coupling arises from a positive outer transformation rather than a separately signed additive interaction.

The fitted values of $$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$2 are not merely small corrections around the Chinchilla case. On Farseer, the full-grid and L-shaped fits yield $$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$3 and $$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$4; on SK-Grid, both fits yield $$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$5. Thus, the data consistently reject $$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$6 on the principal experiments. On the Farseer-code and original Chinchilla measurements, however, $$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$7 is closer to one, approximately $$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$8--$$L(N,D)=\left(A N^{-\alpha}+B D^{-\beta}\right)^k+E.$$9, and the predictive advantage correspondingly weakens. This dataset dependence is an important qualification: Skaling does not imply that all training recipes exhibit equally strong coupling.

Experimental design and sparse profiling

The paper’s second contribution is methodological. A conventional rectangular sweep over logarithmic grids is computationally inefficient because training cost scales approximately with $k=1$0. The upper-right portion of the grid therefore dominates total compute, even though many expensive points are statistically redundant for identifying the functional form.

Skaling motivates an L-shaped design concentrated on the low-compute edges. The D-band varies $k=1$1 using small models, while the N-band varies $k=1$2 at short training horizons. These two bands separately expose the decay associated with data and model size, while the coupled functional form reconstructs the interior and high-compute region.

Figure 4: L-shaped sampling concentrates measurements on low-compute grid edges rather than densely sampling the compute-dominated upper-right corner.

The design is more than a heuristic reduction in run count. It relies on asymptotic structure: at sufficiently large $k=1$3, the data-dependent contribution becomes small and the remaining variation identifies size-related behavior; symmetrically, large $k=1$4 isolates the data-related component. The experiments show that the Skaling form can exploit these boundary measurements more effectively than Chinchilla.

On Farseer, reducing the fitting compute from approximately $k=1$5 FLOPs for the full grid to $k=1$6 FLOPs for the L-shaped grid increases Skaling’s interpolation MAPE from $k=1$7 to $k=1$8, while its far-extrapolation MAPE remains $k=1$9. On SK-Grid, the L-shaped design uses approximately $k\neq 1$0 FLOPs instead of $k\neq 1$1 FLOPs. Skaling retains an interpolation MAPE of $k\neq 1$2 and far-extrapolation MAPE of $k\neq 1$3.

By comparison, Chinchilla degrades sharply under sparse sampling. Its SK-Grid far-extrapolation MAPE rises from $k\neq 1$4 on the full grid to $k\neq 1$5 on the L-shaped grid. The results support the paper’s claim that the relevant benefit of coupling is not simply improved fitting capacity; it is compatibility with an experimental geometry that emphasizes informative low-cost boundaries.

Predictive performance

The principal evaluation uses Farseer, containing 404 configurations across 25 model sizes and 55 data budgets, and SK-Grid, containing 134 configurations across 15 model sizes and 16 data budgets. The evaluation separates interpolation, extrapolation in $k\neq 1$6, extrapolation in $k\neq 1$7, and far extrapolation beyond both axes.

On the full Farseer grid, Skaling achieves MAPE values of $k\neq 1$8 for interpolation, $k\neq 1$9 for larger-$k\approx0.31$0 extrapolation, $k\approx0.31$1 for larger-$k\approx0.31$2 extrapolation, and $k\approx0.31$3 for far extrapolation. Chinchilla obtains $k\approx0.31$4, $k\approx0.31$5, $k\approx0.31$6, and $k\approx0.31$7, respectively. On SK-Grid, Skaling obtains $k\approx0.31$8, $k\approx0.31$9, $0.45$0, and $0.45$1, compared with Chinchilla’s $0.45$2, $0.45$3, $0.45$4, and $0.45$5.

The strongest result concerns the far-extrapolation corner of SK-Grid under sparse sampling: Skaling reduces MAPE from Chinchilla’s $0.45$6 to $0.45$7. This is a reduction of more than $0.45$8, achieved using a fraction of the profiling compute. The paper also reports that, across individual configurations, Skaling is more accurate than Chinchilla on $0.45$9 of runs, with a median error improvement of $1.5$0 and at least a $1.5$1 improvement on approximately one-third of configurations.

The nine-parameter Farseer baseline does not consistently match Skaling. Despite its greater nominal flexibility, it produces larger data-extrapolation errors on the principal full-grid experiments: $1.5$2 on Farseer and $1.5$3 on SK-Grid, compared with Skaling’s $1.5$4 and $1.5$5. This comparison supports an inductive-bias interpretation: a small, structurally appropriate extension can outperform a substantially richer model whose parameterization does not align as well with the geometry of the empirical surface. Nevertheless, optimizer sensitivity and the difficulty of fitting Farseer’s parameterization mean that this conclusion should not be interpreted as a universal ranking of functional expressivity.

Compute-optimal allocation

A notable property of Skaling is that the outer exponent changes the loss surface without changing the stationarity condition for compute-optimal allocation. Under the compute constraint $1.5$6, minimizing the monotone transformation of the inner sum produces the same first-order condition as in the additive law. Consequently, the optimal allocation has the same algebraic form, but its fitted parameters differ because the Skaling and Chinchilla models infer different exponents and amplitudes.

The optimal point is characterized by equality of the logarithmic marginal slopes:

$1.5$7

The authors estimate this frontier nonparametrically using both GP and MLS surrogates, rather than relying solely on a fitted parametric law. Both estimates show a decreasing token-to-parameter ratio with increasing compute on Farseer. Power-law fits to the empirical frontier yield exponents of approximately $1.5$8 and $1.5$9, close to Skaling’s analytic value of approximately $3\times$0 and opposite in sign to Chinchilla’s approximately $3\times$1.

Figure 5: GP- and MLS-derived compute-optimal frontiers agree with Skaling’s decreasing token-to-parameter ratio and diverge from Chinchilla’s nearly flat prediction.

The extrapolated difference is substantial. At approximately $3\times$2 FLOPs, Chinchilla predicts around 380 tokens per parameter, whereas the empirical gradient-based estimates and Skaling predict approximately 20--40 tokens per parameter. The paper describes this as a roughly $3\times$3 discrepancy one order of magnitude beyond the observed data. This is a strong and consequential claim, although it depends on extrapolating fitted power-law trends beyond the measurement range and on the validity of the training recipe used to generate Farseer.

The direction of the allocation trend is not universal. On Farseer, $3\times$4, yielding a decreasing $3\times$5 with compute. On SK-Grid, $3\times$6, so the corresponding optimum increases with compute. The robust conclusion is therefore not that optimal token-to-parameter ratios must decline, but that coupling can materially alter the scale dependence of the allocation rule, with the direction determined by the fitted exponents and training regime.

Iso-ratio compute extrapolation

The paper further evaluates the operational setting in which frontier models are trained along fixed $3\times$7 ratios. The authors hold out the eight highest-compute points from each iso-ratio slice and fit each global law only to the lower-compute remainder.

Skaling achieves pooled MAPE of $3\times$8 over 112 held-out high-compute Farseer runs. This is a $3\times$9 reduction relative to Chinchilla’s $D$00 and is lower than the Farseer baseline’s $D$01. Its error remains below $D$02 in each training-regime partition. The per-ratio power law performs slightly better near the empirically optimal ratios, with $D$03 versus Skaling’s $D$04, but it is separately fitted for each ratio and cannot provide a joint model of allocation across $D$05 and $D$06.

This experiment clarifies the practical scope of the result. Skaling is not necessarily the best one-dimensional model when the ratio is fixed in advance. Its advantage is that it provides a globally coherent two-dimensional surface capable of both compute extrapolation and resource-allocation analysis.

Interpretation of the irreducible loss

The fitted irreducible-loss term $D$07 deserves careful interpretation. On Farseer, Skaling reduces the fitted $D$08 from approximately $D$09 to $D$10 on the full grid and from $D$11 to $D$12 on the L-shaped grid. The authors explicitly reject interpreting this as evidence that the true irreducible loss is nearly zero. Because $D$13 changes the asymptotic curvature of the reducible component, the coupling exponent and the constant floor can compensate for one another over the finite scale range observed.

This is a form of weak identifiability. The measured runs constrain total loss accurately but do not necessarily distinguish whether flattening arises from a true constant floor or from a slowly decaying coupled term. The near-vanishing $D$14 on Farseer therefore reflects parameter tradeoff rather than a direct estimate of the Bayes-optimal loss floor. This caveat is central for using Skaling in regimes substantially beyond the training grid.

The dominated-pair analysis reinforces the issue. Removing $D$15 from pairwise loss differences improves Chinchilla’s far-extrapolation error on Farseer from $D$16 to $D$17, indicating that poor identification of the floor accounts for part of Chinchilla’s boundary error. However, this correction does not consistently improve Skaling and can even worsen some sparse-grid results. The evidence suggests that Skaling’s gains arise from both improved treatment of the floor and a more appropriate interaction structure, with their relative contributions depending on the dataset and sampling scheme.

Practical and theoretical implications

Practically, the paper proposes a lower-cost workflow for scaling-law experiments: use an L-shaped set of inexpensive runs, fit a coupled law in log space, and validate against targeted boundary or iso-ratio extrapolation. This approach can reduce the compute required for empirical profiling by roughly an order of magnitude while preserving prediction quality. It is particularly relevant when the objective is not merely to interpolate among existing runs but to select future model sizes, data budgets, and compute-optimal training configurations.

Theoretically, the work reframes scaling-law selection as a problem of differential structure rather than parameter count. The crucial question is whether the loss surface has a nonzero mixed derivative and whether the proposed parameterization can represent its sign while preserving monotonicity. This provides a principled criterion for extending scaling laws to additional axes, such as data mixture, repeated data, context length, optimizer configuration, or distillation variables.

Several limitations remain. The experiments are based on specific architectures, datasets, validation losses, and hyperparameter prescriptions. The SK-Grid models use a fixed mixture and StepLaw-based optimization settings, while the Farseer and Chinchilla datasets arise from different experimental pipelines. The measured coupling may therefore reflect not only intrinsic model/data interaction but also recipe-dependent effects, including learning-rate schedules, batch-size scaling, and data composition. Moreover, the empirical optimal-allocation frontiers are inferred from finite grids and then extrapolated substantially beyond them. Validation on independent architectures, modalities, data mixtures, and training protocols is necessary before treating the coupling exponent as a universal property.

Future scaling-law systems could combine Skaling-style inductive structure with hierarchical Bayesian fitting, uncertainty-aware active sampling, and multi-axis coupling. A natural extension would replace a single scalar $D$18 with a constrained interaction structure for model size, data volume, context length, and data quality, while retaining monotonicity and interpretable marginal effects. Active-learning procedures could then select new runs where posterior uncertainty in the compute-optimal frontier is greatest rather than sampling a predetermined grid.

Conclusion

The paper identifies a specific structural weakness in the additive Chinchilla law: its enforced zero interaction between model size and training data. Direct mixed-derivative estimates, boundary residuals, and controlled extrapolation experiments support the conclusion that this assumption is violated on the principal datasets. Skaling addresses the problem with one outer coupling exponent while preserving independent size and data exponents, monotonicity, and a tractable compute-optimal allocation rule.

Its empirical advantages are strongest at grid boundaries and under sparse experimental designs. On SK-Grid, far-extrapolation MAPE falls from $D$19 to $D$20 with full-grid fitting and from $D$21 to $D$22 under L-shaped sampling. The iso-ratio experiment yields pooled high-compute MAPE of $D$23, substantially below Chinchilla’s $D$24. These results indicate that functional form and sampling geometry are jointly decisive for reliable scaling prediction.

The paper’s broader contribution is methodological: scaling laws should be evaluated by their derivative structure and extrapolation behavior, not by interior interpolation alone. Subject to validation across training recipes and model families, coupling-aware laws such as Skaling offer a technically simple route toward more reliable and compute-efficient decisions about future model scaling (2608.07222).