In-Context Neural Scaling Laws

• In-Context Neural Scaling Laws are quantitative relations that link inference adaptation performance to resources like demonstration count, prompt length, and model scale.
• They are analyzed through diverse frameworks—including Bayesian, in-context regression, and empirical context-aware models—which reveal saturating gains and resource trade-offs.
• Unified transformer theories integrate architectural parameters and task hierarchy to elucidate in-context adaptation, while unresolved challenges remain in unifying heterogeneous approaches.

In-context neural scaling laws are quantitative relations between inference-time adaptation performance and the resources that make such adaptation possible: the number of in-context examples, prompt length, usable context window, model depth and width, pretraining data, and sometimes training or test-time compute. The literature does not yet present a single universally accepted law. Instead, it offers several partially overlapping formalisms: Bayesian laws in which demonstrations update a posterior over latent tasks, solvable toy theories in which self-attention implements in-context optimization, empirical laws that jointly model downstream performance with training compute and provided context, and broader transformer theories that connect ICL emergence to depth, width, context length, and task structure (Arora et al., 2024).

1. Scope, variables, and problem formulations

The direct literature on in-context neural scaling laws uses different targets and different scaling variables. Some papers scale the probability of the next correct output as a function of the number of demonstrations. Others scale population risk in in-context regression as a function of depth, width, context length, and training time. Others model downstream task performance jointly in terms of training compute and prompt length. A more ambitious line states a transformer ICL test error bound that depends on model scale, training demonstrations, context length, and sequence length (Montgomery et al., 16 Oct 2025).

Framework	Predicted quantity	Main variables
Bayesian ICL	expected next-example / next-token probability	$n$, $P_{X,m}$, $p_m$, $K$
In-context regression	population risk $\mathcal L$	$t$, $N$, $L$, $P$
Context-aware downstream scaling	aggregate task performance $\mathcal P$	$P_{X,m}$0, $P_{X,m}$1, $P_{X,m}$2
Unified transformer ICL theory	ICL test error $P_{X,m}$3	$P_{X,m}$4, $P_{X,m}$5, $P_{X,m}$6, $P_{X,m}$7

This heterogeneity is substantive rather than merely notational. In the Bayesian line, in-context learning is approximate posterior inference over a finite task family. In the in-context regression line, a deep linear self-attention model learns an in-context optimizer. In the context-aware empirical line, context is treated as an inference-time resource that interacts multiplicatively with training compute. In the unified transformer line, ICL emergence is tied to architectural scale and task hierarchy (Arora et al., 2024).

2. Bayesian laws for scaling with demonstrations

A direct probabilistic account models ICL as Bayesian inference over latent tasks $P_{X,m}$8, with prior $P_{X,m}$9 and task-conditional likelihoods $p_m$0. Given a document $p_m$1, the posterior is

$p_m$2

and the expected next-example / next-token probability under $p_m$3 in-context examples is

$p_m$4

where $p_m$5 is the expected probability of a sample from task $p_m$6 under task $p_m$7, $p_m$8, and $p_m$9 is an ICL efficiency coefficient (Arora et al., 2024).

This law predicts saturation rather than indefinite improvement. As $K$0 grows, posterior mass concentrates on the task hypothesis best supported by the examples, so the curve approaches a task-conditional asymptote. That distinguishes it from unbounded power-law fits and makes prior-versus-evidence tradeoffs explicit. In this formalism, $K$1 controls the zero-shot or few-shot starting point, $K$2 controls curvature and asymptotes, and $K$3 controls how many effective Bayesian updates each nominal example contributes (Arora et al., 2024).

The framework is especially notable for controlled studies of many-shot jailbreaking. In synthetic GINC experiments, Bayesian variants matched or exceeded non-Bayesian baselines, and on extrapolation the scoring-wise Bayesian law achieved $K$4 NRMSE compared with about $K$5–$K$6 for the non-Bayesian baselines. In real-world instruction-tuned LLM experiments, Bayesian laws were competitive rather than uniformly dominant, with average NRMSE $K$7 for Bayesian sampling-wise and $K$8 for a logistic baseline. The substantive claim is therefore not only predictive accuracy, but interpretability: many-shot recovery of a suppressed behavior is modeled as likelihood evidence overcoming a prior disadvantage rather than as an opaque prompting effect (Arora et al., 2024).

3. Solvable in-context regression: depth, width, context, and time

A separate direct line studies in-context learning of linear regression in a deep linear self-attention model. A context contains $K$9 labeled examples and $\mathcal L$0 masked evaluation points, and the context loss is

$\mathcal L$1

Under alignment assumptions, the transformer reduces to a $\mathcal L$2-model in which the predictor acts like $\mathcal L$3 steps of preconditioned gradient descent on the in-context regression problem. The core resource variables are depth $\mathcal L$4, width $\mathcal L$5, context length $\mathcal L$6, batch size $\mathcal L$7, and pretraining time $\mathcal L$8, with compute

$\mathcal L$9

This makes width, depth, context, and training time distinct scaling axes rather than collapsing them into total parameter count (Bordelon et al., 1 Oct 2025).

The theory distinguishes three settings. In ISO, both covariates and tasks are isotropic; in FS, covariance structure is fixed across contexts; in RRS, covariances are randomly rotated across contexts. The difference is decisive. In ISO and FS, depth helps only when context is limited, because with large enough context a shallow model suffices or can memorize an effective preconditioner. In RRS, by contrast, covariance orientation changes across contexts, so the model cannot hard-code a universal anisotropic solver in its weights. Depth then remains useful even at infinite context length, because it is performing genuine iterative in-context computation rather than merely compensating for data scarcity (Bordelon et al., 1 Oct 2025).

In the RRS regime, the paper states a separable scaling law

$t$0

together with a compute-optimal shape law

$t$1

The depth exponent depends only on the source exponent $t$2, whereas the width and context exponents depend on $t$3. This is one of the clearest direct formulations of an in-context neural scaling law because the scaled task is itself ICL, the architectural variables include both width and depth, and the statistical variable $t$4 is the number of in-context examples (Bordelon et al., 1 Oct 2025).

4. Context-aware scaling and context-horizon laws

An empirical downstream line treats provided context as a first-class scaling variable. The proposed law for aggregate task performance is

$t$5

where the text describes the third factor as a penalty term for when $t$6. Here $t$7 is non-embedding training compute, $t$8 is prompt length, and $t$9 is the model’s context limit. The product structure encodes the claim that compute and context are complementary rather than additive, and the empirical results show low absolute prediction error on arithmetic reasoning, commonsense reasoning, and machine translation, together with reliable extrapolation to larger context (Montgomery et al., 16 Oct 2025).

This line is directly about downstream in-context performance rather than upstream language-model loss. It also makes task dependence explicit through fitted context exponents. Arithmetic reasoning yields $N$0, commonsense reasoning $N$1, and machine translation $N$2, which the paper interprets as different saturation regimes: arithmetic benefits from many demonstrations over a broader range, whereas commonsense reasoning and translation saturate quickly after the first few demonstrations (Montgomery et al., 16 Oct 2025).

A different but closely related theory derives scaling from language statistics rather than from prompt engineering. In the data-limited regime for autoregressive LLMs, the key quantities are pairwise token-correlation decay

$N$3

and next-token conditional-entropy decay

$N$4

These imply a data-dependent prediction horizon

$N$5

and, in the fast within-horizon learning regime,

$N$6

The same paper also predicts a scaling collapse for $N$7-gram losses,

$N$8

This is not a theory of few-shot ICL in the modern task-conditioning sense, but it is a direct theory of how usable context grows with data and how loss scales as longer effective context becomes exploitable (Cagnetta et al., 7 Feb 2026).

5. Unified transformer theories of ICL emergence

A more ambitious transformer-specific framework states that ICL test error satisfies

$N$9

with

$L$0

Here $L$1 is the hierarchy depth of the task family, $L$2 is the number of training demonstrations, $L$3 is context length, and $L$4 is an effective model-size proxy. The framework further states that transformers can implement gradient-based meta-learning in their forward pass, with effective learning rate

$L$5

and that ICL emergence follows a sigmoid-like transition with critical scale

$L$6

Within the same framework, the stated compute-allocation prescription is

$L$7

The central claim is therefore that model depth, width, context length, and training data participate in a common ICL scaling theory, and that the exponents are determined by task structure rather than being universal constants (Mehta et al., 9 Nov 2025).

The mechanistic core of that theory is constructive. For function classes learnable by $L$8-step gradient descent, the paper states that there exists a transformer with $L$9 and $P$0 such that forward-pass attention computes gradient-like updates and achieves error within $P$1 of the best $P$2-step learner. This gives a direct bridge between architectural depth and the number of implicit in-context adaptation steps (Mehta et al., 9 Nov 2025).

The same paper also reports synthetic-task evidence: measured model/data exponents close to the stated theory on linear regression, sparse linear regression, and decision trees; threshold-like emergence with critical scales that rise sharply with task hierarchy; and a strong depth advantage at fixed budget. At the same time, the paper details also note internal inconsistencies, including a mismatch between the stated $P$3 and the supporting algebra, a tension between the corollary for depth-width scaling and the approximation term used elsewhere, and a discrepancy between the tabulated theoretical $P$4 for $P$5 and the formula $P$6. The framework is therefore significant as an overview, but not yet fully settled in its mathematical details (Mehta et al., 9 Nov 2025).

6. Indirect foundations: duality, geometry, invariance, and data distribution

Several adjacent theories do not analyze ICL directly, but they supply reusable structures for thinking about in-context scaling. One field-theoretic random-feature regression model yields an exact planar result in which the optimal test loss scales as

$P$7

with a corresponding optimal ridge law

$P$8

and an explicit $P$9 duality between model size and sample count. In the noiseless case this duality forces identical scaling exponents with respect to $\mathcal P$0 and $\mathcal P$1; label noise breaks the symmetry. By analogy, this provides a clean capacity-versus-data template for in-context settings where context examples may play the role of temporary data and model parameters the role of stored capacity (Zhang, 2024).

A different adjacent line derives transformer scaling laws on low-dimensional manifolds. Under a manifold hypothesis with intrinsic dimension $\mathcal P$2 and $\mathcal P$3-Hölder targets, the paper states

$\mathcal P$4

This is not an ICL theorem, but it isolates an important geometric claim: scaling exponents are governed by intrinsic rather than ambient dimension. This suggests that heterogeneous in-context exponents across tasks may reflect different effective task or prompt manifolds rather than only different model sizes (Havrilla et al., 2024).

A third adjacent line studies invariance under data transformations. It states that bijective transformations preserve mutual information,

$\mathcal P$5

and therefore preserve scaling exponents, whereas non-bijective transformations change the law through an information-resolution variable

$\mathcal P$6

yielding

$\mathcal P$7

This is a training-time result rather than an ICL result. Still, it suggests that reversible prompt re-encodings could preserve in-context exponents, whereas truncation, lossy retrieval, or prompt compression could alter them through a context-resolution analogue of $\mathcal P$8 (Han et al., 8 May 2026).

A fourth adjacent theory roots scaling laws in a percolation model of the data distribution. It derives one regime with power-law-distributed discrete subtasks and another with a dominant manifold, and explicitly remarks that clustered data with many rare classes can drive emergent in-context learning in transformers. This suggests two possible ICL regimes: a retrieval-like regime over heavy-tailed latent subtasks, and a local interpolation regime over a connected task manifold. The paper stops short of a direct prompt-length theorem, but it makes the dependence of scaling exponents on data-distribution structure explicit (Brill, 2024).

7. Limitations and unresolved questions

The current state of in-context neural scaling laws is structurally plural rather than unified. Some theories are exact but highly stylized: deep linear self-attention on in-context regression, Gaussian teacher-student models, or random-feature kernels. Some are directly about prompt-conditioned behavior but only at the level of empirical curve fitting. Some are mechanistic but presently contain unresolved internal inconsistencies. Some are rigorous transformer scaling theories that are adjacent to ICL rather than directly about it (Bordelon et al., 1 Oct 2025).

A first limitation is task scope. Direct theories often analyze regression, synthetic latent task families, or stylized prompts rather than open-ended autoregressive language modeling. A second is architectural scope. Several results depend on linear attention, random features, NTK-style conditioning, or width proxies such as $\mathcal P$9 rather than on full transformer parameterization. A third is statistical scope. Most theories assume i.i.d. demonstrations, latent task mixtures, or smooth structured tasks, whereas real prompts are often noisy, non-i.i.d., instruction-like, out-of-distribution, or adversarial. A fourth is the distinction between context use and in-context learning proper: some of the strongest context-horizon results concern autoregressive conditioning rather than few-shot task adaptation (Arora et al., 2024).

The central unresolved question is whether a single law can jointly account for parametric knowledge, prompt-conditioned adaptation, usable context horizon, retrieval or external memory, and test-time compute. The current literature supports several narrower conclusions: posterior-style evidence accumulation yields saturating demonstration curves; depth and width are not interchangeable in ICL; context and training compute interact rather than add; language statistics can determine how effective context grows with data; and data geometry or information preservation can alter exponents in principled ways. What remains missing is a broadly validated theory that simultaneously treats sequence structure, prompt semantics, transformer internals, and deployment-time interventions such as retrieval, summarization, and many-shot prompting (Montgomery et al., 16 Oct 2025).