Inverted-U Law in System Performance

• The inverted-U law is a non-linear phenomenon where performance or activation initially rises with capacity, peaks, then declines before potentially recovering.
• Empirical studies in LLMs and brain imaging reveal distinct curve patterns, with easy tasks showing inverted-U trends and hard tasks exhibiting U-shaped scaling.
• Methodologies such as polynomial fits and generalized additive models quantify curvature, enabling forecasting of emergent behaviors in complex systems.

The inverted-U law refers to a robust, recurring nonlinearity in the relationship between system performance and an underlying control variable—such as model size in artificial neural networks or brain activation in cognitive neuroscience. Empirically, this law manifests as performance or activation initially increasing with greater capacity or engagement, then peaking and subsequently decreasing, before potentially recovering again at larger scale or proficiency. Contrasting U-shaped and inverted-U scaling trends in complementary subgroups often underlie the aggregate emergence phenomenon in complex systems such as LLMs and the human brain.

1. Mathematical Formalization of Inverted-U and U-Shaped Scaling

Operationalization of the inverted-U law centers upon fitting performance or neural activation as a function of a control parameter, with the curvature sign determining the qualitative shape:

• In LLMs, downstream performance $P$—measured by accuracy or a continuous metric such as the binary Brier Score—is plotted against effective model size $M = \log_{10}(C/10^{21})$, where $C \approx 6ND$ denotes training FLOPs for $N$ tokens and $D$ parameters (Wu et al., 2024).
• In brain–behavior mapping, regional activation $A_i$ is modeled as a function of task performance $P$ (quantified via the Balanced Integration Score; see Section 3), with curvature assessed via penalized regression splines or (in simple cases) quadratic terms:

$A_i(P) = \alpha_i P^2 + \beta_i P + \gamma_i$

The sign of $\alpha_i$ distinguishes inverted-U ($\alpha_i < 0$) from U-shaped curves ($M = \log_{10}(C/10^{21})$0) (Cao et al., 16 Oct 2025).

In LLM research, low-degree polynomials are fit:

• Easy group: $M = \log_{10}(C/10^{21})$1 (odd degree, accommodates local maxima and subsequent ascent)
• Hard group: $M = \log_{10}(C/10^{21})$2 (quadratic, to capture local minima) No closed-form universal inverted-U law is posited; rather, curve shapes are determined empirically within difficulty strata.

2. Empirical Methodologies and Curve Discovery

LLMs

To decompose observed emergent abilities, downstream questions are grouped by intrinsic difficulty, quantified as follows (Wu et al., 2024):

• For each question $M = \log_{10}(C/10^{21})$3, a difficulty score $M = \log_{10}(C/10^{21})$4 is the average binary-Brier performance across models below a threshold $M = \log_{10}(C/10^{21})$5.
• Questions are sorted by $M = \log_{10}(C/10^{21})$6 and split into $M = \log_{10}(C/10^{21})$7 bins (commonly $M = \log_{10}(C/10^{21})$8 for analysis), creating difficulty stratifications.

For each group, mean performance is plotted across $M = \log_{10}(C/10^{21})$9, revealing:

• U-shaped scaling for hard bins: initial decrement followed by steady ascent beyond a valley.
• Inverted-U scaling for easy bins: initial improvement, then degradation, with a second ascent at larger-scale $C \approx 6ND$0.

Brain Activation

In cognitive neuroscience, the non-linear activation–performance link is captured by generalized additive models (GAMs):

$C \approx 6ND$1

where $C \approx 6ND$2 is a penalized spline of performance $C \approx 6ND$3 (BIS); $C \approx 6ND$4 indexes cortical parcels (Cao et al., 16 Oct 2025). Nonlinearity classification utilizes the mean second derivative $C \approx 6ND$5 of $C \approx 6ND$6, designating concave (inverted-U) for $C \approx 6ND$7, convex (U-shaped) for $C \approx 6ND$8, and linear if $C \approx 6ND$9.

3. Mechanistic and Theoretical Explanations

LLM Double Descent Analogy

The inverted-U scaling on easy tasks in LLMs echoes the double descent phenomenon (Wu et al., 2024):

• Classical regime: Small $N$0, bias dominates, performance rises.
• Intermediate regime: Model capacity increases but underfits, variance rises, performance drops—a trough.
• Modern/interpolating regime: Model size sufficient to fit patterns, performance resumes improvement.

For hard questions, small models guess at random, medium models become distracted by partial motifs, and performance dips; only at larger $N$1 does recovery and understanding occur—resulting in a U-shape.

Non-linear Brain Response Across Macroscale Gradients

In the brain, a gradient from sensorimotor to association cortex determines the nonlinearity of activation–performance curves (Cao et al., 16 Oct 2025):

• Low-order regions: U-shaped responses (activation dips for intermediate proficiency, rises again at high performance).
• High-order association cortices: Inverted-U curves (activation rises with proficiency but drops off for the most proficient—supposedly due to cognitive automation or reduced effort).
• The spatial transition from U to inverted-U aligns with the sensorimotor–association (S–A) cortical axis.

4. Forecasting and Predictive Utility (Slice-and-Sandwich Approach)

The slice-and-sandwich pipeline provides a method for forecasting emergent performance and its threshold in LLMs (Wu et al., 2024):

1. Slice: Rank questions by difficulty and divide into easy, medium, and hard groups.
2. Fit: Model the metric of interest (e.g., binary Brier) as a function of $N$2 via degree-5 polynomial for easy, degree-2 for hard, using only $N$3 data.
3. Sandwich: Aggregate forecast $N$4 for $N$5 calculated as the mean of the easy and hard fits:

$N$6

1. Project: Map $N$7 to accuracy via a fitted linear map on $N$8 data.

This framework anticipates the so-called emergent jump in coarse metrics by capturing the intersection of rising trends in both easy and hard strata.

5. Empirical Findings and Regional Characterization

LLMs: Emergence as Aggregate Artifact

The apparent stagnation followed by a sharp performance gain at threshold $N$9 in overall metrics is an aggregate effect: early improvements on easy questions offset declines on hard ones, creating a plateau beforehand and a joint surge when both types are on the upswing. This outcome cautions against over-interpreting “emergence” seen in aggregate metrics (Wu et al., 2024).

Brain Imaging: Regional Nonlinearities

GAM-based models reveal that across the cortex:

• Face working memory activates a broad S–A axis, yielding a shift from U-shaped (μ'' > 0) in sensory parcels to inverted-U (μ'' < 0) in high-rank association parcels.
• In the place condition, activation remains concentrated in low-order regions, and only U-shaped curves emerge.

In face trials, peak-of-curve values $D$0 in concave (association) regions cluster above mean proficiency; in convex (sensory) regions (place trials), peaks occur below average performance. The curvature-to-S–A gradient correlation—$D$1 (p<0.001) for faces—demonstrates systematization of the inverted-U law across large-scale cortex (Cao et al., 16 Oct 2025).

Parcel Classification Summary (FDR-corrected, face/place tasks)

Condition	Linear Increase	Linear Decrease	Inverted-U (Concave)	U-shaped (Convex)	Other Nonlinear
Faces (n=91)	46	10	28	3	4
Places (n=101)	41	9	1	27	23

Key sensory ROIs (e.g., V1, V2, V3A) and high-order association regions (e.g., d32, 8C, 9-46d) were identified as canonical examples.

6. Implications for Research and Practice

• Apparent sudden emergent abilities in LLMs may represent artifacts of opposing nonlinear trends in stratified data. Leveraging difficulty-aware continuous metrics and subgroup analyses exposes smooth progress and enables pre-emergence forecasting (Wu et al., 2024).
• In systems neuroscience, the inverted-U law is operationalized as negative curvature in activation–performance relationships, and its systematic mapping to macroscale cortical gradients (S–A axis) supports functional specialization hypotheses: association cortices display effortful engagement then automation, while sensory cortices exhibit baseline encoding demands and fine-tuning at expert proficiency (Cao et al., 16 Oct 2025).
• Practical guidelines for engineering or anticipating emergent properties in machine learning include: designing sensitive evaluation metrics, stratifying by intrinsic difficulty, tracking subgroup trends, and employing forecasting pipelines such as slice-and-sandwich rather than naive sigmoidal curve fitting.

7. Limitations and Generalization

No unified parametric law yet governs the inverted-U in either field; both LLM and neuroscience studies rely on empirical regression for curve discovery, and the specificity of the results is tied to chosen metrics, tasks, sample characteristics, and stratification heuristics. In the brain, the robustness of S–A shifts is context-dependent (emerging for faces but not places); generalization to other stimulus domains or populations requires further study. The interpretability of model coefficients in spline-based approaches is limited compared to parametric forms, and physiological mechanisms underlying observed nonlinearities remain open for further research (Wu et al., 2024, Cao et al., 16 Oct 2025).