Epoch Capabilities Index Framework

• Epoch Capabilities Index is a unified statistical framework that quantifies AI model capabilities and benchmark challenges using latent-variable inference.
• It enables temporal alignment and forecasting of AI progress by mapping state-of-the-art models on a common capability scale.
• The framework facilitates cross-era and cross-model comparisons by stitching non-overlapping benchmarks into a continuous progress trajectory.

The Epoch Capabilities Index refers to a principled, statistical framework for quantifying, aligning, and forecasting the evolution of AI model capabilities across diverse benchmarks and time periods. It provides a single numerical scale on which both model “capability” and benchmark “difficulty” co-exist, enabling cross-era, cross-benchmark, and cross-model comparability—even for models assessed on non-overlapping benchmarks. The construct is not tied to any fixed set of benchmarks and does not presuppose any particular scaling laws of capability with respect to compute or time. Grounded in formal latent-variable inference, this framework enables rigorous analyses of AI progress, algorithmic efficiency, and system-wide accelerations in an era when traditional benchmarks rapidly saturate or fragment (Ho et al., 28 Nov 2025).

1. Statistical Framework and Latent Variable Model

Central to the Epoch Capabilities Index is a latent trait model that jointly infers model capabilities and benchmark difficulties. For each AI model $m$, a single “capability” parameter $C_m$ is assigned; for each benchmark $b$, both a “difficulty” $D_b$ and a discrimination (slope) parameter $\alpha_b$ are inferred. The observed score for a model-benchmark pair is modeled as

$$\mathrm{score}(m, b) = \sigma(\alpha_b (C_m - D_b)), \quad \sigma(x) = \frac{1}{1+e^{-x}},$$

where $\sigma$ denotes the logistic sigmoid function. This IRT-like (Item Response Theory) framework allows one to treat model performances as samples from a common probabilistic process, with $C_m$ and $D_b$ existing on the same latent numerical axis. Parameter fitting proceeds via regularized least-squares minimization over all observed (model, benchmark) pairs:

$$\mathcal{L} = \sum_{m, b} [\mathrm{score}(m,b) - \sigma(\alpha_b(C_m - D_b))]^2 + \lambda \sum (C_m^2 + D_b^2 + \alpha_b^2),$$

with a fixed reference benchmark for identifiability. The approach leverages all available score data, regardless of benchmarking overlap or era, and does not require any assumptions about time or compute scaling during inference (Ho et al., 28 Nov 2025).

2. Temporal Alignment and Forecasting Capabilities

After fitting, each model’s inferred capability $C_m$ is attached to its real-world release date $t_m$, yielding a temporally anchored capability trajectory. The “frontier” series—composed of models that were state-of-the-art when released—exhibits an approximately linear growth:

$C_{\mathrm{frontier}}(t) \approx C_0 + r \cdot t,$

with $r \approx 0.55$ capability units per year (95% CI: 0.45–0.67). This linearity emerges empirically from the data and is not imposed. Gaps between major models (e.g., the jump from GPT-4 to GPT-5-high, ≈1.05 units) can be compared directly, and rates of progress (such as ~0.55 units per year) can be mapped to human time-horizon equivalence using regressions such as

$\text{time-horizon} \simeq \exp(3.69 C_m-4.58).$

On this mapping, each 0.55 unit increase in capability corresponds to a doubling of human task time-horizons roughly every 4 months. Simple extrapolations indicate advances of 1.35–1.99 capability units within three years, with error bands reflecting the variability in past progress rates (Ho et al., 28 Nov 2025).

3. Benchmark Stitching and Cross-Era Comparisons

The critical innovation of the Epoch Capabilities Index is its ability to “stitch” together benchmark results with uneven release times and limited overlap. Because all scores $s_{m, b}$ conform to the same latent-variable generative process, retrospective comparison is possible even for models never co-evaluated. After scale-fitting, one can reconstruct unified capability time series, detect regressions or accelerations, and directly compare contemporary models to historical baselines. Empirical R² of 0.86–0.87 under the sigmoid model confirms high fidelity in capturing model-vs-benchmark relations, with alternative models (e.g., piecewise linear) performing similarly (Ho et al., 28 Nov 2025).

4. Applications: Progress Quantification, Efficiency Gains, and Acceleration Detection

The model operationalizes several key analyses:

• AI progress measurement: Frontier capabilities climb at $r=0.55$ units per year, mapping to concrete “model gaps” (e.g., GPT-4 to GPT-5-high ≈1.05 units; GPT-4 to Claude 3.5 Sonnet ≈0.21 units).
• Algorithmic efficiency estimation: For model $m$ with total training FLOPs $F_m$, capabilities are fit as $C_m = k \log F_m + b_m$. Within a family, $k \approx 0.168$. Yearly algorithmic gains ($\Delta b \approx 0.297$ units/year) yield annual compute reductions of $\exp(\Delta b/k) \approx 6\times$ (95% CI: 3×–40×), and even with compute fixed, capabilities improve at $\approx 0.3$ units/year.
• Acceleration detection: Piecewise linear fits of $C_{\mathrm{frontier}}(t)$ identify epochs of acceleration. Post-April 2024, the slope doubles ($0.352 \rightarrow 0.689$, ratio ≈1.95), closely mirroring time-horizon acceleration detected in concurrent studies.

Synthetic experiments confirm the statistical power to detect such accelerations within months, given realistic noise and model coverage (Ho et al., 28 Nov 2025).

5. Integration with Broader Benchmarking Ecosystems

The Epoch Capabilities Index creates a “Rosetta Stone” for inter-benchmark and inter-epoch evaluation. It aligns with methodologies emphasizing end-to-end scenario fidelity, such as AIBench, which builds domain-specific benchmarks, enforces quality-ensured throughput and latency constraints, and can be mapped to other benchmarks via normalized throughput and latency metrics (Gao et al., 2020). Cross-stack benchmarks like AIPerf extend this translation, offering analytically derived hardware-agnostic metrics (operations per second) and workload-driven scaling, crucial for comparisons between HPC and AI systems (Ren et al., 2020). Next-generation “live” benchmarking paradigms (e.g., PeerBench) propose dynamic, proctored, and cohort-normalized protocols for rolling benchmark renewal and composite scoring, potentially providing high-fidelity streams of measurement to feed into the Epoch Capabilities Index (Cheng et al., 8 Oct 2025).

6. Model and Benchmark Score Table

An example of inferred capability scores on the unified scale is as follows:

Model	Release Date	Capability ($C_m$)
GPT-5-high	2025	2.65
o3-high	2024.5	2.51
Claude 3.5 Sonnet	2024.5	1.81
GPT-4 (Mar 2023)	2023.25	1.60

This ordering and spacing permit precise gap analysis, cumulative progress indexing, and horizon forecastings on an absolute scale (Ho et al., 28 Nov 2025).

7. Limitations and Role in Future Evaluation Protocols

The Epoch Capabilities Index depends critically on benchmark quality, calibration, and resistance to overfitting or contamination. Fragmented, non-representative, or strategically gamed benchmarks distort the inferred scale. Proposals such as PeerBench advocate for sealed execution, rolling item banks, and reputation-weighted scoring to ensure the provenance and statistical validity required by the Index (Cheng et al., 8 Oct 2025). Broader adoption of modular, scenario-driven, and hardware-agnostic benchmarking frameworks (e.g., AIBench, AIPerf) is essential to maintain the robustness and cross-domain relevance of the capability scale (Gao et al., 2020, Ren et al., 2020). There is no explicit assumption or enforcement of monotonic improvement; regressions or decelerations are detectable empirically. The Index does not prescribe, but only diagnoses, trends in capability progression.

---

The Epoch Capabilities Index operationalizes a unified, continually updated quantitative scale for AI model capabilities across arbitrary time spans and ever-evolving benchmark suites. By co-inferring latent capabilities and difficulties, it provides a rigorous foundation for measuring, forecasting, and interpreting the trajectory of AI systems in a rapidly growing and diversifying ecosystem (Ho et al., 28 Nov 2025, Cheng et al., 8 Oct 2025, Gao et al., 2020, Ren et al., 2020).