Exponential Progress Curves

• Exponential progress curves are mathematical models that describe performance improvements using exponential functions, incorporating key laws such as Wright’s experience curve and Moore’s law.
• Methodologies involve time-series regression and first-difference models to statistically estimate forecast errors and construct predictive intervals with validated empirical metrics.
• Applications span diverse fields like solar photovoltaics and deep reinforcement learning, providing actionable insights into cost reductions and enhanced sample efficiency.

Exponential progress curves describe the phenomenon wherein key technological, scientific, or algorithmic performance metrics improve at a rate that can be modeled by an exponential function of time or experience. These curves are empirical regularities in domains such as technological cost reduction and sample efficiency in machine learning, notably deep reinforcement learning (DRL). The concept underlies widely cited laws including Wright’s experience curve and “Moore’s law” type time-trend models, and forms the basis for quantitative forecasting frameworks validated on diverse technologies and algorithmic benchmarks.

1. Fundamental Mathematical Formulations

Exponential progress can be described by two principal empirical laws.

Wright’s Experience Curve Model relates reductions in unit cost $C(t)$ to cumulative production $E(t)$, assuming constant elasticity:

$C(t) = C_0 \cdot [E(t)]^{-b}$

Where $b>0$ is the “learning” or “experience” exponent, so that doubling cumulative production reduces cost by a factor $2^{-b}$. In log-linearized form:

$$y_t = \ln C(t) = \ln C_0 - b x_t \qquad x_t = \ln E(t)$$

Exponential (Moore’s Law) Progress Model models cost as decaying exponentially in calendar time:

$C(t) = C_0 \cdot \exp(-k t)$

Where $k>0$ is the exponential decline rate. In logarithmic coordinates:

$y_t = \ln C(t) = \ln C_0 - k t$

These models are often statistically indistinguishable when cumulative production itself grows approximately exponentially over time, with the experience curve parameters mapping onto the exponential time-trend via $k \approx b r$, with $r$ the logarithmic growth rate of experience (Lafond et al., 2017).

2. Statistical Estimation and Forecasting Frameworks

Estimation typically proceeds by time-series regression, most effectively using first-difference models to address nonstationarity and autocorrelation. For Wright’s law:

$$\Delta y_t = -b \Delta x_t + \eta_t,\quad \eta_t \sim N(0,\sigma_\eta^2)$$

and for Moore’s law:

$\Delta y_t = -k + n_t,\quad n_t \sim N(0,K^2)$

Distributional forecasting under Wright’s law involves analytically deriving the variance of $\tau$-step-ahead forecast errors, accounting for observation noise and parameter uncertainty. For a constant experience growth rate, the variance is:

$$\text{Var}[\mathcal{E}_\tau] = \sigma_\eta^2 [\tau + \tau^2/m]$$

where $m$ is the size of the estimation window. Forecasts for $\ln C_{t+\tau}$ are thus Student-t distributed, enabling construction of full predictive intervals (Lafond et al., 2017).

Autocorrelation extensions utilize moving average noise models (MA(1)), further refining interval calibration via analytic variance formulas.

3. Empirical Validation Across Technological Domains

Large-scale panel tests have been performed across 51 heterogeneous technologies spanning chemicals, hardware, energy, and consumer durables (Lafond et al., 2017). Each technology exhibits declines in cost tracked by both experience-curve and time-trend models with nearly identical point forecasts, owing to the prevalence of exponential production growth.

Key metrics for empirical validation include:

• Mean squared forecast error normalized by theoretical predictive variance
• Interval coverage (68%, 95%)
• Probability integral transform (PIT) calibration
• Surrogate-data Monte Carlo tests using synthetic time series from the fitted MA(1) model

Findings establish robust agreement between theoretical forecast distributions and pooled normalized errors, with experience-curve methods yielding marginal improvements only in scenarios of more volatile production growth.

4. Application to Solar Photovoltaic Modules

Analysis of 41 years of PV module prices demonstrates convergence between experience-curve and Moore’s-law based predictions (Lafond et al., 2017). For annual decline rate, estimates are:

• Exponential model: $\hat{\mu} = -0.121$/year (12.1%/yr)
• Experience curve: $\hat{r} = 0.318$/year (32%/yr production growth), $\hat{b} = 0.380$ so that $\hat{b}\hat{r} = 0.121$

Forecasting forward, both models deliver median predicted price declines of 10–15% per annum, with 95% prediction intervals spanning ±30–40%, and nearly coincident point and interval bounds under business-as-usual exponential growth.

5. Exponential Progress in Deep Reinforcement Learning Sample Efficiency

Dorner et al. identify exponential improvement in sample efficiency for DRL algorithms across standard benchmarks, quantifying the number of environment samples $S(t)$ required to reach predefined performance thresholds as a function of publication date:

$S_{\mathrm{SOTA}}(t) = S_0 \cdot 2^{t/T_d}$

where $T_d$ is the "doubling time"—the period over which the sample requirements halve. Linear regression of $\log_2 S_i$ vs $t_i$ across state-of-the-art algorithms produces robust $T_d$ estimates (Dorner, 2021).

Reported doubling times for sample efficiency are:

Benchmark	Doubling Time $T_d$
Atari (DQN, DDQN, Dueling DQN, C51 baselines)	10–18 months
State-based continuous control (Gym/rllab)	5–24 months
Pixel-based continuous control (DM Control)	4–9 months

For each benchmark, the authors trace SOTA improvements via efficient extraction of training curve thresholds, fit exponential models, and assess goodness-of-fit visually via log-scaled plots. Shortest doubling times are found in pixel-based continuous control tasks.

6. Model Limitations, Caveats, and Policy Implications

Several important limitations apply:

• Predictive accuracy relies on the assumption of exponential growth in cumulative experience or production; deviations may degrade model performance.
• Experience-curve models are preferable for policy-driven scenario analysis, e.g., when quantifying cost impact of altered deployment trajectories.
• Small estimation windows inflate uncertainty in predictive intervals.
• Possible omitted variable bias (R&D intensity, input costs, scale economies) and reverse causality can affect parameter estimation.
• In algorithmic settings (DRL), the reliance on published training curves introduces measurement noise and bias; sample counts may not incorporate all development iterations (Dorner, 2021).

A plausible implication is that exponential models suffice for routine, steady-growth forecasting but experience-based models add value by capturing the effects of policy or structural deviations.

7. Relationship Between Experience Curves and Pure Time-Trend Models

A key insight is the observational equivalence between experience curves and exponential time-trend models under exponential production growth conditions. The mapping $k \approx b r$ ensures that both frameworks produce nearly identical point and interval forecasts, so long as cumulative production is locally exponential (Lafond et al., 2017).

This suggests that for technologies and domains exhibiting sustained, exponential scale-up, either model may be used interchangeably for business-as-usual forecasts. However, explicit distributional forecasting under Wright’s law improves uncertainty quantification and offers a principled mechanism for scenario analysis when growth rates shift.

• Lafond et al. “How well do experience curves predict technological progress? A method for making distributional forecasts” (Lafond et al., 2017).
• Dorner et al. “Measuring Progress in Deep Reinforcement Learning Sample Efficiency” (Dorner, 2021).