Algorithmic Moore’s Law

• Algorithmic Moore’s Law is defined as the exponential reduction in computational resources required for fixed performance, leveraging key algorithmic innovations.
• Empirical evidence shows that compute requirements for neural networks and language models halve every 8–16 months, significantly outpacing hardware improvements.
• Statistical and co-design models validate that integrating software innovations with hardware scaling yields transformative gains in cost, energy, and performance.

Algorithmic Moore’s Law refers to the empirically observed, quantifiable trend in which algorithmic and computational efficiency improve at an exponential—or in some metrics, super-exponential—rate, analogous to the classic hardware-driven Moore’s Law for semiconductor technology. Unlike physical transistor scaling, Algorithmic Moore’s Law captures progress in engineering, mathematics, software, and systems design that multiplies or supersedes hardware improvements, with measurable impact across cost, energy, data, and performance for a broad class of computational tasks.

1. Formal Definitions and Models

Algorithmic Moore’s Law is generally expressed through exponential or sub-exponential reductions in computational resources (such as cost, energy, or floating-point operations) required to achieve a fixed level of performance over time. The archetypal formulation is found in (1207.1463), where Moore’s Law describes unit cost as a function of time: $y_t = B \cdot \exp(-mt)$ or in logarithmic form,

$\log y_t = -mt + \log B + n(t)$

with $y_t$ representing inflation-adjusted unit cost, $B, m > 0$, and $n(t)$ an additive noise term.

Algorithmic analogues focus on metrics like the compute needed to reach a threshold accuracy, with doubling times as the key parameter:

• For neural networks, the compute required for AlexNet-level accuracy fell 44× between 2012–2019, corresponding to a doubling of algorithmic efficiency every 16 months (Hernandez et al., 2020).
• In LLMs, effective training compute halves every 8–9 months (Ho et al., 9 Mar 2024), outpacing hardware improvements per traditional Moore’s Law.

This time-based exponential decay is equivalent, under certain conditions (exponential production or usage growth), to a power-law relationship in input variables due to the Sahal equivalence (1207.1463); thus, exponential algorithmic progress is often indistinguishable from learning-by-doing effects in practice.

2. Quantitative Evidence Across Domains

Empirical analyses across diverse domains substantiate Algorithmic Moore’s Law as a distinct driver of technological and scientific progress:

• In computer vision, algorithmic improvements halve compute needs every 9 months, roughly twice the rate of hardware-driven progress (Erdil et al., 2022).
• For LLMs, effective compute requirements for a target performance level halve approximately every 8 months (95% CI: 5–14 months) (Ho et al., 9 Mar 2024).
• In neural network training (ImageNet), efficiency doubled every 16 months (7-year span); the total improvement due to hardware scaling alone would have been only 11×, compared to the observed 44× reduction driven by algorithms (Hernandez et al., 2020).
• Across Chess, Go, weather prediction, protein folding, and oil exploration, performance improvements are overwhelmingly explained by increases in computing power (49–94%) (Thompson et al., 2022). However, for linear improvements in outcome (e.g., Elo points, reduced forecast error), exponential increases in computing power are needed, making algorithmic efficiency a necessity as hardware scaling slows.

A comparative summary:

Domain	Halving Time for Compute	Main Performance Metric	Reference
ImageNet CV	9 months	Top-1 accuracy	(Erdil et al., 2022)
LLMs	8–9 months	Fixed cross-entropy loss	(Ho et al., 9 Mar 2024)
Neural Nets (ImageNet)	16 months	AlexNet-level accuracy	(Hernandez et al., 2020)

Algorithmic advances (e.g., the introduction of transformer architectures) can result in compute-equivalent gains of 7× or greater—effectively substituting years of progress in a single innovation (Ho et al., 9 Mar 2024).

3. Theoretical and Statistical Frameworks

The characterization of Algorithmic Moore’s Law leverages formal statistical models:

• Generalized Moore’s Law as a correlated geometric random walk with drift, $y_t = y_{t-1} + \mu + n_t$, captures the stochastic process of cost or efficiency improvement, where $\mu$ is the drift rate and $n_t$ are uncorrelated or autocorrelated noise increments (Farmer et al., 2015).
• Forecasting errors for technological progress follow closed-form distributions (e.g., Student $t$-distribution for normalized errors), enabling rigorous quantification of forecast reliability and the statistical significance of trends (Farmer et al., 2015).
• In the decomposition of progress, models attribute proportions of performance gain to algorithmic, compute, and data innovations (via Shapley values), systematically quantifying the algorithmic share (Erdil et al., 2022, Ho et al., 9 Mar 2024).

Moreover, the effect of self-organization in complex systems can be mathematically related to Algorithmic Moore’s Law. Here, key system properties (transistor count, energetics, action efficiency) exhibit mutual exponential growth, modeled by coupled matrix differential equations (Georgiev et al., 2016). These relationships result in empirically observed power laws and reinforce the conceptual unity of exponential hardware and algorithmic improvement.

4. Algorithmic Mechanisms and Practical Strategies

The core mechanisms underlying Algorithmic Moore’s Law include:

• Algorithmic innovations that reduce the compute or energy needed for a fixed result (“compute-augmenting” advances); e.g., batch normalization, more efficient architectures, improved optimizers, and sparsity techniques (Hernandez et al., 2020, Erdil et al., 2022).
• Precision modulation and inexactness: Lowering arithmetic precision for non-critical parts of computations and reinvesting saved energy in higher-accuracy phases, thereby improving overall algorithmic efficiency under fixed energy budgets (Leyffer et al., 2016, Augustine et al., 2017).
• Tailored energy allocation by exploiting problem asymmetry: Prioritizing resource allocation for input bits or operations with greater impact, especially effective for asymmetric problems (Augustine et al., 2017).
• Hardware–software co-design: Integrated development cycles where hardware is specifically optimized for the dominant algorithms and vice versa, mitigating the “hardware lottery” and accelerating progress in emerging domains (e.g., generative AI) (Yazdanbakhsh, 9 Apr 2025).

The multiplicative effect of hardware and algorithmic improvements is crucial: gains in algorithmic efficiency multiply hardware-based speedups, resulting in super-exponential progress when both scale concurrently (Hernandez et al., 2020).

5. Limitations, Slowdown, and Generalizations

While Algorithmic Moore’s Law has held for several decades, evidence suggests that exponential rates are not sustained indefinitely:

• Quantitative modifications propose models where growth gradually slows, with performance following a logarithmic integral ($li(t) \sim t/\ln t$) rather than pure exponential doubling (Zhang, 2022).
• As physical scaling approaches hard limits (e.g., set by the Landauer bound, quantum tunneling, and the Compton wavelength) (Kumar, 2015), further algorithmic improvements must compensate for hardware slowdowns, but the rate of improvement is expected to “decelerate.”
• The effectiveness of algorithmic inexactness, for instance, is constrained by the underlying problem symmetry; only asymmetric problems allow non-uniform resource allocation to yield dramatic improvements (Augustine et al., 2017).

Broader metrics, such as the holistic CLEAR metric (Capability-to-Latency-Energy-Amount-Resistance), have been proposed as more robust measures of technological and algorithmic system value—subsuming both classic hardware scaling and algorithmic performance (Sun et al., 2016).

6. Implications for Technology, Policy, and Future Paradigms

Algorithmic Moore’s Law has led to implications across theory, practice, and policy:

• Societal Impact: Algorithmic improvements have multiplied the effect of hardware scaling, enabling transformative applications in AI, computational science, and industry despite stagnating transistor scaling (Conte et al., 2021, Thompson et al., 2022).
• Forecasting and Planning: Predictability and the statistical basis for future forecasting (with quantifiable error bars) give stakeholders tools for scenario planning—critical for domains like climate mitigation policy and national computing infrastructure (1207.1463, Hager et al., 2020).
• Co-Design and Interdisciplinary Development: The convergence of hardware and software design paradigms, especially for AI and high-performance computing, is now recognized as essential for sustaining exponential progress (Yazdanbakhsh, 9 Apr 2025, Friedman et al., 2021).
• Quantum Computing & New Metrics: For quantum computers, no direct analogue of Moore’s Law applies; progress must be measured through more complex, multi-factor metrics (qubit fidelity, quantum volume, system integration), and algorithmic advances in error correction and application methods represent a key lever for further acceleration (Ezratty, 2023).

Theoretical perspectives such as the Feynman Mandate advocate moving beyond single-metric exponential progress (e.g., transistor count) to encompassing efficiency, energy, and economic cost as the foundational metrics for both hardware and algorithmic contributions (Friedman et al., 2021).

7. Synthesis: The Nature and Limits of Algorithmic Moore’s Law

Algorithmic Moore’s Law represents a measurable, statistical regularity in which algorithmic advancements deliver exponential (or sub-exponential) improvements in the cost, performance, and energy efficiency of computation. Distinct from, but often multiplicative with, hardware advances, these trends have become central as traditional physical scaling reaches its limits.

However, empirical and theoretical evidence indicates that the pace of algorithmic improvement may also slow—either due to saturation of readily accessible innovation, fundamental limits of computational complexity, or the compounding difficulty of optimizing across increasingly complex, co-designed hardware-software stacks. Future trajectories depend on continued interdisciplinary innovation, robust forecasting frameworks, and flexibility in metric selection to capture meaningful progress in a post-Moore era.