Classical and Quantum Speedups for Non-Convex Optimization via Energy Conserving Descent

Abstract: The Energy Conserving Descent (ECD) algorithm was recently proposed (De Luca & Silverstein, 2022) as a global non-convex optimization method. Unlike gradient descent, appropriately configured ECD dynamics escape strict local minima and converge to a global minimum, making it appealing for machine learning optimization. We present the first analytical study of ECD, focusing on the one-dimensional setting for this first installment. We formalize a stochastic ECD dynamics (sECD) with energy-preserving noise, as well as a quantum analog of the ECD Hamiltonian (qECD), providing the foundation for a quantum algorithm through Hamiltonian simulation. For positive double-well objectives, we compute the expected hitting time from a local to the global minimum. We prove that both sECD and qECD yield exponential speedup over respective gradient descent baselines--stochastic gradient descent and its quantization. For objectives with tall barriers, qECD achieves a further speedup over sECD.

• The paper demonstrates that ECD, by conserving a modified energy, enables trajectories to escape local minima efficiently, yielding polynomial escape times versus the exponential delays seen in SGD.
• The study extends ECD to stochastic (sECD) and quantum (qECD) frameworks, providing exact analytical expressions for escape (hitting) times in double-well potential landscapes.
• Quantum ECD exhibits a robust speedup with a 1/V₀ scaling in tall-barrier regimes, significantly outperforming classical methods and underscoring its potential for global optimization.

Classical and Quantum Speedups in Non-Convex Optimization via Energy Conserving Descent

Introduction and Motivation

This paper presents the first analytical investigation of Energy Conserving Descent (ECD), an optimization method inspired by Hamiltonian dynamics, in the context of one-dimensional non-convex optimization, with an emphasis on double-well potentials. ECD offers an alternative to traditional gradient-based optimizers by introducing a dynamics that conserves a modified energy and, crucially, exhibits a mechanism for escaping from strict local minima. The study extends to stochastic (sECD) and quantum (qECD) variants, aiming to characterize and compare classical and quantum speedups in escaping local minima, and benchmarks these against established methods such as stochastic gradient descent (SGD) and its quantum analogue, the quantum tunneling walk (QTW).

Theoretical Formulation

The authors formalize deterministic ECD as a first-order discretization of a Hamiltonian system with position-dependent mass inversely proportional to the objective potential $V(\theta) = F(\theta) - F_0$, where $F_0$ is the user’s guess for the global optimum. Unlike SGD, which dissipates its "energy" and slows irreversibly near local minima, ECD artificially preserves an energy invariant $E = \|\Pi_0\|^2 V(\Theta_0)$, which, for certain parameter regimes, guarantees that trajectories can escape local minima.

A key insight is that the behavior of ECD is driven by the gap between the guessed minimum $F_0$ and the true minimum $\min F$, partitioned into exact-guessing, over-guessing, and under-guessing regimes, with the under-guessing regime ($F_0 < \min F$) displaying recurrent, non-trapped dynamics.

Stochastic and Quantum Dynamics

• Stochastic ECD (sECD): Noise is introduced via energy-preserving, randomized momentum “rotations,” and in 1D, direction flips governed by a Poisson process of rate $\lambda_c$, countering the non-ergodicity of the deterministic dynamics.
• Quantum ECD (qECD): The ECD Hamiltonian is quantized using symmetric ordering to yield $$H = -\hbar^2 \partial_\theta [V(\theta) \partial_\theta]$$. The dynamics is then governed by the time-dependent Schrödinger equation, with the hitting time defined by detection in a small window around the global minimum, using a protocol akin to that used in quantum walk analysis.

Analysis of Escape Times in Double-Well Potentials

The core analytical results concern the expected time for the sECD and qECD dynamics to transition from a local minimum to a global minimum, specifically for double-well potentials in the under-guessing regime:

Figure 1: Symmetrical double well in the under-guessing regime, the primary landscape for analyzing escape dynamics.

The main object of study is the expected "hitting time" from the local minimum at $-a$ to the global minimum at $+a$. The hitting time for both sECD and qECD is computed exactly in terms of the potential $F_0$0, noise rates, and the initial conditions, leveraging changes of variables (Liouville transform) and Markov/semi-Markov process analysis for sECD, and WKB/saddle-point analysis for qECD in the semiclassical regime.

Key Results

• Classical (sECD) vs. SGD: For one-dimensional symmetric double-well potentials, sECD yields a polynomial-in-barrier escape time, versus the exponential-in-barrier escape time characteristic of SGD. Formally, for barrier height $F_0$1 and under-guessing error $F_0$2, sECD achieves

$F_0$3

where the omitted terms are $F_0$4-dependent but subdominant in relevant regimes.

• Quantum (qECD) Advantage: qECD achieves a strictly lower hitting time, with

$F_0$5

yielding an $F_0$6 advantage over sECD as $F_0$7, with even stronger separation in the tall-barrier ($F_0$8) regime.

• Scaling Regimes:
  • When $F_0$9 (small under-guessing error), both classical and quantum ECD transition from exponential to polynomial scaling in escape time, in sharp contrast to mere exponentially-slow diffusive escapes seen in SGD.
  • For $E = \|\Pi_0\|^2 V(\Theta_0)$0 (large under-guessing error), classical escape times are even further improved but remain super-polynomial, while quantum ECD attains a $E = \|\Pi_0\|^2 V(\Theta_0)$1 scaling.

These results are made explicit for the symmetric potential $E = \|\Pi_0\|^2 V(\Theta_0)$2.

Numerical and Analytical Strengths

The authors derive all results analytically, obtaining closed-form and asymptotic expressions for escape times, and demonstrate exponential improvement of sECD and qECD over SGD and QTW, respectively. The quantum advantage is robust across initialization protocols, further enhanced in the tall-barrier regime, and persists under strategies for adaptively improving $E = \|\Pi_0\|^2 V(\Theta_0)$3 via iterative bisection.

Contradictory and Strong Claims

• ECD Provably Escapes Local Minima Without Structural Assumptions: Unlike SGD, which requires convexity, one-point-convexity, or smoothing for any guarantee of escaping strict traps, ECD does not require such assumptions in the under-guessing regime.
• Quantum Speedup Over All Classical Protocols Considered: The quantum advantage persists even when sECD’s energy is optimally tuned post-hoc.
• Ergodicity via Energy-Preserving Noise: The introduction of direction-flip noise generates turnarounds even in 1D, making sECD a viable global optimizer in a setting where naive stochasticity would be insufficient.

Theoretical and Practical Implications

The work establishes an explicit theoretical foundation for energy-conserving optimization dynamics as a viable alternative to noisy gradient descent, both in classical and quantum regimes. In practical machine learning optimization, this suggests possible algorithms that more efficiently escape multi-modal traps, particularly in settings where diffusion-based methods are exponentially slow.

Quantum Hamiltonian simulation, as formalized here, provides a pathway for realizing provable quantum speedups for global non-convex optimization, especially in landscapes characterized by large potential barriers—a quantized analogue to the known advantage of quantum walks for traversing classically hard graphs.

Future Directions

• Extension to Higher Dimensions: The present analysis, restricted to $E = \|\Pi_0\|^2 V(\Theta_0)$4, motivates generalization to high-dimensional objective functions, where the geometry is more complex.
• Algorithmic and Oracle Complexity: Beyond hitting times, a full analysis of query/model complexity, discretization artifacts, and resource overhead in the quantum setting will be necessary for benchmarking practical implementations.
• Robustness and Mixing: Future studies will analyze initialization sensitivity and convergence to stationary distributions in both stochastic and quantum settings, laying the groundwork for systematic comparison of robustness.
• Landscape Generalization: Analyses of ECD in non-doubly-well or more degenerate multi-well landscapes, and in settings with sign-indefinite potentials, are important for applicability to realistic non-convex ML objectives and quantum optimization landscapes.

Conclusion

This paper rigorously establishes, through exact and asymptotic analysis, that both stochastic and quantum energy-conserving descent algorithms yield exponential improvements in escaping local minima compared to standard (stochastic) gradient descent, with qECD offering further substantial quantum speedups over all classical analogues in the tall-barrier regime. These results significantly reinforce the prospect of energy-conserving (and quantum-augmented) dynamical frameworks for global non-convex optimization, both improving practical optimization strategies and informing the theoretical study of quantum-accelerated learning dynamics.

---

Reference: "Classical and Quantum Speedups for Non-Convex Optimization via Energy Conserving Descent" (2604.13022)