Marginal Energy Landscape Insights

• Marginal energy landscapes are high-dimensional random surfaces defined by a continuous spectrum of near-zero Hessian eigenvalues, leading to flat directions in the landscape.
• They play a crucial role in glassy dynamics, jamming, neural network optimization, and constraint satisfaction by governing relaxation and convergence phenomena.
• Advanced methods like random matrix theory, Kac–Rice complexity, and replica calculations provide precise characterization of their scaling properties, geometry, and marginal stability.

A marginal energy landscape is a high-dimensional random energy or loss surface in which an extensive set of stationary points (minima, sometimes saddles) are characterized by marginal stability: their Hessian spectra degrade continuously down to zero, with the lowest eigenvalue at or vanishingly close to zero, and a nontrivial density of near-flat directions (“pseudogap”). Such landscapes play a central role in glassy systems, constraint satisfaction problems, neural network optimization, jamming of amorphous solids, and diffusion in temporally and spatially tuned potentials. Despite their rarity compared to stably “gapped” minima or saddles, marginal minima are favored both by physical relaxational dynamics and by typical algorithms, often governing long out-of-equilibrium behavior. Their statistical structure, scaling properties, geometric arrangement, and impact on aging and convergence phenomena have been systematically characterized by recent developments in random matrix theory, Kac–Rice complexity, replica calculations, and precise large-deviation conditioning techniques.

1. Definition and Characterization of Marginal Minima

Marginal minima are stationary points $x^\star$ of a smooth high-dimensional random energy (or loss) function $H(x)$, such that

• $H$ is stationary: $\nabla H(x^\star) = 0$,
• the Hessian $\mathrm{Hess}\, H(x^\star)$ is positive semidefinite but with its smallest eigenvalue exactly zero,
• the eigenvalue density $\rho(\lambda)$ of the Hessian has a continuous pseudogap down to $\lambda=0$, typically scaling as $\rho(\lambda) \sim A \lambda^\gamma$ for some $\gamma > 0$ as $\lambda \to 0^+$.

Marginal stationarity implies flat directions in configuration space, meaning these critical points sit at the threshold between true minima and saddles. In the context of p-spin spherical spin glasses, the condition for marginality is that the Wigner semicircle edge of the Hessian spectrum touches zero, i.e., the “radial-reaction” parameter $\mu$ equals its critical value $\mu_{\mathrm{mg}}$.

The statistical weight (complexity) of marginal minima at energy $E$ is captured via $\Sigma_{\rm m}(E)$, the logarithmic complexity conditioned on marginality:

$\Sigma_{\rm m}(E) = \Sigma_0(E, \mu_{\rm m}(E)),$

where $\mu_{\rm m}(E)$ is the solution of the stationary condition $$\partial_{\lambda^\ast} \Sigma_{\lambda^\ast}(E, \mu) |_{\lambda^\ast = 0} = 0$$ (Kent-Dobias, 2024).

2. Complexity, Threshold Energies, and Conditioning Techniques

The Kac–Rice complexity formalism is used to compute the number of stationary points with fixed energy, trace, and minimum eigenvalue. For a given landscape, the total number of such points is:

$$\mathcal N_{H}(E, \mu, \lambda^\ast) = \int d\nu_H(x, \omega) \, \delta(NE - H(x)) \, \delta(N\mu - \mathrm{Tr}\ \mathrm{Hess}) \, \delta(N\lambda^\ast - \lambda_{\min}(\mathrm{Hess})).$$

The “marginal” complexity is then defined by tuning to $\lambda^\ast = 0$, leading to a precise count of marginal minima. In canonical models such as pure p-spin spherical glasses,

$$\Sigma_{\rm m}(E) = \Sigma(E, \mu_{\mathrm{mg}}), \qquad \mu_{\mathrm{mg}} = 2\sqrt{f''(1)}$$

with $f(q)$ the covariance function of the field (Kent-Dobias, 2024, Folena et al., 2019).

The special “threshold energy” $E_{\mathrm{th}}$ is defined as the highest energy for which the complexity of marginal minima is nonnegative,

$\Sigma_{\rm m}(E_{\mathrm{th}}) = 0.$

At $E_{\mathrm{th}}$, marginal minima proliferate exponentially; for $E < E_{\mathrm{th}}$ only a vanishing minority exists, and for $E > E_{\mathrm{th}}$ they are absent. In mixed spherical p-spin models, marginal minima exist over a continuous band of energies, but only at $E_{\mathrm{th}}$ do they form a dense, pseudomanifold structure (Kent-Dobias, 2023).

3. Geometry, Hierarchy, and Ultrametric Structure

In the corrugated energy landscapes of glasses and jamming, marginal minima organize not as isolated attractors but as a highly structured hierarchy. In the mean-field fullRSB (“Gardner”) phase, every basin subdivides recursively into sub-basins, arising at all scales. This fractal hierarchy has been verified numerically for finite-dimensional soft sphere packings (Dennis et al., 2019):

• The contact-configuration distance $d(a, b)$ between minima provides a metric embedding.
• Hierarchical clustering of minima reveals a nested block structure in the distance matrix: in marginal landscapes, ultrametricity emerges, i.e., for any triple $(x, y, z)$, $d(x, z) \le \max\{d(x, y), d(y, z)\}$.
• The deviation from perfect ultrametricity $\mathcal D$ vanishes as system size increases, especially near jamming.
• This hierarchical, ultrametric structure is a direct signature of a marginal manifold and distinguishes the marginal phase from both the 1RSB (simple glass) and replica-symmetric (liquid) phases.

A summarizing table delineating the phases found via landscape structure:

Phase	Geometry of Minima	Complexity Scaling
RS (liquid)	Single basin	1
1RSB (glass)	Finite number of basins	Polynomial ($O(N)$)
fullRSB (marginal)	Hierarchical, ultrametric	Exponential (in $N$)

(Dennis et al., 2019)

4. Marginal Landscapes in Glassy Dynamics and Optimization

Marginal energy landscapes dominate the asymptotic regime of slow dynamics such as gradient descent, simulated annealing, and glassy relaxation:

• In high-dimensional p-spin models, gradient descent initialized randomly converges to marginal minima at $E_{\mathrm{th}}$, and out-of-equilibrium (aging) dynamics explores the marginal manifold (Folena et al., 2019).
• In deep network loss surfaces modeled by spherical spin glasses, the band of energies corresponding to marginal minima determines the transition between phases with exponentially many, polynomially many, or only a single minimum as the regularization is varied (Chaudhari et al., 2015).
• AnnealSGD exploits this phase structure, starting optimization in a trivial regime (unique minimum), transiting through the marginal (polynomial) regime, and finally refining in the full exponential regime, to accelerate convergence and avoid high-index saddles.

A plausible implication is that the flatness associated with marginal minima correlates with wider basins, which empirically improves generalization in neural networks (Chaudhari et al., 2015).

5. Proximity Structure and Breakdown of the Marginal Manifold

Recent two-point complexity calculations in mixed landscapes reveal that, except at $E_{\mathrm{th}}$, marginal minima are generally isolated in configuration space:

• Only at $E = E_{\mathrm{th}}$ does the set of marginal minima form a continuous pseudomanifold: the two-point complexity at high overlap ($q \rightarrow 1$) is nonzero only at this fine-tuned energy (Kent-Dobias, 2023).
• For $E \ne E_{\mathrm{th}}$, marginal minima are separated by finite gaps in overlap, i.e., they are local attractors or “islands”, not smoothly connected.
• This geometric isolation explains why aging in mixed models is not controlled by a single threshold energy, but rather depends globally on the basin structure encountered during relaxation (Kent-Dobias, 2023, Folena et al., 2019).
• An isolated eigenvalue in the Hessian spectrum, induced by overlap constraints or finite-size effects, can occasionally produce additional marginal directions, but does not alter the broad picture of isolated marginal minima.

6. Marginal Landscapes in Non-Gaussian and Constrained Models

Beyond Gaussian spherical models, the formalism extends to generalized, constrained, or non-Gaussian landscapes:

• In multi-spherical or bipartite spin glasses, the Hessian block structure leads to more complex marginality conditions but the essential pseudogap criterion persists (Kent-Dobias, 2024).
• For sums-of-squares of spherical random functions (relevant in certain constraint satisfaction and regression problems), the spectrum and marginal complexity can be explicitly obtained by combining replica and spherical integral formalisms. Marginal minima are again exponentially abundant only in a specific energy band.
• The square-root vanishing of the Hessian edge, $\rho(\lambda) \sim \sqrt{\lambda}$ as $\lambda \to 0$, is a universal signature of marginality when the Hessian ensemble is in the shifted GOE class (Kent-Dobias, 2024).

7. Marginal Potentials in Time-dependent Landscapes

A distinct example appears in diffusion in time-dependent parabolic potentials of the form $U(x, t) = C x^2 t^{-1}$, where $\omega=1$ is the marginal case:

• For this landscape, the Fokker–Planck equation admits exact solutions for all moments and survival probabilities (Turban, 2010).
• The diffusion regime, dynamical exponent $z$, and survival probability exponents vary continuously with the amplitude $C$, separating normal diffusion ($z=2$), superdiffusion ($z=1/|C|<2$), and a threshold regime with logarithmic corrections.
• The $\omega=1$ line supports a continuum of nontrivial fixed points parametrized by $C$, and only $p=2$ (parabolic) perturbations sustain this marginality under rescaling.
• Physically, the time-decaying coupling ensures the “strength” of the potential is renormalization-invariant at large scales—realizing a nonequilibrium realization of a marginal landscape in a dynamical context.

• "Conditioning the complexity of random landscapes on marginal optima" (Kent-Dobias, 2024)
• "Arrangement of nearby minima and saddles in the mixed spherical energy landscapes" (Kent-Dobias, 2023)
• "Jamming Energy Landscape is Hierarchical and Ultrametric" (Dennis et al., 2019)
• "Rethinking mean-field glassy dynamics and its relation with the energy landscape: the awkward case of the spherical mixed p-spin model" (Folena et al., 2019)
• "On the energy landscape of deep networks" (Chaudhari et al., 2015)
• "Anomalous diffusion in a space- and time-dependent energy landscape" (Turban, 2010)