Expansive Gradient Dynamics Overview

• Expansive Gradient Dynamics is a multifaceted concept describing distinct regimes such as orbit separation in flows, infinite series gradient expansions, adaptive optimization, and high-dimensional jamming.
• It contrasts non-expansive, Lyapunov-dissipative systems with mechanisms that amplify gradients through state-dependent dynamics, stochastic exploration, and geometric reformulations.
• The framework has practical implications in designing adaptive learning algorithms, analyzing critical transitions in jamming, and evolving neural architectures via Hilbert-space flows.

In the cited literature, the phrase expansive gradient dynamics appears in several technically distinct senses. It can denote orbit-separating dynamics in the sense of expansiveness for flows and maps; all-order gradient expansions whose resummation reconstructs non-perturbative attractors; optimization laws that amplify, regularize, or geometrize gradient motion through gradient statistics, momentum, or adaptive architecture growth; and expanding fronts driven by self-generated gradients in continuum models. A recurrent counterpoint is that many classical gradient systems are intrinsically non-expansive: under detailed balance, nonlinear graph dynamics can be written as passive gradient descents of strictly convex sum-separable energies, with monotone decay of a Lyapunov functional (Mangesius et al., 2015).

1. Passive graph-gradient structure and the non-expansive baseline

A central reference point is the class of nonlinear dynamics on a weighted directed graph

$$\dot{x}_i=\sum_{j:(j,i)\in B} w_{ij}\,\phi(x_j,x_i),$$

with strongly connected graph, positive left eigenvector $c$ satisfying $c^\top L=0^\top$, and detailed balance

$$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$

After the mass-preserving change of variables $q=Cx$, these dynamics admit the gradient form

$\dot q=-K(q)\,\nabla E(q),$

where $K(q)$ is a symmetric irreducible Laplacian and

$E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$

is a sum-separable energy. If $H$ is strictly convex, then $E$ is a Lyapunov function and

$c$0

For normalized $c$1, these energies coincide with Csiszár–Ali–Silvey information divergences; examples include the quadratic energy and relative entropy. The same structure yields a circuit interpretation in which passivity of each nonlinear resistor is equivalent to strict convexity of the stored energy, and a Markov-chain interpretation in which the Csiszár divergence to the stationary distribution decreases monotonically (Mangesius et al., 2015).

This framework is important because it specifies a regime that is explicitly not expansive. Energy expansion is ruled out by $c$2; the induced dynamics are passive, mass-preserving, and asymptotically convergent to the uniform agreement state

$c$3

A plausible implication is that any discussion of expansive gradient dynamics must distinguish carefully between systems that are gradient descents in a state-dependent metric and systems in which expansiveness is meant in a topological, asymptotic-series, or adaptive-optimization sense.

2. Expansiveness in the dynamical-systems sense

In smooth dynamics, expansiveness refers to orbit separation rather than monotone energy dissipation. For flows on a compact connected Riemannian manifold, the paper on kinematic $c$4-expansive flows defines

$c$5

and calls the flow kinematic $c$6-expansive when $c$7 uniformly. The robust form is highly rigid: if a vector field is $c$8-robustly kinematic $c$9-expansive, then it is quasi-Anosov; the chain recurrent set is Axiom A without cycles; robustly kinematic $c^\top L=0^\top$0-expansive homoclinic classes are hyperbolic; and kinematic $c^\top L=0^\top$1-expansive flows have no singularities (Lee et al., 2018). This implies that classical gradient flows on compact manifolds, which generically possess singularities at critical points, are structurally incompatible with robust kinematic $c^\top L=0^\top$2-expansiveness.

For nonautonomous discrete-time systems, strong uniform expansivity is the condition that there exists $c^\top L=0^\top$3 such that for every $c^\top L=0^\top$4 one can find $c^\top L=0^\top$5 with

$c^\top L=0^\top$6

for all times $c^\top L=0^\top$7. In this setting, uniformly expanding $c^\top L=0^\top$8 sequences admit explicit entropy formulas in terms of Jacobians, while strongly uniformly expansive systems admit time-dependent analogues of classical positive-expansive results, including an equivalent family of metrics $c^\top L=0^\top$9 in which each map locally expands distances uniformly: $$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$0 for fixed $$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$1, $$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$2 (Kawan, 2014).

At the linear level, an integer matrix $$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$3 is expansive when all eigenvalues satisfy $$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$4. It is strictly expansive relative to a compact fundamental domain $$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$5 when $$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$6; for compact, convex, centrally symmetric $$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$7, this is equivalent to

$$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$8

Strongly connected diagonally dominant integer matrices are strictly expansive, whereas determinant-two matrices are not strictly expansive relative to certain “nice” sets such as convex, centrally symmetric, or connected-boundary tiles (Speegle, 9 Jun 2025). These results concern geometric separation and tiling-compatible expansion, not Lyapunov descent.

3. All-order gradient expansions and hydrodynamic attractors

A second meaning of expansive gradient dynamics concerns taking a gradient expansion seriously as an infinite series, studying its large-order structure, and reconstructing the full dynamics from resummation. In the non-relativistic BGK model, the spatial Chapman–Enskog expansion of the hydrodynamic dispersion relation is

$$c_i w_{ij}=c_j w_{ji}\quad\Leftrightarrow\quad CL=L^\top C.$$9

with all-order coefficients obtained by Lagrange inversion: $q=Cx$0 The first coefficients are

$q=Cx$1

and satisfy the large-order asymptotics

$q=Cx$2

The series is factorially divergent, but its Borel transform has its nearest singularity at $q=Cx$3, on the negative real axis, so the spatial gradient series is strictly Borel summable. In the relativistic Anderson–Witting analogue, bounded velocities $q=Cx$4 imply bounded equilibrium moments and hence a convergent spatial expansion with finite nonzero radius of convergence (Kooshkbaghi, 23 Mar 2026).

This result is noteworthy because it sharply contrasts spatial and temporal gradient expansions. Temporal series, such as those for Bjorken flow, are non-Borel-summable along the physical direction and require transseries sectors; the spatial series in the BGK setting does not. A concrete consequence is that a diagonal Borel–Padé approximant of order $q=Cx$5, built from the first 30 Borel-transformed Chapman–Enskog coefficients, reproduces the non-perturbative attractor within numerical precision (Kooshkbaghi, 23 Mar 2026).

A related large-order program appears in the diagrammatic analysis of gradient flow for learning Canonical Polyadic decompositions. There the loss evolution is expanded as

$q=Cx$6

and each coefficient is represented by diagram mergers and Wick contractions. For Gaussian initialization, the averaged coefficients are polynomials in the size parameters and scale as monomials $q=Cx$7. Pareto-optimal monomials organize the large-size asymptotics and define distinct regimes such as free evolution, NTK, and under- and over-parameterized mean-field behavior. Recurrences for the coefficients are then converted to PDEs for generating functions and, in a wide range of cases, solved by the method of characteristics (Yarotsky et al., 4 Feb 2026). In this usage, “expansive” refers not to trajectory separation but to the controlled unfolding of nonlinear gradient flow into a structured hierarchy of high-order terms.

4. Adaptive optimization, stochastic exploration, and bounded amplification

In optimization and learning theory, expansive gradient dynamics often refers to gradient laws that enlarge motion along selected directions, either by geometry, by stochasticity, or by state-dependent gain. A covariant geometric version is the continuous-time system

$q=Cx$8

where both the covariant force $q=Cx$9 and the metric tensor $\dot q=-K(q)\,\nabla E(q),$0 are built from exponentially weighted first and second moments of the gradient. In this framework, gradient descent, SGD, RMSProp, and Adam arise as special cases, while full-covariance choices such as

$\dot q=-K(q)\,\nabla E(q),$1

define genuinely coordinate-covariant preconditioned flows. The full metric uses the entire second-moment tensor rather than just its diagonal, but this introduces $\dot q=-K(q)\,\nabla E(q),$2 memory and at least $\dot q=-K(q)\,\nabla E(q),$3–$\dot q=-K(q)\,\nabla E(q),$4 cost (Guskov et al., 7 Apr 2025).

In reinforcement learning, policy-gradient dynamics in a bandit setting admit an Itô drift–diffusion form. For a two-armed bandit,

$\dot q=-K(q)\,\nabla E(q),$5

with drift and diffusion coefficients depending on the learning rate $\dot q=-K(q)\,\nabla E(q),$6. The leading drift scales as $\dot q=-K(q)\,\nabla E(q),$7, while diffusion scales as $\dot q=-K(q)\,\nabla E(q),$8. In the non-convex $\dot q=-K(q)\,\nabla E(q),$9-dimensional $K(q)$0-armed construction, the learning problem maps to a disordered Hamiltonian, and the learning rate acts as an effective temperature,

$K(q)$1

so increasing $K(q)$2 smooths rugged landscapes and broadens exploration (Fabbricatore et al., 2022). This is a precise stochastic sense in which gradient dynamics become more expansive.

A different mechanism appears in the AGEM algorithm with energy and momentum. Its high-resolution ODE is

$K(q)$3

with Lyapunov function

$K(q)$4

The system is globally well posed; trajectories remain bounded; $K(q)$5, $K(q)$6, and $K(q)$7. Under the Polyak–Łojasiewicz condition, the objective converges exponentially (Demircigil, 2022). Here momentum creates richer transients, but the energy variable imposes global dissipation rather than true expansiveness.

State-dependent learning rates based on terminal attractors provide yet another interpretation. Classical terminal-attractor choices produce error dynamics

$K(q)$8

with $K(q)$9, leading to finite-time convergence of the scalar error. However, the associated parameter dynamics can become unbounded near minima because the learning rate divides by powers of $E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$0. The placid terminal attractor and placid fast terminal attractor modify the gain by a sigmoid factor of $E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$1, so that escape steps near local minima remain finite and nonzero, while $E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$2 near the global minimum (Zhao et al., 2024). This is a controlled form of expansiveness: enough amplification to leave poor basins, but not enough to destabilize the target equilibrium.

An empirical manifestation of unwanted expansiveness appears in gradient normalization for deep networks. Layer-wise Z-score normalization rescales by $E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$3, so layers with tiny gradient standard deviation can be amplified severely. The proposed alternative computes the global gradient standard deviation $E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$4 and uses the autoscaling factor

$E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$5

with a safeguard choosing $E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$6. Because $E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$7, this reduces the effective stepsize rather than amplifying it, and the scaled SGD update

$E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$8

retains standard smoothness-based convergence bounds. On CIFAR-100, the method improves or matches the AdamW baseline on ResNet-20, ResNet-56, and VGG-16-BN while avoiding the degradation observed with per-layer Z-score normalization (Yun, 3 Sep 2025).

5. Expanding neural manifolds and natural gradient flows

A more literal notion of expansive gradient dynamics is developed for neural-network–parameterized subsets of an ambient Hilbert space $E(q)=\sum_i c_i H(c_i^{-1}q_i)=\sum_i c_i H(x_i)$9. The ideal problem is the minimization of a quadratic affine energy

$H$0

whose unique minimizer is the Riesz lift $H$1, and whose ideal Hilbert-space gradient flow is explicitly

$H$2

Restricting to a neural manifold $H$3, the tangent space is

$H$4

and the induced neural flow matrix is the Gram matrix

$H$5

The projected natural gradient dynamics solve

$H$6

or, with Levenberg–Marquardt regularization,

$H$7

at each step (Dahmen et al., 17 Jul 2025).

The expansive mechanism enters when the current tangent space is poorly aligned with the ambient Hilbert-space gradient. The framework therefore enlarges the architecture step by step by either adding a residual layer before the output or widening the last hidden layer. Expansion is triggered by loss saturation, and new tangent directions are selected to maximize alignment with the Hilbert-space gradient through quantities of the form

$H$8

where $H$9 collects derivatives with respect to the new parameters. In supervised learning, elliptic PDEs, and model reduction, this produces a hybrid scheme in which regularized natural-gradient steps on selected layers are interleaved with architecture growth and Adam updates on all layers. The importance of assembling the flow matrix with the inner product of the ambient Hilbert space, rather than an arbitrary Euclidean proxy, is a central empirical conclusion (Dahmen et al., 17 Jul 2025).

6. High-dimensional landscapes, jamming, and critical gradient descent

In complex high-dimensional energy landscapes, expansive features can arise even under plain gradient descent. For soft repulsive particles in the $E$0 limit, the many-body gradient flow

$E$1

admits a dynamical mean-field reduction to an effective one-gap process with self-consistent memory kernels and colored noise. The resulting observables include the energy $E$2, pressure $E$3, contact number $E$4, two-time correlation and response, and mean-square displacement (Manacorda et al., 2022).

The central phenomenon is a jamming transition. At low rescaled density $E$5, gradient descent reaches zero-energy states in which overlaps vanish; at high $E$6, the asymptotic energy remains positive and overlaps persist. At the transition the dynamics becomes critical. In the unjammed phase, the long-time response kernel can be solved explicitly, and the Hessian spectrum around the final configuration is a Marčenko–Pastur law with a zero-mode delta peak of weight $E$7. As $E$8, the lower band edge tends to zero, the decay of the memory kernel crosses from exponential to algebraic $E$9, and the response develops marginal behavior (Manacorda et al., 2022).

This is relevant to expansive gradient dynamics because the geometry of the basin becomes progressively flatter and more weakly restoring near jamming. Small perturbations can induce persistent displacements along zero or near-zero modes, the integrated response diverges, and the mean-square displacement grows with density in the Random Lorentz Gas proxy. The same paper identifies a direct analogy with the capacity transition in supervised learning, and explicitly notes that the perceptron appendix realizes the same SAT/UNSAT structure as soft-sphere jamming (Manacorda et al., 2022).

The literature therefore supports a layered conclusion. Expansive gradient dynamics is not a single theory but a family of regimes. In some settings it means topological orbit separation; in others it means factorially rich gradient expansions; in optimization it may mean stochastic broadening, state-dependent amplification, or Hilbert-metric manifold growth; and in physical landscapes it may mean marginal exploration near jamming. A recurring distinction is that these usages are technically disjoint from passive gradient systems, where detailed balance, strict convexity, and Lyapunov dissipation enforce the opposite behavior (Mangesius et al., 2015). Open directions stated in the cited works include scalable full-covariance covariant descent (Guskov et al., 7 Apr 2025), rigorous analysis of architecture-expanding Hilbert-space flows (Dahmen et al., 17 Jul 2025), extension of gradient-dynamics diagnostics to Vision Transformers (Yun, 3 Sep 2025), explicit asymptotics in the jammed phase of the mean-field soft-sphere problem (Manacorda et al., 2022), and the role of transseries versus strict Borel summability beyond the spatial BGK setting (Kooshkbaghi, 23 Mar 2026).