Local Hessian Projections

• Local Hessian projections are methods that approximate full Hessian matrices with locally restricted curvature data, enabling efficient computations in high-dimensional settings.
• They are applied in turbulence closures, block-local estimations in variational quantum algorithms, and stochastic Hessian-vector products in machine learning to capture essential second-order structure.
• By focusing on tangent spaces, selected eigenspaces, or probe directions, these projections balance computational tractability with preserving key curvature features, despite potentially overlooking global effects.

Searching arXiv for relevant papers on local Hessian projections and adjacent uses of Hessian projection methods. Local Hessian projections are procedures that replace an explicit full Hessian by second-order information restricted to a neighborhood, a tangent space, a selected eigenspace, or a small family of probe directions. In the literature summarized here, the term covers spatially local closures for the anisotropic pressure Hessian in turbulence, block-sparse Hessian estimation in variational quantum algorithms, stochastic Hessian-vector supervision in machine-learning interatomic potentials, low-curvature Gauss–Newton editing of LLMs, Hessian-direction loss-landscape projections, and intrinsic projections of Euclidean Hessians onto manifolds (Chevillard et al., 2011, Huang et al., 31 Jan 2026, Rodriguez et al., 4 Mar 2026, Ikram et al., 17 Feb 2026, Böttcher et al., 2022, Birtea et al., 2018).

1. Common mathematical structure

A recurring formulation treats the Hessian without assembling its full matrix. In a finite-dimensional inner-product space, the Hessian can be defined basis-free as the symmetric bilinear form

$H(x)[u,v] := d^2 f|_x(u,v),$

with associated self-adjoint Hessian operator $\mathcal{H}(x)$ defined by

$H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$

This makes it possible to work directly with Hessian–vector products or Hessian–pair products rather than with an $n\times n$ array (Carlsson et al., 5 Feb 2025).

Projection enters in several distinct ways. In Euclidean-to-manifold transfer, one projects an ambient Hessian or gradient onto a tangent space. On the orthogonal Stiefel manifold $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$, the orthogonal projector onto $T_X St(p,n)$ is

$P_X(A)=A-X\,\mathrm{sym}(X^\top A),$

and the Riemannian Hessian is

$$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$

At a critical point $X$, $\Lambda(X)=0$, so the curvature term drops out and $\mathcal{H}(x)$0 (Birtea et al., 2018).

Projection can also mean reducing the Hessian to a low-dimensional coordinate system. For a loss $\mathcal{H}(x)$1, random directions collected in $\mathcal{H}(x)$2 yield a projected Hessian $\mathcal{H}(x)$3. In that setting,

$\mathcal{H}(x)$4

so an expected random projection sees only the trace, or equivalently the mean curvature after normalization by $\mathcal{H}(x)$5 (Böttcher et al., 2022). This is a central distinction between random projection and targeted Hessian-direction projection.

A plausible unifying interpretation is that local Hessian projections preserve only the second-order structure judged operationally relevant for the task: spatial locality in turbulence, backward-lightcone locality in quantum circuits, probe-direction curvature in atomistic learning, or low-curvature eigenspaces in constrained editing.

2. Spatial locality in turbulence closures

In isotropic and homogeneous turbulence, the Lagrangian dynamics of the velocity gradient tensor $\mathcal{H}(x)$6 depend on the joint action of the self-stretching term and the pressure Hessian $\mathcal{H}(x)$7. The relevant scalar invariant is

$\mathcal{H}(x)$8

with Hessian $\mathcal{H}(x)$9. The deviatoric part of the pressure Hessian is $H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$0, and the corresponding deviatoric Hessian of $H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$1 is $H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$2 (Chevillard et al., 2011).

Starting from the Poisson equation $H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$3, one obtains the exact nonlocal relation

$H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$4

The local Hessian projection model assumes that the dominant contribution comes from a ball $H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$5, with $H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$6 of order the Kolmogorov scale $H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$7, and Taylor-expands $H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$8. This yields the local closure

$H(x)[u,v] = \langle \mathcal{H}(x)u, v\rangle.$9

Equivalently,

$n\times n$0

with $n\times n$1 and $n\times n$2 (Chevillard et al., 2011).

The scale $n\times n$3 is fixed by matching the root-mean-square norm of $n\times n$4,

$n\times n$5

In DNS at $n\times n$6, this gives $n\times n$7. Quantitative validation reported Pearson correlations between DNS eigenvalues of $n\times n$8 and those of $n\times n$9 of $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$0 for the smallest eigenvalue, $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$1 for the intermediate eigenvalue, and $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$2 for the largest eigenvalue. The corresponding eigenvector-alignment PDFs peak sharply at $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$3 for all three $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$4, and the component pair $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$5 has a joint PDF elongated along the diagonal with component-wise Pearson correlation $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$6 (Chevillard et al., 2011).

The same model reproduces several statistical features in the $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$7 plane: the counter-action on the Vieillefosse tail, the void of flux in the dissipation-production region $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$8, and the redistribution towards enstrophy-production $$St(p,n)=\{X\in\mathbb{R}^{n\times p}\mid X^\top X=I_p\}$$9. It also reproduces the nonzero $T_X St(p,n)$0 at $T_X St(p,n)$1 and the near-linear trend in a neighborhood of $T_X St(p,n)$2, in contrast to older closures proportional to $T_X St(p,n)$3 (Chevillard et al., 2011). The central modeling conclusion is that the local effects appear to be the most important ones, even though the exact pressure Hessian is nonlocal.

3. Block-local Hessian assembly in variational quantum algorithms

In variational quantum algorithms, local Hessian projection arises from the observation that second-order information is accessible only within the causal support of local observables. For a parameterized circuit

$T_X St(p,n)$4

with each $T_X St(p,n)$5 having spectrum $T_X St(p,n)$6, the cost is

$T_X St(p,n)$7

The Hessian entries admit exact second-order parameter-shift rules. For $T_X St(p,n)$8,

$T_X St(p,n)$9

and for $P_X(A)=A-X\,\mathrm{sym}(X^\top A),$0,

$P_X(A)=A-X\,\mathrm{sym}(X^\top A),$1

In every case, $P_X(A)=A-X\,\mathrm{sym}(X^\top A),$2, with $P_X(A)=A-X\,\mathrm{sym}(X^\top A),$3 and coefficients $P_X(A)=A-X\,\mathrm{sym}(X^\top A),$4 (Huang et al., 31 Jan 2026).

The variance structure is controlled by a constant-size covariance matrix. If $P_X(A)=A-X\,\mathrm{sym}(X^\top A),$5 and $P_X(A)=A-X\,\mathrm{sym}(X^\top A),$6 under $P_X(A)=A-X\,\mathrm{sym}(X^\top A),$7, then

$P_X(A)=A-X\,\mathrm{sym}(X^\top A),$8

For the termwise local cost

$P_X(A)=A-X\,\mathrm{sym}(X^\top A),$9

with each $$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$0 $$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$1-local and $$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$2, bounded-depth $$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$3-local circuits imply a backward-lightcone radius $$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$4. If two terms sit at graph distance $$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$5, their expectation values depend on disjoint parameter sets and are independent under i.i.d. uniform initialization. The resulting dependency-graph argument yields

$$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$6

If $$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$7 and $$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$8, then $$\mathrm{Hess}\,f(X)[\xi]=P_X(D^2F(X)[\xi])-\xi\,\Lambda(X), \qquad \Lambda(X)=\mathrm{sym}(X^\top\nabla F(X)).$$9 (Huang et al., 31 Jan 2026).

This locality leads directly to a projected Hessian assembly rule. For each local term $X$0, one defines the backward-lightcone parameter set

$X$1

One then estimates only those entries $X$2 for which there exists at least one $X$3 with $X$4, assembling a block-diagonal or block-sparse approximation from the local subblocks $X$5. In the local-cost regime, the number of shots required to resolve each entry at the typical fluctuation scale is $X$6, whereas for global objectives Hessian variances are exponentially suppressed and the number of shots must scale as $X$7 to maintain a constant signal-to-noise ratio (Huang et al., 31 Jan 2026).

4. Stochastic probe projections and low-curvature editing in machine learning

In machine-learning interatomic potentials, the Hessian of the potential energy surface is a $X$8 matrix,

$X$9

and a full Hessian loss

$\Lambda(X)=0$0

is often impractical because obtaining all unique second derivatives requires solving $\Lambda(X)=0$1 response equations, storing $\Lambda(X)=0$2 scales as $\Lambda(X)=0$3, and back-propagating through the full matrix is prohibitively expensive beyond a few dozen atoms. Projected Hessian Learning replaces this by stochastic Hessian-vector products. With

$\Lambda(X)=0$4

Hutchinson’s identity gives

$\Lambda(X)=0$5

The projected loss for one configuration is

$\Lambda(X)=0$6

and $\Lambda(X)=0$7 is computed with automatic differentiation at a cost comparable to $\Lambda(X)=0$8–$\Lambda(X)=0$9 force evaluations. Randomized probes per minibatch allow both HVP-based schemes to match full-Hessian training in energy, force, and Hessian accuracy while delivering $\mathcal{H}(x)$00 epoch speedups for the small molecular systems studied; on typical $\mathcal{H}(x)$01 molecules, full Hessian training takes $\mathcal{H}(x)$02, while both E–F–HVP one-hot and E–F–PHL take $\mathcal{H}(x)$03, and E–F alone takes $\mathcal{H}(x)$04. On normal-mode-sampled structures, PHL reduces energy RMSE by $\mathcal{H}(x)$05–$\mathcal{H}(x)$06, force RMSE by $\mathcal{H}(x)$07–$\mathcal{H}(x)$08, and Hessian RMSE by $\mathcal{H}(x)$09–$\mathcal{H}(x)$10 relative to E–F (Rodriguez et al., 4 Mar 2026).

A different projection principle appears in model editing. CrispEdit formulates editing as

$\mathcal{H}(x)$11

and measures capability drift using a Bregman divergence induced by the pointwise loss $\mathcal{H}(x)$12. A second-order Taylor expansion around $\mathcal{H}(x)$13 yields the quadratic approximation

$\mathcal{H}(x)$14

where

$\mathcal{H}(x)$15

is the Generalized Gauss–Newton Hessian. If

$\mathcal{H}(x)$16

then choosing an energy threshold $\mathcal{H}(x)$17 and removing the top-$\mathcal{H}(x)$18 eigenspace defines the low-curvature projector

$\mathcal{H}(x)$19

CrispEdit makes this practical with K-FAC and a matrix-free projector that exploits Kronecker structure, avoiding construction of the full projection matrix. Across standard model-editing benchmarks, it achieves high edit success while keeping capability degradation below $\mathcal{H}(x)$20 on average across datasets (Ikram et al., 17 Feb 2026).

These examples share a common operational pattern: the full Hessian is replaced either by its action on probe vectors or by updates confined to a curvature-selected subspace.

5. Tangent-space, manifold-learning, and visualization perspectives

Local Hessian projection also appears in manifold methods, where curvature information is restricted by intrinsic dimension. In Hessian Locally Linear Embedding, each data point $\mathcal{H}(x)$21 is assigned a $\mathcal{H}(x)$22-nearest-neighbor set $\mathcal{H}(x)$23, with

$\mathcal{H}(x)$24

After local PCA, one constructs

$\mathcal{H}(x)$25

orthonormalizes its columns, and keeps the last $\mathcal{H}(x)$26 columns as $\mathcal{H}(x)$27. Each column $\mathcal{H}(x)$28 of $\mathcal{H}(x)$29 satisfies

$\mathcal{H}(x)$30

so the global embedding solves

$\mathcal{H}(x)$31

Qiao et al. reinterpret this construction as a variant of LLE in which the special Hessian weights can be replaced by arbitrary orthonormalized vectors satisfying the same local constraints, leading to Tangential LLE. They also show that HLLE may produce projection-like results when the embedding dimension $\mathcal{H}(x)$32 exceeds the manifold dimension $\mathcal{H}(x)$33, and propose always performing local PCA to $\mathcal{H}(x)$34, even when the final embedding is sought in $\mathcal{H}(x)$35 with $\mathcal{H}(x)$36 (Lin et al., 2021).

A closely related but analytically distinct use of Hessian projections appears in loss-landscape visualization. For a critical point $\mathcal{H}(x)$37, the local quadratic expansion

$\mathcal{H}(x)$38

shows that eigenvalues of $\mathcal{H}(x)$39 are the principal curvatures. Random Gaussian projections preserve the trace in expectation but do not preserve saddle structure: saddle points in the original space are rarely correctly identified as such in expected lower-dimensional representations if random projections are used. The proposed remedy is to project along the dominant Hessian directions, typically the eigenvectors corresponding to the largest positive and most negative eigenvalues, and to visualize

$\mathcal{H}(x)$40

For image classifiers and a function approximator with up to about $\mathcal{H}(x)$41 parameters, these Hessian projections reveal clear saddles and negative-curvature escape directions, whereas random projections show only convex contours (Böttcher et al., 2022).

Together, these results emphasize that the projection space itself is part of the model. Orthogonality to the local tangent plane, restriction to intrinsic dimension, or alignment with extremal Hessian directions changes the qualitative meaning of the projected curvature.

6. Computational regimes, advantages, and limitations

The main computational motivation for local Hessian projections is the cost of explicit second-order structure. In large-scale optimization, forming and storing the full Hessian on $\mathcal{H}(x)$42 costs $\mathcal{H}(x)$43 memory and $\mathcal{H}(x)$44 or worse time, whereas each Hessian–vector product via an analytic formula or directional derivative can cost $\mathcal{H}(x)$45 or $\mathcal{H}(x)$46, and the finite-difference directional-derivative route uses $\mathcal{H}(x)$47–$\mathcal{H}(x)$48 gradient calls per Hessian–vector product. This basis-free regime supports Newton, Daniel’s conjugate-gradient rule, and quasi-Newton schemes without explicit matrix formation (Carlsson et al., 5 Feb 2025).

For inverse problems governed by PDEs, the same idea is developed through hierarchical matrix approximations of Hessians with hierarchically low-rank local block structure. The randomized $\mathcal{H}(x)$49-construction requires $\mathcal{H}(x)$50 Hessian–vector products, and $\mathcal{H}(x)$51 mat–vec operations cost $\mathcal{H}(x)$52. In representative inverse problems, the method demonstrates up to an order of magnitude speedup compared to globally low-rank approximations (Ambartsumyan et al., 2020).

The limitations are domain-specific but structurally similar. In turbulence, neglect of nonlocal Green’s-function tails may underpredict extreme events, and computing high-order derivatives from noisy data or coarse DNS is challenging (Chevillard et al., 2011). In variational quantum algorithms, the local-cost regime admits $\mathcal{H}(x)$53 sampling, but global objectives require exponentially many shots to resolve Hessian entries against shot noise (Huang et al., 31 Jan 2026). In loss-landscape visualization, random projections erase saddle information because they return mean curvature in expectation rather than extremal curvature (Böttcher et al., 2022). In manifold learning, HLLE can produce projection patterns when $\mathcal{H}(x)$54 unless the local PCA dimension is fixed to the manifold dimension (Lin et al., 2021).

This suggests a general trade-off. Local Hessian projections gain tractability by discarding global or high-dimensional curvature structure, but the validity of the approximation depends on whether the retained locality, probe set, tangent model, or eigenspace matches the governing geometry of the problem.