Normalized Directional Sharpness (NDS)

• Normalized Directional Sharpness (NDS) is a directional sharpness construct that normalizes raw curvature measures to mitigate confounding factors such as learning rate, step magnitude, and contrast.
• It is applied in diverse settings including stochastic optimization, optimizer comparison via Hessian Rayleigh quotients, and dynamic quality assessment in satellite imagery.
• By isolating directional signals with tailored normalization protocols, NDS provides actionable insights into training stability and image sharpness evaluation.

Searching arXiv for the cited papers to ground the article in current preprints. Normalized Directional Sharpness (NDS) is a direction-sensitive sharpness construct that appears in several distinct technical settings rather than as a single universal formula. In recent arXiv usage, it denotes at least four related normalizations: a learning-rate-normalized mini-batch curvature statistic for the Edge of Stochastic Stability (EoSS) in momentum training, $\mathrm{NDS}=\eta\,\mathrm{BS}$; a Hessian Rayleigh quotient along an optimizer’s realized update direction, $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$; a normalized fluctuation statistic over short SAM or ASAM probe trajectories, $\mathrm{Std}(\{\log(r_t+\varepsilon)\})$ with $r_t=s_t^2/\mathcal L_{\xi_t}$; and, in satellite image quality assessment, a normalized decay rate of gradients along pronounced edges (Andreyev et al., 15 Apr 2026, Wang et al., 3 Jun 2026, Tan et al., 23 Jun 2026, Antonel, 2024). The shared theme is normalization of a directional sharpness signal so that operative comparisons are less confounded by learning rate, step magnitude, loss scale, parameterization, contrast, or exposure.

1. Taxonomy of definitions

The term covers multiple non-equivalent quantities whose common structure is directional measurement plus explicit normalization. In stochastic optimization near EoSS, the underlying directional statistic is Batch Sharpness,

$$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$

and the natural normalization is

$$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$

In optimizer comparison, NDS is the Hessian Rayleigh quotient along the actual update,

$$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$

In certification, the central object is a directional-sharpness time series generated by short SAM-style dynamics and normalized by per-step loss,

$$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$

In satellite imagery, NDS is used as a compact name for the normalized decay rate of gradients along pronounced edges, with directional scores

$$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$

These definitions are not interchangeable, but each treats sharpness as a quantity that must be evaluated along an operational direction rather than by a purely global worst-case surrogate (Andreyev et al., 15 Apr 2026, Wang et al., 3 Jun 2026, Tan et al., 23 Jun 2026, Antonel, 2024).

Setting	Direction	Normalization
Momentum EoSS	Mini-batch gradient direction $g_B$	$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$0
Optimizer comparison	Realized update $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$1	Divide by $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$2
Certification	SAM/ASAM probe trajectory	$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$3
Satellite imagery	Pronounced edge directions	Divide gradient decay by original edge gradient

A common misconception is to treat NDS as a synonym for maximum curvature or for a single Hessian-based scalar. The relevant papers instead define it through optimizer directions, stochastic mini-batch directions, SAM probe directions, or edge-normal directions, and the normalization target changes with the application.

2. Learning-rate-normalized directional sharpness at the Edge of Stochastic Stability

In "Momentum Further Constrains Sharpness at the Edge of Stochastic Stability" (Andreyev et al., 15 Apr 2026), the operative curvature statistic is Batch Sharpness, the expected directional mini-batch curvature along the stochastic descent direction. The paper emphasizes that this quantity differs from $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$4, the largest eigenvalue of the full-batch Hessian. In full-batch GD, $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$5 marks the deterministic Edge of Stability, but in mini-batch training $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$6 may plateau far below $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$7 and does not change instantaneously with batch-size interventions, whereas catapult events align with Batch Sharpness crossing its operating plateau. This is why the directional statistic, rather than the maximum curvature, is treated as the empirically relevant quantity for EoSS (Andreyev et al., 15 Apr 2026).

The normalization

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$8

removes the trivial $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$9 scaling that repeatedly appears in the observed curvature thresholds. Under SGDM and SGDN, the paper identifies two batch-size-dependent plateau regimes. In the small-batch, noise-dominated regime,

$\mathrm{Std}(\{\log(r_t+\varepsilon)\})$0

In the large-batch regime, the plateau depends on the momentum variant. For SGDM,

$\mathrm{Std}(\{\log(r_t+\varepsilon)\})$1

whereas for SGDN,

$\mathrm{Std}(\{\log(r_t+\varepsilon)\})$2

For vanilla SGD, $\mathrm{Std}(\{\log(r_t+\varepsilon)\})$3 recovers

$\mathrm{Std}(\{\log(r_t+\varepsilon)\})$4

These plateau values are central because they show that momentum does not merely shift a single stability threshold. In small batches, momentum lowers the operative normalized plateau to $\mathrm{Std}(\{\log(r_t+\varepsilon)\})$5, whereas in large batches SGDM raises it to $\mathrm{Std}(\{\log(r_t+\varepsilon)\})$6. The paper explicitly highlights this qualitative flip: momentum favors flatter regions in the noise-dominated regime but sharper regions in the near-deterministic regime (Andreyev et al., 15 Apr 2026).

The measurement protocol is also directional. At parameter iterate $\mathrm{Std}(\{\log(r_t+\varepsilon)\})$7, one computes or approximates $\mathrm{Std}(\{\log(r_t+\varepsilon)\})$8 and $\mathrm{Std}(\{\log(r_t+\varepsilon)\})$9 on random mini-batches $r_t=s_t^2/\mathcal L_{\xi_t}$0, forms the Rayleigh quotient

$r_t=s_t^2/\mathcal L_{\xi_t}$1

and averages over batches to obtain $r_t=s_t^2/\mathcal L_{\xi_t}$2. Tracking $r_t=s_t^2/\mathcal L_{\xi_t}$3 over training reveals progressive sharpening followed by stabilization at the batch-regime-dependent plateau; multiplying by $r_t=s_t^2/\mathcal L_{\xi_t}$4 yields NDS, which exposes the optimizer dependence through $r_t=s_t^2/\mathcal L_{\xi_t}$5.

3. Linear stability interpretation and hyperparameter coupling

The same EoSS work derives the momentum thresholds from linearized dynamics near a minimizer $r_t=s_t^2/\mathcal L_{\xi_t}$6. With $r_t=s_t^2/\mathcal L_{\xi_t}$7 the velocity and $r_t=s_t^2/\mathcal L_{\xi_t}$8, SGDM is written as

$r_t=s_t^2/\mathcal L_{\xi_t}$9

where $$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$0 is the random mini-batch Hessian under quadratic approximation and interpolation (Andreyev et al., 15 Apr 2026).

In one dimension, with random curvature $$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$1 of mean $$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$2 and variance $$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$3, the dominant eigenvalue of the mean-square operator expands as

$$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$4

The resulting exact boundary for mean-square stability interpolates between deterministic and stochastic limits: $$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$5 As $$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$6, this recovers the deterministic heavy-ball threshold $$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$7. As $$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$8, the effective step size

$$\mathrm{BS}(\theta) := \mathbb{E}_{B\sim \mathcal P_b}\!\left[ \frac{g_B(\theta)^\top H_B(\theta)\,g_B(\theta)}{\|g_B(\theta)\|_2^2} \right],$$9

governs stability, yielding the small-batch threshold $$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$0 and thus $$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$1.

The multidimensional slow-mode constraint in the noise-dominated regime is

$$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$2

with $$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$3, $$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$4, and $$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$5. The paper notes that this coincides with the vanilla-SGD mean-square stability condition evaluated at $$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$6, explaining why the small-batch momentum threshold matches SGD’s EoSS evaluated at the amplified effective step size (Andreyev et al., 15 Apr 2026).

This stability picture has direct tuning implications. In the small-batch regime, keeping $$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$7 approximately fixed maintains the NDS plateau and avoids crossing the stochastic edge. Mid-run changes that lower the effective threshold, such as increasing $$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$8, increasing $$\mathrm{NDS}(\theta):=\eta\cdot \mathrm{BS}(\theta).$$9, or decreasing $$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$0, produce catapults exactly when Batch Sharpness rises above the new plateau. Stabilizing changes reopen progressive sharpening until $$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$1 approaches the new, higher plateau. The paper’s empirical demonstrations on MLPs and CNNs up to ResNet-18 on CIFAR-10 and SVHN therefore present NDS not only as a descriptive statistic but also as a control variable for instability-adjacent training (Andreyev et al., 15 Apr 2026).

The main limitation is theoretical. Unlike vanilla SGD, where crossing $$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$2 is a proved sufficient instability criterion on quadratics, the momentum analysis does not provide a direct momentum-specific theorem making BS a formal certificate. Instead, the small-batch SGDM result is reduced to vanilla SGD with $$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$3.

4. Hessian Rayleigh-quotient NDS in optimizer comparison

In "Why Muon Outperforms Adam: A Curvature Perspective" (Wang et al., 3 Jun 2026), NDS is defined as the Rayleigh quotient of the Hessian along the optimizer’s realized update direction: $$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$4 The local second-order model is

$$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$5

so the curvature penalty can be decomposed as

$$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$6

Within this framework, the first-order gain is $$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$7, and NDS isolates the second-order curvature actually paid by the realized step rather than by a global spectral extremum (Wang et al., 3 Jun 2026).

This formulation is used to explain optimizer differences at matched validation loss on a 124M-parameter NanoGPT trained on FineWeb-10B. The paper reports that Adam and Muon have comparable first-order decreases, while Muon consistently incurs a smaller curvature penalty. Decomposing the penalty shows that update norms are comparable, so the curvature-penalty gap is driven by lower NDS for Muon rather than by smaller steps. At matched validation loss, the Adam/Muon ratio of NDS averages $$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$8, whereas the ratio of squared update norm stays near $$\mathrm{NDS}(\Delta\theta):=\frac{\Delta\theta^\top H\Delta\theta}{\|\Delta\theta\|^2}.$$9; over aligned training steps, the mean NDS ratio is $$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$0 (Wang et al., 3 Jun 2026).

The computational methodology is explicitly directional and avoids full Hessian materialization. At a training step $$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$1, with parameters $$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$2 and optimizer update $$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$3, the experiments compute the Hessian–vector quadratic form $$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$4 and normalize by $$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$5. Directional sharpness is computed every $$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$6 steps through Hessian–vector products. For Muon, $$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$7, where $$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$8 is the spectrally normalized momentum matrix, the polar factor $$r_t:=\frac{s_t^2}{\mathcal L_{\xi_t}}, \qquad \mathrm{NDS}_{\mathcal C}(\vec w_0,D,r) := \mathrm{Std}\Big(\{\log(r_t+\varepsilon)\}_{t=0}^{T-1}\Big).$$9 of $$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$0; Muon uses Newton–Schulz orthogonalization with $$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$1 iterations, momentum warmed from $$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$2 to $$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$3, and leaves embeddings and $$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$4 to Adam in the reported run (Wang et al., 3 Jun 2026).

The paper also decomposes total NDS into within-layer and cross-layer components,

$$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$5

with

$$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$6

$$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$7

For Muon, the cross-layer component drops faster, so the within-layer fraction rises from about $$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$8 early to about $$\mathrm{NDS}_x=\frac{1}{n}\sum_{i=1}^n \Delta G_{x,i}, \qquad \mathrm{NDS}_y=\frac{1}{n}\sum_{i=1}^n \Delta G_{y,i}.$$9 later, while Adam’s fraction stays comparatively stable at approximately $g_B$0. A layerwise localization analysis attributes roughly $g_B$1 of the within-layer Adam–Muon NDS gap to boundary layers $g_B$2 and $g_B$3, approximately $g_B$4 to deep layers $g_B$5–$g_B$6, and approximately $g_B$7 to middle layers $g_B$8–$g_B$9 (Wang et al., 3 Jun 2026).

The same paper links NDS to data imbalance. On Zipf-Probabilistic Context-Free Grammar data with imbalance exponent $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$00, the trajectory-averaged NDS rises with imbalance for both optimizers but much more for Adam. After normalization by Muon’s NDS at $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$01, Adam’s normalized NDS increases from $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$02 to $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$03, whereas Muon’s increases from $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$04 to $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$05; the normalized gap grows from $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$06 to $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$07. The theoretical explanation is given through a quadratic model with heterogeneous positive curvatures $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$08 and gradient alignment toward top-curvature modes. Muon’s spectral normalization equalizes amplitudes across active singular modes, so its per-step NDS becomes a group-size average,

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$09

whereas GD’s NDS is a residual-energy-weighted average,

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$10

Under the stated heterogeneity and alignment conditions, Muon has smaller average NDS than GD for any finite horizon and, when curvature heterogeneity is sufficiently strong, also attains lower local quadratic loss after the same number of steps (Wang et al., 3 Jun 2026).

5. Dynamic NDS for certification, auditing, and SAM stability

"Certification of Machine Learning Models via Directional Sharpness" (Tan et al., 23 Jun 2026) introduces directional sharpness as a dynamic, SAM-based generalization metric and adopts an empirical normalization by default. The procedure starts from initial parameters $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$11, dataset $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$12, public seed $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$13, batch size $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$14, number of steps $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$15, a per-step sharpness function $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$16, and a SAM update operator $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$17. For each step $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$18, it samples a mini-batch $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$19, computes the mini-batch loss $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$20, computes per-step sharpness $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$21, and updates

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$22

For SAM,

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$23

and for ASAM,

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$24

The probe history is then aggregated by a fluctuation statistic. The default normalization is

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$25

which is intended to reduce sensitivity to overall loss scale (Tan et al., 23 Jun 2026).

The paper’s theoretical analysis places this dynamic NDS in a local linearization regime near a minimum $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$26, under smoothness, a PL inequality, and bounded gradient noise: $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$27

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$28

For mini-batch SAM sharpness with $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$29 and radius $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$30, the per-step squared sharpness satisfies the sandwich bound

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$31

with

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$32

If a minimum is linearly SAM-stable, then $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$33 remains bounded by a constant times $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$34; conversely, exponential growth of $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$35 implies SAM-instability (Tan et al., 23 Jun 2026).

The detection mechanism is explicitly mini-batch-sensitive. With per-example gradients $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$36, average gradient $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$37, and coherence

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$38

the gap between RMS mini-batch SAM sharpness and full-batch sharpness is

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$39

When $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$40 and $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$41, this is approximately

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$42

This explains why dynamic directional sharpness can expose incoherence that static full-batch sharpness averages away (Tan et al., 23 Jun 2026).

Empirically, on CIFAR-10/100 with VGG-13/16/19-BN and WRN28-10 across SGD, Adam, SAM, and ASAM, the dynamic measure outperforms static baselines in correlation with generalization. Reported values include Spearman $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$43, Kendall $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$44, Kendall $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$45 for directional sharpness with $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$46, $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$47, compared with lower values for ASAM and magnitude-aware worst-case sharpness. In matched-accuracy benign/faulty pairs, the benign/faulty ratio is substantially smaller for the directional metric than for static sharpness in noisy labels, spurious-feature overfitting, backdoors, post-quantization quality, and small-test-set settings. The same work also makes the metric proof-compatible: a prover can commit to $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$48 and $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$49, execute the public NDS computation in-circuit, and prove the predicate

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$50

without revealing the data. For $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$51 and $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$52, proving NDS with the cited backend is reported as up to $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$53 faster than proving an entire training run (Tan et al., 23 Jun 2026).

6. NDS in no-reference satellite image sharpness assessment

In "A Novel No-Reference Image Quality Metric For Assessing Sharpness In Satellite Imagery" (Antonel, 2024), NDS is a compact name for the paper’s "normalized decay rate of gradients along pronounced edges." The grayscale image is

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$54

with spatial gradient

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$55

Edge detection and gradient estimation are performed with a Sobel operator $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$56. At an edge pixel, the unit normal and tangent are

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$57

The directional gradient along the normal is

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$58

The operational pipeline begins with preprocessing. High-frequency anomalies are filtered by a neighbors-based rule,

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$59

where

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$60

A low-high intensity mask excludes saturated and underexposed pixels,

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$61

Gradients on $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$62 are filtered by percentile masks $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$63, typically in the $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$64th to $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$65th percentiles, to isolate pronounced edges.

The core directional score is based on the change in edge gradients after controlled Gaussian blur. With Gaussian kernel $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$66 of size $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$67 and $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$68,

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$69

Using the same masks, the normalized decays are

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$70

and the directional sharpness scores in the paper’s percent scale are

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$71

The normalized scores are

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$72

so $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$73 and $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$74 (Antonel, 2024).

The normalization divides decay by the original edge gradient and therefore acts as a local contrast normalization. The paper also defines a representativeness indicator using a heavier blur with $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$75,

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$76

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$77

to filter images lacking sufficient usable edges. The intended application is constellation-wide quality monitoring, and the directional separation $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$78 supports diagnosis of motion blur versus defocus (Antonel, 2024).

The paper further provides an analytical step-edge model. For a unit step of contrast $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$79 blurred by a Gaussian PSF of width $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$80,

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$81

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$82

Applying additional Gaussian blur of width $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$83 changes the effective width to $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$84, so the peak normalized decay becomes

$\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$85

Sharper edges, corresponding to smaller $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$86, therefore exhibit larger normalized decay upon additional blur (Antonel, 2024).

7. Conceptual relations, distinctions, and limitations

Across these papers, NDS is consistently a directional quantity, but the direction and the normalization target are domain-specific. In momentum EoSS analysis, the direction is the stochastic mini-batch gradient and the normalization removes $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$87 scaling. In Muon-versus-Adam analysis, the direction is the optimizer’s actual update and the normalization divides out step magnitude. In certification, the direction is induced by a short SAM or ASAM trajectory, and normalization divides out per-step loss scale before taking a temporal fluctuation statistic. In satellite imagery, the directions are pronounced edge normals or their axis-aligned approximations, and normalization divides gradient decay by original edge strength. This suggests that NDS is better understood as a design pattern for directional sharpness than as a single canonical scalar.

Several distinctions are technically important. First, directional sharpness is not the same as $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$88. The EoSS paper explicitly shows that $\Delta\theta^\top H\Delta\theta/\|\Delta\theta\|^2$89 may fail to diagnose mini-batch instability transitions, whereas Batch Sharpness tracks catapult events (Andreyev et al., 15 Apr 2026). Second, the optimizer-comparison NDS of (Wang et al., 3 Jun 2026) is not a dynamic generalization certificate; it is a local Rayleigh quotient tied to a realized update. Third, the certification NDS of (Tan et al., 23 Jun 2026) is not a Hessian quadratic form at all, but a normalized fluctuation statistic over short SAM-style dynamics. Fourth, the image-quality NDS of (Antonel, 2024) is unrelated to learning dynamics despite sharing the same directional-normalized motif.

The limitations are likewise definition-specific. The EoSS momentum analysis relies on linearization near a minimizer, quadratic approximation of per-sample losses, interpolation, and mean-square stability with random curvature; extending the analysis to very large models remains future work (Andreyev et al., 15 Apr 2026). The Muon analysis studies local second-order behavior and stylized quadratic problems with heterogeneous curvature; its empirical results are reported for NanoGPT-scale models and controlled Zipf-PCFG settings (Wang et al., 3 Jun 2026). The certification framework gives a one-way stability-to-bounded-sharpness implication and requires threshold calibration for binary certificates, while dataset forgery is out of scope (Tan et al., 23 Jun 2026). The satellite metric can become unreliable under sparse edges, cloud cover, near-Nyquist texture, extreme noise, heavy compression, or spatially varying blur (Antonel, 2024).

Taken together, these works establish NDS as a versatile but non-unified concept. In each case, it is the normalization of a directionally meaningful sharpness signal that makes the resulting quantity operational: learning-rate invariant for EoSS boundaries, step-size invariant for optimizer-local curvature penalties, loss-scale stabilized for dynamic generalization auditing, and contrast-normalized for no-reference image sharpness assessment.