Sharpness-Aware Learnability (SAL)

• SAL is a research perspective that analyzes how neighborhood-robust objectives reshape learnability and generalization in models.
• It integrates techniques such as adjusting perturbation radii, adaptive step frequencies, and learning rate scaling to optimize training dynamics.
• SAL underpins advances in domains like meta-learning, self-supervision, and continual learning through practical strategies that improve downstream performance.

Sharpness-Aware Learnability, or SAL (Editor’s term), denotes an interpretive research perspective in which learnability is studied through the local geometry induced by sharpness-aware optimization: whether learning dynamics, generalization, adaptation, or forgetting improve when training is made robust to parameter perturbations or when the degree of sharpness-awareness is itself learned, scheduled, or decomposed. The cited literature does not use SAL as a standardized name, but it repeatedly addresses closely related questions in supervised learning, self-supervision, meta-learning, continual learning, machine unlearning, hybrid modeling, and spiking neural networks (Wang et al., 2024, Tran et al., 2023, Abbas et al., 2022).

1. Scope and conceptual status

SAL is best treated as an umbrella over several technical programs rather than as a single canonical framework. One line starts from Sharpness-Aware Minimization (SAM) and asks how neighborhood-robust objectives alter optimization and generalization. A second line asks how the strength, frequency, or parameterization of sharpness-aware perturbations should be chosen. A third line studies whether sharpness-aware objectives improve downstream learnability in settings where the target is not ordinary ERM, such as linear evaluation after self-supervised pretraining, bilevel meta-learning, or retain-versus-forget optimization (Wang et al., 2024, Zhao et al., 2022, Tran et al., 2023, Abbas et al., 2022).

A recurring feature of this literature is that “sharpness-awareness” is operationalized through local perturbation robustness, but the object whose learnability is at stake varies by domain. In self-supervised learning it is the downstream classifier induced by a representation; in meta-learning it is the adapted task predictor after few-shot updates; in unlearning it is retained utility under active forget interference; in hybrid models it is the recoverability of scientific parameters; and in spiking networks it is the transfer from surrogate-forward training to hard-spike deployment (Tran et al., 2023, Abbas et al., 2022, Tang et al., 16 Jun 2025, Takeishi, 6 Feb 2026, Nicholson, 14 Mar 2026).

Direction	Representative mechanism	Papers
Canonical sharpness-aware training	Neighborhood worst-case loss and normalized gradient perturbation	(Wang et al., 2024, Long et al., 2023)
Adaptive sharpness control	Learned radius, randomized schedules, adaptive stepsizes	(Wang et al., 2024, Zhao et al., 2022, Cheng et al., 30 May 2025, Yue et al., 2020)
Learnability theory under sharpness	Downstream SSL bounds, bilevel bounds, noise-learning separation	(Tran et al., 2023, Abbas et al., 2022, Chen et al., 2023)
Task-specific SAL instantiations	CL decomposition, unlearning, hybrid models, SNN transfer	(Chen et al., 12 Jan 2026, Tang et al., 16 Jun 2025, Takeishi, 6 Feb 2026, Nicholson, 14 Mar 2026)

This suggests that SAL is less a settled doctrine than a family of questions about how local robustness criteria reshape what is learnable, how much signal is needed, and which directions of parameter change are beneficial or harmful.

2. Foundational objectives and sharpness notions

The common starting point is the SAM min-max problem

$$\min_{\theta} \max_{\|\epsilon\| \leq \rho } L(D^{tr}; \theta + \epsilon),$$

with practical update

$$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$

so the perturbation radius $\rho$ determines the neighborhood over which local sharpness is probed (Wang et al., 2024). The literature repeatedly treats $\rho$ as the critical control variable: if it is too small, sharpness is weakly penalized; if too large, optimization can be distorted or over-regularized (Wang et al., 2024).

A more dynamical interpretation is given by the SAM edge-of-stability analysis. Under a local quadratic model, gradient descent has the usual stability threshold $\|H\|_{op} \approx 2/\eta$, whereas SAM yields a moving threshold that depends on $\eta$, $\rho$, and the current gradient norm $\|g\|$. The resulting “SAM-edge” shrinks as $\|g\|$ decreases, so the admissible curvature becomes progressively stricter during training (Long et al., 2023). In SAL terms, this makes sharpness-awareness a trajectory constraint rather than merely a property of final minima.

Several later papers argue that the fixed-radius formulation is itself not unique. “Loss-Equated SAM” replaces fixed parameter-space radius by a fixed loss-space budget $\sigma$, using

$$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$0

and derives the surrogate

$$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$1

The explicit claim is that standard SAM is dominated by the first-order term $$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$2, whereas LE-SAM removes that variable first-order contribution and shifts the effective signal toward curvature (Wang et al., 11 May 2026). This suggests that SAL depends not only on whether perturbations are used, but on how sharpness is parameterized.

A different criticism comes from the analysis of SAM’s approximation gap. “1st-Order Magic” argues that more precise approximations of the original SAM objective can degrade generalization performance, and that practical benefits may arise from the first-order boundary-point approximation rather than faithful optimization of the exact worst-case neighborhood loss (Tiwary et al., 2024). That claim weakens any simple identification of SAL with exact local max-loss minimization. Under this view, the learnability effect may come from the algorithmic surrogate actually optimized, not only from the idealized min-max objective.

3. Learning or controlling the sharpness mechanism

A major SAL theme is that sharpness-awareness need not be fixed. In LETS, the perturbation radius itself becomes a learned hyperparameter through bilevel optimization: $$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$3 subject to a SAM lower-level problem (Wang et al., 2024). LETS learns a single global radius, enforces positivity via $$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$4, and alternates weight updates with radius updates. On CIFAR-100 with WRN-28-10 it reports $$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$5 for LETS-SAM over SAM, and on ImageNet with ResNet-50 it reports $$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$6 (Wang et al., 2024). This is one of the clearest direct embodiments of SAL as “learning how much sharpness-awareness should be enforced.”

A second route is to adapt the frequency of sharpness-aware steps rather than the radius. Randomized Sharpness-Aware Training (RST) performs a Bernoulli trial at each iteration and chooses SGD or SAM with probability $$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$7, yielding expected propagation count $$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$8 per iteration (Zhao et al., 2022). Its generalized form, G-RST, decouples update frequency from effective sharpness regularization and is reported to outperform SAM in most cases while saving $$\theta_{t+1} = \theta_t - \eta \nabla L\!\left( D^{tr}; \theta_t + \rho \frac{\nabla L(D^{tr};\theta_t)} {\|\nabla L(D^{tr};\theta_t)\|} \right),$$9 extra computation cost (Zhao et al., 2022). This suggests that sharpness-aware learnability may be stage-dependent and intermittent rather than tied to “SAM every step.”

A third route is to adapt optimizer scale directly. SALR computes a local sharpness proxy and sets

$\rho$0

so learning rate increases in sharp regions and decreases in flatter ones (Yue et al., 2020). The method is reported to improve generalization, converge faster, and drive solutions to significantly flatter regions (Yue et al., 2020). LightSAM pursues a related goal from a different angle: it uses AdaGrad-Norm, AdaGrad, and Adam inside SAM so that perturbation radius and learning rate are adaptive, and claims that under weak assumptions it can converge ideally with any choices of perturbation radius and learning rate, achieving parameter-agnostic behavior (Cheng et al., 30 May 2025).

Method	Controlled quantity	Reported implication
LETS	Global radius $\rho$1 via squared generalization gap	Radius becomes learnable rather than hand-tuned (Wang et al., 2024)
RST / G-RST	Probability $\rho$2 of sharpness-aware steps	Intermittent sharpness-awareness can preserve gains with lower cost (Zhao et al., 2022)
SALR	Learning rate from local sharpness	Escapes sharper regions by enlarging steps there (Yue et al., 2020)
LightSAM	Adaptive perturbation radius and learning rate	SAM becomes parameter-agnostic under the paper’s assumptions (Cheng et al., 30 May 2025)

Taken together, these methods broaden SAL from a fixed objective into a control problem over extent, timing, and optimization scale.

4. Theoretical formulations of learnability under sharpness

The sharpest formal SAL statement in self-supervision appears in Sharpness & Shift-Aware Contrastive Learning. For contrastive SSL followed by linear evaluation, the paper proves that downstream supervised loss is upper-bounded by a sharpness-aware empirical InfoNCE term, together with complexity terms and a positive-pair distribution shift term (Tran et al., 2023). Its practical method, SSA-CLR, combines SAM-like perturbation with Fourier-based augmentation and reports linear-evaluation gains from $\rho$3 to $\rho$4 on CIFAR-10 and from $\rho$5 to $\rho$6 on CIFAR-100 over SimCLR (Tran et al., 2023). In SAL language, that result ties flatter self-supervised encoders to better downstream linear learnability.

In bilevel learning, Sharp-MAML imports SAM into both the lower adaptation and upper meta-update. For the one-step MAML objective $\rho$7, the paper proves

$\rho$8

and gives a PAC-Bayes-style bound of the form

$\rho$9

so meta-generalization depends explicitly on neighborhood empirical performance and lower-level stability (Abbas et al., 2022). Empirically, Mini-Imagenet 5-way 1-shot improves from $\rho$0 for reproduced MAML to $\rho$1 for Sharp-MAML$\rho$2, and 5-shot improves from $\rho$3 to $\rho$4 (Abbas et al., 2022). This makes SAL concrete for meta-learning: a learnable initialization is one that remains good under local perturbations throughout bilevel adaptation.

A different theoretical line studies sharpness-aware learnability in a noisy feature-learning regime. For a two-layer convolutional ReLU network with label-flip noise, the paper “Why Does Sharpness-Aware Minimization Generalize Better Than SGD?” shows that SGD achieves low test error only when

$\rho$5

whereas SAM can still achieve low test error in the weaker-signal regime

$\rho$6

(Chen et al., 2023). Its mechanism is not Hessian smoothing but perturbation-induced deactivation of noise-fitting neurons in the early stage of training. This implies that SAL can be a theory of feature selection under perturbation, not merely of converging to flatter minima.

These results share a common structure: local robustness enters the theory not only as a geometric regularizer, but as a constraint on which signals can accumulate reliably during learning.

5. Domain-specific instantiations

In continual learning, FLAD decomposes sharpness-aware perturbations into a gradient-aligned component and a stochastic-noise component, then keeps only the noise component (Chen et al., 12 Jan 2026). The method defines perturbations

$\rho$7

and uses the update $\rho$8 (Chen et al., 12 Jan 2026). Full FLAD requires $\rho$9 forward and $\|H\|_{op} \approx 2/\eta$0 backward passes per iteration, but the paper reports that applying FLAD in only $\|H\|_{op} \approx 2/\eta$1 of epochs already yields substantial gains over vanilla SGD, and that using FLAD for only $\|H\|_{op} \approx 2/\eta$2 of epochs reduces overhead by at least $\|H\|_{op} \approx 2/\eta$3 relative to full-method training (Chen et al., 12 Jan 2026). SAL here becomes directional: not every perturbation direction is equally beneficial for continual generalization.

In hybrid scientific machine learning, the paper “Learning Deep Hybrid Models with Sharpness-Aware Minimization” applies SAM only to the ML parameters $\|H\|_{op} \approx 2/\eta$4 in

$\|H\|_{op} \approx 2/\eta$5

leaving scientific parameters $\|H\|_{op} \approx 2/\eta$6 unperturbed (Takeishi, 6 Feb 2026). The induced objective is effectively

$\|H\|_{op} \approx 2/\eta$7

The reported effect is improved scientific-parameter recoverability without bespoke architecture-specific regularizers. On the wind-tunnel task, $\|H\|_{op} \approx 2/\eta$8-error moves from $\|H\|_{op} \approx 2/\eta$9 for ERM to $\eta$0 for SAM and $\eta$1 for FSAM; on the light-tunnel task, cosine similarity moves from $\eta$2 for ERM to $\eta$3 for SAM, ASAM, and FSAM (Takeishi, 6 Feb 2026). This suggests a variant of SAL in which flatness in the neural subspace improves learnability of the intended scientific decomposition.

In spiking neural networks, SAST applies SAM to a smooth surrogate-forward SNN so that the training gradient is exact for the optimized auxiliary model. The surrogate objective

$\eta$4

is then trained with the SAM objective

$\eta$5

and update

$\eta$6

using an independent second minibatch (Nicholson, 14 Mar 2026). The clearest empirical effect is transfer-gap reduction: on N-MNIST, hard-spike accuracy rises from $\eta$7 to $\eta$8 with best $\eta$9; on DVS Gesture, it improves from $\rho$0 to $\rho$1 with best $\rho$2 (Nicholson, 14 Mar 2026). Here SAL is tied to learnability of a deployed hard-spike model through flatness of a smooth surrogate training objective.

Machine unlearning produces perhaps the strongest interference-based SAL picture. One paper selects parameters for unlearning via the diagonal Hessian of the original learning loss, arguing that the least-sharp directions are the best coordinates to update; on CIFAR-10 targeted poisoning, “SA 1st order” reaches $\rho$3 versus $\rho$4 for last-layer first-order unlearning and $\rho$5 for fine-tuning (Malekmohammadi et al., 8 Apr 2025). A second paper, “Sharpness-Aware Machine Unlearning,” shows that under retain/forget interference SAM enlarges the signal regime in which retain information remains learnable, but loses its denoising effect on the forget set itself. It then proposes Sharp MinMax, which learns retain signals with SAM and unlearns forget signals with sharpness maximization, and reports the best performance across forget-set difficulty levels measured by memorization (Tang et al., 16 Jun 2025). These results make SAL explicitly asymmetric: flatness helps retention and generalization, while sharper fitting can help stringent sample-specific forgetting.

6. Empirical picture, controversies, and open directions

Across the literature, SAL is associated with lower generalization gaps, flatter landscapes, or lower curvature proxies, but these benefits are neither uniform nor uncontested. In LETS, smaller validation-versus-training discrepancy is tied to better performance and flatter loss landscapes, yet the method depends on validation data and learns only a single global scalar radius (Wang et al., 2024). In RST and FLAD, intermittent sharpness-awareness often preserves gains with lower overhead, indicating that sharpness control may be important mainly at certain stages rather than continuously (Zhao et al., 2022, Chen et al., 12 Jan 2026).

A major controversy concerns what mechanism actually explains SAM. One line argues that the practical first-order approximation itself is the source of benefit and that more accurate optimization of the original max-loss objective can degrade generalization (Tiwary et al., 2024). Another line argues that fixed-radius SAM is mismatched to the second-order nature of flatness and should be replaced by fixed loss-budget perturbations, as in LE-SAM (Wang et al., 11 May 2026). These positions are compatible only at a high level: both reject a simplistic reading of SAL as mere faithful minimization of the canonical SAM objective.

A second controversy concerns memorization and privacy. “On Memorization and Privacy Risks of Sharpness Aware Minimization” reports that SAM’s gains are concentrated on atypical, low-influence-entropy test points that rely more heavily on memorized training support, and that best single-query membership attack accuracy rises under SAM: on CIFAR-100, from $\rho$6 for SGD to $\rho$7 for SAM; on CIFAR-10, from $\rho$8 to $\rho$9 (Kim et al., 2023). This implies that sharpness-aware learnability can improve accuracy while redistributing learnability toward memorization-heavy or privacy-sensitive examples.

A third controversy concerns what aspect of a perturbation is useful. FLAD argues that the stochastic-noise component, rather than the gradient-aligned component, is the beneficial part of SAM/GAM-style perturbations in continual learning (Chen et al., 12 Jan 2026). The unlearning literature goes further and shows that different subsets of parameters or different sharpness biases may be desirable for retain and forget objectives (Malekmohammadi et al., 8 Apr 2025, Tang et al., 16 Jun 2025). This weakens any one-size-fits-all notion of sharpness-aware training.

The open directions are correspondingly consistent across papers. Repeatedly identified problems include learning richer sharpness-control structures beyond a single scalar radius, reducing or eliminating dependence on held-out validation signals, characterizing the effect of crude Hessian approximations, adapting schedules online rather than fixing them a priori, and deriving stronger theory that links learned sharpness-control variables to true generalization rather than only to optimization convergence or surrogate criteria (Wang et al., 2024, Zhao et al., 2022). A broader implication is that SAL is likely to remain a plural concept: some settings favor flatness, some favor selective sharpness, and many require learning not only model parameters but the geometry of the training rule itself.