Basis-Agnostic Curvature Penalty

• Basis-Agnostic Curvature Penalty is a regularization concept where curvature is measured on intrinsic geometric or functional objects independent of any coordinate or basis representation.
• In Kolmogorov–Arnold networks, the approach penalizes edge-wise curvature directly, reducing oscillations and kinks to enhance the model’s smoothness and interpretability.
• Related formulations extend the idea to optimizer analysis, geometric corrections, and discrete settings, demonstrating its versatility across neural networks, control systems, and combinatorial structures.

Basis-agnostic curvature penalty denotes a curvature-sensitive construction whose value, geometric meaning, or regularizing effect is defined independently of a particular coordinate basis, parameter basis, or overcomplete representation. In the present literature, the term is most explicitly realized as an edge-wise smoothness regularizer for Kolmogorov–Arnold networks (KANs), where curvature is penalized at the level of the learned activation functions rather than at the level of a specific spline parameterization (Bagrow, 4 May 2026). Closely related ideas also appear as a directional Hessian penalty in optimizer analysis (Wang et al., 3 Jun 2026), as a geometric correction determined by the curvature of a moving electronic subspace (Halliday et al., 2021), as a metric deformation of one-hot label space (Sheehan et al., 2018), as a minimal-basis problem for curvature-dependent total derivatives in six-dimensional anomaly integration (Ferreira et al., 2018), as an intrinsic variational curvature operator in optimal control (Agrachev et al., 2013), and as Ollivier–Ricci curvature for basis exchange walks on matroids (Detherage, 23 Sep 2025).

1. Conceptual scope

The literature does not use a single universal definition across all domains. Instead, several papers employ the same structural idea: curvature should be attached to an intrinsic object—such as a learned function, an update direction, a bundle connection, a metric tensor, or a minimal invariant basis—rather than to an arbitrary presentation. A plausible synthesis is that a basis-agnostic curvature penalty is any construction in which curvature is measured on the underlying geometric or functional object and only subsequently expressed in a chosen representation.

Domain	Curvature quantity	Basis-agnostic mechanism
KANs	$\sum_e \int_{\Omega_e} (\phi_e''(z))^2\,dz$	any basis with a curvature Gram matrix
Optimizer analysis	$\frac12\langle Z,\mathcal H[Z]\rangle$ and NDS	curvature along the realized update direction
Moving-basis Ehrenfest dynamics	$\Theta^\mu_{\ \nu jk}$ and Pulay terms	connection, curvature, and projector of the subspace manifold
Label-space geometry	metric tensor $g_{\alpha\beta}$	geometry changes while one-hot coordinates remain
6D anomaly integration	independent total-derivative basis	redundant curvature directions are removed

In the explicit regularization setting, the goal is usually to suppress oscillation, sharpness, or pathological roughness without tying the regularizer to a specific basis expansion. In the geometric setting, the point is different: curvature terms arise as intrinsic consequences of the manifold or bundle structure, and basis-agnostic language is used to distinguish geometric content from coordinate artifacts. In the algebraic setting, basis-agnosticity means eliminating redundant invariant directions before assigning coefficients to curvature-dependent terms.

2. Edge-wise curvature regularization in Kolmogorov–Arnold networks

The clearest explicit basis-agnostic curvature penalty appears in the analysis of KANs, where learned edge activations often develop pathologically high-curvature oscillations and kink-like artifacts that do not usually hurt predictive accuracy but do make the edge functions hard to interpret (Bagrow, 4 May 2026). Standard KAN regularizers act only on activation magnitudes and edge sparsity, so they do not control derivatives. The proposed remedy is to penalize the curvature of each learned univariate edge function directly.

For a sufficiently smooth function $g$, the paper uses the bending-energy surrogate

$\int (\nabla^2 g(x))_F^2\,dx,$

which in the univariate case becomes

$\int (g''(z))^2\,dz.$

For a KAN edge $e$ with activation $\phi_e$ supported on $\Omega_e$, the conceptual penalty is

$\frac12\langle Z,\mathcal H[Z]\rangle$0

This is already basis-agnostic in the functional sense: the penalty is defined on the activation itself, not on a particular coefficientization.

The paper then specializes to the standard KAN parametrization

$\frac12\langle Z,\mathcal H[Z]\rangle$1

with spline basis $\frac12\langle Z,\mathcal H[Z]\rangle$2, learnable coefficients $\frac12\langle Z,\mathcal H[Z]\rangle$3, and scalar weights $\frac12\langle Z,\mathcal H[Z]\rangle$4. Expanding the curvature gives

$\frac12\langle Z,\mathcal H[Z]\rangle$5

where $\frac12\langle Z,\mathcal H[Z]\rangle$6 and $\frac12\langle Z,\mathcal H[Z]\rangle$7. The practical regularizer uses the P-spline second-difference approximation for the spline term and drops the cross term, yielding a penalty of the form

$\frac12\langle Z,\mathcal H[Z]\rangle$8

with

$\frac12\langle Z,\mathcal H[Z]\rangle$9

The inclusion of the SiLU term is essential in the paper’s formulation, because a spline-only penalty would leave the base function unconstrained.

The paper’s basis-agnostic claim is stronger than the spline implementation. If the basis has square-integrable second derivatives, then

$\Theta^\mu_{\ \nu jk}$0

Thus the curvature penalty is a quadratic form determined by the basis-specific curvature Gram matrix $\Theta^\mu_{\ \nu jk}$1, not by any special property of B-splines. The appendix gives the corresponding construction for FastKAN’s Gaussian-RBF basis.

A major technical contribution is the connection between this edge-wise penalty and the curvature of the full composed model. For scalar composition,

$\Theta^\mu_{\ \nu jk}$2

the second derivative is

$\Theta^\mu_{\ \nu jk}$3

The first term exhibits the amplification effect produced by steep intermediate activations. For a depth-$\Theta^\mu_{\ \nu jk}$4 vector-valued composition $\Theta^\mu_{\ \nu jk}$5, the output Hessian decomposes as

$\Theta^\mu_{\ \nu jk}$6

and, for KAN layers with diagonal Hessians,

$\Theta^\mu_{\ \nu jk}$7

This yields the composition-level curvature

$\Theta^\mu_{\ \nu jk}$8

together with the upper bound

$\Theta^\mu_{\ \nu jk}$9

Under bounded coefficient-of-variation of the path weights, bounded density of activation inputs on each support interval, and reasonable knot spacing, the paper proves

$g_{\alpha\beta}$0

with

$g_{\alpha\beta}$1

The weighted variant

$g_{\alpha\beta}$2

retains the same smoothness notion while incorporating compositional importance. Empirically, the weighted version outperformed the uniform one in 9 of 10 seeds and more than halved mean test RMSE in the reported comparison. The paper also reports that curvature-penalized KANs were the smoothest model in every equation of the Feynman benchmark, had the lowest total edge curvature in all 14 equations, and competitive or better RMSE in 9/14. In the overparameterized regime, at $g_{\alpha\beta}$3 the curvature penalty lowered test RMSE for all $g_{\alpha\beta}$4 values relative to the KAN penalty and enabled stable single-stage training without grid extension, for both Adam and L-BFGS.

3. Directional curvature as optimizer-induced penalty

A different but closely related use of the idea appears in the curvature-based comparison of Muon and Adam, where the relevant quantity is not function-space bending energy but the second-order loss term incurred by an optimizer’s update direction (Wang et al., 3 Jun 2026). The paper starts from the second-order Taylor approximation

$g_{\alpha\beta}$5

where $g_{\alpha\beta}$6 is the realized one-step loss decrease, $g_{\alpha\beta}$7 is the first-order decrease, and

$g_{\alpha\beta}$8

is the second-order curvature penalty.

The paper’s key claim is that Muon’s advantage over Adam is not a substantially larger first-order gain but a smaller second-order curvature penalty. To separate curvature from step magnitude, it defines Normalized Directional Sharpness (NDS),

$g_{\alpha\beta}$9

so that

$g$0

This is basis-agnostic in the paper’s sense because curvature is evaluated along the actual update direction produced by the optimizer, rather than through coordinate-wise sharpness.

Empirically, Muon and Adam have comparable update norms and similar first-order decreases at matched validation loss, but Muon incurs a much smaller second-order term. In the main matched-validation-loss plot, the Adam-to-Muon NDS ratio averages about $g$1, while the squared-norm ratio stays near $g$2. The paper therefore attributes Muon’s advantage to lower NDS rather than smaller step size.

The same framework is used to study how data structure and network structure affect curvature. On Zipf-PCFG data with controllable imbalance, Adam’s normalized NDS increases from about $g$3 to $g$4 as the imbalance parameter $g$5 goes from $g$6 to $g$7, while Muon’s increases only from $g$8 to $g$9; the gap grows from $\int (\nabla^2 g(x))_F^2\,dx,$0 to $\int (\nabla^2 g(x))_F^2\,dx,$1. The paper concludes that data imbalance amplifies Muon’s NDS advantage.

A within-/cross-layer decomposition further refines the curvature picture. Writing the Hessian in layer blocks $\int (\nabla^2 g(x))_F^2\,dx,$2, the paper decomposes NDS into within-layer and cross-layer parts:

$\int (\nabla^2 g(x))_F^2\,dx,$3

For Muon, cross-layer NDS drops quickly during training, and in middle and late stages the lower total NDS is mainly sustained by smaller within-layer curvature. The paper also reports that the within-layer Adam–Muon gap is concentrated in boundary layers and deeper layers, especially the first and last layers.

The quadratic theory offered in the paper gives a structural explanation. Under a low Kronecker-rank Hessian, approximate simultaneous diagonalization, curvature heterogeneity with high- and low-curvature groups, and gradient alignment toward high-curvature modes, Muon equalizes singular-mode amplitudes while GD concentrates more energy in sharper modes. The resulting theorem states that Muon attains smaller average NDS than GD over every finite horizon and, under sufficient curvature heterogeneity, also yields lower local quadratic loss after the same number of steps. This suggests that, in optimizer analysis, a basis-agnostic curvature penalty is best understood as curvature encountered along the realized trajectory rather than curvature attached to fixed coordinates.

4. Geometric formulations beyond explicit regularization

Several papers develop basis-agnostic curvature constructions that are not penalties added by hand but exact geometric or metric consequences of the chosen formalism. In moving-basis Ehrenfest dynamics, the electronic states span a family of subspaces $\int (\nabla^2 g(x))_F^2\,dx,$4 or $\int (\nabla^2 g(x))_F^2\,dx,$5 forming a fiber bundle over nuclear configuration space, and ordinary derivatives are replaced by covariant ones (Halliday et al., 2021). The bundle curvature is

$\int (\nabla^2 g(x))_F^2\,dx,$6

and the Ehrenfest force becomes

$\int (\nabla^2 g(x))_F^2\,dx,$7

The velocity-dependent term is interpreted as intrinsic curvature, while Pulay terms are associated with extrinsic curvature through the complement projector $\int (\nabla^2 g(x))_F^2\,dx,$8. The paper explicitly states that this is not a penalty added by hand; rather, the geometric correction is an exact consequence of working on a curved moving subspace. Its basis-agnostic character comes from the use of the connection $\int (\nabla^2 g(x))_F^2\,dx,$9, curvature $\int (g''(z))^2\,dz.$0, and projector $\int (g''(z))^2\,dz.$1 as geometric objects attached to the manifold of electronic subspaces.

In curved label-space geometry, the basis is left unchanged but the metric is modified (Sheehan et al., 2018). One-hot labels remain the coordinate system, yet the Euclidean metric is replaced by a symmetric metric tensor $\int (g''(z))^2\,dz.$2 so that

$\int (g''(z))^2\,dz.$3

With $\int (g''(z))^2\,dz.$4, the distance between one-hot classes becomes

$\int (g''(z))^2\,dz.$5

The paper defines curved quadratic error and curved cross-entropy, with off-diagonal metric entries parameterized by a confusion-derived similarity,

$\int (g''(z))^2\,dz.$6

and smooths the confusion statistics by an exponential moving average. This is basis-agnostic because the representation basis is unchanged while the geometry is altered. It is also learning-algorithm-agnostic in the paper’s terminology because it can be bolted onto existing models. The reported experiments on CIFAR-10 and CIFAR-100 found no significant improvement in performance compared with a flat label space.

A more abstract geometric formulation appears in the variational definition of curvature for affine optimal control systems, which the paper describes as coordinate free, state-feedback invariant, intrinsic, and independent of the chosen presentation (Agrachev et al., 2013). The curvature operator is extracted from the small-time asymptotics

$\int (g''(z))^2\,dz.$7

with

$\int (g''(z))^2\,dz.$8

In the Riemannian case, the paper recovers

$\int (g''(z))^2\,dz.$9

while the same construction extends to Finsler, sub-Riemannian, and sub-Finsler settings. The paper does not introduce a regularizer, but its framework provides an intrinsic curvature operator from which a penalty functional could naturally be constructed. This suggests a broad geometric meaning of basis-agnostic curvature: curvature is derived from intrinsic second variation rather than from a preferred frame.

5. Minimal invariant bases and redundant curvature directions

In six-dimensional conformal-anomaly integration, the relevant basis-agnostic issue is not a smoothness prior but the elimination of redundant curvature-dependent total derivatives before assigning coefficients to them (Ferreira et al., 2018). The anomaly in even dimension $e$0 is decomposed schematically as

$e$1

where the $e$2 are total-derivative curvature terms. Because the anomaly-induced effective action integrates these surface terms through local pieces, an overcomplete set of $e$3 would introduce spurious ambiguity.

In four dimensions there is only one such term, $e$4. In six dimensions the literature had used eight curvature-dependent total derivatives, $e$5, including terms of schematic form

$e$6

together with covariant double divergences of quadratic curvature tensors. The paper shows that these eight objects are not linearly independent. The key identity is

$e$7

Equivalently,

$e$8

Thus the true basis has seven elements rather than eight.

The derivation expands each $e$9 into scalar invariants $\phi_e$0 and solves for linear combinations

$\phi_e$1

finding

$\phi_e$2

up to normalization. The appendix also provides related total-derivative identities, such as

$\phi_e$3

This suggests that any curvature-dependent functional on the six-derivative surface sector should be written on an independent basis only. In the paper’s own anomaly-integration context, the reduction from eight coefficients to seven clarifies the structure of the effective action and the local counterterm freedom. More generally, it exemplifies a central basis-agnostic principle: curvature-dependent constructions should be defined on the independent content of the invariant sector, not on an overcomplete parametrization.

6. Discrete and combinatorial curvature

A discrete analogue appears in the study of Ollivier–Ricci curvature for basis exchange walks on matroids (Detherage, 23 Sep 2025). The state space is the set of bases $\phi_e$4 of a matroid $\phi_e$5, and the basis exchange walk chooses $\phi_e$6 uniformly, deletes it, and then chooses uniformly a basis containing $\phi_e$7. Curvature is defined via Wasserstein contraction:

$\phi_e$8

where $\phi_e$9 is the graph metric induced by the chain.

The paper proves a universal lower bound. For a rank-$\Omega_e$0 matroid over a ground set of size $\Omega_e$1,

$\Omega_e$2

while for $\Omega_e$3,

$\Omega_e$4

A sharper corollary makes the local exchange geometry explicit:

$\Omega_e$5

This formulation shows that high overlap in the local neighbor sets promotes positive curvature.

The paper also gives an upper bound that can certify negative curvature:

$\Omega_e$6

and therefore a sufficient condition for negative curvature when

$\Omega_e$7

Examples of nonnegative curvature include rank-2 matroids, any matroid with $\Omega_e$8, the uniform matroid, the Vámos matroid, and graphic matroids induced by graphs with edge-disjoint cycles. Examples of negative curvature include a rank-3 $\Omega_e$9-linear matroid with

$\frac12\langle Z,\mathcal H[Z]\rangle$00

and the graphic matroid on $\frac12\langle Z,\mathcal H[Z]\rangle$01 with

$\frac12\langle Z,\mathcal H[Z]\rangle$02

The paper does not introduce a formal basis-agnostic curvature penalty by name. However, its structural message is closely aligned with that language: curvature is controlled by local exchange asymmetry rather than by representability, strong log-concavity, or any preferred presentation of the matroid. This suggests a discrete version of the same principle seen elsewhere in the literature: curvature is most naturally attached to intrinsic transition geometry, not to a chosen basis description.