ASP-KAN-HAQ: Active Subspace Embedded KAN

• The paper introduces asKAN, a hierarchical framework that embeds active subspace analysis within KAN architecture to better model ridge functions without expanding network size.
• It interleaves lightweight, non-parametric linear projections with standard KAN blocks, reducing errors by up to an order of magnitude across benchmarks in function fitting and equation solving.
• The approach maintains a constant parameter count while adaptively aligning internal network representations with principal input directions, thereby significantly enhancing performance.

The active subspace-embedded Kolmogorov–Arnold Network (asKAN) is a hierarchical neural network framework that synergizes the Kolmogorov–Arnold Network (KAN) functional representation with active subspace methodology. asKAN is designed specifically to overcome representational inflexibility observed in standard KANs when modeling ridge functions, which commonly underlie the relationships in physical systems. The architecture interleaves KAN blocks with explicit linear projections that adaptively align network representations along principal “ridge” directions, as identified by active subspace analysis. This hierarchical, iterative approach preserves the original model size, introducing only lightweight, non-parametric projections while substantially enhancing performance across canonical benchmarks in function fitting, equation-solving, and scientific data reconstruction (Zhou et al., 7 Apr 2025).

1. Kolmogorov–Arnold Theorem and Network Implementation

The Kolmogorov–Arnold superposition theorem guarantees that any continuous function $f:\;[0,1]^N\;\to\;\mathbb{R}$ can be expressed as

$$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$

where each $\phi_{q,m}$ and $\varphi_q$ is a univariate function. In Kolmogorov–Arnold Networks (KAN), these univariate maps are parameterized as linear combinations of local B-spline basis functions: $$\phi_{q,m}(x_m)\approx\sum_{i=1}^K c_{q,m}^i\,B^i(x_m)\,,$$ with $B^i$ the fixed splines and $c_{q,m}^i$ the learned coefficients.

A KAN layer of width $N_{l+1}$ mapping $\mathbf{x}^l\in\mathbb{R}^{N_l}$ computes

$$x_j^{l+1} = \sum_{i=1}^{N_l}\phi^l_{j,i}\bigl(x_i^l\bigr), \quad j=1,\dots,N_{l+1}\,,$$

and deep KANs are compositions of such layers: $$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$0 Standard KANs thus approximate multivariate functions by summing univariate coordinate-wise splines. The limitation is that such architectures inadequately capture dependencies that require linear combinations of inputs—a scenario prevalent in ridge functions (Zhou et al., 7 Apr 2025).

2. Ridge Functions and the Active Subspace Method

A ridge function is defined as

$$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$1

depending solely on a single linear combination of the input vector. Ridge functions frequently occur in physical contexts where intrinsic dependencies are aligned with particular directions in input space. Standard KANs, relying on univariate mappings, require considerably expanded architectures to represent such dependencies efficiently.

The active subspace method (ASM) determines principal directions along which a function $$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$2 exhibits maximal variation. It constructs the covariance matrix of gradients: $$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$3 where the expectation is taken over the input domain. The dominant eigenvectors $$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$4 (columns of $$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$5) span the “active subspace”, capturing the directions most pertinent for function approximation. ASM thus furnishes an efficient, data-driven dimensionality reduction, particularly suited for functions of ridge type.

3. asKAN Architecture: Hierarchical Embedding of Active Subspaces

The asKAN architecture interleaves KAN blocks and active subspace projections across $$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$6 levels. At level $$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$7, a KAN block $$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$8 maps the data, after which an active subspace analysis determines the projection matrix $$f(\mathbf{x}) = \sum_{q=1}^{2N+1} \varphi_q\Bigl(\,\sum_{m=1}^{N}\phi_{q,m}(x_m)\Bigr)\,, \quad \mathbf{x}=(x_1,\dots,x_N)\,,$$9. The procedure at each level is as follows:

1. Train $\phi_{q,m}$0 on current inputs $\phi_{q,m}$1: $\phi_{q,m}$2.
2. Estimate the active subspace covariance:

$\phi_{q,m}$3

and compute the orthogonal eigenbasis $\phi_{q,m}$4.

1. Project inputs:

$\phi_{q,m}$5

The complete asKAN mapping is

$\phi_{q,m}$6

This approach adapts the network’s internal representation at each step, aligning subsequent training to active directions and facilitating efficient ridge-function modeling (Zhou et al., 7 Apr 2025).

4. Iterative Subspace Projection and Training Workflow

At each level, having trained the KAN on the projected data, the method computes the local active subspace, projects data accordingly, and repeats. The resulting workflow can be described as:

$\varphi_q$8

At each level, all KAN blocks are architecturally identical to the baseline model; the only addition is a non-parametric linear projection. The projection matrices $\phi_{q,m}$7 are recalculated using the locally trained KAN’s output, allowing the network to adaptively align with principal features of the data through successive iterations.

5. Model Size and Parameter Efficiency

A distinguishing property of asKAN is that it maintains constant network size with respect to the baseline KAN. Each KAN block (width, depth, spline basis cardinality) remains fixed. The additional active subspace projections $\phi_{q,m}$8 introduce no new neurons, trainable parameters, or additional layers. This results in an architecture whose parameter count and spline coefficient inventory are unchanged from the original KAN, while providing enhanced representational capabilities (Zhou et al., 7 Apr 2025). The computational overhead from the projections is negligible for both inference and training.

6. Empirical Performance on Benchmark Problems

asKAN has been validated across canonical scientific computing tasks where ridge structure is prominent:

Problem	Network Shape	KAN Final Error	asKAN Final Error
2D Ridge Function Fitting	{2→5→1}	≈ $\phi_{q,m}$9	≈ $\varphi_q$0
Poisson Equation Solver	{2→3→1}	≈ $\varphi_q$1	≈ $\varphi_q$2
Sound-Field Reconstruction	{3→5→1}	≈ $\varphi_q$3	≈ $\varphi_q$4

Details:

• Function fitting: Fitting $\varphi_q$5 on 2D data with cubic B-spline KAN, asKAN reduced test error by an order of magnitude and progressively aligned hidden nodes with the true ridge direction.
• Poisson equation: Solution expressed as a sum of two ridge waves. asKAN decreased total loss from $\varphi_q$6 to $\varphi_q$7, showing a substantial reduction in pointwise absolute error.
• Sound-field reconstruction: Predicting spatio-temporal pressure fields using 20,000 data points, asKAN attained converged losses an order of magnitude lower than the baseline, and substantially reduced snapshot errors (Zhou et al., 7 Apr 2025).

For all tasks, the network topology (layers, neurons, spline degree) was strictly controlled to isolate the impact of active subspace embedding.

7. Significance, Limitations, and Prospective Directions

The asKAN method demonstrates that the explicit incorporation of active subspace projections within KAN architectures yields significantly improved performance on ridge-structured learning problems, all without increasing model complexity. This substantiates the hypothesis that interleaving function-approximation blocks and data-driven, non-trainable projections systematically realigns network representations to the most informative directions in input space.

A plausible implication is that asKAN’s core approach may benefit other architectures that suffer from similar directional inflexibility. However, asKAN’s empirical benefits were demonstrated on low- and moderate-dimensional scientific applications, and further testing is required for larger-scale or non-ridge dominated scenarios. The methodology preserves exact architectural match to the baseline, ensuring that observed improvements derive exclusively from the iterative subspace reorientation, not increased expressive capacity through parameter expansion (Zhou et al., 7 Apr 2025).