Don't Fix the Basis -- Learn It: Spectral Representation with Adaptive Basis Learning for PDEs

Abstract: Spectral neural operators achieve strong performance for PDE learning, but rely on fixed global bases that limit their ability to represent spatially heterogeneous and multiscale dynamics. We propose Adaptive Basis Learning (ABLE), a framework that learns data-dependent spectral representations instead of relying on predefined bases. ABLE constructs a spatially adaptive Parseval frame via a learned ancillary density, enabling the operator to act in a lifted spectral space while preserving invertibility and maintaining $O(N\log N)$ complexity through FFT-based implementation. This shifts the source of expressivity from spectral coefficients to the representation itself, allowing the model to capture localized structures and non-translation-invariant interactions more efficiently. ABLE integrates seamlessly into existing neural operator architectures as a drop-in replacement for spectral layers. Across a range of benchmarks ABLE improves accuracy over strong baselines, with the largest gains in regimes characterized by sharp gradients and multiscale behavior. Moreover, augmenting existing models (e.g., U-FNO, HPM) with ABLE further enhances their performance, demonstrating its role as a general and complementary spectral refinement. Our results highlight that the data-driven choice of representation, rather than operator complexity alone, is a key bottleneck in neural operator design. By learning the basis itself, ABLE provides a principled and efficient framework for improving spectral methods in PDE learning.

• The paper introduces an adaptive basis learning (ABLE) framework that leverages spatially modulated spectral representations to overcome the limitations of fixed bases in modeling heterogeneous PDE solutions.
• The paper employs a parameterizable density function and an MLP-Softmax scheme to learn non-translation-invariant basis functions, significantly reducing error in challenging benchmarks like Burgers', Darcy flow, and turbulent Navier-Stokes.
• The paper demonstrates that the ABLE framework preserves FFT compatibility and computational efficiency while achieving substantial accuracy improvements over traditional spectral neural operators.

Adaptive Basis Learning for Spectral Neural Operators: A Review of "Don't Fix the Basis -- Learn It" (2605.10451)

Motivation and Background

Spectral neural operators, particularly the Fourier Neural Operator (FNO), have become foundational for learning solution operators of parametric PDEs, enabling efficient and scalable surrogates across scientific and engineering domains. These approaches project functions onto predetermined global spectral bases (e.g., Fourier, Chebyshev, Laplacian eigenfunctions), and operate by learning to modify spectral coefficients in that fixed representation. While this structure ensures efficiency, stability, and favorable computational scaling (typically $\mathcal{O}(N \log N)$ via FFT), it fundamentally restricts expressivity—especially in regimes dominated by spatial heterogeneity, sharp transitions, or multiscale dynamics. The fixed, global basis cannot efficiently or robustly encode localized or non-translation-invariant phenomena, often requiring prohibitively many high-frequency modes and suffering from truncation artifacts. The work under review introduces a distinct paradigm: learning the basis itself, thereby shifting the locus of modeling expressivity from operator design to representation learning.

The ABLE Framework: Theory and Construction

The authors introduce Adaptive Basis Learning (ABLE), a framework in which the representation basis is parameterized and learned from data, functioning as a spatially adaptive, input-dependent Parseval frame. This learnable basis is constructed through a parameterizable density $p(x, y)$, inducing an overcomplete, spatially modulated set of basis functions:

$e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$

with normalization $\int p(x, y) d y = 1$ for all $x$ in the spatial domain. This construction forms a Parseval tight frame, ensuring invertibility, energy preservation, and compatibility with FFT-based implementation. The transformation

$$A: f(x) \mapsto f_{k, y} = \langle f, e_{k, y} \rangle,$$

and its inverse, jointly define a bijective, isometric map between $L^2$ spaces, generalizing classical spectral decompositions. Crucially, by learning $p(x, y)$ (typically via an MLP-Softmax scheme with temperature control), ABLE can modulate the representation to fit localized, multiscale, or irregular structures in the data.

Importantly, the associated neural operator,

$$\mathcal{G}_{\text{ABLE}} = A^{-1} \circ R \circ A,$$

acts as a spectral multiplier composed in the adaptive spectral space. The resulting kernel is generally non-translation-invariant and spatially heterogeneous, permitting modeling of operators far beyond convolutional or global spectral mechanisms. When $p(\cdot, \cdot)$ is constant, the standard FNO is recovered as a strict special case, establishing the theoretical strict inclusion of FNO and its fixed-basis spectral counterparts as a subset of ABLE.

The temperature parameter in the Softmax induces an emergent "phase-like" behavior: at $p(x, y)$0, the basis selection becomes deterministic, yielding highly localized representation suitable for discontinuities; at $p(x, y)$1, it approaches a uniform mixture, akin to multi-head FNOs but without adaptivity.

Empirical Results and Numerical Performance

Empirical evaluation spans 1D Burgers', 2D Darcy flow, and 2D Navier-Stokes (NS, low-viscosity, turbulent regime) benchmarks. The experiments demonstrate several strong claims:

• Substantial reduction in error in regimes requiring high-fidelity localized or discontinuous representation. For example, on the challenging Burgers’ equation with low viscosity ($p(x, y)$2), ABLE reduces the relative $p(x, y)$3 error from approximately $p(x, y)$4 (FNO) to $p(x, y)$5, outperforming all tested spectral and hybrid baselines at comparable computational complexity.
• Consistent improvements even on elliptic, diffusion-dominated problems (Darcy). Despite the lack of high-frequency, localized phenomena, ABLE reduces error by $p(x, y)$6 relative to FNO, and further improves strong hybrid models such as U-FNO.
• Best-in-class results in highly multiscale, turbulent Navier-Stokes flows (relative $p(x, y)$7 error reduced by roughly $p(x, y)$8 vs. FNO2D, and $p(x, y)$9 improvement when augmenting HPM). Notably, ABLE integrates with FFT-compatible latent-spectral models at negligible overhead, enabling both standalone and plug-and-play use.

Ablation studies show that moderate increases in learned basis size $e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$0 and tuning of the temperature $e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$1 can improve results, but excessive capacity can lead to overfitting; optimal adaptivity is achieved at intermediate temperature values, affirming the inductive bias induced by soft basis selection.

Theoretical Implications and Approximation Theory

The paper provides formal analysis showing that ABLE admits a strictly larger class of operators than FNO, enabling spatially heterogeneous, nonlinear, and non-translation-invariant kernel structures. Approximation theory results show $e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$2 convergence in $e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$3 for BV-class functions for $e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$4 by solely increasing basis size $e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$5, whereas FNO can be order-limited for non-smooth targets and lacks such convergence for $e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$6. Hence, ABLE is particularly suited for operator learning in settings where solutions are in non-smooth or even non-$e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$7 function classes.

The endogenous nonlinearity induced by an input-dependent basis (i.e., $e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$8) provides a new avenue for implicit nonlinearity and input-conditional adaptation, distinct from stacking nonlinear layers.

Integration with Existing Architectures and Computational Considerations

A central technical advantage is that ABLE retains $e_{k,y}(x) = \sqrt{p(x, y)} e^{i k \cdot x},$9 computational complexity—the same scaling as FNO up to a small multiplicative factor—by preserving FFT compatibility in its adaptive basis transforms. This property differentiates ABLE from most prior attempts at introducing adaptivity or attention to operator learning, which often raise complexity to $\int p(x, y) d y = 1$0 and impair scalability.

ABLE serves as a general refinement: it can enhance standard spectral, hybrid, and even recent latent spectral transformer-based operators (e.g., U-FNO, HPM, SAOT), simply by replacing the spectral component with the adaptive lift. This modularity greatly expands its impact potential across scientific ML pipelines.

Implications and Future Directions

The work underscores that representation learning—particularly, the choice of spectral basis—is a primary bottleneck for expressivity and accuracy in neural operator design, often eclipsing gains from architectural complexity alone. Practically, ABLE's improved operator learning is relevant for turbulent, multi-scale PDE simulation, scientific computing surrogates, and potentially any process where operator non-stationarity or heterogeneity is present. The adaptive basis philosophy can inform further work on unstructured domains, non-Euclidean manifolds, and physical systems where standard global bases are inadequate.

Open challenges include extending ABLE to geometrically irregular domains, unstructured meshes, and scaling to massive 3D turbulent problems; the foundations laid here suggest that learnable representations may systematically overcome the limitations of fixed harmonic expansions in these regimes.

Conclusion

"Don't Fix the Basis -- Learn It" (2605.10451) introduces ABLE, a learnable, adaptive spectral representation that generalizes and extends the classical Fourier-based spectral neural operator paradigm. By parameterizing the basis, ABLE enables rigorous, invertible, and efficient function-to-spectral lifts that are data- and locality-aware. Empirical and theoretical analysis supports the claim that adaptive representation, rather than operator complexity per se, can be the principal bottleneck—and driver of progress—in PDE operator learning and related areas. ABLE establishes a new direction toward principled, efficient, and expressive spectral representations for scientific machine learning applications.