HadaCore: Walsh-Hadamard Neural Operator

• HadaCore is a neural operator framework that utilizes Walsh-Hadamard transforms to accurately approximate PDE solution operators with discontinuous coefficients.
• It replaces traditional Fourier layers with WHT-based spectral layers, enabling efficient low-sequency spectral compression and mitigating Gibbs phenomena.
• The framework benefits from ensemble strategies with FNOs, significantly reducing errors in complex physics problems like heat conduction and 2D Burgers.

The Walsh-Hadamard Neural Operator (WHNO) is a neural operator framework designed for efficient and accurate approximation of solution operators to partial differential equations (PDEs) with discontinuous coefficients and sharp interfaces. The method leverages the Walsh-Hadamard transform as its spectral backbone, facilitating superior preservation of discontinuities compared to Fourier-based neural operators, particularly by mitigating the Gibbs phenomenon and enabling low-sequency spectral compression. Weighted ensembles of WHNO and traditional Fourier Neural Operators (FNO) further enhance performance, capturing complementary aspects of piecewise-constant and smoothly varying features (Cavallazzi et al., 10 Nov 2025).

1. Mathematical Foundations

WHNO is grounded in the mathematical properties of Walsh-Hadamard (WH) basis functions, $\{w_k(x)\}$. Each $w_k(x)$ is a piecewise-constant "rectangular" wave on $[0,1]$, taking values $\pm1$ on dyadic subintervals. The basis is orthonormal, satisfying

$\int_0^1 w_k(x) w_\ell(x) dx = \delta_{k\ell}$

where $\delta_{k\ell}$ is the Kronecker delta. The sequency ordering indexes $k$ by the number of zero-crossings per unit interval, providing a natural separation between global (low-$k$) and localized (high-$k$) features.

The two essential forms of the WH transform used are:

• Continuous Walsh-Hadamard Transform on $[0,1]$:

$w_k(x)$0

• Discrete Walsh-Hadamard Transform on a uniform $w_k(x)$1 grid:

$w_k(x)$2

Key algorithmic benefit is the Fast Walsh-Hadamard Transform (FWHT), which computes WH coefficients in $w_k(x)$3 time, matching Fourier FFT complexity. Notably, rectangular WH basis functions enable exact representation of step discontinuities without the overshoot—the Gibbs phenomenon—prevalent in Fourier expansions (Cavallazzi et al., 10 Nov 2025).

2. Operator Network Architecture

WHNO replaces the Fourier spectral layers in FNOs with Walsh-Hadamard spectral layers. Each WHNO block maps an input $w_k(x)$4 to an output $w_k(x)$5 via the following sequence:

1. Forward 2D WHT:

$w_k(x)$6

where $w_k(x)$7 is the normalized Hadamard matrix.

2. Low-Sequency Truncation: Retain a $w_k(x)$8 block around the origin for spectral compression:

$w_k(x)$9

3. Spectral Weights and Bias: Apply learnable weights $[0,1]$0 and biases $[0,1]$1 per sequency:

$[0,1]$2

1. Zero Padding: Restore the original $[0,1]$3 dimension by zero-padding the remaining coefficients.
2. Inverse 2D WHT: Revert to the spatial domain via:

$[0,1]$4

The complete WHNO encoder-decoder consists of input lifting, two WHT-based spectral layers with residual connections, intermediate $[0,1]$5 convolution, and a dilated convolutional decoder. Example hyperparameters for heat conduction and Burgers problems include $[0,1]$6, $[0,1]$7, $[0,1]$8, and $[0,1]$9k parameters.

Forward pass pseudocode: $\int_0^1 w_k(x) w_\ell(x) dx = \delta_{k\ell}$5

3. Training Procedures

Training employs mean squared error loss: $\pm1$0 with AdamW optimizer (default $\pm1$1 parameters, weight decay $\pm1$2), batch size $\pm1$3, $\pm1$4 epochs, and cosine annealing of learning rate from $\pm1$5 to $\pm1$6.

Three canonical datasets are used:

• Darcy flow: Four random rectangular obstacles with binary permeability on $\pm1$7 grids (Jacobi solver).
• Heat conduction: Random rectangular inclusions of varied conductivity, central square heat source, explicit timestepping to quasi-steady state.
• 2D Burgers: Three random non-overlapping block initial conditions, $\pm1$8, $\pm1$9 grid, periodic BC, $\int_0^1 w_k(x) w_\ell(x) dx = \delta_{k\ell}$0 time-steps.

4. Empirical Results and Quantitative Comparisons

WHNO establishes the following performance benchmarks (MSE=mean squared error; MAE=mean absolute error):

Problem	Method	MSE	MAE	Max Error
Heat conduction	WHNO	0.000113	0.00675	0.101
	FNO	0.000148	0.00793	0.104
	Ensemble (50:50)	0.000073	0.00539	0.093
2D Burgers	WHNO	0.000113±0.000049	0.00675±0.00158	0.101±0.039
	FNO	0.000148±0.000092	0.00793±0.00249	0.104±0.034
	Ensemble (60:40)	0.000073±0.000037	0.00539±0.00143	0.093±0.037

• Darcy flow: WHNO with 16 sequency modes ($\int_0^1 w_k(x) w_\ell(x) dx = \delta_{k\ell}$1) achieves 0.88% relative error; errors are localized primarily at obstacle interfaces.
• Heat conduction: A 50-50 ensemble of WHNO and FNO reduces MSE by 35% and MAE by 20% relative to WHNO, and by 51% and 32% relative to FNO.
• 2D Burgers: In a 60:40 (WHNO:FNO) ensemble, MSE and error variance are reduced by approximately 35% over WHNO alone and 51% over FNO. The optimal ensemble weights ($\int_0^1 w_k(x) w_\ell(x) dx = \delta_{k\ell}$2) shift toward WHNO for problems dominated by discontinuities.

5. Interpretation and Theoretical Insights

Walsh-Hadamard basis functions exhibit unique efficacy in representing step discontinuities without ringing, by virtue of their rectangular, orthonormal structure. In the WHNO, low-sequency WH coefficients can encode sharp jumps with minimal error spread near discontinuities, leading to uniform error distributions rather than error concentration at interfaces, as exhibited in FNO outputs.

A crucial finding is the complementarity between WHNO and FNO: WHNO excels in piecewise-constant and discontinuous fields (e.g., binary permeability, block initial conditions), efficiently capturing sharp interfaces, whereas FNO more effectively represents smooth variations and subtle fine-scale wavenumbers. Weighted ensembles, parameterized by a single scalar weight $\int_0^1 w_k(x) w_\ell(x) dx = \delta_{k\ell}$3, optimally combine these strengths for a given PDE configuration.

Both WHNO and FNO have comparable computational complexity ($\int_0^1 w_k(x) w_\ell(x) dx = \delta_{k\ell}$4 inference). Ensembles incur double inference cost but require no additional training and consistently yield substantial (∼35%) improvements in MSE—a justified trade-off in high-accuracy regimes.

6. Practical Implications and Outlook

The WHNO framework achieves state-of-the-art accuracy on PDEs with abrupt coefficients and sharp interface phenomena, a regime where Fourier-based neural operators are suboptimal due to Gibbs effects and slow decay of spectral coefficients. The methodology is validated across elliptic, parabolic, and nonlinear advection-diffusion models, indicating broad applicability to physical systems with discontinuous parameters. Notably, the use of simple weighted ensemble methods systematically harnesses the complementary spectral representations inherent to the WH and Fourier domains (Cavallazzi et al., 10 Nov 2025).

A plausible implication is that WHNO constitutes a foundational tool for data-driven operator learning in computational physics, engineered systems with multi-material interfaces, and inverse problems involving abrupt parameter changes. The ensemble principle demonstrated within WHNO and FNO architectures suggests a generalizable strategy for integrating diverse spectral representations in neural-operator frameworks.