Walsh-Hadamard Transform: Principles and Applications

• The Walsh-Hadamard Transform is a self-inverse, binary orthogonal transform defined on dimensions N=2^n, featuring a recursive butterfly structure.
• It enables efficient, multiplication-free computations with O(N log N) complexity, facilitating signal processing, sparse recovery, and deep neural network optimizations.
• Applications span coding theory, quantum information, and image compression, with innovations in sparse transforms and adaptive quantization enhancing performance.

The Walsh-Hadamard Transform (WHT) is a canonical orthogonal, self-inverse transform of dimension $N = 2^n$, whose matrix consists exclusively of entries in $\{+1, -1\}$ with multiplicative structure determined by the binary inner product. The WHT, known under variants as the (discrete) Hadamard transform, Sylvester-Hadamard transform, and closely related to the class of square-wave bases, is fundamental in signal processing, coding theory, fast algorithms, combinatorial mathematics, deep learning, and quantum information theory. The transform admits an efficient radix-2 “butterfly” decomposition, variants for sparse and blockwise computation, and generalizations to finite fields and higher-dimensional analogues.

1. Definition and Mathematical Properties

The $N \times N$ Hadamard matrix $H_N$ is defined recursively as

$$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$

with base case $H_1 = 1$. For $N = 2^n$, the entries obey $[H_N]_{k,\ell} = (-1)^{\langle k, \ell \rangle}$, where $\langle k, \ell \rangle$ is the binary inner product in $\mathbb{F}_2$, and indices are enumerated in binary.

Given $\{+1, -1\}$0, the WHT is

$\{+1, -1\}$1

and is typically normalized by division by $\{+1, -1\}$2 or absorbed into normalization in subsequent layers (Jeong et al., 2019, Jeon et al., 22 Sep 2025, Li et al., 2015, Pan et al., 2021).

Key properties:

• Orthogonality: $\{+1, -1\}$3
• Involution: $\{+1, -1\}$4
• Self-inverse (orthogonal form): $\{+1, -1\}$5
• All entries in $\{+1, -1\}$6, enabling multiplier-free computation

The 2D WHT is constructed via Kronecker products: for a matrix $\{+1, -1\}$7, $\{+1, -1\}$8.

2. Fast Algorithms and Complexity

The classic Fast Walsh-Hadamard Transform (FWHT) uses a butterfly recursion with $\{+1, -1\}$9 additions and subtractions, requiring no multiplication operations:

$H_1 = 1$3 (Pan et al., 2021, Jeon et al., 22 Sep 2025, Alman, 2022, Bomfin et al., 2023).

Variants include:

• Blockwise/blocked FWHT: optimized for memory hierarchies and parallel systems (Lu, 2016). With appropriate blocking parameter $N \times N$0, data fits into L2/L3 caches or disk pages, enabling multi-terabyte-scale computation.
• Lookup Table Accelerated FWHT: Over finite fields of constant size, the bit complexity is improved to $N \times N$1 via precomputed Kronecker blocks and table lookups. (Alman, 2022).
• Matrix non-rigidity acceleration: By decomposing Hadamard matrices into a low-rank plus a sparse component, one obtains an operation count of $N \times N$2 (Alman et al., 2022), outperforming the standard $N \times N$3 bound for all practical input sizes.
• Haar-wavelet-structured FWHT: The CHW algorithm cascades size-$N \times N$4 Haar transforms; total cost remains $N \times N$5 but may offer implementation or parallelization advantages (Thompson, 2016).

3. Sparse Walsh-Hadamard Transform and Sublinear Regimes

For signals whose Walsh spectrum is $N \times N$6-sparse ($N \times N$7), sublinear algorithms aim to recover the spectrum in sample and time complexities scaling with $N \times N$8 rather than $N \times N$9:

• SparseFHT (Scheibler et al., 2013): For noiseless settings, achieves $H_N$0 samples and $H_N$1 runtime, using random subsampling (aliasing), induced parity constraints, and BP-style “peeling” decoders on the sparse-graph induced by hashing.
• SPRIGHT (Li et al., 2015): Robustifies these ideas to Gaussian noise, maintaining $H_N$2 sample and $H_N$3 runtime with high-probability exact recovery, with additional bin-detection and code-decoding mechanisms. Decoding succeeds with probability $H_N$4 under random support and constant SNR.
• Practical implementation at tera-scale (Lu, 2016): Blocked, cache/disk-optimized general FWHT forms the backbone for (noisy-)sparse WHTs by providing thresholded spectral support initialization, then sparse recovery.
• In communication systems, sparse WHT precoding achieves a favorable complexity–performance regime for iterative detectors over ISI channels (Bomfin et al., 2023).

4. Applications in Deep Learning and Signal Compression

The WHT and its fast algorithms have recently been integrated into DNN architectures, offering improvements in both efficiency and—nontrivially—accuracy:

• Fixed WHT for pointwise convolution: In MobileNet-V1, replacing the $H_N$5 pointwise convolution in the top layers with a DWHT achieves 79.1% reduction in parameter count, 48.4% reduction in FLOPs, and a 1.49% accuracy increase on CIFAR-100. The cross-channel mixing power is attributed to the full N-channel coverage per output and the logarithmic dataflow (Jeong et al., 2019).
• Quantization-aware WHT adapters (QWHA): By expressing adapter weights in the Walsh basis with adaptively-allocated sparse spectral coefficients, quantization error is focused into a small number of WHT coefficients, enabling high-accuracy sub-4-bit quantization and rapid parameter-efficient fine-tuning of large models (Jeon et al., 22 Sep 2025).
• Block and 2D WHT layers: Both 1D and 2D WHTs, with trainable smooth-thresholding nonlinearities, can replace $H_N$6 and $H_N$7 convolutions. 2D-WHT layers operate 24× faster than baseline convolutions with a minor accuracy tradeoff and reduced RAM usage on embedded hardware (Pan et al., 2022, Pan et al., 2021).
• Feature compression: Two-stage 2D WHT (column then row) followed by fixed/adaptive region selection and nonlinear pooling provides a 35× compression in feature map size and a 4–5× speedup in CNN-based underwater object classification, with accuracy improvement over DCT and uncompressed approaches (Zhao et al., 2021).
• Binary and multiplication-free networks: The binary nature of WHT kernels enables multiplier-free convolution layers, further reducing computational and energy costs.

5. Extensions to Quantum Information and Coding Theory

• Quantum circuits: The generalized Hadamard/WHT is crucial as the prototypical single-qutrit quantum Fourier transform, implemented with high fidelity in superconducting qutrits and generalized to higher-dimensional quantum systems through two-step decompositions and simultaneous multi-level driving strategies (Yurtalan et al., 2020).
• Hybrid classical–quantum WHT: Quantum circuits enable the WHT in $H_N$8 time by merging $H_N$9 classical preparation/post-processing with a single quantum Hadamard layer; this reduces classical $$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$0 cost provided state preparation can be made efficient. Polar WHT basis functions (sequency/natural orderings generalized to polar domains) enable the suppression of circular or azimuthal noise patterns in images, with substantial speedup and demonstrable improvements in SSIM/PSNR metrics (Rohida et al., 2024).
• Generalization to finite fields: WHT extends via the Vilenkin–Chrestenson transform, supporting arbitrary prime powers and maximal non-proportional vector selection to compute weight distributions and covering radii of linear codes with reduced complexity. For a code over $$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$1, computation of Hamming weight enumerators and covering radii can be performed in $$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$2 steps (Piperkov et al., 2022).

6. Comparative Features and Complexity

Algorithm/Variant	Arithmetic Complexity	Special Features
Classical FWHT	$$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$3	Additions/subtractions only; cache-friendly
Non-rigidity-accelerated FWHT (Alman et al., 2022)	$$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$4	Block low-rank+sparse merge per step
Lookup-Table-accelerated FWHT (Alman, 2022)	$$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$5	Requires random-access tables; bit-complexity saving
Blockwise/External FWHT (Lu, 2016)	$$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$6, disk/mem-optimized	Enables $$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$7 on commodity hardware
SparseFHT (Scheibler et al., 2013), SPRIGHT (Li et al., 2015)	$$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$8–$$H_1 = [1], \qquad H_{2N} = \begin{pmatrix} H_N & H_N \ H_N & -H_N \end{pmatrix}$$9	Sublinear, robust to noise, iterative decoding
CHW (Haar-cascade) (Thompson, 2016)	$H_1 = 1$0	Pipelined Haar block decomposition
QWHA Deep Adapter (Jeon et al., 22 Sep 2025)	$H_1 = 1$1 + adaptive updates	Sparse, quantization-aware, per-channel spectral

7. Principal Domains of Impact and Limitations

• Deep learning: WHT layers, adapters, and compression enable significant reductions in parameter count, memory usage, and computational cost with minor or even positive effects on accuracy, especially in resource-constrained and real-time settings (Jeong et al., 2019, Pan et al., 2022, Jeon et al., 22 Sep 2025).
• Sparse and compressive sensing: Sublinear algorithms (SparseFHT, SPRIGHT) furnish provably efficient recovery for signals with sparse Walsh spectra under noisy or adversarial conditions (Scheibler et al., 2013, Li et al., 2015).
• Tera-scale and distributed data: WHT is emblematic of arithmetic kernels that remain feasible up to terabyte-scale by careful blocking, streaming I/O, and distributed memory architectures (Lu, 2016).
• Coding theory: Extensions to finite fields (Vilenkin-Chrestenson) enable efficient reconstruction of code parameters (weight enumerators, covering radii) over $H_1 = 1$2 for large codes (Piperkov et al., 2022).
• Quantum information: The WHT is central in both algorithmic construction (quantum Fourier transforms) and scalable gate implementations for qutrits and higher-dimensional systems (Yurtalan et al., 2020).
• Limitations: For extremely high sparsity or structural priors, sublinear algorithms presuppose randomness or mild structure in the spectral support. Fully quantum WHT remains bottlenecked by state-preparation and measurement on current devices (Rohida et al., 2024). Lookup-table accelerated variants assume sublinear random access, which may not translate into all hardware environments (Alman, 2022).

---

The WHT and its algorithmic ecosystem illustrate the depth and versatility of binary-orthogonal transforms in modern data science, bridging classical signal processing, scalable computation, deep learning efficiency, coding theory, and quantum technology.