---
title: Fast Walsh–Hadamard Transform (FWHT)
url: https://www.emergentmind.com/topics/fast-walsh-hadamard-transform-fwht
type: topic
---

# Fast Walsh–Hadamard Transform (FWHT)

The Fast Walsh–Hadamard Transform (FWHT) is an in-place, divide-and-conquer algorithm for multiplying an input vector by the Walsh–Hadamard matrix $H_N$, where $N$ is a power of two. The algorithm’s simplicity, reliance only on addition/subtraction, and optimal $O(N\log N)$ arithmetic complexity have made it a staple in domains ranging from digital communications and compressed linear algebra to quantum information processing.

## 1. Mathematical Definition and Structure

Let $N = 2^n$. The (unnormalized) Walsh–Hadamard matrix $H_N \in \{\pm1\}^{N\times N}$ is defined recursively as:
- $H_1 = [1]$
- $H_{2n} = \begin{pmatrix} H_n & H_n \\ H_n & -H_n \end{pmatrix}$

Given $x\in \mathbb{R}^N$, the FWHT computes the transform $y = H_N x$, i.e., $y_i = \sum_{j=0}^{N-1} (H_N)_{i,j}x_j$. Equivalently, its entries satisfy $y_k = \sum_{j=0}^{N-1} (-1)^{\langle k,j\rangle} x_j$, where $\langle k,j\rangle$ is the bitwise dot product modulo 2. The normalized form, $(1/\sqrt{N})H_N$, renders $H_N$ orthonormal: $H_NH_N^\top = N I_N$, so $H_N^{-1} = (1/N)H_N$ [2601.09477, 2512.24685].

## 2. Core Algorithms and Fast Implementations

### Standard In-Place “Butterfly” FWHT

The classical FWHT, also known as Yates’ algorithm, employs butterfly operations:
- For each stage $\ell$ (from $0$ to $n-1$), pair elements whose indices differ at bit $\ell$.
- For each such pair $(a, b)$, update to $(a+b, a-b)$.

Formally, for array $F \in \mathbb{C}^{N}$:
```python
def FWHT_inplace(F):
    N = len(F)
    half = 1
    while half < N:
        for i in range(0, N, 2*half):
            for j in range(half):
                u = F[i+j]
                v = F[i+j+half]
                F[i+j] = u + v
                F[i+j+half] = u - v
        half *= 2
```
This routine requires $O(N\log N)$ additions/subtractions, no multiplications beyond $\pm1$ [2512.24685, 1404.1148]. The transform is entirely in-place, using $O(1)$ auxiliary space.

### Improved Arithmetic and Bit Complexities

Recent work has established theoretical improvements:
- **Operation count reduction:** Using a decomposition of $H_8$ as the sum of a rank-$1$ and a sparse matrix, it is possible to reduce the leading constant for the FWHT to $(23/24)N \log_2 N + O(N)$ arithmetic operations, compared to the folklore $N\log_2 N$ [2211.06459].
- **Lookup tables for bit operations:** Over constant-sized fields, precomputing and using lookup tables for small blocks enables an FWHT in $O(N\log N/\log\log N)$ bit operations [2211.04643].

## 3. Practical Optimizations and Parallelization

Efficient implementations leverage modern computer architectures:
- **In-place permutation and memory locality:** Butterfly operations touch only contiguous memory, ensuring cache efficiency; no scatter/gather is required. For multi-dimensional applications, storing data so that axes are transformed in contiguous memory regions maximizes bandwidth [2408.06206, 2512.24685, 2601.09477].
- **Multi-threaded and SIMD:** The independence of different butterfly operations at each stage enables vectorization and threading. On GPUs, butterfly stages map naturally to SIMT warps/blocks. FWHT can be parallelized at both the butterfly and batch level [2412.08832, 2512.24685, 2601.09477].
- **Hardware acceleration:** Algorithms such as HadaCore exploit GPU Tensor Cores by restructuring the FWHT to map to hardware matrix-multiply-accumulate primitives, giving speedups up to $3.5\times$ for size-256 vectors. Larger transforms are tiled and organized via in-register transposes and shared-memory staging [2412.08832].

## 4. Statistical and Group-Theoretic Properties

The FWHT admits several crucial mathematical properties:
- **Self-inverse (up to scaling):** $H_N^{-1} = (1/N)H_N$
- **Energy preservation:** $\|H_N x\|_2^2 = N\|x\|_2^2$, so the normalized transform is isometric.
- **Group convolution:** On $Z_2^n$, the FWHT diagonalizes the XOR-convolution:
  $$(f \oplus g)(w) = \sum_{x \oplus y = w} f(x)g(y)$$
  under the transform $\widehat{f \oplus g} = \hat{f} \cdot \hat{g}$ (pointwise product), enabling $O(N\log N)$ convolution for functions on Boolean cubes [2512.24685, 2601.09477].

## 5. Major Application Domains

### Quantum Information Processing

- **Pauli decomposition:** All Pauli string coefficients of an arbitrary $2^n \times 2^n$ matrix $A$ can be computed in $O(N^2\log N)$ time and $O(1)$ extra memory via an FWHT-based batched algorithm, which applies an in-place XOR-permutation, row-wise FWHT, and phase correction [2408.06206].
- **Stabilizer Rényi entropy:** The computation of the second-order stabilizer entropy is reduced from $O(8^N)$ to $O(N4^N)$ using $2^N$ FWHTs of length $2^N$, exploiting natural parallelism and in-place operations [2512.24685].

### Compressed Matrix Computation

- **Compressed multiplication via sketching:** Pagh’s algorithm for compressed matrix multiplication can substitute FFT with FWHT, preserving unbiasedness and variance bounds. In practice, the FWHT-based variant is up to $4\times$ faster than FFT-based sketching and can yield $40\times$ speedups over dense DGEMM (Intel MKL) when the product matrix is output-sparse [2601.09477].

### Digital Communications

- **Hadamard coded modulation (HCM):** The FWHT enables efficient mapping of data onto Hadamard codewords, resulting in transmission schemes with low peak-to-average-power ratios (PAPR), outperforming OFDM in high-power and nonlinear regimes. Interleaving mitigates inter-symbol interference (ISI) in dispersive channels. HCM achieves PAPR $=2$, whereas OFDM’s PAPR grows with $\log N$ [1404.1148].

### Sparse Signal Processing

- **Sparse FWHT (SparseFHT):** For $K$-sparse signals in the Hadamard domain, a graph-code-inspired algorithm computes the WHT using $O(K\log K\,\log(N/K))$ time and $O(K\log(N/K))$ samples, via subsampling with random hashing and a belief-propagation (peeling) decoder. This is strictly sub-linear in $N$ for $K\ll N$ [1310.1803].

## 6. Algorithmic Extensions and Specialized Methods

### Table: Notable FWHT Variants and Theoretical Innovations

| Paper           | Main Innovation                                      | Complexity Improvement     |
|-----------------|-----------------------------------------------------|---------------------------|
| [2211.06459]    | Low-rank + sparse decomposition for FWHT            | $23/24\,N\log N$ ops      |
| [2211.04643]    | Lookup tables for $F_q$-valued WHT                  | $O(N\log N/\log\log N)$ bits |
| [2601.09477]    | FWHT for compressed matrix sketching                | $2\text{--}4\times$ speedup over FFT |
| [1404.1148]     | FWHT in HCM for PAPR reduction in communications    | PAPR $=2$ vs. OFDM        |
| [1310.1803]     | SparseFHT for $K$-sparse signals                    | $O(K\,\log K\log(N/K))$   |

Algorithmic improvements include blockwise recursion with higher-radix kernels, leveraging matrix non-rigidity, tailored bit-block algorithms for finite fields, and the use of hardware-specific kernels (e.g., Tensor Core blocked FWHTs) [2412.08832, 2211.06459, 2211.04643].

## 7. Numerical Benchmarks and Implementation Comparisons

Empirical comparisons on CPUs (AMD EPYC, 64-core nodes) and GPUs (NVIDIA A100/H100) reveal that:
- FWHT-based implementations outperform FFT for integer-valued and real transforms where only $\pm1$ arithmetic is needed.
- Blocked, in-place, and hardware-optimized FWHT kernels yield $1.1\text{--}3.6\times$ gains over previous libraries (e.g., Dao fast-hadamard-transform), with minimal numerical error even in low-precision FP16/BF16 [2412.08832, 2601.09477].
- In Pauli decomposition for $n\geq5$ qubits, the FWHT method is $1.4\times$ to $3.6\times$ faster than previous tensorized and C++ FWHT implementations [2408.06206].

## References

- "Pauli Decomposition via the Fast Walsh–Hadamard Transform" [2408.06206]
- "Engineering Compressed Matrix Multiplication with the Fast Walsh–Hadamard Transform" [2601.09477]
- "A fast and exact algorithm for stabilizer Rényi entropy via XOR-FWHT" [2512.24685]
- "HadaCore: Tensor Core Accelerated Hadamard Transform Kernel" [2412.08832]
- "Hadamard Coded Modulation: An Alternative to OFDM for Optical Wireless Communications" [1404.1148]
- "Faster Walsh-Hadamard and Discrete Fourier Transforms From Matrix Non-Rigidity" [2211.06459]
- "Faster Walsh-Hadamard Transform and Matrix Multiplication over Finite Fields using Lookup Tables" [2211.04643]
- "A Fast Hadamard Transform for Signals with Sub-linear Sparsity in the Transform Domain" [1310.1803]

Source: https://www.emergentmind.com/topics/fast-walsh-hadamard-transform-fwht