Fourier Holographic Reduced Representation

• FHRR is a high-dimensional vector symbolic architecture that encodes symbols as unit-magnitude complex vectors with phases on the unit circle.
• Its operations—binding, unbinding, and bundling—leverage Fourier transforms for robust, efficient, and mathematically interpretable symbolic manipulation.
• Extensions like GHRR introduce non-commutative binding to enhance expressivity for representing order-sensitive and hierarchical data.

Fourier Holographic Reduced Representation (FHRR) is a Vector Symbolic Architecture (VSA) framework in which high-dimensional complex unit vectors encode symbols as phases on the unit circle, supporting algebraically robust compositional operations such as binding, unbinding, and superposition. FHRR leverages the Fourier domain for highly efficient computation of associative operations, and provides a mathematically principled approach for manipulating structured or symbolic data within differentiable, neural, and neuromorphic systems. Its core properties—unitarity, invertibility, robustness, kernel interpretability, and quasi-orthogonality—enable FHRR to serve as a foundation for both general-purpose world models and attention-based deep architectures, and motivate extensions such as Generalized Holographic Reduced Representation (GHRR) for richer compositionality.

1. Mathematical and Algebraic Formulation

In FHRR, each atomic symbol is encoded as a $d$-dimensional complex vector of unit magnitude:

$$\mathbf{x} = \left[ e^{i\theta_1}, \; e^{i\theta_2}, \ldots, e^{i\theta_d} \right]^\top$$

where $\theta_j \in [0,2\pi)$. Randomly sampled phase vectors are nearly orthogonal in high dimensions, facilitating the representation of large symbolic vocabularies (Yeung et al., 2024).

Binding of two symbols is effected by element-wise complex multiplication (“Hadamard product”):

$$(\mathbf{x} \otimes \mathbf{y})_k = x_k y_k = e^{i(\theta_k^x + \theta_k^y)}$$

This corresponds to circular convolution in the real domain, computed efficiently via FFT:

$$\mathbf{x} \otimes \mathbf{y} = \text{IFFT}( \text{FFT}(\mathbf{x}) \odot \text{FFT}(\mathbf{y}) )$$

Unbinding is performed by binding to the complex conjugate:

$$\mathbf{x} \oslash \mathbf{y} = \mathbf{x} \otimes \mathbf{y}^*$$

ensuring invertibility.

Bundling (superposition) is achieved by vector addition and normalization, or angle-based renormalization in the complex plane:

$$\mathbf{z} = \operatorname{angle}( \mathbf{x} + \mathbf{y} + \cdots )$$

The similarity metric is the normalized real part of the Hermitian inner product, or phase cosine:

$$\text{sim}(\mathbf{x}, \mathbf{y}) = \frac{1}{d} \sum_{k=1}^d \cos(\theta_k^x - \theta_k^y)$$

Fractional Power Encoding (FPE) enables continuous values to be encoded as exponentiated phase vectors:

$$\phi(x) = (e^{i\boldsymbol{\theta}})^x = e^{i x \boldsymbol{\theta}}$$

and generalized to $d$-dimensional continuous vectors via $A \in \mathbb{R}^{n \times d}$:

$\phi(\mathbf{x}) = e^{i A \mathbf{x}}$

(Bremer et al., 2024).

2. Theoretical Properties, Invertibility, and Kernel Connections

FHRR operations are unitary (norm-preserving), exactly invertible via conjugation, robust to additive phase noise, and quasi-orthogonal for random symbols ($\text{sim} \approx 0$ for unrelated vectors). The binding operation is commutative and associative ($$\mathbf{x} \otimes \mathbf{y} = \mathbf{y} \otimes \mathbf{x}$$), which restricts representation of order-sensitive structures—this is addressed by higher-order constructs such as permutations (Yeung et al., 2024).

FHRR can be formally interpreted via its connection to random Fourier features, providing a kernel perspective by Bochner’s theorem. The encoding

$\phi(\mathbf{x}) = e^{i M \mathbf{x}}$

implies that inner products in FHRR space approximate shift-invariant kernels (Chung et al., 25 Feb 2026). This kernel-theoretic view connects FHRR to established statistical learning methods and allows principled design of learning objectives for structured representations.

3. Cleanup, Decoding, and Robust Computation

FHRR supports a range of noise-robust cleanup and decoding mechanisms necessary for robust symbolic computation. After superposition or unbinding, the resulting vectors may deviate from the valid embedding manifold due to noise, interference, or implementation artifacts. Cleanup projects the corrupted vector back onto the valid codebook.

For continuous encodings (FPE), the corrupted phase vector is modeled as a sample from a von Mises (circular Gaussian) distribution. Decoding is cast as maximum-likelihood estimation:

$$\hat{\mathbf{x}} = \arg\max_{\mathbf{x}} \frac{1}{n}\sum_{k=1}^n \cos(\phi_k - (A \mathbf{x})_k )$$

Using a two-stage optimization combining composite likelihood estimation (CLE) and direct MLE enables convergence to the unique global optimum while avoiding phase-wrapping local minima. Empirical evaluations demonstrate that gradient-based cleanup approaches (“CSim”) approach the accuracy of exhaustive search but at much lower cost, outperforming denoising autoencoders and resonator networks for both additive phase and bundling-generated noise (Bremer et al., 2024).

The high-dimensional quasi-orthogonality of FHRR vectors allows for effective bundle cleanup and decoding, supporting compositional memory and robust iterative rollouts.

4. Integration into Neural and Deep Architectures

FHRR operations can be embedded within differentiable neural architectures for both symbolic and sub-symbolic tasks. The “phasor activation” PB layer generalizes bundling and complex-valued matrix projections:

$$\operatorname{PB}(\mathbf{A}; W_r, W_p, b) = \operatorname{angle}(W_r \cdot \exp(i n \mathbf{A}) \cdot W_p + b)$$

FHRR-based residual blocks use binding (rather than summation) as a skip connection, initializing as the identity via near-zero phases (Bazhenov, 2022).

Attention mechanisms are constructed entirely from FHRR primitives:

• QKV projections as PB layers
• Real-valued similarity matrix via FHRR cosine
• Weighted complex sum as attention output
• Skip-bind residual via element-wise binding

These modules support both self-attention (for sequences) and cross-attention (Perceiver-style, with fixed queries) architectures, delivering models where all latent computations remain within the uniform high-dimensional complex phase space. This design provides direct compatibility with neuromorphic systems implementing VSA primitives.

5. Generalization, Group-Theoretic Structure, and Interpretable World Models

When applied to structured world modeling, FHRR leverages group-theoretic foundations: if environment transitions are generated by a group action, then the embedding space $(S^1)^D$ itself forms a continuous commutative group, and symbolic encoders can be trained to be equivariant under group homomorphism (Chung et al., 25 Feb 2026).

Learning state and action encoders as FHRR random Fourier layers parameterized by phase projections yields closed-form multi-step rollout, strong zero-shot accuracy, and generalization to unseen action compositions. Experimental results in a discrete grid world demonstrate:

• One-step zero-shot: 87.5% (FHRR) vs ≈1% (MLPs)
• 20-step rollout: 34.6% (FHRR) without cleanup, 61.4% with cleanup
• >4× robustness to noise compared to standard MLPs

These gains derive from explicit algebraic structure: latent trajectories follow group composition via repeated binding, and the codebook geometry mirrors the true environment manifold. Cleanup mechanisms exploit quasi-orthogonality to correct drift.

6. Expressivity, Limitations, and Extensions

FHRR provides robust, interpretable, high-capacity symbolic representations; however, standard FHRR binding is commutative, which limits order-sensitivity and nested compositional encodings. This requires explicit use of permutations or positional tags for ordered or hierarchical data (Yeung et al., 2024).

Generalized Holographic Reduced Representation (GHRR) extends FHRR by promoting phase scalars to unitary matrices (block diagonal), enabling non-commutative binding, native encoding of hierarchical trees and graphs, and enhanced decoding capacity for nested structures. GHRR interpolates between FHRR ($m=1$) and full tensor-product representations ($m \rightarrow D$), without sacrificing capacity or robustness (Yeung et al., 2024).

A summary table contrasting FHRR and GHRR:

Property	FHRR ($m=1$)	GHRR ($m>1$)
Binding commutativity	Commutative	Non-commutative
Expressive power	Limited for order/Nesting	Rich (trees, graphs)
Computational complexity	$O(D)$	$O(D m^3)$
Capacity	$O(D)$	$O(D m^2)$
Use cases	Simple key-value	Nested/compositional

FHRR’s computational efficiency (all operations $O(D)$ per symbol, $O(D \log D)$ for binding via FFT), differentiable structure, and neural compatibility make it well-suited for scalable and interpretable architectures across deep learning and neuromorphic domains. Limitations include the need for explicit structural augmentation in certain applications, motivating ongoing research into non-commutative and higher-order VSA frameworks.

7. Applications and Empirical Results

FHRR has enabled the construction of residual and attention-based neural architectures where all network computations are symbolic and algebraically structured. Demonstrations include:

• Multi-domain deep architectures (image, molecular data) using the same FHRR-based network topology, paralleling Perceiver-style universality (Bazhenov, 2022)
• Structured world models with strong zero-shot composition and long-range rollout accuracy (Chung et al., 25 Feb 2026)
• Robust continuous value decoding and bundle cleanup via optimization-based methods, with near-zero failure up to moderate noise (Bremer et al., 2024)

A plausible implication is that FHRR enables a route to integrating symbolic computation, geometric priors, and deep networks in a unified, hardware-compatible space suitable for large-scale reasoning and memory.

Advances in group-structured VSAs, efficient cleanup algorithms, and extensions such as GHRR continue to expand the scope and versatility of FHRR for structured AI and cognitive modeling.