Generalized Holographic Reduced Representations

• Generalized HRRs are hyperdimensional computing frameworks that use m×m unitary matrices to enable non-commutative binding and improved encoding of compositional data.
• They extend Fourier HRRs by interpolating between scalar representations and full tensor-product models, resulting in enhanced fidelity and robust noise tolerance.
• GHRRs maintain key properties such as exact invertibility, similarity invariance, and a tunable balance between computational efficiency and representational expressivity.

Generalized Holographic Reduced Representations (GHRR) define a class of Hyperdimensional Computing (HDC) frameworks that extend the capabilities of Fourier Holographic Reduced Representations (FHRR) by introducing a flexible, non-commutative binding operation. Each GHRR hypervector is structured as a sequence of $m \times m$ unitary matrices, providing a parameterization that interpolates between scalar (commutative) FHRR and full tensor-product representations. The approach enables improved encoding fidelity for complex, nested, or compositional data structures, while retaining canonical HDC features such as robustness to cross-talk noise, explicit invertibility of binding, and transparent similarity metrics. GHRR maintains data and computational efficiency, offering a powerful representational toolkit at the intersection of connectionist and symbolic AI paradigms (Yeung et al., 2024).

1. Formal Structure and Core Operations

A GHRR hypervector is a length-$D$ tuple of $m\times m$ unitary matrices, notated as

$$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$

where $U(m)$ denotes the group of $m\times m$ unitary matrices. For a hypervector $H = [a_1, a_2, \ldots, a_D]^T$ with $a_j \in U(m)$, the three canonical HDC operations are defined componentwise:

• Bundling (Superposition): $H_1 \oplus H_2 := [a_j + b_j]_{j=1\ldots D}$
• Binding (Non-commutative, denoted "$\star$"): $D$0; matrix multiplication enforces non-commutativity when $D$1.
• Inverse Binding (Unbinding): $D$2, guaranteeing $D$3.

Similarity for $D$4 is defined by

$D$5

For $D$6, GHRR reduces to the FHRR inner product, confirming the generalization.

2. Relationship to Fourier Holographic Reduced Representations

The FHRR model arises as the special case $D$7, with each $D$8 a unit modulus complex scalar: $D$9. In FHRR, bundling is elementwise addition, binding is elementwise multiplication, and unbinding applies elementwise complex conjugation. Specifically,

$m\times m$0

In GHRR, each scalar $m\times m$1 is replaced by $m\times m$2; binding is matrix multiplication, and unbinding uses the Hermitian transpose, yielding a non-commutative algebra for $m\times m$3 and enabling richer compositional encodings (Yeung et al., 2024).

3. Theoretical Properties

GHRR preserves the central theoretical properties necessary for robust HDC:

• Quasi-Orthogonality: For $m\times m$4, with each $m\times m$5 ($m\times m$6 Haar random in $m\times m$7, $m\times m$8 diagonal with phases $m\times m$9), two independently sampled $$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$0 satisfy $$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$1 and $$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$2. This implies nearly orthogonal codewords for high dimension.
• Exact Invertibility: For any $$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$3, $$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$4.
• Similarity Preservation under Binding: For any fixed $$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$5, $$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$6, showing that the binding operation respects the similarity structure of superposed codewords.
• Robustness and Capacity: The noise amplitude from bundling $$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$7 quasi-orthogonal hypervectors scales as $$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$8, while signal strength is $$\mathcal{H}_{D,m} = (U(m))^D \subset \mathbb{C}^{D \times m \times m},$$9. Decoding remains reliable up to $U(m)$0. Empirically, the maximal number of storable bundles obeys $U(m)$1 with $U(m)$2–$U(m)$3, depending on retrieval criteria.

4. Kernel and Binding Characteristics

Kernel Perspective

Given a real vector $U(m)$4, an encoding into GHRR form is achieved by

$U(m)$5

where $U(m)$6 are sampled from symmetric distributions. The expected similarity between $U(m)$7 and $U(m)$8 satisfies

$U(m)$9

where each $m\times m$0 is a shift-invariant kernel (e.g., RBF), with weights $m\times m$1 reflecting the distribution of $m\times m$2.

Binding and Non-Commutativity

GHRR binding corresponds to a block-diagonal projection of an outer product followed by unitary mixing, generalizing the diagonal projection in FHRR. The degree of non-commutativity is measured by

$m\times m$3

Lower diagonality in $m\times m$4-matrices leads to reduced $m\times m$5, supporting more distinctive role-filler bindings and promoting the representation of hierarchical or nested data.

5. Empirical Observations

Empirical experimentation with fixed total effective dimension ($m\times m$6) demonstrates the following:

• Non-Commutativity: Nested dictionary retrieval tasks confirm that FHRR ($m\times m$7) retrieves ambiguous targets under commutative binding, while GHRR ($m\times m$8) enables unambiguous recovery at correct indices with similarity $m\times m$9 for targets and $H = [a_1, a_2, \ldots, a_D]^T$0 for distractors.
• Commutativity Control: The commutativity parameter correlates with $H = [a_1, a_2, \ldots, a_D]^T$1-diagonality (Pearson $H = [a_1, a_2, \ldots, a_D]^T$2), providing explicit tuning of algebraic structure from commutative to non-commutative regimes.
• Decoding Depth: For binary tree dictionary depths $H = [a_1, a_2, \ldots, a_D]^T$3, FHRR ($H = [a_1, a_2, \ldots, a_D]^T$4) exhibits accuracy $H = [a_1, a_2, \ldots, a_D]^T$5 for $H = [a_1, a_2, \ldots, a_D]^T$6 and collapses by $H = [a_1, a_2, \ldots, a_D]^T$7. GHRR ($H = [a_1, a_2, \ldots, a_D]^T$8) sustains exact decoding to $H = [a_1, a_2, \ldots, a_D]^T$9 and degrades gradually to $a_j \in U(m)$0 at $a_j \in U(m)$1. Permutation operations induce thresholded failure across all representations, but GHRR provides a smooth interpolation depending on $a_j \in U(m)$2 or $a_j \in U(m)$3 configuration.
• Capacity: Unpermuted, bound hypervectors with $a_j \in U(m)$4 show FHRR ($a_j \in U(m)$5) capacity $a_j \in U(m)$6; GHRR ($a_j \in U(m)$7) achieves $a_j \in U(m)$8. For base vectors or commutative scenarios, all $a_j \in U(m)$9 perform equivalently, scaling linearly with total dimension.

6. Position within Hyperdimensional Computing

GHRR establishes a continuous spectrum of representational expressivity. As $H_1 \oplus H_2 := [a_j + b_j]_{j=1\ldots D}$0 increases from $H_1 \oplus H_2 := [a_j + b_j]_{j=1\ldots D}$1 to $H_1 \oplus H_2 := [a_j + b_j]_{j=1\ldots D}$2, the system transitions from "holographic" (diagonal, commutative FHRR) to full tensor-product representations (maximal block size, highly non-commutative), offering practitioners a tunable vehicle for matching representational needs to compositional task requirements. This structure preserves invertibility, robustness, and similarity invariance, with empirical advantages in representational fidelity and memory capacity for hierarchical or deeply compositional structures (Yeung et al., 2024).