Hyperdimensional Computing (HDC)

• Hyperdimensional Computing is a computational framework that uses high-dimensional random vectors and operations like bundling, binding, and permutation to represent and process information.
• Its quasi-orthogonal and distributed representations provide robustness to noise and enable efficient, parallel processing ideal for edge and embedded hardware.
• The framework supports diverse applications—from image recognition and graph learning to bioinformatics—demonstrating significant gains in speed, efficiency, and accuracy.

Hyperdimensional Computing (HDC) is a computational paradigm inspired by properties of neural circuits, leveraging very high-dimensional random vectors—typically 1,000 to 100,000 dimensions—for robust, efficient, and massively parallel information processing. Data, concepts, and relationships are mapped to hypervectors, with mathematical operations that include binding (association/multiplication), bundling (addition/superposition), and (optionally) permutation (sequence/role encoding). Key characteristics such as distributed representation, quasi-orthogonality, and inherent robustness to noise make HDC a compelling framework for learning, reasoning, and classification, particularly on resource-limited and error-prone hardware platforms (Neubert et al., 2021, Nunes et al., 2022, Karunaratne et al., 2019, Basaklar et al., 2021, Fayza et al., 2023, Yu et al., 2022, Yan et al., 2023, Vergés et al., 2023, Arbore et al., 19 Oct 2024, Angioli et al., 28 Jan 2025, Pu et al., 2023, Smets et al., 2023, Aygun et al., 2023, Piran et al., 30 Sep 2025, Khan et al., 2021, Dewulf et al., 2023, Raviv, 5 Mar 2024, Kazemi et al., 2021, Stock et al., 27 Feb 2024, Thomas et al., 2022).

1. Core Principles and Algebraic Operations

HDC represents objects, symbols, and structures as dense random hypervectors in either binary, bipolar, or real-valued hyperspaces. The canonical set of operations is as follows:

• Bundling (Superposition or Summation): For two hypervectors $A, B \in \mathbb{R}^D$, bundling is $C = A \oplus B$ defined as element-wise addition (or majority in binary). Bundling preserves similarity; $C$ retains resemblance to each constituent.
• Binding (Association or Multiplication): For $A,B$, binding is $C = A \otimes B$, either component-wise multiplication (bipolar/real) or XOR (binary). Binding yields a hypervector nearly orthogonal to either input and is typically self-inverse: $A \otimes A \approx \mathbf{1}$.
• Permutation (Optional Role/Position Encoding): A fixed bijective permutation $\pi$ shuffles coordinates, used to encode order (not always required).
• Similarity Measures: Cosine similarity is prevalent for real/bipolar hypervectors:

$$\mathrm{sim}(A, B) = \frac{A \cdot B}{\|A\| \|B\|},$$

with Hamming distance for binary cases.

Distributed, high-dimensional representations enable $>99$\% chance of quasi-orthogonality between distinct random vectors, facilitating robust set operations and information recovery even under high levels of random bit errors (Neubert et al., 2021, Karunaratne et al., 2019, Smets et al., 2023).

2. Encoding Strategies and Model Construction

• Random Hypervector Generation: Atomic symbols and roles are mapped to hypervectors by random sampling per component from $\{\pm1\}$ or $\{0,1\}$, with near-orthogonality ensured by design (Neubert et al., 2021).
• Preprocessing for Real-Euclidean Data: Inputs are processed via random Gaussian projection to $D$-dimensions, L2-normalization, and (optionally) mean centering. For categorical or structured data, quantization and mapping to codebooks or via streaming hash-based approaches are used (Neubert et al., 2021, Thomas et al., 2022).
• Composite Representation: Sets, tuples, or associations are encoded by alternately binding and bundling—e.g., in image aggregation, each local descriptor $L_i$ is bound to a position hypervector $P_i$ (encoding spatial role) and then all results are bundled:

$$L = \bigoplus_{i=1}^k (L_i \otimes P_i) = \sum_{i=1}^k (L_i \otimes P_i)$$

(Neubert et al., 2021).

Tables of core operations:

Operation	Binary/Bipolar ($\{\pm1\}$)	Real-valued	Effect
Bundling	Majority (or sum + sign)	Sum	Preserves similarity, forms prototype
Binding	XOR (or component mult.)	Component-product	Associates (nearly orthogonal)
Permutation	Circular shift/permute	Circular shift	Encodes role/order

3. Theoretical Properties and Expressivity

• Quasi-Orthogonality: As $D \to \infty$, random hypervectors are mutually orthogonal with inner product tending to zero, a property that underpins robustness and set capacity (Neubert et al., 2021, Dewulf et al., 2023).
• Distributivity: Binding distributes over bundling, yielding efficient algebraic manipulation for compositional structured data (Neubert et al., 2021, Dewulf et al., 2023).
• Similarity Preservation: Binding with the same vector preserves similarity:

$$\mathrm{dist}(X \otimes A, X \otimes B) \approx \mathrm{dist}(A,B)$$

(Neubert et al., 2021).

• Kernel and Random-Feature View: HDC can be seen as a random-feature map for kernel methods, where the induced similarity approximates a universal kernel, as in Random Fourier Feature–based HVs (Yu et al., 2022).
• Trade-offs: There is a non-monotonic relationship between hypervector dimension and expressivity/accuracy; after a certain point, increasing $D$ does not always enhance accuracy and may degrade majority-based classification. Optimization for dimension and bit-width has led to ultra-low cost models with competitive accuracy (Basaklar et al., 2021, Yan et al., 2023).

4. Practical Applications and Empirical Results

• Image and Signal Aggregation: HDC enables aggregation of deep or hand-crafted local descriptors with spatial/temporal roles (via binding) into a single holistic hypervector for image or place recognition. Empirical studies show a 20% increase in mean average precision (mAP) and 3.6$\times$ better worst-case mAP in mobile-robotics place recognition (DELF-HDC vs NetVLAD, DenseVLAD, etc.) (Neubert et al., 2021).
• Graph Learning: Encodings where vertices are assigned hypervectors by PageRank ranking, with graph-level hypervectors formed by binding neighboring vertex codes and then bundling, enable classification competitive with GNNs but with $14.6\times$ speedup in training and $2\times$ faster inference (Nunes et al., 2022).
• Bioinformatics and Molecular Data: HDC offers a unified and efficient framework for sequence encoding, omics data fusion, and fast similarity search with competitive or superior accuracy to traditional methods under large-scale or hardware-constrained conditions (e.g., 15$\times$ speedup in mass-spec clustering, robust performance in federated omics data integration) (Stock et al., 27 Feb 2024).
• Edge and Embedded Hardware: Optimization of dimension and hypervector construction allows for HDC classifiers to reside in $<200$ kB and infer in $<1$ ms on 200 MHz microcontrollers, with $10$–$100\times$ model compression and $1$–$2\times$ accuracy/robustness gains (Basaklar et al., 2021).
• Streaming and Large-Scale Data: Hash-based (e.g., Bloom-filter) encoding algorithms allow HDC to process high-cardinality features with $O(\ln m)$ overhead, overcoming the scaling limitations of traditional codebook sampling approaches (Thomas et al., 2022).
• Hardware Implementations: In-memory and photonic accelerators can realize HDC primitives (binding/bundling/similarity) natively, demonstrated via phase-change memory crossbars, FeFETs, and photonic MZM pipelines. On-chip solutions achieve up to $6\times$ energy and $3.7\times$ area reduction, or $10^4$–$10^5\times$ improvement in energy-delay-product relative to DNNs (Karunaratne et al., 2019, Fayza et al., 2023).

5. Hardware Robustness, Optimization, and Scaling

• Error Resilience: The distributed, random representation endows HDC with natural resilience: stochastic bit-flips or conductance drift in hardware cause negligible loss in accuracy up to $\pm 20\%$ variation or $20\%$ stuck-at failures in PCM (Karunaratne et al., 2019).
• Hardware-Aware Design: Static analysis frameworks such as Heim optimize hypervector size and thresholding for given error models, yielding $1.15$–$7.14\times$ reduction in vector size at $99$% accuracy, with $30$–$10^5\times$ speedup in tuning (Pu et al., 2023).
• Multi-bit and Quantized HDC: Lowering the number of bits per dimension (e.g., to 2- or 3-bits) can realize $78\times$ reductions in training/inference energy with minor impact on accuracy; optimal quantization and retraining routines mitigate hardware-induced computational drift (Kazemi et al., 2021).
• Compiler and Programming Systems: Compiler toolchains (HDCC, HPVM-HDC) enable efficient, multithreaded and SIMD-generation of HDC classification code for embedded and HPC targets, with up to $132\times$ speedup and $158\times$ memory reduction versus interpreted HDC libraries (Vergés et al., 2023, Arbore et al., 19 Oct 2024).

6. Domain-Specific Encoding and Generalization

• Domain Sensitivity: The optimal encoder, projection variance, and hypervector dimension are highly dependent on input data characteristics—e.g., temporal signals favor Random Fourier Features (RFF) with larger spread, while spatial/image data favor linear random projections with moderate dimension (Piran et al., 30 Sep 2025). Uniform recipes do not suffice across domains.
• Multi-Objective Optimization: Joint tuning for accuracy, latency, and training energy is performed via Bayesian optimization, resulting in edge-deployable models that beat or match the best deep/transformer models at $6\times$–$40\times$ savings in inference/training energy (Piran et al., 30 Sep 2025).
• Expressive Power and Theoretical Limits: Hyperdimensional transformations generalize HDC operations as kernel mean embeddings. The induced vector-space structure enables classification, regression, Bayesian inference, and deconvolution via sums and inner products, with random-feature approximations converging to universal kernels for sufficiently large $D$ (Dewulf et al., 2023, Yu et al., 2022).

7. Extensions, Limitations, and Future Directions

• Key–Value and Exact Recovery: Encoding via random linear codes and subcodes allows algebraically exact recovery of bound/bundled components (solving the HDC “factorization” problem) for key–value stores, sets, and trees, outperforming resonator networks in exactness and speed (Raviv, 5 Mar 2024).
• Hardware-Practicality: Ongoing efforts include tailored compilation, in-memory and photonic HDC accelerators, approaches for robust online learning under severe analog drift, and adaptation to rarely-failing and emerging non-volatile devices (Fayza et al., 2023, Arbore et al., 19 Oct 2024, Pu et al., 2023, Khan et al., 2021).
• Limitations: Current algebra and code-theoretic recovery methods for bundled representations are exact in the absence of noise; noise-tolerant decoding for analog HDC remains an open research challenge (Raviv, 5 Mar 2024).
• Biological and Neuromorphic Relevance: HDC offers a mathematically grounded, cognitive-inspired bridge between connectionist and symbolic modes of computation, with algorithmic analogues to kernel methods, associative memory, and holographic storage (Stock et al., 27 Feb 2024, Dewulf et al., 2023).

Hyperdimensional Computing, as unified by its algebraic operations and high-dimensional probabilistic geometry, continues to expand in expressive power, algorithmic diversity, and hardware applicability. Its theoretical guarantees and energy/latency scaling on modern hardware substantiate its utility as a robust, interpretable, and efficient alternative for a broad class of AI and signal processing tasks across domains.