FactorHD: Hyperdimensional Factorization

• FactorHD is a neuro-symbolic hyperdimensional computing model that factorizes complex multi-object, multi-class hierarchies with embedded memorization clauses for selective recoverability.
• It employs a bundling-binding-bundling symbolic encoding and clause-driven factorization algorithm to overcome combinatorial search challenges and avoid superposition catastrophe and the problem of 2.
• Empirical evaluations show FactorHD scales efficiently with significant speedups and maintains high factorization accuracy across various representational settings and neural integration.

Searching arXiv for the specified paper and closely related HDC factorization work. arxiv_search: query: "FactorHD: A Hyperdimensional Computing Model for Multi-Object Multi-Class Representation and Factorization" max_results: 5 Searching for "FactorHD" on arXiv. {"query":"FactorHD: A Hyperdimensional Computing Model for Multi-Object Multi-Class Representation and Factorization","max_results":5} FactorHD is a neuro-symbolic hyperdimensional computing (HDC) model designed to represent and factorize multi-object, multi-class hierarchies with class–subclass relations efficiently and accurately. It was introduced to address a specific failure mode in prior HDC formulations: when multiple objects occupy different levels of classes and subclasses, factorization—the recovery of constituent symbol hypervectors from a composite representation—becomes difficult because existing class–instance and class–class schemes encounter combinatorial search, information loss, and ambiguity under superposition. FactorHD introduces a bundling–binding–bundling symbolic encoding with an embedded memorization clause, together with a clause-driven factorization algorithm that selectively eliminates redundant classes and recovers target associations. The formulation is explicitly intended to avoid both “superposition catastrophe” and “the problem of 2,” while preserving scalability to large representational sizes (Zhou et al., 16 Jul 2025).

1. Problem setting and representational objective

In the FactorHD formulation, HDC operates in a high-dimensional space of quasi-orthogonal hypervectors (HVs) representing symbols such as classes, subclasses, and their compositions. Let $F$ denote the number of classes, or factors, and let $M$ denote the number of items, or subclasses, per class codebook. A multi-object, multi-class scene is treated as compositional data in which each object may occupy one or multiple subclasses across one or multiple classes, potentially at different hierarchy levels, as in the example animal $\rightarrow$ dog $\rightarrow$ spaniel $\rightarrow$ Fido (Zhou et al., 16 Jul 2025).

Factorization is defined as the operation that, given a composite HV $H$ representing one or multiple objects, recovers the constituent symbol HVs—specifically, class labels and subclass items—for the target class or classes and object or objects. In classic class–class HDC representations, this operation often requires combinatorial search over $M^F$ candidate tuples and iterative unbinding, which degrades both efficiency and accuracy at scale. When multiple objects are represented by bundling, their components superpose; this creates ambiguity described in the paper as superposition catastrophe. If identical items occur, information can be lost through the problem of 2. FactorHD addresses this setting by embedding an explicit per-class memorization clause inside each class bundle so that unbinding by the labels of other classes isolates the target class bundle without exhaustive search (Zhou et al., 16 Jul 2025).

The central representational objective is therefore not merely compositional encoding, but selective recoverability. FactorHD is constructed so that class-specific information remains distinguishable even when multiple objects and multiple hierarchy levels are superposed in the same composite HV. A plausible implication is that the model treats recoverability as a first-class design criterion rather than as a secondary property of symbolic composition.

2. Hypervector space, codebooks, and operators

FactorHD uses bipolar HVs for item memory, with $\vec{V} \in \{-1,+1\}^D$, while bundled single-object clauses are clipped into ternary HVs with $\vec{V} \in \{-1,0,+1\}^D$. When bundling across objects, intermediate sums may remain in $\mathbb{Z}^D$ before later thresholding or clipping; in the reported experiments, FactorHD uses ternary HVs and therefore 2 bits per dimension (Zhou et al., 16 Jul 2025).

For each class $M$0, the model defines a codebook

$M$1

with each $M$2 quasi-orthogonal. Each class also has a label HV, $M$3, quasi-orthogonal to other items and labels. A global $M$4 can be used when a class is absent for an object. Similarity is measured by normalized dot product:

$M$5

The paper states that other metrics such as cosine or Hamming are compatible (Zhou et al., 16 Jul 2025).

The core VSA operators are standard but used in a specific way. Bundling is component-wise addition. Within a single-object bundle, component values are clipped to $M$6 for storage, while multi-object accumulations can temporarily remain in $M$7. Binding is component-wise multiplication, denoted $M$8, and is self-inverse, so unbinding uses the same operation. The paper notes that $M$9 and that $\rightarrow$0, the all-ones HV. Permutation $\rightarrow$1 is available in the VSA toolbox for sequences, but is not used in FactorHD’s basic class–subclass representation. Thresholding is application-specific and applies both to similarity values and to component values during storage conversion (Zhou et al., 16 Jul 2025).

Dimensionality is treated as a practical design parameter. The baselines use typical values such as $\rightarrow$2 for $\rightarrow$3 and $\rightarrow$4 for $\rightarrow$5, whereas FactorHD halves $\rightarrow$6 when using ternary HVs to match the storage footprint of binary baselines. This design couples symbolic encoding to memory-efficiency considerations rather than treating dimensionality as a purely abstract capacity parameter (Zhou et al., 16 Jul 2025).

3. Symbolic encoding and the memorization clause

FactorHD represents multi-class, multi-level hierarchies through a bundling–binding–bundling form. For each class $\rightarrow$7, all subclass items associated with the scene under that class are bundled together with the class label:

$\rightarrow$8

where $\rightarrow$9 indexes the set of subclass items under class $\rightarrow$0 that appear in the current scene. If no items under class $\rightarrow$1 are present, the model still preserves the clause using

$\rightarrow$2

The full representation is then the binding of the per-class clauses:

$\rightarrow$3

The extra memorization clause is precisely the inclusion of $\rightarrow$4 inside each $\rightarrow$5 (Zhou et al., 16 Jul 2025).

This clause is the mechanism that makes independent recovery possible. Because the label is embedded as a summand in each class bundle, unbinding all other labels from $\rightarrow$6 leaves a residual HV that contains primarily the target class label together with its subclass items. In the single-object, single-level case, the paper writes:

$\rightarrow$7

The labels therefore act as anchors that preserve class-specific information despite superposition across objects (Zhou et al., 16 Jul 2025).

For multi-level hierarchies, FactorHD treats different subclass levels equally at encoding time by bundling all relevant subclass HVs for a class into $\rightarrow$8. If a hierarchy contains, for example, a class, a level-1 subclass, and a level-2 subclass, both levels are included in the class-specific set $\rightarrow$9. In factorization, however, recovery proceeds top-down: the method first identifies the level-1 subclass under a class, then restricts similarity comparisons to the children of the identified item at the next level, and so on (Zhou et al., 16 Jul 2025). This separation between flat encoding and hierarchical decoding is one of the model’s distinctive design choices.

The paper’s synthetic example uses two classes, Animal and Color, with the scene consisting of a red spaniel dog and a white siamese cat. The encoding is

$\rightarrow$0

$\rightarrow$1

$\rightarrow$2

This example is used to illustrate that the encoded representation contains enough information to recover both class-local content and cross-class object associations without directly storing each object as a separate bound tuple (Zhou et al., 16 Jul 2025).

4. Factorization algorithm and decision rules

The factorization algorithm begins by isolating each class through unbinding with the labels of all other classes. For class $\rightarrow$3, the paper defines

$\rightarrow$4

Because $\rightarrow$5, this operation eliminates the product contributions of the other classes and preserves the bundling clause for class $\rightarrow$6. The resulting $\rightarrow$7 is then matched against the item memory of class $\rightarrow$8 by computing, for every $\rightarrow$9,

$H$0

In single-object mode, the method chooses the $H$1 item and then recursively descends through the hierarchy by restricting the next-stage comparisons to the children of the selected item. In multi-object mode, the method selects the set

$H$2

using either class-specific thresholds or a shared threshold $H$3 (Zhou et al., 16 Jul 2025).

After per-class candidate selection, the method forms cross-class candidate combinations. For candidate sets $H$4, it chooses one item per class and constructs an object proposal

$H$5

Each proposal is then scored by

$H$6

If $H$7, the combination is accepted as an actual object. Accepted objects may be explicitly reconstructed and optionally excluded from further consideration, after which the process can repeat until no new objects are found (Zhou et al., 16 Jul 2025).

The decision rules are thus two-stage. First, class-local evidence is extracted through $H$8 or $H$9 selection. Second, cross-class consistency is tested through $M^F$0. This dual screening is the mechanism by which FactorHD removes algebraically possible but semantically absent combinations. In the synthetic example, four candidate combinations arise from two animal candidates and two color candidates, but only the combinations corresponding to dog–red and cat–white are accepted because only those exceed the acceptance threshold in the combination test (Zhou et al., 16 Jul 2025).

This algorithmic structure is also the paper’s explicit answer to the two major HDC failure modes. Superposition catastrophe is addressed because unbinding isolates class-specific bundles and thresholding prunes spurious overlaps, while cross-class validation removes false associations. The problem of 2 is addressed because bundling across objects accumulates counts per component in $M^F$1 before thresholding or clipping, so similarity amplitudes increase with multiplicity. In multi-object mode, multiple items per class may therefore surpass threshold, and repeated instances can be inferred through accepted cross-class proposals. The paper notes, however, that exact counting of identical objects may require additional amplitude calibration or explicit integer accumulation before clipping (Zhou et al., 16 Jul 2025).

5. Complexity, thresholding, and scaling behavior

The computational profile of FactorHD is expressed in terms of encoding cost, per-class isolation, similarity scanning, and combination testing. Building one per-class clause $M^F$2 is $M^F$3 for addition and clipping. Binding across $M^F$4 classes to obtain $M^F$5 is $M^F$6. During factorization, one unbinding via the labels of other classes is $M^F$7, and the similarity scan is $M^F$8 per class or per level. If the candidate sets are $M^F$9, then the number of cross-class combinations is $\vec{V} \in \{-1,+1\}^D$0, and each combination test is $\vec{V} \in \{-1,+1\}^D$1. The overall time is therefore approximately

$\vec{V} \in \{-1,+1\}^D$2

which the paper reports as empirically behaving as $\vec{V} \in \{-1,+1\}^D$3, where $\vec{V} \in \{-1,+1\}^D$4 is the maximum number of subclass items per class, because the candidate sets remain small and only a few accepted combinations are tested (Zhou et al., 16 Jul 2025).

The comparison offered in the paper is primarily against resonator networks and the IMC factorizer. Those methods are described as requiring iterative searches across the $\vec{V} \in \{-1,+1\}^D$5 space and repeated updates, whereas FactorHD’s time complexity is approximately $\vec{V} \in \{-1,+1\}^D$6 and the IMC factorizer and resonator are larger than $\vec{V} \in \{-1,+1\}^D$7 under typical scaling (Zhou et al., 16 Jul 2025). In the context of the paper, this is not merely a runtime observation but part of the representational argument: the memorization clause reduces the search problem from a global combinatorial exploration to a class-local isolation plus a small candidate validation stage.

Threshold selection is formalized through an empirically fitted expression:

$\vec{V} \in \{-1,+1\}^D$8

where $\vec{V} \in \{-1,+1\}^D$9 is the number of objects, $\vec{V} \in \{-1,0,+1\}^D$0 the number of factors, $\vec{V} \in \{-1,0,+1\}^D$1 the number of items per codebook, and $\vec{V} \in \{-1,0,+1\}^D$2 the dimensionality. The paper states that $\vec{V} \in \{-1,0,+1\}^D$3 increases with $\vec{V} \in \{-1,0,+1\}^D$4 and decreases with $\vec{V} \in \{-1,0,+1\}^D$5, and that it is approximately linear with $\vec{V} \in \{-1,0,+1\}^D$6 and $\vec{V} \in \{-1,0,+1\}^D$7. Values near $\vec{V} \in \{-1,0,+1\}^D$8 achieve high accuracy even if not optimal (Zhou et al., 16 Jul 2025). Thresholding is therefore treated as a calibrated operational parameter rather than as a generic tuning heuristic.

Capacity and robustness are attributed to quasi-orthogonality and label anchoring. As $\vec{V} \in \{-1,0,+1\}^D$9 and $\mathbb{Z}^D$0 grow, the paper reports that FactorHD’s accuracy declines more slowly than the baselines and that the method scales to representation sizes up to $\mathbb{Z}^D$1. Larger $\mathbb{Z}^D$2 increases robustness by improving quasi-orthogonality and reducing error rates; in particular, FactorHD maintains at least $\mathbb{Z}^D$3 factorization accuracy for Rep 1 even at lower $\mathbb{Z}^D$4 (Zhou et al., 16 Jul 2025). The stated significance is that representational size, memory footprint, and factorization reliability remain jointly manageable within the same symbolic framework.

6. Empirical evaluation, neural integration, and limitations

The empirical evaluation includes three representational settings. Rep 1 is single object with a single subclass level. Rep 2 is single object with multiple subclass levels. Rep 3 is multiple objects with multiple subclass levels. The experimental baselines are the resonator network of Frady et al. (2020), the IMC factorizer of Langenegger et al. (2023), and C-I designs of Camposampiero et al. (2024). The reported dimensions are $\mathbb{Z}^D$5 for $\mathbb{Z}^D$6 and $\mathbb{Z}^D$7 for $\mathbb{Z}^D$8 in the binary baselines, while FactorHD uses ternary HVs with halved $\mathbb{Z}^D$9 for comparable memory. The datasets include RAVEN, CIFAR-10, and CIFAR-100 (Zhou et al., 16 Jul 2025).

For Rep 1, FactorHD maintains more than $M$00 factorization accuracy even at lower $M$01, while the resonator network fails when problem size reaches $M$02. The IMC factorizer remains accurate but requires thousands of iterations; the paper gives the example of 3312 iterations at $M$03, $M$04, $M$05. The factorization time of FactorHD scales negligibly with problem size relative to the baselines, with speedups of $M$06 at $M$07 and up to approximately $M$08 at $M$09 (Zhou et al., 16 Jul 2025).

For Rep 2, FactorHD reaches $M$10 accuracy at $M$11, and the top-down factorization strategy reduces the number of similarity computations by restricting lower-level searches to the children of already identified higher-level items. For Rep 3, the method does not require prior knowledge of the number of objects, although higher $M$12 is needed for very high accuracy. Thresholding limits the number of candidate combinations, and only combinations satisfying $M$13 are inferred as actual objects (Zhou et al., 16 Jul 2025).

In the RAVEN experiments, training and factorization achieve more than $M$14 accuracy for most patterns at $M$15, and reasonable accuracy is maintained even at reduced $M$16. For vision tasks, ResNet-18 is used as the neuro front-end for feature extraction; features are then encoded into HVs and factorized. On CIFAR-10, images are encoded by binding the image label with a dummy label. On CIFAR-100, coarse and fine labels are bound. With this integration, FactorHD achieves $M$17 factorization accuracy on CIFAR-10, with less than $M$18 loss relative to typical classification accuracy of plain ResNet-18 on CIFAR-10. The paper also states that CIFAR-100 factorization of coarse and fine labels is supported and that FactorHD maintains high factorization accuracy even with bundled, or superposed, image inputs, improving training efficiency (Zhou et al., 16 Jul 2025).

The comparison with prior HDC designs is explicit. C-I models, which bind class with instance and bundle across classes, can factorize single-object partial information but suffer superposition catastrophe and the problem of 2 for multiple objects and deeper hierarchies. C-C models, which bind across classes and bundle across objects, can represent multiple objects but usually require iterative search across $M$19 candidates or nonlinear dynamics, which causes inefficiency and parameter sensitivity. FactorHD differs by embedding $M$20 into each class clause, retaining only items above threshold, and then removing extraneous associations through cross-class validation (Zhou et al., 16 Jul 2025).

The paper also lists several limitations and edge cases. Threshold mis-setting can miss items if too high or admit spurious candidates if too low, thereby increasing the load of combination testing. Very deep hierarchies and small $M$21 reduce quasi-orthogonality and increase noise, requiring either larger $M$22 or stricter thresholds for high accuracy. Exact multiplicity estimation for identical objects may require additional amplitude calibration or explicit integer accumulation before clipping. Finally, the paper notes that the underlying operations—addition, multiplication, thresholding, and similarity—are well suited to neuromorphic or compute-in-memory accelerators, including FeFET-based associative memory and in-memory HV operations, and that ternary HVs reduce storage bandwidth (Zhou et al., 16 Jul 2025). This suggests a hardware-relevant interpretation of FactorHD as a symbolic factorization scheme aligned with streaming similarity scans and low-bit-width memory representations.