FedUHD: Unsupervised Federated HDC

• FedUHD is an unsupervised federated learning framework that uses high-dimensional hypervectors for clustering without relying on labels.
• It employs client-side kNN-based cluster removal and server-side weighted aggregation to effectively handle non-iid data and communication noise.
• Empirical evaluations show significant efficiency gains with up to 173.6x speedup and enhanced robustness over traditional neural network methods.

FedUHD is an unsupervised federated learning framework based on Hyperdimensional Computing (HDC) that replaces the usual neural-network backbone with high-dimensional hypervectors and cluster-centric aggregation. It is designed for three practical difficulties in federated unsupervised learning: non-independent and identically distributed data across clients, expensive computation and communication at the edge, and vulnerability to communication noise. Rather than learning class prototypes from labels, FedUHD learns a set of global cluster hypervectors, $$\mathcal{G} = \{\mathbf{g}_j \in \mathbb{R}^{D}\}_{j=1}^{J}$$, with $J > K$ because the true number of classes $K$ is unknown in unsupervised learning, and it combines client-side kNN-based cluster hypervector removal with server-side weighted HDC aggregation (Lee et al., 16 Aug 2025).

1. Problem formulation and conceptual basis

FedUHD is situated in federated unsupervised learning (UFL), where clients collaboratively learn from unlabeled local data without centralizing raw samples. In this setting, client $i$ holds data $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$, with total sample count $n = \sum_i n_i$. The absence of labels makes non-iid heterogeneity especially difficult, because there is no label signal to align local and global representations. The framework is explicitly motivated by the tendency of global models to become biased toward dominant client distributions and to underrepresent minority clusters under heterogeneous sampling (Lee et al., 16 Aug 2025).

The architectural premise is that HDC is a brain-inspired computing scheme with lightweight training and inference operations, much smaller model size, and robustness to communication noise. FedUHD therefore shifts the basic learning substrate from backpropagation over neural weights to operations on hypervectors, including random projection or encoding, bundling by vector addition, and similarity search. This substitution is not merely an implementation detail: it defines the framework’s claims of edge efficiency, small communication payloads, and robustness under noisy transmission (Lee et al., 16 Aug 2025).

A central conceptual distinction is that FedUHD does not assume known class identities. Instead, it learns global cluster hypervectors. This moves the learning target away from supervised prototype estimation and toward unsupervised cluster consensus. The choice $J > K$ reflects the paper’s treatment of cluster centroids as an overcomplete unsupervised representation rather than a direct one-to-one encoding of semantic classes (Lee et al., 16 Aug 2025).

2. Representation learning and training pipeline

FedUHD begins by converting each input into a hypervector. For a sample $\mathbf{x}_{in}$, the encoded representation is

$H_{in} \gets r(f(\mathbf{x}_{in})).$

Here $f$ is a feature extractor, used for images, and $J > K$0 is a random projection function. For simple tabular or sensor data such as HAR, standard random projection is sufficient. For images such as CIFAR, a pretrained and frozen feature extractor is used before HDC encoding. The resulting encoded set is denoted $J > K$1 (Lee et al., 16 Aug 2025).

Local learning is then performed by clustering in HDC space: $J > K$2 Here $J > K$3 is the number of local epochs and $J > K$4 denotes the initialization centroids for client $J > K$5. In the initial round, clients use random global centroids $J > K$6; in later rounds, they use server-provided centroids. This gives FedUHD a standard federated round structure, but with local k-means over encoded hypervectors rather than local neural optimization (Lee et al., 16 Aug 2025).

At the protocol level, the framework proceeds in five stages. The server initializes centroids and sends them to clients. Each client encodes its local data into hypervectors, applies kNN-based removal, runs local k-means, and computes local centroids and cluster sizes. The client then uploads $J > K$7 and $J > K$8 to the server. The server computes weighted global centroids $J > K$9 and broadcasts them back to clients. This cycle repeats until convergence (Lee et al., 16 Aug 2025).

One consequence of this design is that the transmitted model state is cluster-centric rather than parameter-centric. The exchanged objects are local centroid hypervectors and cluster sizes, not deep neural weights. This is the main structural reason that FedUHD claims lower communication overhead than NN-based UFL (Lee et al., 16 Aug 2025).

3. Mechanisms for non-iid robustness

FedUHD’s two main technical innovations are both aimed at the non-iid setting. The first is a client-side kNN-based cluster hypervector removal method. The intuition is that some server-provided global centroids may be harmful for a given client if they do not correspond to any nearby local structure. For each global centroid, the client finds its $K$0 nearest encoded samples $K$1, inspects their cluster IDs, and treats the centroid as an outlier if none of the $K$2 neighbors matches the centroid’s cluster ID. The corresponding global centroid and local centroid for that cluster ID are then removed before local retraining, and the number of client-side clusters is updated as

$K$3

This mechanism is described as removing “detrimental outliers” and is intended to prevent misleading initialization, improve local alignment with actual client data, and stabilize training without labels (Lee et al., 16 Aug 2025).

The second innovation is server-side weighted HDC aggregation. In unsupervised FL, label identities are unknown, so label-based aggregation strategies are unavailable. FedUHD therefore weights each client contribution to cluster $K$4 by the number of local samples assigned to that cluster. The server aggregates local centroid hypervectors $K$5 into a global centroid $K$6 using cluster-size weights derived from $K$7. The operational meaning is direct: clients with more samples assigned to cluster $K$8 contribute more to the corresponding global centroid. The stated purpose is to balance heterogeneous client distributions and make the global model more representative even without labels (Lee et al., 16 Aug 2025).

Taken together, these mechanisms act on opposite sides of the federated loop. The client-side procedure prunes globally supplied centroids that are locally implausible, while the server-side procedure amplifies centroids that are supported by more local assignments. This suggests a two-level correction scheme for non-iidness: local exclusion of mismatched structure and global weighting of supported structure. The paper presents these two mechanisms as the basis for FedUHD’s accuracy gains over neural-network baselines in unsupervised federated settings (Lee et al., 16 Aug 2025).

4. Communication model, computational profile, and robustness

FedUHD’s efficiency claims derive from the properties of HDC rather than from a new communication topology. The protocol is still a standard federated loop, but the local workload consists mainly of encoding, kNN filtering, k-means clustering, and vector aggregation. No backpropagation is performed during federated training. The exchanged model state comprises only centroid hypervectors and cluster sizes. This is materially smaller than transmitting neural-network weights, especially when contrasted with NN backbones such as ResNet18, which the paper cites as having millions of parameters (Lee et al., 16 Aug 2025).

The framework also attributes robustness benefits to the “holistic” representation of hypervectors: small perturbations do not drastically change meaning. In the paper’s framing, this property matters under packet loss and additive noise, where deep models are more fragile. FedUHD therefore claims an advantage not only in nominal efficiency but also in operation over unreliable communication links (Lee et al., 16 Aug 2025).

A common misconception is that replacing the federated learner with HDC eliminates all feature extraction costs. FedUHD does not make that claim. For image data such as CIFAR, it uses a pretrained and frozen feature extractor before HDC encoding, whereas for simple tabular or sensor data standard random projection is sufficient. The crucial distinction is that the federated training and communication concern the HDC cluster learner, not a trainable neural backbone (Lee et al., 16 Aug 2025).

The framework’s efficiency argument is thus layered. Smaller model state reduces payload size. Absence of backpropagation reduces local compute and energy. HDC operations are described as naturally parallel and memory-friendly. Robustness to communication noise is claimed as an additional systems property rather than merely a by-product of smaller models (Lee et al., 16 Aug 2025).

5. Empirical evaluation

The experimental evaluation uses HAR, CIFAR-10, and CIFAR-100. Data are distributed non-iid using a Dirichlet prior $K$9 with $i$0. The client counts are 10 for HAR, 10 and 100 for CIFAR-10, and 10 and 100 for CIFAR-100. HDC dimensions are $i$1 for HAR and $i$2 for CIFAR-10 and CIFAR-100. The number of clusters is $i$3 for HAR, $i$4 for CIFAR-10, and $i$5 for CIFAR-100. Neighborhood size $i$6 takes values $i$7 depending on dataset, and the baselines are Orchestra, f-BYOL, f-specloss, f-simsiam, and f-simclr, with Orchestra identified as generally the strongest baseline (Lee et al., 16 Aug 2025).

The primary accuracy metric is unsupervised clustering accuracy (ACC), defined through the best one-to-one mapping between predicted clusters and true classes. Additional reported metrics are training time, energy consumption, communication cost, and robustness under noise (Lee et al., 16 Aug 2025).

Setting	ACC
HAR, 10 clients	76.85%
CIFAR-10, 10 clients	67.54%
CIFAR-10, 100 clients	67.77%
CIFAR-100, 10 clients	31.11%
CIFAR-100, 100 clients	26.86%

Across these settings, FedUHD improves accuracy by an average of 15.50% relative to Orchestra. On efficiency, the reported average gains over Orchestra are $i$8 speedup and $i$9 energy efficiency improvement, while the abstract reports up to $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$0 better speedup and $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$1 energy efficiency. Communication savings are reported as up to $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$2 lower communication cost on HAR, $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$3 lower on CIFAR-10, and $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$4 lower on CIFAR-100 (Lee et al., 16 Aug 2025).

Robustness experiments consider two communication failure scenarios: packet loss and Gaussian noise. Under packet loss, accuracy degradation is 15.68% for FedUHD versus 64.95% for Orchestra. Under Gaussian noise, degradation is 24.46% for FedUHD versus 66.48% for Orchestra. These comparisons are central to the paper’s claim that HDC-based federated unsupervised learning tolerates noisy communication better than NN-based alternatives (Lee et al., 16 Aug 2025).

Sensitivity analysis studies neighborhood size $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$5 and number of clusters $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$6. FedUHD is reported to perform well over a broad $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$7 range. Larger $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$8 generally improves ACC because finer cluster granularity better matches true classes, and FedUHD consistently outperforms Orchestra at the same $\mathcal{X}_i = \{\mathbf{x}_j\}_{j=1}^{n_i}$9. This suggests that cluster granularity is a meaningful design knob rather than a fixed nuisance parameter (Lee et al., 16 Aug 2025).

6. Position in the literature, terminology, and significance

FedUHD belongs to a broader line of federated HDC research, but its problem formulation is distinct. FedHDC introduced federated supervised HDC with class prototype hypervectors, and FHDnn extended it by combining a fixed self-supervised SimCLR feature extractor with federated training of only the HDC learner. RE-FHDC then focused on resource-efficient supervised federated HDC by training multiple smaller sub-models and refining their concatenation. FedUHD differs from both by targeting unsupervised federated learning and by learning global cluster hypervectors rather than class prototypes (Ergun et al., 2023, Zeulin et al., 2023).

The framework’s significance is summarized in five points in the underlying paper: no dependence on neural backpropagation, much smaller communication payloads, lower training energy and latency, better robustness to packet loss and noise, and explicit mechanisms for handling non-iid data without labels. In that sense, its contribution is not only a faster model family but an alternative systems-and-learning design for UFL on resource-constrained edge devices (Lee et al., 16 Aug 2025).

The name also admits a potential ambiguity in adjacent literature. A separate 2022 work on federated unsupervised domain adaptation for face recognition is titled “Federated Unsupervised Domain Adaptation for Face Recognition” and names its method FedFR, although one summary sentence refers to “FedFR/FedUHD.” That method addresses privacy-preserving source-target face adaptation via clustering and domain constraint loss rather than HDC-based unsupervised federated clustering, and it should be distinguished from FedUHD as defined in the 2025 HDC paper (Zhuang et al., 2022).

Several practical implications follow from the reported results. Because FedUHD uses only centroid hypervectors and cluster sizes, it is particularly well matched to settings where model transmission dominates cost. Because it uses kNN-based centroid removal and weighted aggregation, it directly targets the absence of labels in non-iid federated learning rather than assuming supervised alignment. The sensitivity results further suggest that deployment requires choices for $n = \sum_i n_i$0 and $n = \sum_i n_i$1, and a plausible implication is that image performance will remain coupled to the quality of the pretrained and frozen feature extractor used before HDC encoding (Lee et al., 16 Aug 2025).