Fed-REACT: Federated Representation Learning

• Fed-REACT is a federated learning framework designed for heterogeneous and evolving client data, using a two-stage process that decouples self-supervised representation learning from task-specific model training.
• It employs federated self-supervised contrastive learning to train a shared encoder and then leverages adaptive evolutionary clustering to dynamically group clients based on task-model weight similarities.
• Empirical results on diverse real-world datasets demonstrate that Fed-REACT achieves superior accuracy, robustness, and efficiency while addressing privacy and resource concerns.

Searching arXiv for the specified paper and closely related federated learning work to ground the article in current literature. Search results will be used only to verify the paper and contextualize citations; factual details in the article will remain constrained to the supplied data block. Performing arXiv query for ([2509.07198](/papers/2509.07198)) and Fed-REACT. Querying arXiv. Fed-REACT is a federated learning framework for heterogeneous and evolving client data. It is introduced in "Fed-REACT: Federated Representation Learning for Heterogeneous and Evolving Data" (Chen et al., 8 Sep 2025) and is motivated by the high resource costs and privacy concerns associated with centralized machine learning, together with the fact that in real-world deployments, client data distributions often evolve over time and differ significantly across clients. Fed-REACT combines representation learning with evolutionary clustering in a two-stage process: in the first stage, each client learns a local model to extract feature representations from its data; in the second stage, the server dynamically groups clients into clusters based on these representations and coordinates cluster-wise training of task-specific models for downstream objectives such as classification or regression. The source description further states that the method provides a theoretical analysis of the representation learning stage and empirically demonstrates superior accuracy and robustness on real-world datasets.

1. Problem setting and overall design

Fed-REACT proceeds in two sequential phases. In Phase I each client learns a shared encoder via self-supervised contrastive learning. In Phase II the server clusters clients by their task-model weights through evolutionary clustering and then performs cluster-wise aggregation of lightweight task models.

The target setting is federated learning under both heterogeneity and temporal evolution. The source description identifies two distinct difficulties: client data distributions differ significantly across clients, and those distributions evolve over time. Standard FL algorithms are presented as degrading under these conditions. Fed-REACT addresses this by separating representation learning from downstream task training.

A common misconception is to treat Fed-REACT as a single global supervised model. The source description instead defines a two-stage pipeline in which the encoder is learned in a federated self-supervised manner, after which clients train small task models and the server carries out dynamic grouping and cluster-wise aggregation. Another common misconception is to treat the grouping rule as data-driven in the sense of raw-example access; the source description states that the server groups clients by task-model weights and that only model weights and similarities are shared.

2. Phase I: federated representation learning

In Phase I, each client $k$ holds an encoder $f(\cdot\,;\theta)$, for example a causal dilated CNN. The encoder maps a multivariate time series $x\in\mathbb{R}^{d\times T}$ to a feature vector $f(x;\theta)\in\mathbb{R}^{\hat d}$.

For an anchor $x^\mathrm{ref}$, a positive $x^\mathrm{pos}$, and negatives $\{x^{\mathrm{neg}_r}\}_{r=1}^R$, the contrastive loss is defined as

$$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$

where $\sigma(u)=1/(1+e^{-u})$.

The federated optimization rule in this phase is standard Federated Avg. At each round $t$, the server sends $f(\cdot\,;\theta)$0 to all clients; client $f(\cdot\,;\theta)$1 performs local SGD on $f(\cdot\,;\theta)$2 and returns $f(\cdot\,;\theta)$3; the server aggregates according to

$f(\cdot\,;\theta)$4

This design makes the encoder the shared object across clients, while downstream supervision is deferred to Phase II. A plausible implication is that the framework attempts to decouple representation formation from the non-stationary task structure that later appears in the clustering stage.

3. Phase II: evolutionary clustering and cluster-wise task training

After learning the encoder, clients extract feature representations for labeled samples and train a small task model, for example an SVM for classification or a single linear layer for regression. The server then groups clients and aggregates within each cluster (Chen et al., 8 Sep 2025).

At round $f(\cdot\,;\theta)$5, client $f(\cdot\,;\theta)$6 uploads its task-model weights $f(\cdot\,;\theta)$7. The observed similarity matrix is

$f(\cdot\,;\theta)$8

The underlying “true” similarity $f(\cdot\,;\theta)$9 is unknown, and the source description models

$x\in\mathbb{R}^{d\times T}$0

with noise $x\in\mathbb{R}^{d\times T}$1.

To smooth similarity over time, Fed-REACT uses adaptive evolutionary smoothing, identified in the source description as AFFECT. The smoothed similarity is

$x\in\mathbb{R}^{d\times T}$2

where $x\in\mathbb{R}^{d\times T}$3 is chosen to minimize

$x\in\mathbb{R}^{d\times T}$4

Clustering is then performed by applying agglomerative hierarchical clustering to $x\in\mathbb{R}^{d\times T}$5 to produce $x\in\mathbb{R}^{d\times T}$6 clusters $x\in\mathbb{R}^{d\times T}$7. For each cluster $x\in\mathbb{R}^{d\times T}$8, the per-round aggregate is

$x\in\mathbb{R}^{d\times T}$9

where $f(x;\theta)\in\mathbb{R}^{\hat d}$0 is the set of labeled samples client $f(x;\theta)\in\mathbb{R}^{\hat d}$1 uses at round $f(x;\theta)\in\mathbb{R}^{\hat d}$2.

The framework includes two temporal smoothing strategies for cluster parameters. Strategy A1 is simple temporal averaging: $f(x;\theta)\in\mathbb{R}^{\hat d}$3 Strategy A2 is weighted averaging with forgetting: $f(x;\theta)\in\mathbb{R}^{\hat d}$4

Full Phase II repeats for $f(x;\theta)\in\mathbb{R}^{\hat d}$5 rounds: clients train local task models; the server collects $f(x;\theta)\in\mathbb{R}^{\hat d}$6, computes $f(x;\theta)\in\mathbb{R}^{\hat d}$7, clusters to get $f(x;\theta)\in\mathbb{R}^{\hat d}$8, aggregates per-cluster parameters, applies A1 or A2, and broadcasts cluster models back to members.

4. Theoretical analysis of the representation-learning stage

The theoretical analysis is confined to Phase I. Assumption 3.1 states: for every client $f(x;\theta)\in\mathbb{R}^{\hat d}$9 and time $x^\mathrm{ref}$0, the local loss $x^\mathrm{ref}$1 is bounded above by $x^\mathrm{ref}$2; $x^\mathrm{ref}$3 is $x^\mathrm{ref}$4-Lipschitz and $x^\mathrm{ref}$5-smooth; stochastic gradients $x^\mathrm{ref}$6 are unbiased, with variance $x^\mathrm{ref}$7; and the projection error satisfies $x^\mathrm{ref}$8.

The main convergence result introduces

$x^\mathrm{ref}$9

and considers time-smoothed federated GD with step size $x^\mathrm{pos}$0. As $x^\mathrm{pos}$1,

$x^\mathrm{pos}$2

The source description states that, in particular, for large window $x^\mathrm{pos}$3 so that $x^\mathrm{pos}$4, the dominant error is the projection error $x^\mathrm{pos}$5 (Chen et al., 8 Sep 2025).

The key lemmas are summarized as follows. Lemma 1 gives a one-step descent inequality with smoothed gradient: $x^\mathrm{pos}$6 Lemmas 2 and 3 provide bounds on objective drift. The proof outline is: show $x^\mathrm{pos}$7 is $x^\mathrm{pos}$8-smooth; expand one GD step, isolate $x^\mathrm{pos}$9, and take expectations to handle stochasticity; telescopically sum over $\{x^{\mathrm{neg}_r}\}_{r=1}^R$0, bound the drift terms using boundedness $\{x^{\mathrm{neg}_r}\}_{r=1}^R$1; then let $\{x^{\mathrm{neg}_r}\}_{r=1}^R$2 and simplify.

This suggests that the formal guarantee is directed at the representation-learning stage rather than at the full clustering-and-task-training pipeline.

5. Empirical evaluation

The empirical evaluation uses three datasets. RTD consists of 3D air-writing trajectories with 10-client and 50-client partitions via Dirichlet($\{x^{\mathrm{neg}_r}\}_{r=1}^R$3), with heterogeneity in class distributions. Its evolving scenarios are: Strategy 1, where each cluster alternates between two Dirichlet-sampled label distributions via a Markov chain with transition $\{x^{\mathrm{neg}_r}\}_{r=1}^R$4; Strategy 2, which continuously resamples within cluster support plus rare cross-cluster drift; and Strategy 3, which uses permanent client migration with small probability $\{x^{\mathrm{neg}_r}\}_{r=1}^R$5. EEG motor-imagery uses 26-channel, 70-step sequences with 10 clients in 3 clusters: left-hand only, right-hand only, and mixed. SUMO EV is an urban mobility regression task on per-vehicle time series predicting battery percentage, with heterogeneity in sample size and usage patterns.

For comparison to supervised FL baselines, the source description states that Table 1 reports average test accuracy on RTD and EEG and RMSE on SUMO EV. Fed-REACT uses Causal CNN encoder plus SVM or linear regressor, with $\{x^{\mathrm{neg}_r}\}_{r=1}^R$6.

Setting	Fed-REACT	Baselines
RTD (10 clients, LSTM, acc)	0.992	FedAvg 0.732, FedProx 0.804, Ditto 0.863, APFL 0.828
RTD (10 clients, TimesNet, acc)	0.992	FedAvg 0.793, FedProx 0.883, Ditto 0.863, APFL 0.755
RTD (10 clients, PatchTST, acc)	0.992	FedAvg 0.918, FedProx 0.903, Ditto 0.991, APFL 0.991
RTD (10 clients, Causal CNN, acc)	0.992	FedAvg 0.982, FedProx 0.988, Ditto 0.989, APFL 0.990
RTD (50 clients, Causal CNN, acc)	0.988	FedAvg 0.986, FedProx 0.984, Ditto 0.895, APFL 0.650
EEG (10 clients, Causal CNN, acc)	0.796	FedAvg 0.559, FedProx 0.605, Ditto 0.516, APFL 0.606
SUMO EV (Causal CNN, RMSE)	1.3	FedAvg 39.8, FedProx 38.2, Ditto 40.1, APFL 38.6

The source description explicitly states that Fed-REACT outperforms all baselines by large margins.

For comparison to clustered FL methods, the source description reports average cluster-model accuracy and Rand score versus ground-truth clusters, with baselines IFCA, FLSC, FLACC, and two ablations of Fed-REACT without temporal memory: SC+MMA snapshot clustering and EC+MMA evolutionary clustering but no model memory.

Setting	Fed-REACT	Other methods
RTD 10 clients (Strategy 1)	A1 0.918, A2 0.870	SC+MMA 0.827, EC+MMA 0.826, IFCA 0.883, FLSC 0.887, FLACC 0.876
RTD 100 clients (Strategy 1)	A1 0.790, A2 0.791	SC+MMA 0.724, EC+MMA 0.733, IFCA 0.701, FLSC 0.695, FLACC 0.693
RTD 100 clients (Strategy 2)	A1 0.856, A2 0.858	SC+MMA 0.803, EC+MMA 0.838, IFCA 0.684, FLSC 0.581, FLACC 0.408
EEG 10 clients	A1 0.802, A2 0.808	SC+MMA 0.799, EC+MMA 0.800, IFCA 0.513, FLSC 0.513, FLACC 0.565

The source description further states that figures of Rand score over rounds show that Fed-REACT’s evolutionary clustering quickly converges to the true partition even under heavy non-stationarity, whereas others oscillate or fail (Chen et al., 8 Sep 2025).

The ablation and sensitivity studies vary the number of clusters $\{x^{\mathrm{neg}_r}\}_{r=1}^R$7, the heterogeneity parameter $\{x^{\mathrm{neg}_r}\}_{r=1}^R$8, and the Markov transitions $\{x^{\mathrm{neg}_r}\}_{r=1}^R$9. For stationary RTD with 100 clients and Dirichlet $$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$0, the accuracy-versus-$$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$1 table reports:

• $$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$2: Fed-REACT(A1) $$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$3, Fed-REACT(A2) $$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$4, SC+MMA $$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$5, EC+MMA $$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$6, IFCA $$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$7, FLSC $$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$8, FLACC $$L_{\mathrm{cl}}(\theta) = -\log\sigma\bigl(f(x^\mathrm{ref};\theta)^\top f(x^\mathrm{pos};\theta)\bigr) -\sum_{r=1}^R \log\sigma\bigl(-f(x^\mathrm{ref};\theta)^\top f(x^{\mathrm{neg}_r};\theta)\bigr),$$9.
• $\sigma(u)=1/(1+e^{-u})$0: Fed-REACT(A1) $\sigma(u)=1/(1+e^{-u})$1, Fed-REACT(A2) $\sigma(u)=1/(1+e^{-u})$2, SC+MMA $\sigma(u)=1/(1+e^{-u})$3, EC+MMA $\sigma(u)=1/(1+e^{-u})$4, IFCA $\sigma(u)=1/(1+e^{-u})$5, FLSC $\sigma(u)=1/(1+e^{-u})$6, FLACC $\sigma(u)=1/(1+e^{-u})$7.
• $\sigma(u)=1/(1+e^{-u})$8: Fed-REACT(A1) $\sigma(u)=1/(1+e^{-u})$9, Fed-REACT(A2) $t$0, SC+MMA $t$1, EC+MMA $t$2, IFCA $t$3, FLSC $t$4, FLACC $t$5.
• $t$6: Fed-REACT(A1) $t$7, Fed-REACT(A2) $t$8, SC+MMA $t$9, EC+MMA $f(\cdot\,;\theta)$00, IFCA $f(\cdot\,;\theta)$01, FLSC $f(\cdot\,;\theta)$02, FLACC $f(\cdot\,;\theta)$03.

For accuracy versus $f(\cdot\,;\theta)$04, the reported values are:

• $f(\cdot\,;\theta)$05: Fed-REACT(A1) $f(\cdot\,;\theta)$06, Fed-REACT(A2) $f(\cdot\,;\theta)$07, SC+MMA $f(\cdot\,;\theta)$08, EC+MMA $f(\cdot\,;\theta)$09, IFCA $f(\cdot\,;\theta)$10, FLSC $f(\cdot\,;\theta)$11, FLACC $f(\cdot\,;\theta)$12.
• $f(\cdot\,;\theta)$13: Fed-REACT(A1) $f(\cdot\,;\theta)$14, Fed-REACT(A2) $f(\cdot\,;\theta)$15, SC+MMA $f(\cdot\,;\theta)$16, EC+MMA $f(\cdot\,;\theta)$17, IFCA $f(\cdot\,;\theta)$18, FLSC $f(\cdot\,;\theta)$19, FLACC $f(\cdot\,;\theta)$20.
• $f(\cdot\,;\theta)$21: Fed-REACT(A1) $f(\cdot\,;\theta)$22, Fed-REACT(A2) $f(\cdot\,;\theta)$23, SC+MMA $f(\cdot\,;\theta)$24, EC+MMA $f(\cdot\,;\theta)$25, IFCA $f(\cdot\,;\theta)$26, FLSC $f(\cdot\,;\theta)$27, FLACC $f(\cdot\,;\theta)$28.

The source description adds that varying Markov transitions $f(\cdot\,;\theta)$29 in Strategy 1 also confirms robustness, with details omitted there for brevity.

6. Computational profile, privacy, deployment, and interpretation

The source description compares encoder complexity against sequence length $f(\cdot\,;\theta)$30. For Causal CNN, the cost is $f(\cdot\,;\theta)$31 FLOPs for kernel size $f(\cdot\,;\theta)$32 and $f(\cdot\,;\theta)$33 layers. For PatchTST with self-attention, the cost is $f(\cdot\,;\theta)$34. Fed-REACT uses Causal CNN for linear scaling.

Model	Parameters $f(\cdot\,;\theta)$35	FLOPs $f(\cdot\,;\theta)$36
Causal CNN	156	32
LSTM	398	80
PatchTST	131	2.7
TimesNet	89	17.6

The communication cost per round is split by phase. In Phase I, one shared encoder $f(\cdot\,;\theta)$37 is sent per client, giving standard Federated Avg cost $f(\cdot\,;\theta)$38. In Phase II, the server clusters centrally and then sends each client only its cluster model $f(\cdot\,;\theta)$39, giving cost $f(\cdot\,;\theta)$40. The source description contrasts this with IFCA and FLSC, which require sending $f(\cdot\,;\theta)$41 models to every client, namely $f(\cdot\,;\theta)$42 (Chen et al., 8 Sep 2025).

The stated privacy property is that only model weights and similarities, specifically cosine values, are shared; no raw data or labels are shared. Evolutionary clustering uses only weight vectors and an adaptive forgetting factor. This does not claim stronger privacy guarantees than those statements; a plausible implication is that the privacy argument is architectural rather than based on a formal privacy definition in the supplied description.

The deployment recommendations are explicit. When the number of clusters is unknown, estimate $f(\cdot\,;\theta)$43 via the elbow rule on within-cluster sum-of-squares or by maximizing the silhouette score. The framework supports intermittent client participation by reusing last-known task weights for absent clients when forming $f(\cdot\,;\theta)$44. For strategy selection, use A2 for highly non-stationary settings to discount outdated models.

In summary, the source description characterizes Fed-REACT through three components: federated self-supervised contrastive learning for a shared encoder, dynamic client clustering via adaptive evolutionary smoothing of task-model similarities, and cluster-wise aggregation of lightweight task models with temporal smoothing. This suggests that its central methodological claim is not merely improved local adaptation, but the explicit integration of representation learning, evolving similarity estimation, and cluster-level task-model coordination under heterogeneous and evolving data.