Federated Task Vector Aggregation

• Federated task vector aggregation is a method that integrates heterogeneous, task-specific model updates from distributed clients, enabling personalized multi-task learning.
• It employs clustering, similarity-weighted, and selective averaging techniques to align diverse updates and overcome challenges in non-IID environments.
• This approach enhances convergence, communication efficiency, and overall accuracy compared to traditional federated averaging methods.

Federated task vector aggregation refers to a collection of protocols, algorithms, and theoretical frameworks that enable the aggregation of heterogeneous, task-specific model vectors or updates in federated learning (FL) environments. These frameworks are designed to address the technical challenges imposed by multi-task, multi-domain, or non-IID federated learning, where local clients may have divergent objectives, architectures, or data distributions, and plain averaging (as in FedAvg) is suboptimal.

1. Foundations and Scope

Federated task vector aggregation generalizes standard FL parameter averaging by explicitly recognizing the local vector differences dictated by client-specific or task-specific objectives. Unlike classical aggregation, which assumes a shared task or label space, modern federated applications may demand:

• Aggregation across multiple tasks, domains, or output spaces (heterogeneous objectives).
• Incorporation of structurally pruned or otherwise misaligned local models.
• Personalization at the client or subgroup level by leveraging similarities in local optimization directions.

Formally, these setups rely on constructing per-client (or per-task) "task vectors," typically defined as the model parameter difference between local fine-tuning and a shared model reference. Aggregation combines these vectors, often using alignment-aware, cluster-based, or adaptive-weighted procedures, to produce one or more global models or adapters fit for downstream re-personalization (Yuan et al., 4 Aug 2025, Yang et al., 16 Sep 2025, Tsouvalas et al., 10 Feb 2025, Shi et al., 20 Mar 2025, Wei et al., 30 May 2025).

2. Mathematical Formulation of Task Vector Aggregation

Let $\theta_g$ denote the global model in round $t$, and $\theta_k^t$ the local model trained by client $k$. The canonical "task vector" for client $k$ is

$\tau_k^t = \theta_k^t - \theta_g^t$

This vector encodes the client-specific parameter update direction, capturing both local data statistics and task characteristics. In multi-task federated learning (FMTL), these vectors may be further partitioned, masked, recalibrated, or combined.

Common aggregation schemes include:

• Weighted averaging: Assign weights $\lambda_k$ and compute $\sum_k \lambda_k \tau_k^t$.
• Similarity-weighted aggregation: Compute pairwise similarities (e.g., cosine) between task vectors and aggregate with weights reflecting affinity (e.g., $\lambda_k \propto \cos(\tau_k, \tau_g)$) (Yang et al., 16 Sep 2025, Shi et al., 20 Mar 2025).
• Clustered or task-specific aggregation: Cluster clients by similarity in task-vector space and aggregate within clusters (Yuan et al., 4 Aug 2025, Tsouvalas et al., 10 Feb 2025).
• Selective/Sparse aggregation: Mask or select only task-relevant dimensions/subspaces before averaging (Wei et al., 30 May 2025, Tsouvalas et al., 10 Feb 2025).

Task vector aggregation is often mathematically expressed by

$$\theta_g^{t+1} = \theta_g^t + \sum_{k=1}^K \lambda_k^t \tau_k^t$$

where $\lambda_k^t$ may be determined by client dataset size, alignment, or optimization objectives (Shi et al., 20 Mar 2025).

3. Core Methodological Variants

The principal variants and methodological innovations within federated task vector aggregation are:

a) Clustering-Based, Task-Aware Aggregation

Methods such as FedAPTA handle clients submitting possibly pruned or structurally heterogeneous model vectors. Aggregation proceeds by:

1. Model recovery: Expand pruned vectors using the last global as template;
2. Task-vector formation: Compute $\Delta W_i = W_i - W^g_{\text{ref}}$ for each client $i$;
3. Distance calculation & clustering: Use cosine distance between $\Delta W_i$ and $\Delta W_j$ to cluster clients (e.g., via HDBSCAN);
4. Cluster-local aggregation: Aggregate $W_i$ within each cluster to form task-specific global models (Yuan et al., 4 Aug 2025).

b) Similarity-Weighted and Personalized Aggregation

Frameworks such as FedAWA and bi-level personalization in federated foundation models assign higher influence to client vectors aligned with the consensus or a personalization criterion. This typically follows an optimization over aggregation weights:

$$\boldsymbol{\lambda}^t = \operatorname{arg\,min}_{\lambda_1,\ldots,\lambda_K} \sum_{k=1}^K \lambda_k \|\tau_k^t - \tau_g^t\|_2 + d\left(\sum_k \lambda_k \theta_k^t, \theta_g^t \right)$$

with constraints $\lambda_k\ge 0$, $\sum_k \lambda_k=1$, and $d(\cdot,\cdot)$ a divergence penalty (Shi et al., 20 Mar 2025, Yang et al., 16 Sep 2025).

Personalized aggregation may additionally compute, for each client $i$,

$$p_{i,k}^t = \frac{g(\tau_i^t, \tau_k^t)}{\sum_{j=1}^K g(\tau_i^t, \tau_j^t)}$$

and broadcast $$\bar{\theta}_i^{\,t+1} = \bar{\theta}_i^t + \sum_k p_{i,k}^t \tau_k^t$$ (Yang et al., 16 Sep 2025).

c) Subspace Decoupling and Sign-Based Merging

Several approaches (MaTU, FedDEA) recognize that different tasks may activate disjoint parameter subspaces and accommodate this via:

• Masking client updates to retain only strong (task-relevant) response dimensions (Wei et al., 30 May 2025).
• Aggregating sign and magnitude in a merged task vector (unified adapter) to avoid destructive interference when clients operate on multiple disparate tasks (Tsouvalas et al., 10 Feb 2025).

In MaTU, a client with task set $\mathcal{T}_n$ forms a unified vector by taking the sign-agreement and maximal amplitude per coordinate:

$\tau_n = \sigma_n \odot \mu_n$

where $$\sigma_n = \operatorname{sgn}\left(\sum_{t \in \mathcal{T}_n} \tau_n^t\right)$$, and $\mu_n$ is the largest matching-magnitude among $\{\tau_n^t\}$.

d) SVM or Support-Vector-Based Aggregation

TurboSVM-FL fits support vector machines on the "class-embedding" rows of client models. Only embeddings serving as support vectors (i.e., lying close to class boundaries) are aggregated, and a further max-margin regularization step increases class separation in the server-updated model (Wang et al., 2024).

4. Algorithmic and System Implementations

Table: Representative Federated Task Vector Aggregation Algorithms

Method	Aggregation Principle	Personalization	Key Features
FedAPTA	Clustered weighted mean	Task-specific	Model infilling, HDBSCAN clusters
TurboSVM-FL	Selective (SVM support vector)	No	SVM boundary focus, margin regularize
FedAWA	Adaptive geometric weighting	No	Convex-opt alignment weights
MaTU	Sign-masked vector fusion	Task+Unified	Lightweight modulator masks
FedDEA	Magnitude-based subspace decoupling	No	Top-k threshold, recalibration
Bi-level Personalization	Cosine-weighted, per-client	Yes	Client-specific model broadcast
PF-MSMTrec	Decoupled, multi-phase aggregation	Yes	Parameter template, conflict-QP

Algorithmic realization generally involves local client computation of task vectors, possibly further masked or refined, followed by server- or leader-based aggregation using the selected protocol. Some methods support heterogeneous or pruned models and design server-side recovery and alignment strategies (Yuan et al., 4 Aug 2025); others assume uniform architectures (Shi et al., 20 Mar 2025, Yang et al., 16 Sep 2025).

Decentralized variants such as ColNet partition aggregation among rotating group leaders using conflict-averse subroutines to combine group-wise backbone parameters (Feng et al., 17 Jan 2025).

5. Security and Privacy-Preserving Aggregation

Efficient and secure aggregation of task vectors is essential for privacy guarantees. The Partial Vector Freezing (PVF) framework compresses the cost of secure aggregation protocols by freezing most vector entries and only integrating a fraction by carefully designed linear transformations. This offers:

• 70–99.5× acceleration in secure aggregation with compression factor $\lambda=100$.
• Retention of semi-honest and active adversary security of underlying SAPs.
• Consistency and verification extensions using Pedersen commitments and validity checks.

PVF is protocol-agnostic and integrates with all major secure aggregation protocols (Zhang et al., 2023).

6. Experimental Insights and Benchmarks

Task vector aggregation frameworks consistently outperform FedAvg and other baseline aggregation strategies across diverse benchmarks and heterogeneity scenarios. Notable findings include:

• FedAPTA outperforms competing multi-task FL algorithms by up to 4.23%, particularly in heterogeneous deployment settings (Yuan et al., 4 Aug 2025).
• TurboSVM-FL accelerates convergence rates by 40–60% and yields substantial improvements in F1 and accuracy in user-level non-IID settings (Wang et al., 2024).
• MaTU achieves state-of-the-art accuracy in multi-task settings with 60–70% communication savings relative to grouped MaT-FL baselines (Tsouvalas et al., 10 Feb 2025).
• FedDEA delivers large gains in task mIoU and stability (e.g., +10.6% on NYUD-V2 segmentation) by suppressing interference at the aggregation stage (Wei et al., 30 May 2025).
• Bi-level personalization with task-vector similarity yields 1–3% accuracy improvements (e.g., 93.4% vs ~90.0%) and enhanced convergence in vision and NLP federated fine-tuning (Yang et al., 16 Sep 2025).
• Decentralized approaches such as ColNet reduce gradient conflict and outperform centralized, naïve, or intra-task only aggregation (Feng et al., 17 Jan 2025).

Server or leader overhead remains tractable, with most methods designed for zero extra client compute and minimal communication increases.

7. Practical Considerations and Limitations

Current federated task vector aggregation frameworks assume common parameter spaces or partial architectural alignment for vector operations. Extending adaptive aggregation protocols to fully heterogeneous client architectures—beyond parameter masking or recovery—remains an open research challenge (Shi et al., 20 Mar 2025, Yuan et al., 4 Aug 2025). Theoretical convergence guarantees are established in the convex or smooth case but remain unproven for many complex, weighted, or masked aggregation schemes (Yang et al., 16 Sep 2025, Tsouvalas et al., 10 Feb 2025). Choice of masking ratio, cluster algorithm, and similarity thresholds significantly impact stability and efficiency. Deployment in privacy-critical settings can leverage PVF for fast, secure summation without substantial computation or communication inflation (Zhang et al., 2023).

8. Conclusion

Federated task vector aggregation provides the technical underpinnings for robust, adaptive federated learning in the presence of heterogeneous, multi-task, or personalized client workloads. By constructing, comparing, and aggregating task vectors—via alignment-aware, similarity-weighted, or sparsity-enhanced mechanisms—modern protocols outperform traditional averaging in both accuracy and efficiency, while supporting decentralization, structural heterogeneity, and scalable privacy guarantees. Foundational works include FedAPTA (Yuan et al., 4 Aug 2025), TurboSVM-FL (Wang et al., 2024), FedAWA (Shi et al., 20 Mar 2025), MaTU (Tsouvalas et al., 10 Feb 2025), Bi-level personalization (Yang et al., 16 Sep 2025), ColNet (Feng et al., 17 Jan 2025), FedDEA (Wei et al., 30 May 2025), and practical secure aggregation modules such as PVF (Zhang et al., 2023). The evolution of federated task vector aggregation marks a decisive step toward general, communication- and privacy-efficient federated multi-task optimization.