SpectralKrum: Byzantine-Robust Federated Learning

• SpectralKrum is a federated learning aggregation rule that uses spectral subspace estimation to identify benign model updates amidst Byzantine attacks.
• It estimates a low-dimensional principal subspace from a rolling buffer via PCA and projects incoming updates, filtering those with high orthogonal residual energy.
• The method shows robust performance against directional and subspace-aware attacks while facing challenges with label-flip, min-max, and backdoor scenarios.

SpectralKrum is a Byzantine‐robust aggregation rule for Federated Learning (FL) that integrates spectral subspace estimation with geometric neighbor‐based selection. The design exploits the observation that, even under highly heterogeneous (non‐IID) client data, optimization trajectories of honest aggregates tend to concentrate in a low‐dimensional manifold. SpectralKrum operates by estimating this historical subspace via PCA, projecting new client model updates into this subspace, applying Krum selection in compressed coordinates, and filtering candidates whose orthogonal residual energy is abnormally large. This process leverages only model updates, preserves FL privacy properties, and targets attacks that inject deviations orthogonal to benign optimization dynamics (Tripathi et al., 12 Dec 2025).

1. Algorithmic Procedure

SpectralKrum maintains a rolling buffer of the last $B$ robustly aggregated model updates $X \in \mathbb{R}^{B \times d}$. The buffer is used to estimate a rank‐$r$ principal component subspace. Each incoming client update $d_i \in \mathbb{R}^d$ is projected into this subspace ($z_i = U^\top d_i$), yielding compressed coordinates. The orthogonal residual energy $\rho_i = \|d_i - U z_i\|_2$ quantifies deviation from the benign subspace. Krum selection is applied to the compressed set $\{z_i\}$ to identify a subset $S$ most tightly clustered in the subspace. From $S$, only candidates with residual energy below a quantile‐derived threshold $\tau$ are retained. If no candidates survive, the one with minimal residual is selected. The output is the mean of these filtered updates. The PCA basis and threshold are periodically recomputed via robust trimming of extremes in the buffer.

2. Spectral Subspace Estimation

The estimation uses the rolling buffer $X = [g^{(t-B+1)}, \dots, g^{(t)}]$ of previous aggregates. Centering is achieved by subtracting the mean or median from each row. To attenuate historical Byzantine effects, the top $\alpha$‐fraction of rows with largest/smallest norms are trimmed, resulting in $X_k$. The covariance $C = \frac{1}{|X_k|} X_k^\top X_k$ is computed, and the dominant $r$ PCA directions $U \in \mathbb{R}^{d \times r}$ are extracted as the top $r$ eigenvectors ($$C u_j = \lambda_j u_j, \, j=1 \dots r, \lambda_1 \geq \dots \geq \lambda_r$$). The quantile threshold $\tau$ is constructed by measuring the orthogonal residuals of buffer aggregates and selecting the $q$‐th quantile.

3. Projection, Filtering, and Geometric Selection

Each new update $\Delta_i$ is decomposed:

• Projection: $P_i = U U^\top \Delta_i$
• Residual: $r_i = \Delta_i - P_i$, with energy $$\|r_i\|_2^2 = \|\Delta_i\|_2^2 - \|U^\top \Delta_i\|_2^2$$

In the compressed space, Krum selection is performed:

• For each $z_i = U^\top \Delta_i \in \mathbb{R}^r$, calculate pairwise distances to other projected updates.
• For each $i$, sum the smallest $(n - f - 2)$ distances to identify geometric clustering.
• The index minimizing this sum is retained if its residual energy is within $\tau$; otherwise, the lowest residual in $S$ is selected.
• The aggregate $a$ is the average of the filtered candidates.

This hybrid combines geometric proximity under manifold compression with explicit filtering against spectral anomalies.

4. Privacy and Data Constraints

SpectralKrum requires only client‐submitted model updates (gradients/parameter deltas), never accessing raw data, labels, or any trusted external hold‐out. Subspace estimation is fully based on past robust aggregates rather than reference data. No intermediate projections or internal statistics are disclosed to clients, enabling standard differential‐privacy or secure aggregation frameworks to remain operational.

5. Theoretical Guarantees

SpectralKrum’s robustness is conditional, not universal. Its correctness is established under the following assumptions:

• At most $f$ out of $n$ client updates are Byzantine.
• Honest updates satisfy $$\Delta_h = U U^\top \Delta_h + \epsilon_h,\, \|\epsilon_h\|_2 \leq \sigma$$.

Key properties:

• Robust Subspace Lemma: Given buffer size $B$ and mild trimming, $U$ approximates the true benign subspace to error $O(\sigma)$.
• Krum Selection Guarantee: If honest projected coordinates ${z_h}$ are tightly clustered and Byzantines are distant, Krum reliably selects an honest update ($\|z_s - \text{mean}(z_h)\|_2 \leq O(\rho)$).
• Aggregate Error Bound: Aggregate $a$ is an average over $G \subseteq S$, for honest $\Delta_h$ with $\|\Delta_h - \mu\|_2 \leq \delta$ leading to $\|a - \mu\|_2 \leq \delta$.
• Convergence: Under $L$‐smoothness and bounded honest variance, coupling SpectralKrum with FedAvg yields $O(1/\sqrt{T})$ convergence to a stationary point.

Byzantine guarantees degrade when malicious updates mimic the benign subspace.

6. Empirical Evaluation and Performance

Experiments are performed on CIFAR‐10, with data partitioned Dirichlet($\alpha=0.1$) across 100 clients, creating significant non‐IID skew. Each round, $n=10$ clients, with up to $f=2$ Byzantines, contribute TinyCNN updates. Seven attack families (Sign‐Flip, Label‐Flip, Min‐Max, Buffer‐Drift, Adaptive‐Steer, Semantic Backdoor, None) and eight baseline defenses are benchmarked. Metrics include per‐round accuracy, mean AUC, attack success rate (ASR), and computation overhead.

Key outcomes:

Attack Type	SpectralKrum Performance	Leading Baseline(s)
Directional/Subspace-aware	≈50% accuracy	Matches DnC-PMF, MultiKrum
Label-Flip/Min-Max	39–49% accuracy	TrimmedMean 55–56%
Semantic Backdoor	≈47% accuracy	All aggregation defenses limited

SpectralKrum achieves rapid convergence and stable accuracy in the presence of directional and subspace-aware attacks, outperforming FullKrum and matching cluster-based baselines. In label-flip and min-max scenarios, coordinate-wise methods demonstrate superior robustness. Backdoor attacks remain unsuppressed by aggregation-only defenses.

Computationally, SpectralKrum’s overhead ($\approx$1.3 s/round) exceeds that of FullKrum (7 ms) but remains practical relative to Bulyan (400 ms).

7. Strengths and Limitations

SpectralKrum’s principal strength is its capacity to detect updates with large orthogonal innovation, effectively mitigating sign-flip and subspace-aware attacks and yielding tight clustering for geometric selection. However, when adversarial perturbations reside entirely within the benign subspace (as in label-flip and min-max attacks), residual filtering is ineffective. Backdoor attacks structured to mimic benign local training evade detection. The PCA estimation introduces latency and computational cost due to buffer accumulation and subspace extraction.

A plausible implication is that, while SpectralKrum demonstrates substantial gains for certain attack families under federated heterogeneity, comprehensive defense necessitates hybrid strategies combining spectral, coordinate-wise, and activation-level analyses (Tripathi et al., 12 Dec 2025).