Loss-Based Aggregation (FedLol)

• Loss-Based Aggregation (FedLol) is a federated learning strategy that weights client updates using loss metrics rather than static sample proportions.
• It employs methods like hard/soft loss selection, inverse-risk weighting, and greedy inclusion to adapt aggregation based on performance indicators.
• Empirical results demonstrate improved convergence and enhanced robustness against data heterogeneity and adversarial attacks in various evaluation settings.

Searching arXiv for the cited papers and topic-specific context. I’ll look up the relevant arXiv records for loss-based aggregation, FedLol, and closely related aggregation schemes. Searching arXiv for "loss-based aggregation federated learning FedLol". Loss-based aggregation is a class of federated and distributed learning strategies that replaces static coordinate-wise averaging with aggregation rules driven by loss information. Rather than assigning client influence solely by sample proportion, these methods use local loss, validation loss, empirical risk on trusted or private data, or related reliability surrogates to determine how updates enter the next model. In the federated learning literature, this design space includes hard and soft selection of high-loss or low-loss clients, inverse-risk weighting, and greedy acceptance rules based on server-side evaluation. The label “FedLol” appears in related discussions for federated loss-based aggregation, although adjacent work also uses the same abbreviation for federated logarithmic pooling in probabilistic aggregation (Mansour et al., 2022, Bhowmick et al., 8 Jan 2025, Neyman et al., 2022).

1. Definition, scope, and naming

In its most direct form, loss-based aggregation differentiates client contributions according to the value of their losses, rather than treating model updates as interchangeable objects whose weights are fixed by data size alone. This shift is motivated by the observation that sample-proportion weighting can be suboptimal under statistical heterogeneity, noisy data, or adversarial behavior, because recent client performance may contain information about which updates should be emphasized or suppressed (Mansour et al., 2022).

The term is not associated with a single weighting direction. One line of work explicitly gives more influence to clients with larger recent loss gaps, as in FedWorse and FedSoftWorse, on the premise that underperforming clients can accelerate correction of poorly fitted regions of the objective (Mansour et al., 2022). Another line assigns larger weight to models with lower empirical risk or lower validation loss, interpreting low loss as a trust signal; this includes inverse-risk peer-to-peer aggregation and server-side greedy selection based on a trusted dataset (Bhowmick et al., 8 Jan 2025, Kritharakis et al., 25 Aug 2025). A common misconception is therefore that loss-based aggregation necessarily means downweighting high-loss clients. The literature instead contains both correction-oriented and trust-oriented schemes.

A second source of ambiguity is nomenclature. In the federated learning setting, “FedLol” is described as federated loss-based aggregation, where server aggregation weights are tied to inverse validation loss or, more broadly, to loss-derived trust measures (Bhowmick et al., 8 Jan 2025). In a distinct expert-aggregation setting, “FedLol” is also used in discussion of federated logarithmic pooling, where the object being aggregated is a probability forecast rather than a model parameter vector (Neyman et al., 2022). The shared abbreviation does not imply identical algorithms.

2. Canonical formulations

A generic federated update with adaptive aggregation weights can be written as

$$w_{t+1} = \sum_{i \in I} \alpha_t^i \left(w_t^i - \eta_t g_i(w_t^i)\right),$$

where $\alpha_t^i$ are aggregation weights satisfying $\sum_i \alpha_t^i = 1$ (Mansour et al., 2022). Loss-based methods differ mainly in how $\alpha_t^i$ is computed.

Method	Signal used	Weight construction
FedWorse / FedWorse($k$)	$F_i(w_t) - F_i^\star$	Highest-loss client(s) receive nonzero weight
FedSoftWorse	$F_i(w_t) - F_i^\star$	Softmax weighting by loss gap
P2P adaptive aggregation	$r_k(\hat{w}_l^t)$ on private data	Inverse-risk weighting over neighbors
FedGreed	$f_{\mathrm S}(\widehat{x}_i^{t+1})$ on trusted server data	Greedy inclusion if aggregate loss decreases

For FedWorse, only the worst client set $J_t = \arg\max_i (F_i(w_t) - F_i^\star)$ receives nonzero weight, with uniform mass over that set. FedWorse($\alpha_t^i$0) extends this to the top $\alpha_t^i$1 worst clients. FedSoftWorse smooths the selection by assigning

$\alpha_t^i$2

which preserves loss sensitivity while reducing the volatility of hard-max selection (Mansour et al., 2022).

In decentralized peer-to-peer learning, adaptive aggregation is performed locally at each worker. After SGD,

$\alpha_t^i$3

worker $\alpha_t^i$4 aggregates neighbor models as

$\alpha_t^i$5

The weights are computed from empirical risks on worker $\alpha_t^i$6’s private data,

$\alpha_t^i$7

with an optional selective variant that restricts aggregation to

$\alpha_t^i$8

This makes the loss signal personalized: each node evaluates neighbors using its own private dataset rather than a shared validation set (Bhowmick et al., 8 Jan 2025).

FedGreed uses a trusted server dataset rather than local client losses. Each returned client model $\alpha_t^i$9 is scored by

$\sum_i \alpha_t^i = 1$0

the empirical loss on the server’s trusted set. Updates are sorted by increasing $\sum_i \alpha_t^i = 1$1, and the server greedily enlarges the averaged subset only when the candidate aggregate further decreases trusted loss. If $\sum_i \alpha_t^i = 1$2, FedGreed selects

$\sum_i \alpha_t^i = 1$3

and sets $\sum_i \alpha_t^i = 1$4 (Kritharakis et al., 25 Aug 2025).

3. Theoretical properties and trade-offs

The convergence analysis of loss-based aggregation makes explicit that reweighting interacts with both optimization speed and heterogeneity. For federated averaging with differentiated loss-based weights, the principal bound is

$\sum_i \alpha_t^i = 1$5

where $\sum_i \alpha_t^i = 1$6 is a lower bound on the weighting skew and $\sum_i \alpha_t^i = 1$7 captures data heterogeneity, with $\sum_i \alpha_t^i = 1$8 in the IID case (Mansour et al., 2022). In the IID regime, maximizing $\sum_i \alpha_t^i = 1$9 speeds convergence without an additional heterogeneity penalty. In the non-IID regime, the same skew can worsen the residual error term, which explains the empirical advantage of softened or hybrid rules over uncompromising hard selection.

The peer-to-peer adaptive scheme provides a distinct guarantee under non-convex loss functions and non-iid data distributions. Under local strong convexity around stationary points, Lipschitz gradients, unbiased SGD, and bounded second moments,

$\alpha_t^i$0

for any normal worker $\alpha_t^i$1 (Bhowmick et al., 8 Jan 2025). The paper further states that the method is independent of specific attacks and does not assume a bound on the number of adversarial neighbors.

FedGreed establishes a different form of control: monotonic reduction, or non-increase, in the trusted loss $\alpha_t^i$2 per round, along with bounded optimality gaps under strong adversarial behavior (Kritharakis et al., 25 Aug 2025). Its robustness statement is notable because it does not require any assumption about the fraction of adversarial participants. This guarantee is tied to the availability of a trusted server dataset; the loss signal is centralized but externally validated.

These analyses illustrate that “loss-based” does not denote a single theorem template. Some guarantees concern Euclidean contraction with heterogeneity terms, some concern asymptotic risk on each peer, and some concern monotonic decrease of a trusted validation objective.

4. Empirical behavior under IID, non-IID, and adversarial settings

The empirical literature reports that loss-based aggregation can improve convergence behavior, but the benefit is strongly design-dependent. In IID experiments on MNIST, FedSoftWorse slightly outperforms FedAvg after 100 rounds, with approximately $\alpha_t^i$3 versus $\alpha_t^i$4 accuracy, consistent with the theory that emphasizing high-loss clients can accelerate convergence when heterogeneity is negligible (Mansour et al., 2022). In non-IID FMNIST, hard loss-based rules are unstable, whereas FedSoftBetter reaches $\alpha_t^i$5 accuracy faster than FedAvg and FedSoftWorse, with $\alpha_t^i$6 compared with $\alpha_t^i$7 and $\alpha_t^i$8, respectively; a hybrid method further improves this to $\alpha_t^i$9 (Mansour et al., 2022).

In peer-to-peer learning, empirical evaluation covers UCI HAR, MNIST, and Spambase under Sign-flipping, Arbitrary Byzantine, Fall-of-Empire, and A-little-is-enough attacks. The reported metric is worst-case accuracy of normal workers on the held-out test set, and the adaptive inverse-risk method outperforms BRIDGE variants, Average, and Medoid across all tasks and all attack types. Under strong attacks such as ALIE, the paper states that all baselines fail except the proposed adaptive aggregation (Bhowmick et al., 8 Jan 2025).

FedGreed is evaluated on MNIST, FMNIST, and CIFAR-10 with 10 clients, up to $k$0 adversarial clients, and attacks beginning after the 10th round. Against Mean, Trimmed Mean, Median, Krum, and Multi-Krum, it outperforms the baselines in the majority of adversarial scenarios considered, including label flipping and Gaussian noise injection attacks (Kritharakis et al., 25 Aug 2025). The paper attributes this behavior to the trusted-dataset loss criterion, which is less sensitive to the geometric distortions that undermine update-space robust estimators under strong non-IID skew.

Taken together, these studies suggest that the main empirical determinants are the source of the loss signal, the hardness or smoothness of the weighting rule, and whether the setting is homogeneous, heterogeneous, or Byzantine.

5. Robust, personalized, and trust-based variants

One important axis of variation is where the loss is measured. In the personalized peer-to-peer setting, a node evaluates neighbor models on its own private data. This avoids raw data exchange and yields a personalized notion of trust: lower risk on my data implies stronger alignment with my objective (Bhowmick et al., 8 Jan 2025). In server-centric Byzantine-robust FL, a trusted central dataset plays the same role, but now the trust criterion is common across all clients (Kritharakis et al., 25 Aug 2025).

A second axis is whether the mechanism is purely loss-based or merely adjacent to that family. FedHAW, for example, dynamically updates aggregation weights online using hypergradient descent, but it does not require access to explicit client loss values on the server. The method uses model parameter differences rather than direct client losses, and is explicitly contrasted with FedLol on that basis (Nakai-Kasai et al., 1 May 2026). This distinction matters because not every adaptive weighting rule belongs to loss-based aggregation in the narrow sense.

A third axis concerns whether aggregation is the only place where loss enters the method. FedLD decomposes the global loss into local loss, distribution shift loss, and aggregation loss, then combines margin control regularization with principal gradient-based server aggregation to jointly reduce those three terms (Zeng et al., 2024). FedDUAL likewise combines an adaptive client-side loss with dynamic server aggregation via a Wasserstein barycenter, focusing especially on final layers affected by label skew (Sahoo et al., 2024). These methods are not presented as FedLol, but they show that the surrounding FL literature increasingly treats loss structure and aggregation structure as coupled design variables rather than separate modules.

6. Broader aggregation theory and cross-domain analogues

Loss-based aggregation in FL is part of a broader theory of how losses should be combined. In online learning under expert advice, an axiomatic treatment shows that reasonable aggregation functions satisfying continuity, strict monotonicity, associativity, and loss compatibility must be quasi-sums,

$k$1

for a continuous, strictly increasing $k$2 with $k$3 (Pacheco et al., 2024). This framework interprets the aggregation generator as encoding the learner’s attitude toward losses. A plausible implication is that federated loss-based weighting schemes can be viewed as operational instantiations of a broader choice over how errors, risks, or disagreements should be combined.

The neighboring literature on logarithmic pooling provides a second theoretical perspective. There, experts supply probability distributions, the aggregate is

$k$4

the loss is $k$5, and Online Mirror Descent with a Tsallis entropy regularizer achieves $k$6 expected regret against the best fixed weights in hindsight under calibration assumptions (Neyman et al., 2022). This is not parameter averaging, but it is another formally grounded instance of learning aggregation weights from loss.

The contrast with bound-based weighting is also instructive. One federated alternative replaces raw loss with the disagreement between upper and lower generalization bounds on the second-order moment under distribution shift, then weights clients by

$k$7

The paper explicitly contrasts this with “loss-based schemes (like FedLol)” that weight clients inversely to empirical loss (Xu et al., 2023). This suggests that FedLol occupies one point in a larger design space whose weighting signals range from empirical loss, to validation loss, to risk on private data, to bound slackness.

Finally, the rationale extends beyond federated learning. In RLVR, $k$8 Normalization reformulates loss aggregation as a minimum-variance unbiased estimator under dynamic generation lengths, with inverse-length weighting

$k$9

The paper explicitly notes that FedLol and similar approaches also combine updates based on local loss information and frames $F_i(w_t) - F_i^\star$0 as a variance-optimal weighting scheme tailored to heteroskedastic trajectory lengths (He et al., 9 Sep 2025). The shared principle is not the specific formula but the statistical idea: aggregate updates in proportion to expected information reliability.