---
title: 'TFLlib: Trustworthy Federated Learning Library'
url: https://www.emergentmind.com/topics/tfllib
type: topic
---

# TFLlib: Trustworthy Federated Learning Library

TFLlib, short for **Trustworthy Federated Learning Library**, is a federated-learning security evaluation framework introduced to study poisoning risks under conditions intended to resemble deployed cross-device FL rather than idealized academic simulation. It is the central system in “Unveiling the Security Risks of Federated Learning in the Wild: From Research to Practice” [2603.20615]. The framework standardizes attack re-implementation, workload coverage across image, text, and tabular tasks, modeling of hybrid heterogeneity, and a utility-aware metric suite. Its stated purpose is not to propose another attack or another defense, but to measure how unrealistic poisoning threat models, the omission of hybrid heterogeneity, and incomplete metrics distort the apparent severity of FL poisoning risks in practice [2603.20615].

## 1. Research context and motivating thesis

TFLlib was created in response to what the underlying paper characterizes as a **research-to-practice gap** in federated-learning security evaluation [2603.20615]. In that account, much of the poisoning literature evaluates attacks under assumptions that are stronger for the attacker and cleaner for the system than those found in commercial or production-like FL deployments. The paper singles out three sources of mismatch.

First, it argues that **unrealistic poisoning threat models** inflate reported attack strength. Examples named in the source include fixed attacker participation across rounds, very large compromised-client fractions, advance knowledge of attack schedules, access to benign clients’ updates, and aggressive weight scaling in settings where such behavior would be conspicuous or operationally damaging [2603.20615]. Second, it argues that many benchmarks omit **hybrid heterogeneity**, even though real FL combines data heterogeneity, device heterogeneity, and communication heterogeneity rather than varying them in isolation. Third, it argues that the prevailing metric practice is incomplete because it emphasizes peak or final-round attack success while ignoring temporal stability and the damage inflicted on the benign task.

Within that framing, TFLlib serves as a **measurement system**. The framework is presented as a uniform practical evaluation environment in which representative poisoning attacks can be compared under common system assumptions and with a common metric suite. The paper explicitly states that it is the first implemented system in this line of work that integrates representative poisoning attacks together with utility-aware security metrics under one practical FL framework [2603.20615].

## 2. Architecture, workload coverage, and attack support

The framework is organized into three main parts: **FL Base**, **FL Core**, and **FL Attacker** [2603.20615]. The published description makes FL Base and FL Attacker explicit. FL Base supports minimal FL simulation for three modalities—**image**, **text**, and **tabular**—and three model families—**CNNs**, **Transformers**, and **MLPs**. FL Attacker inherits from the client class and is used to launch poisoning attacks in a unified way. The framework is built in **PyTorch** and includes parallel modules for **GPU acceleration** [2603.20615].

TFLlib is modality-aware rather than image-only. The benchmark tasks named in the source are **CIFAR10** and **FEMNIST** for image, **AGNews** and **IMDB** for text, and **Purchase100** and **Texas100** for tabular evaluation. The corresponding backbones are **ResNet18** for image tasks, **TinyBert** for text tasks, and **MLP** for tabular tasks [2603.20615].

| Dataset | Backbone | Training configuration |
|---|---|---|
| CIFAR10 | ResNet18 | $T=2000$, $\eta=0.01$ |
| FEMNIST | ResNet18 | $T=2000$, $\eta=0.001$ |
| AGNews | TinyBert | $T=500$, $\eta=0.0001$ |
| IMDB | TinyBert | $T=500$, $\eta=0.0001$ |
| Purchase100 | MLP | $T=500$, $\eta=0.01$ |
| Texas100 | MLP | $T=500$, $\eta=0.01$ |

The attack coverage is deliberately broad. For backdoor settings, TFLlib re-implements **DBA**, **CerP**, **EdgeCase**, **A3FL**, **Model Replacement / Replace**, and **Neurotoxin**, with the poison ratio swept over $\{1,3,5,7,10\}\%$ in the main backdoor experiments [2603.20615]. For byzantine or availability-oriented settings, the described implementation includes **IPM**, **Noise**, **Fang**, **LabelFlip**, **SignFlip**, **UpdateFlip**, **MinMax**, **MedianTailored**, **LIE**, and **SignGuard**. The source further notes that attack generation is aligned with the practical threat model: if an attack requires auxiliary information, malicious clients are restricted to what is plausible in deployment, such as their own state and historical information from other compromised clients, rather than direct access to benign clients’ gradients when such access would be unrealistic [2603.20615].

## 3. Practical threat model and hybrid heterogeneity

A defining feature of TFLlib is that it operationalizes a **practical** FL setting rather than an idealized one. The framework evaluates poisoning under **random client sampling**, **modest malicious-client fractions**, **no special access to benign updates**, **FedAvg-centric evaluation**, and heterogeneous client and data conditions [2603.20615]. This setup is intended to model cross-device FL systems in which attackers cannot reliably force multiple compromised clients to be selected in every round.

The paper illustrates the participation issue with a simple example: if $N=100$ clients exist and $K=10$ are selected per round, then attacks that require several malicious clients in every round imply a very large compromised fraction of the overall client population, which the paper treats as implausible in many deployments [2603.20615]. This is central to the framework’s measurement philosophy: the question is not merely whether an attack can succeed under some configuration, but whether it can do so under assumptions consistent with contemporary FL systems.

TFLlib also formalizes **hybrid heterogeneity** rather than limiting evaluation to IID or homogeneous clients. The framework models:

- **Statistical heterogeneity** with a Dirichlet distribution $Dir(\tau^s)$ for client data splits.
- **Device heterogeneity** with
  $$
  \tau_i^d \sim \mathcal{N}_{[0,1]}(0,1-\tau^d).
  $$
- **Communication heterogeneity** with
  $$
  \tau_i^c \sim \mathcal{N}_{[0,1]}(0,1-\tau^c).
  $$

These parameters are used to assign lower local training epochs or higher rejection likelihood to weaker clients [2603.20615]. In the framework’s **practical** setting, the parameters are $\tau^s = 0.9$, $\tau^c = 0.9$, and $\tau^d = 0.9$. In the **idealized** setting, they are $\tau^s = \infty$, $\tau^c = 1$, and $\tau^d = 1$, with FEMNIST treated as a special case [2603.20615]. The substantive implication is that malicious updates are evaluated in an environment where they may fail to appear, may be weakened by heterogeneous participation, and may interact with nonuniform data and system conditions.

## 4. Formalization of training, poisoning, and evaluation metrics

The framework adopts the standard local-training and FedAvg aggregation formalism. For client $i$ at round $t$, local updates are written as
$$
W_{i, t}^{j+1}=W_{i, t}^{j}-\eta\cdot\nabla\mathcal{L}(W_{i, t}^{j},\mathcal{D}_i),
$$
with initialization
$$
W_{i,t}^{0} = G_{t-1},
$$
and submitted update
$$
\nabla_{i,t} = W_{i,t}^{S_i} - G_{t-1}.
$$
The server then aggregates with
$$
G_{t}=G_{t-1}+\sum_{i=1}^{K}w_{i}\nabla_{i,t},
$$
where
$$
w_i = \frac{\|\mathcal{D}_i\|}{\sum_{i=1}^{K}\|\mathcal{D}_i\|}.
$$
This choice is significant because the paper explicitly centers TFLlib on **FedAvg**, arguing that many industry FL platforms still rely on simple efficiency-oriented configurations rather than the more elaborate robust defenses often emphasized in academic work [2603.20615].

To unify attack implementations, the framework uses the generic poisoning model
$$
\nabla_{i,t}^{adv} = \mathcal{F}_{\epsilon}(G_{t},\mathcal{L}_{adv}(\mathcal{T},\mathcal{D}_{i}),\mathcal{I}_{aux}),
$$
where $G_t$ is the current global model, $\mathcal{T}$ is the poisoning or backdoor transformation, $\mathcal{D}_i$ is the attacker’s local data, $\mathcal{L}_{adv}$ is the attacker’s objective, $\mathcal{I}_{aux}$ is auxiliary information available to the attacker, and $\mathcal{F}_{\epsilon}$ denotes a constraint or transformation that can be used to scale or bound the update [2603.20615]. This abstraction is used to place backdoor and byzantine attacks in one evaluative frame.

The metric design is the other major formal contribution. For byzantine settings, the paper defines per-round utility degradation as
$$
ud_t = acc_t - \hat{acc}_t,
$$
where $acc_t$ is the normally trained model’s accuracy and $\hat{acc}_t$ is the accuracy under poisoning [2603.20615]. More broadly, TFLlib recommends measuring the **converged tail** of training rather than a single final point. With $\alpha = 0.9$, the framework averages and computes variances over the final $10\%$ of rounds. The recommended metrics are **BSA** and **BSV** for backdoor effectiveness and temporal stability, **BDA** and **BDV** for byzantine degradation and its variance, and **ACC** and **ACCV** for benign-task utility and its variance [2603.20615]. The paper’s central evaluative claim is that practical security assessment must jointly consider **effectiveness**, **temporal stability**, and **collateral utility loss**.

## 5. Experimental protocol and empirical findings

The main benchmark configuration uses **$N = 100$ clients**, **join ratio $\alpha = 0.1$**, **local batch size $B = 64$**, **local epochs $E = 5$**, **10\% of each dataset held out for evaluation**, and **FedAvg** aggregation [2603.20615]. Within this protocol, the paper reports a consistent pattern: **idealized evaluation often overstates poisoning risk**.

One of the most prominent findings is **dataset dependence** under practical settings. On **Texas100**, **EdgeCase** remains effective, whereas **DBA**, **Replace**, and **Neurotoxin** are much weaker. On **Purchase100**, **DBA** can briefly achieve high success, but the trend is reported as non-monotonic and unstable. On **CIFAR10**, **IMDB**, and **AGNews**, several attacks still work well, which the paper uses to argue that the risk is real, but not uniformly strong across tasks or modalities [2603.20615]. This undermines any interpretation of poisoning severity as a modality-independent or architecture-independent constant.

A second finding is **temporal instability**. Even when backdoor success is high, its variance over the converged tail may also be high, so a strong final-round result may reflect a narrow or transient regime rather than a stable compromise [2603.20615]. This is precisely the failure mode that TFLlib’s BSV and BDV metrics are designed to reveal.

A third finding is **collateral benign-task degradation**. The paper identifies **DBA on Purchase100** as a representative case in which high backdoor success at one practical poison ratio coincides with a sharp collapse of the main task, making the attack conspicuous and less stealthy [2603.20615]. The framework therefore treats “attack works” and “attack works without breaking the base task” as different measurements.

The experiments also report **non-monotonic behavior** with respect to poison ratio. Increasing attacker budget does not necessarily yield stronger or more reliable attacks under realistic sampling and heterogeneity [2603.20615]. This directly challenges the simplified intuition that more malicious participation must monotonically improve attack efficacy.

## 6. Position within the FL tooling landscape

TFLlib occupies a specific layer in the federated-learning software ecosystem: **practical security measurement**. It is therefore distinct from libraries whose primary goal is experiment bootstrapping or model orchestration. For example, **TorchFL** is presented as a performant, plug-and-play library for bootstrapping FL experiments with ready-to-use abstractions for datasets, models, samplers, aggregators, logging, and profiling [2211.00735]. TFLlib, by contrast, is centered on poisoning evaluation under realistic assumptions and on utility-aware metric reporting [2603.20615].

It is also distinct from system-control proposals such as **TiFL**, which addresses straggler mitigation through tier-based client scheduling under heterogeneity [2001.09249]. TFLlib does model device and communication heterogeneity, but its goal is not to optimize training throughput; it is to determine how those heterogeneities alter the measured severity of poisoning attacks [2603.20615]. A plausible implication is that TFLlib and TiFL-style scheduling address different layers of the same FL stack: the former measures security posture, while the latter manages runtime behavior.

A common terminological ambiguity is the similarity between **TFLlib** and **TLib**, a flexible C++ tensor framework for numerical tensor calculus with runtime-variable order, extents, layouts, and contraction modes [1711.10912]. The two systems are unrelated in domain and purpose. TFLlib belongs to federated-learning security evaluation; TLib belongs to numerical multilinear computation.

The broader significance assigned to TFLlib by its authors is methodological. The paper concludes that practical FL security is a **measurement problem before it is an optimization problem**, and it offers concrete guidance accordingly: future work should use realistic attacker participation, avoid implausible access to benign updates, evaluate across image, text, and tabular tasks, include hybrid heterogeneity, and report effectiveness, stability, and utility cost together rather than relying on a single peak or final attack score [2603.20615]. In that sense, TFLlib is best understood not as a new poisoning method, nor as a defense library, but as a standardized framework for making poisoning claims more transferable to real deployments.

Source: https://www.emergentmind.com/topics/tfllib