TFLlib: Trustworthy Federated Learning Library

• TFLlib is a practical federated learning security evaluation framework that measures poisoning risks under realistic conditions using hybrid heterogeneity and utility-aware metrics.
• It standardizes a range of poisoning attack implementations and supports diverse tasks across image, text, and tabular data using varied model architectures.
• Its evaluations reveal that poisoning risk severity varies with dataset dependencies, temporal instability, and collateral benign-task degradation, challenging simplified threat models.

TFLlib, short for Trustworthy Federated Learning Library, is a federated-learning security evaluation framework introduced to study poisoning risks under conditions intended to resemble deployed cross-device FL rather than idealized academic simulation. It is the central system in “Unveiling the Security Risks of Federated Learning in the Wild: From Research to Practice” (Chen et al., 21 Mar 2026). The framework standardizes attack re-implementation, workload coverage across image, text, and tabular tasks, modeling of hybrid heterogeneity, and a utility-aware metric suite. Its stated purpose is not to propose another attack or another defense, but to measure how unrealistic poisoning threat models, the omission of hybrid heterogeneity, and incomplete metrics distort the apparent severity of FL poisoning risks in practice (Chen et al., 21 Mar 2026).

1. Research context and motivating thesis

TFLlib was created in response to what the underlying paper characterizes as a research-to-practice gap in federated-learning security evaluation (Chen et al., 21 Mar 2026). In that account, much of the poisoning literature evaluates attacks under assumptions that are stronger for the attacker and cleaner for the system than those found in commercial or production-like FL deployments. The paper singles out three sources of mismatch.

First, it argues that unrealistic poisoning threat models inflate reported attack strength. Examples named in the source include fixed attacker participation across rounds, very large compromised-client fractions, advance knowledge of attack schedules, access to benign clients’ updates, and aggressive weight scaling in settings where such behavior would be conspicuous or operationally damaging (Chen et al., 21 Mar 2026). Second, it argues that many benchmarks omit hybrid heterogeneity, even though real FL combines data heterogeneity, device heterogeneity, and communication heterogeneity rather than varying them in isolation. Third, it argues that the prevailing metric practice is incomplete because it emphasizes peak or final-round attack success while ignoring temporal stability and the damage inflicted on the benign task.

Within that framing, TFLlib serves as a measurement system. The framework is presented as a uniform practical evaluation environment in which representative poisoning attacks can be compared under common system assumptions and with a common metric suite. The paper explicitly states that it is the first implemented system in this line of work that integrates representative poisoning attacks together with utility-aware security metrics under one practical FL framework (Chen et al., 21 Mar 2026).

2. Architecture, workload coverage, and attack support

The framework is organized into three main parts: FL Base, FL Core, and FL Attacker (Chen et al., 21 Mar 2026). The published description makes FL Base and FL Attacker explicit. FL Base supports minimal FL simulation for three modalities—image, text, and tabular—and three model families—CNNs, Transformers, and MLPs. FL Attacker inherits from the client class and is used to launch poisoning attacks in a unified way. The framework is built in PyTorch and includes parallel modules for GPU acceleration (Chen et al., 21 Mar 2026).

TFLlib is modality-aware rather than image-only. The benchmark tasks named in the source are CIFAR10 and FEMNIST for image, AGNews and IMDB for text, and Purchase100 and Texas100 for tabular evaluation. The corresponding backbones are ResNet18 for image tasks, TinyBert for text tasks, and MLP for tabular tasks (Chen et al., 21 Mar 2026).

Dataset	Backbone	Training configuration
CIFAR10	ResNet18	$T=2000$, $\eta=0.01$
FEMNIST	ResNet18	$T=2000$, $\eta=0.001$
AGNews	TinyBert	$T=500$, $\eta=0.0001$
IMDB	TinyBert	$T=500$, $\eta=0.0001$
Purchase100	MLP	$T=500$, $\eta=0.01$
Texas100	MLP	$\eta=0.01$0, $\eta=0.01$1

The attack coverage is deliberately broad. For backdoor settings, TFLlib re-implements DBA, CerP, EdgeCase, A3FL, Model Replacement / Replace, and Neurotoxin, with the poison ratio swept over $\eta=0.01$2 in the main backdoor experiments (Chen et al., 21 Mar 2026). For byzantine or availability-oriented settings, the described implementation includes IPM, Noise, Fang, LabelFlip, SignFlip, UpdateFlip, MinMax, MedianTailored, LIE, and SignGuard. The source further notes that attack generation is aligned with the practical threat model: if an attack requires auxiliary information, malicious clients are restricted to what is plausible in deployment, such as their own state and historical information from other compromised clients, rather than direct access to benign clients’ gradients when such access would be unrealistic (Chen et al., 21 Mar 2026).

3. Practical threat model and hybrid heterogeneity

A defining feature of TFLlib is that it operationalizes a practical FL setting rather than an idealized one. The framework evaluates poisoning under random client sampling, modest malicious-client fractions, no special access to benign updates, FedAvg-centric evaluation, and heterogeneous client and data conditions (Chen et al., 21 Mar 2026). This setup is intended to model cross-device FL systems in which attackers cannot reliably force multiple compromised clients to be selected in every round.

The paper illustrates the participation issue with a simple example: if $\eta=0.01$3 clients exist and $\eta=0.01$4 are selected per round, then attacks that require several malicious clients in every round imply a very large compromised fraction of the overall client population, which the paper treats as implausible in many deployments (Chen et al., 21 Mar 2026). This is central to the framework’s measurement philosophy: the question is not merely whether an attack can succeed under some configuration, but whether it can do so under assumptions consistent with contemporary FL systems.

TFLlib also formalizes hybrid heterogeneity rather than limiting evaluation to IID or homogeneous clients. The framework models:

• Statistical heterogeneity with a Dirichlet distribution $\eta=0.01$5 for client data splits.
• Device heterogeneity with

$\eta=0.01$6

• Communication heterogeneity with

$\eta=0.01$7

These parameters are used to assign lower local training epochs or higher rejection likelihood to weaker clients (Chen et al., 21 Mar 2026). In the framework’s practical setting, the parameters are $\eta=0.01$8, $\eta=0.01$9, and $T=2000$0. In the idealized setting, they are $T=2000$1, $T=2000$2, and $T=2000$3, with FEMNIST treated as a special case (Chen et al., 21 Mar 2026). The substantive implication is that malicious updates are evaluated in an environment where they may fail to appear, may be weakened by heterogeneous participation, and may interact with nonuniform data and system conditions.

4. Formalization of training, poisoning, and evaluation metrics

The framework adopts the standard local-training and FedAvg aggregation formalism. For client $T=2000$4 at round $T=2000$5, local updates are written as

$T=2000$6

with initialization

$T=2000$7

and submitted update

$T=2000$8

The server then aggregates with

$T=2000$9

where

$\eta=0.001$0

This choice is significant because the paper explicitly centers TFLlib on FedAvg, arguing that many industry FL platforms still rely on simple efficiency-oriented configurations rather than the more elaborate robust defenses often emphasized in academic work (Chen et al., 21 Mar 2026).

To unify attack implementations, the framework uses the generic poisoning model

$\eta=0.001$1

where $\eta=0.001$2 is the current global model, $\eta=0.001$3 is the poisoning or backdoor transformation, $\eta=0.001$4 is the attacker’s local data, $\eta=0.001$5 is the attacker’s objective, $\eta=0.001$6 is auxiliary information available to the attacker, and $\eta=0.001$7 denotes a constraint or transformation that can be used to scale or bound the update (Chen et al., 21 Mar 2026). This abstraction is used to place backdoor and byzantine attacks in one evaluative frame.

The metric design is the other major formal contribution. For byzantine settings, the paper defines per-round utility degradation as

$\eta=0.001$8

where $\eta=0.001$9 is the normally trained model’s accuracy and $T=500$0 is the accuracy under poisoning (Chen et al., 21 Mar 2026). More broadly, TFLlib recommends measuring the converged tail of training rather than a single final point. With $T=500$1, the framework averages and computes variances over the final $T=500$2 of rounds. The recommended metrics are BSA and BSV for backdoor effectiveness and temporal stability, BDA and BDV for byzantine degradation and its variance, and ACC and ACCV for benign-task utility and its variance (Chen et al., 21 Mar 2026). The paper’s central evaluative claim is that practical security assessment must jointly consider effectiveness, temporal stability, and collateral utility loss.

5. Experimental protocol and empirical findings

The main benchmark configuration uses $T=500$3 clients, join ratio $T=500$4, local batch size $T=500$5, local epochs $T=500$6, 10\% of each dataset held out for evaluation, and FedAvg aggregation (Chen et al., 21 Mar 2026). Within this protocol, the paper reports a consistent pattern: idealized evaluation often overstates poisoning risk.

One of the most prominent findings is dataset dependence under practical settings. On Texas100, EdgeCase remains effective, whereas DBA, Replace, and Neurotoxin are much weaker. On Purchase100, DBA can briefly achieve high success, but the trend is reported as non-monotonic and unstable. On CIFAR10, IMDB, and AGNews, several attacks still work well, which the paper uses to argue that the risk is real, but not uniformly strong across tasks or modalities (Chen et al., 21 Mar 2026). This undermines any interpretation of poisoning severity as a modality-independent or architecture-independent constant.

A second finding is temporal instability. Even when backdoor success is high, its variance over the converged tail may also be high, so a strong final-round result may reflect a narrow or transient regime rather than a stable compromise (Chen et al., 21 Mar 2026). This is precisely the failure mode that TFLlib’s BSV and BDV metrics are designed to reveal.

A third finding is collateral benign-task degradation. The paper identifies DBA on Purchase100 as a representative case in which high backdoor success at one practical poison ratio coincides with a sharp collapse of the main task, making the attack conspicuous and less stealthy (Chen et al., 21 Mar 2026). The framework therefore treats “attack works” and “attack works without breaking the base task” as different measurements.

The experiments also report non-monotonic behavior with respect to poison ratio. Increasing attacker budget does not necessarily yield stronger or more reliable attacks under realistic sampling and heterogeneity (Chen et al., 21 Mar 2026). This directly challenges the simplified intuition that more malicious participation must monotonically improve attack efficacy.

6. Position within the FL tooling landscape

TFLlib occupies a specific layer in the federated-learning software ecosystem: practical security measurement. It is therefore distinct from libraries whose primary goal is experiment bootstrapping or model orchestration. For example, TorchFL is presented as a performant, plug-and-play library for bootstrapping FL experiments with ready-to-use abstractions for datasets, models, samplers, aggregators, logging, and profiling (Khimani et al., 2022). TFLlib, by contrast, is centered on poisoning evaluation under realistic assumptions and on utility-aware metric reporting (Chen et al., 21 Mar 2026).

It is also distinct from system-control proposals such as TiFL, which addresses straggler mitigation through tier-based client scheduling under heterogeneity (Chai et al., 2020). TFLlib does model device and communication heterogeneity, but its goal is not to optimize training throughput; it is to determine how those heterogeneities alter the measured severity of poisoning attacks (Chen et al., 21 Mar 2026). A plausible implication is that TFLlib and TiFL-style scheduling address different layers of the same FL stack: the former measures security posture, while the latter manages runtime behavior.

A common terminological ambiguity is the similarity between TFLlib and TLib, a flexible C++ tensor framework for numerical tensor calculus with runtime-variable order, extents, layouts, and contraction modes (Bassoy, 2017). The two systems are unrelated in domain and purpose. TFLlib belongs to federated-learning security evaluation; TLib belongs to numerical multilinear computation.

The broader significance assigned to TFLlib by its authors is methodological. The paper concludes that practical FL security is a measurement problem before it is an optimization problem, and it offers concrete guidance accordingly: future work should use realistic attacker participation, avoid implausible access to benign updates, evaluate across image, text, and tabular tasks, include hybrid heterogeneity, and report effectiveness, stability, and utility cost together rather than relying on a single peak or final attack score (Chen et al., 21 Mar 2026). In that sense, TFLlib is best understood not as a new poisoning method, nor as a defense library, but as a standardized framework for making poisoning claims more transferable to real deployments.