FLsim: Federated Learning Simulation Framework
- FLsim is a modular and library-agnostic simulation framework for federated learning that facilitates reproducible experiments and custom configurations.
- It employs an object-oriented, layered architecture with components like the Job Orchestrator, Logic Controller, and Dataset Distributor to manage training, aggregation, and synchronization.
- FLsim supports diverse scenarios including blockchain integration, multiple network topologies, and consensus mechanisms to fairly benchmark and compare FL algorithms.
Searching arXiv for FLsim and closely related federated-learning simulation frameworks to ground the article in current papers. FLsim is a modular, library-agnostic simulation framework for federated learning designed to make it easier to prototype, benchmark, reproduce, and compare a wide range of FL algorithms and system designs. It was introduced to address fragmentation in the FL research ecosystem, where methods differ in topology, local training logic, aggregation rules, communication patterns, and trust or verification mechanisms such as blockchain. FLsim provides a Python-based, object-oriented, layered architecture in which users specify FL requirements through configuration and strategy files while the framework handles orchestration, synchronization, data distribution, aggregation, and logging (Mukherjee et al., 15 Jul 2025).
1. Definition and scope
FLsim was proposed as a research-oriented FL simulation framework that is modular, scalable, resource-efficient, reproducible, ML-library agnostic, topology-flexible, and blockchain-pluggable. The framework supports different datasets and distributions, different ML libraries, different local training algorithms, different network topologies, different aggregation rules, optional consensus protocols, and optional blockchain integration. Its intended role is to provide a single experimental scaffold in which many FL proposals can be recreated and compared consistently (Mukherjee et al., 15 Jul 2025).
The framework is explicitly positioned against the difficulty of fair benchmarking in federated learning. The paper identifies a broad methodological landscape including FedAvg, FedAvgM, SCAFFOLD, MOON, Differentially private FL, Personalized FL, Hierarchical/clustered FL, Blockchain-based FL, Communication-efficient FL, and Decentralized FL. These proposals can differ in how data is split across clients, what model or library is used, how local learning is done, how aggregation happens, what communication topology is used, whether multiple aggregators or workers exist, and whether consensus or blockchain verification is needed. A plausible implication is that FLsim’s principal contribution is not a single FL algorithm, but an experimental substrate for systematically varying these dimensions.
The framework is distinct from the earlier FLSim testbed developed for evaluating IronForge, which was described as a flexible and efficient testbed for GoogleFL, AsyncFL, BlockFL, and IronForge under heterogeneous, asynchronous, and adversarial conditions (Yu et al., 2023). That distinction matters because FLsim is presented as a general-purpose simulation framework, whereas the IronForge FLSim is the experimental backbone of a specific decentralized FL protocol family.
2. Motivation in the federated-learning tooling landscape
The paper argues that existing tools supported only subsets of the requirements that contemporary FL research imposes. TFF and PySyft are described as mainly client-server and limited in topology support, often single-machine oriented. FATE and FedBioMed are characterized as more deployment-oriented, but less flexible for algorithmic innovation. Flower, FedLab, and FedML are described as useful, but limited in controlled reproducibility, logging, or certain modular customizations. Fedstellar supports decentralized FL, but is described as narrower in flexibility for multi-worker consensus and blockchain-aided decision-making (Mukherjee et al., 15 Jul 2025).
Within that context, FLsim was proposed to streamline benchmarking and rapid experimentation. The framework’s easy-to-use interface allows users to specify customized FL requirements through job configuration, including customized data distributions ranging from non-iid to iid data, selection of local learning algorithms according to user preferences, choice of network topology illustrating communication patterns among nodes, definition of model aggregation and consensus algorithms, and pluggable blockchain support for enhanced robustness. This suggests that FLsim treats an FL experiment as a composition of separable system and algorithmic choices rather than as a monolithic training script.
The paper further emphasizes controlled reproducibility. FLsim supports deterministic execution through synchronized seeds across nodes, deterministic modes for ML libraries, and environment variables such as DETERMINISTIC=true and RANDOM_SEED=<integer>. At the same time, it acknowledges that reproducibility in distributed FL is difficult because hardware differences and floating-point behavior can still slightly perturb results. That framing is important: reproducibility is treated as a system objective, but not as an absolute guarantee under heterogeneous hardware.
3. Architecture, workflow, and configuration model
FLsim is built around an object-oriented, layered architecture. The overall workflow is described as follows: the user defines job configuration files and strategy definitions; the Job Orchestrator loads the configuration; the orchestrator forwards setup information to the Dataset Distributor and the Logic Controller; the Dataset Distributor prepares and shards the data; the Logic Controller assigns roles, synchronizes nodes, and manages phases of FL; clients or workers download data and begin training or aggregation; and performance metrics and resource usage are sent to the Performance Logger or FL-dashboard (Mukherjee et al., 15 Jul 2025).
The framework identifies six core components:
| Component | Function |
|---|---|
| Job Orchestrator | Loads job configuration and FL strategy; creates the overlay network specification; bundles required Python modules; sends setup instructions |
| Logic Controller | Coordinates and synchronizes clients and workers; manages training, aggregation, and consensus phases |
| Dataset Distributor | Archives and indexes dataset chunks; splits datasets according to the chosen distribution algorithm |
| Client / Worker Nodes | Act as clients, workers, or both depending on the experiment |
| Key-Value Store | Used as a broker for state sharing in a pub-sub style |
| Performance Logger and FL-Dashboard | Collects accuracy, loss, CPU, memory, and bandwidth metrics |
To run an experiment, users define three files: a job configuration file in .yaml, a dataset definition file, and an FL strategy file. The dataset interface requires prepare_root_dataset(), distribute_into_chunks(), and preprocess_data(). The strategy interface requires train(dataset), aggregate(client_models), and test(). The strategy class also holds the dataset state instance. Because the framework does not hardwire PyTorch, TensorFlow, or Scikit-Learn into the FL algorithmic logic, users can write the local training loop in the library of choice (Mukherjee et al., 15 Jul 2025).
The node implementation uses Python with communication handled by REST APIs using Flask. The node architecture has five layers: an infrastructure layer, a container base, a networking layer, a functional layer, and an application layer. Above the node stack sit the FLsim Orchestrator and Controller and the CLI/Web frontend. The paper also states that nodes can communicate through a key-value store or pub-sub broker rather than direct node-to-node messaging, the controller polling rate can be reduced, and the controller can be load-balanced using NGINX with multiple controller instances. This suggests that resource efficiency is handled as a systems-design problem rather than only as an optimization of model transmission.
4. Topology control, synchronization, consensus, and blockchain support
A central feature of FLsim is topology agnosticism. The job configuration includes an overlay topology or cluster section, and the framework supports client-server, hierarchical, fully-connected or decentralized, and peer-to-peer communication patterns (Mukherjee et al., 15 Jul 2025). This is presented as one of the major differences from TFF and PySyft, which are described as largely client-server oriented.
Aggregation is user-defined through the strategy’s aggregate() method. In multi-worker settings, FLsim also supports a separate consensus mechanism to choose the next global model. The consensus function accepts a collection of aggregated model parameters and a dictionary of extra hyperparameters, and returns a single parameter set to be used as the next global model. The paper describes a four-phase consensus pipeline: local parameter sharing, aggregated parameter voting, final global parameter setting, and global parameter distribution (Mukherjee et al., 15 Jul 2025).
The synchronization algorithm formalizes FL process control through phase variables. It defines
with $0 =$ System Initializing, $1 =$ In Local Learning, and $2 =$ In Model Aggregation, and
with $0 =$ Nodes not Ready, $1 =$ Nodes Ready for Job, $2 =$ Nodes Ready with Dataset, $3 =$ Clients busy in Training / Workers busy in Aggregation, and $4 =$ Clients Waiting for Next Round / Aggregation Complete. The workflow initializes controller state, brings nodes to ready state, distributes datasets, initializes the global model, and then iterates over rounds in which clients train, workers aggregate, consensus selects the next global model, and the round counter advances. In encyclopedic terms, this is the mechanism by which FLsim turns high-level algorithm specifications into controlled distributed execution.
Blockchain support is optional rather than intrinsic. FLsim includes a pluggable blockchain API for blockchain-based FL, with out-of-the-box support for Ethereum and Hyperledger Fabric. To support a new blockchain platform, the user must provide a wrapper around the FLsim blockchain API, smart contracts, and optionally an auto-orchestration script for private chains. The paper attributes to this feature functions such as model parameter verification, traceability, auditability, provenance of the global model, reputation tracking, and attack prevention. A plausible implication is that FLsim is designed to accommodate both conventional FL benchmarks and trust-augmented FL workflows within the same orchestration framework.
5. Experimental evidence and demonstrated capabilities
The experimental evaluation is organized around research questions and is intended to show support for diverse FL frameworks, support for different ML libraries, multi-worker aggregation under adversarial conditions, topology variation, reproducibility, and large-scale simulation (Mukherjee et al., 15 Jul 2025).
For support of diverse FL frameworks, the paper implements and compares FedAvg, FedAvgM, SCAFFOLD, MOON, differential privacy FL, hierarchical clustering FL, and Fedstellar using CIFAR-10, Dirichlet $0 =$0, 10 clients, PyTorch, a model with 3 CNN layers plus classification head, batch size 64, and learning rate 0.001. The reported findings are that MOON and SCAFFOLD achieve the highest accuracy, SCAFFOLD has the lowest loss in the mid-rounds, Hierarchical FL takes the longest time, Fedstellar uses the most bandwidth due to decentralized communication, and MOON, FedAvg, and differential privacy FL use relatively less bandwidth. The significance of this result lies less in the absolute ranking than in the claim that FLsim can reproduce FL proposals with extra states, custom communication, special training logic, and decentralized architectures.
For support of different ML libraries, the paper tests PyTorch, TensorFlow, and Scikit-Learn under the same FL workflow, with PyTorch and TensorFlow using CNNs and Scikit-Learn using flattened inputs and an MLP. The findings are that PyTorch achieves the highest accuracy, Scikit-Learn achieves the lowest accuracy because of the model differences, TensorFlow takes the longest runtime, PyTorch is fastest, and Scikit-Learn requires the highest communication bandwidth. This section is central to the framework’s claim of being library-agnostic.
For multi-worker aggregation and malicious-worker scenarios, the paper evaluates 10 clients and 1–4 workers, with one malicious worker in several cases and consensus based on the method from Chowdhury et al. (fedrlchain). The scenarios are 1M-0H, 1M-1H, 1M-2H, and 1M-3H. The reported result is that when honest workers are more than 50%, the consensus mechanism suppresses the malicious worker, whereas with a 1:1 malicious-to-honest ratio, the learning trajectory becomes unstable. This demonstrates that FLsim can model multi-worker consensus and capture adversarial behavior.
For topology comparisons, the paper contrasts FedAvg as client-server, hierarchical FL, and Fedstellar as decentralized. Accuracy is broadly similar across the three; Hierarchical FL has somewhat higher loss; hierarchical and decentralized setups consume more CPU and memory; and decentralized FL uses the highest bandwidth. For reproducibility, repeated experiments on the same hardware are identical across three repeated trials, while cross-hardware accuracy variation is small, up to about 0.6% at the 10th round. For scaling, FLsim is tested on MNIST with 100, 250, 500, and 1000 clients using logistic regression and uniform data distribution, with findings that global accuracy stays roughly comparable across client counts while bandwidth and runtime increase as the number of clients increases. Taken together, these evaluations position FLsim as a framework for exposing systems-level trade-offs as much as algorithmic performance.
6. Position in the literature, limitations, and interpretation
The paper’s comparison table positions FLsim as more flexible than many existing tools because it supports multiple FL architectures—CFL, HFL, SDFL, and DFL—multiple libraries including PyTorch, TensorFlow, and Scikit-Learn, simulation-oriented use, blockchain support, and flexible topology support (Mukherjee et al., 15 Jul 2025). Its stated main contributions are the FLsim framework itself, the layered architecture and workflow design, and a comprehensive evaluation covering multiple FL algorithms, multiple ML libraries, consensus-based multi-worker FL, diverse topologies, reproducibility, and large-scale simulation.
The discussion identifies several ongoing improvements: improving efficiency and memory management, optimizing job scheduling, and expanding the repository of demo experiments and templates. The paper also notes a practical limitation in reproducibility: different hardware architectures can still introduce slight numerical differences because of floating-point and implementation variation. Another practical limitation is that, while blockchain support is pluggable, integrating a new blockchain platform still requires custom wrappers and smart contracts. These constraints are notable because they qualify two of the framework’s most visible design claims—reproducibility and extensibility—without negating them.
In the broader FL simulation landscape, FLsim can be understood as a general-purpose experimental workbench rather than a deployment platform or a specialized benchmark harness. This suggests that its significance lies in unifying algorithmic, topological, and trust-related variations within a single configurable system. For researchers, its value is the ability to examine trade-offs among accuracy, loss, CPU, memory, bandwidth, consensus behavior, topology, and reproducibility without rebuilding experimental infrastructure for each new FL proposal.