Differentiable Agent-Based Simulation

• Differentiable agent-based simulation refers to rewriting ABMs as differentiable programs, allowing gradients to flow through simulation trajectories.
• It replaces discrete operations with smooth approximations like logistic sigmoids and Gumbel-Softmax relaxations to facilitate gradient-based calibration and control.
• These methods yield significant speedups and scalability improvements in domains such as epidemiology, traffic, autonomous driving, and supply chains.

Searching arXiv for papers on differentiable agent-based simulation and closely related differentiable ABM frameworks. arXiv search query: "differentiable agent-based simulation" Differentiable agent-based simulation denotes a class of agent-based models in which the full simulation trajectory is written as a differentiable program, allowing automatic differentiation of losses or objectives with respect to model parameters, policy variables, or latent states. Across the literature, this is achieved by expressing agent updates, interaction rules, and readout functions as differentiable tensor operations, or by replacing discrete constructs such as conditional branching, argmax, Bernoulli sampling, and categorical choice with smooth surrogates or reparameterized relaxations. The resulting simulators support gradient-based calibration, forecasting, control, and policy optimization in settings including epidemiology, traffic, autonomous driving, supply chains, bike-sharing systems, and surrogate modeling of non-differentiable agent-based models (Andelfinger, 2021, Chopra et al., 2022, Chopra et al., 2021, Nachkov et al., 14 Nov 2025).

1. Concept and formalization

An agent-based model is commonly written as a time-stepped simulator in which each agent’s state is updated from its previous state, its interactions, and model parameters. In differentiable formulations, the simulator becomes an explicit differentiable map. For autonomous driving, the simulator is written as

$$\mathrm{Sim}:\;\mathcal S\times\mathcal A\to\mathcal S,\quad (s_t,a_t)\mapsto s_{t+1},$$

or equivalently

$s_{t+1} \;=\; f_\theta(s_t,a_t),$

with state and action defined over all interacting vehicles (Nachkov et al., 14 Nov 2025). In graph-based epidemic simulation, a stepwise population update is written as

$x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$

where $F$ represents one GNN layer comprising aggregation and update (Chopra et al., 2021). More generally, differentiable ABM work writes a stochastic simulator as $f_\theta : \Theta \to X$, then reparameterizes randomness by $\epsilon \sim q(\epsilon)$ and $x = g(\epsilon;\theta)$, so that a loss can be differentiated through the simulator by the pathwise gradient (Quera-Bofarull et al., 3 Sep 2025).

This formulation makes the simulation trajectory itself part of a computation graph. In time-driven ABMs, reverse-mode automatic differentiation can be applied to the whole simulation once discrete logic is relaxed, making it possible to compute $\nabla f$ or $\nabla L$ in a single backward pass rather than by finite differences or gradient-free search (Andelfinger, 2021). In the autonomous-driving DSS formulation, the planning loss is $L(a_{0:T-1})=C(s_{1:T})$, and the action gradient is obtained by backpropagation-through-time through imagined future trajectories (Nachkov et al., 14 Nov 2025). In inverse ABM settings, observed macro-scale quantities can be compared to simulator outputs via a loss such as

$s_{t+1} \;=\; f_\theta(s_t,a_t),$0

which is then minimized by gradient descent directly through the unrolled simulator (Chopra et al., 2021).

A central implication is that differentiable ABMs unify simulation, inference, and optimization inside a single computational object. This suggests a shift from treating simulation as a black-box oracle toward treating it as a trainable, calibratable, or controllable differentiable module.

2. Core differentiability mechanisms

The main technical obstacle is that conventional ABMs contain non-differentiable operations: discrete state transitions, logical branching, neighbor selection, event triggers, sorting, Bernoulli infection events, categorical choices, and hard collision or capacity constraints. The literature addresses these obstacles with a recurring set of constructions (Andelfinger, 2021, Quera-Bofarull et al., 2023, Quera-Bofarull et al., 3 Sep 2025).

A basic strategy is smoothing discrete control flow. In time-driven differentiable ABMs, a hard branch

$s_{t+1} \;=\; f_\theta(s_t,a_t),$1

is replaced by a smooth branching rule using a logistic sigmoid $s_{t+1} \;=\; f_\theta(s_t,a_t),$2, yielding

$s_{t+1} \;=\; f_\theta(s_t,a_t),$3

with $s_{t+1} \;=\; f_\theta(s_t,a_t),$4 (Andelfinger, 2021). The same work introduces smooth max/min and periodic steps for event logic, while DEpiABS uses several selective relaxations, including

$s_{t+1} \;=\; f_\theta(s_t,a_t),$5

$s_{t+1} \;=\; f_\theta(s_t,a_t),$6, and $s_{t+1} \;=\; f_\theta(s_t,a_t),$7, depending on the required precision (Gao et al., 12 Feb 2026).

A second strategy is surrogate gradients or reparameterizations for discrete sampling. In differentiable epidemiological and large-scale ABM frameworks, Bernoulli and categorical draws are replaced by straight-through estimators or Gumbel-Softmax relaxations (Chopra et al., 2022, Quera-Bofarull et al., 2023, Quera-Bofarull et al., 3 Sep 2025). For example, a Gumbel-Softmax relaxation for a Bernoulli-type update is written as

$s_{t+1} \;=\; f_\theta(s_t,a_t),$8

making transmission differentiable with respect to $s_{t+1} \;=\; f_\theta(s_t,a_t),$9 (Chopra et al., 2022). In traffic digital twins, next-link choice is modeled with a standard logit utility and then sampled using Gumbel-Softmax with a straight-through estimator so that the forward pass uses exact discrete moves while the backward pass remains differentiable (Makinoshima et al., 26 Mar 2026).

A third strategy is differentiable tensorization of state and interactions. DeepABM represents the entire population as a graph $x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$0, with message passing

$x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$1

so that all agent interactions appear as differentiable GPU-friendly tensor operations (Chopra et al., 2021). DEpiABS similarly tensorizes health states, decisions, locations, and encounter matrices in PyTorch (Gao et al., 12 Feb 2026).

A fourth strategy is differentiable surrogacy rather than direct relaxation of the original simulator. Graph Diffusion Networks construct a differentiable surrogate $x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$2 of a non-differentiable ABM by combining a GNN neighborhood encoder with a conditional diffusion model that learns the distribution of each agent’s next state (Cozzi et al., 27 May 2025). This preserves individual-level stochastic transitions while making the rollout end-to-end differentiable.

These mechanisms differ in fidelity and bias. Some frameworks keep the forward pass exactly discrete and alter only the tangent or backward pass (Quera-Bofarull et al., 3 Sep 2025), while others relax the forward dynamics themselves into smooth approximations (Andelfinger, 2021, Gao et al., 12 Feb 2026). A plausible implication is that the choice between these strategies reflects a trade-off between mechanistic fidelity, gradient quality, and implementation complexity.

3. Simulation architectures and domains

Differentiable agent-based simulation is not a single architecture but a family of implementations adapted to different domains.

In epidemiology, GradABM models a population of $x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$3 agents with disease states $x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$4, sparse contact matrices, differentiable transmission hazards, and Gumbel-Softmax transmission events, enabling calibration, forecasting, and policy evaluation on million-size populations (Chopra et al., 2022). DeepABM instead formulates epidemic interaction as message passing on graphs with node features, edge features, and MPNN updates, achieving real-time GPU performance on $x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$5K agents over $x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$6 days (Chopra et al., 2021). DEpiABS extends this line by constructing a fully mechanistic, individual-level ABM with Society, Epidemic, and Population components, adding mutation, reinfection dynamics, and a z-score scaling method for transferring small simulated populations to larger real populations (Gao et al., 12 Feb 2026).

In traffic and mobility, several distinct paradigms appear. DSS for autonomous vehicle planning uses the differentiable simulator Waymax as both next-state predictor and critic, with hardcoded kinematic bicycle dynamics and differentiable collision and offroad surrogates (Nachkov et al., 14 Nov 2025). ITRA uses a differentiable multi-agent simulator for trajectory prediction in which each vehicle executes steering and acceleration actions under a kinematic bicycle model, and the full scene is rendered into ego-centric bird’s-eye-view images by a differentiable rasterizer (Scibior et al., 2021). At larger urban-network scale, a differentiable traffic digital twin represents the state as a tensor $x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$7 over platoons and links, with differentiable car-following, differentiable node choice, trajectory grafting, and differentiable counting for nowcasting and control on the Chicago network (Makinoshima et al., 26 Mar 2026). Bike-sharing pricing models also fit this category: user demand, destination choice, and trip durations are embedded into a differentiable ABM using GenGS and Gumbel-Softmax, while station inventories evolve through differentiable departure and return flows (Mitomi et al., 31 Jul 2025).

In economics and production networks, a differentiable supply-chain ABM tracks inventories, orders, shipments, production, and shock recovery through array-wise operations, then calibrates parameters by automatic differentiation through time (Hamid et al., 7 Nov 2025). The model’s production rule uses Leontief-style constraints with min operators that are either handled by JAX subgradients or replaced by soft-min approximations (Hamid et al., 7 Nov 2025).

In surrogate modeling, Graph Diffusion Networks learn a differentiable surrogate of arbitrary ABMs by modeling individual-agent transitions directly rather than only aggregate statistics (Cozzi et al., 27 May 2025). This preserves the decentralized, bottom-up character of ABMs, with empirical micro-level fidelity measured by Earth Mover’s Distance and macro-level fidelity measured by symmetric MAPE (Cozzi et al., 27 May 2025).

Across these domains, the common denominator is not a specific state representation but the requirement that agent interactions, temporal recurrences, and loss readouts be embedded in an autodiff-compatible program.

4. Gradient-based planning, calibration, and control

A defining feature of differentiable ABMs is that optimization is performed by backpropagating through the simulator rather than by treating simulation output as an opaque objective.

In planning, DSS optimizes action sequences using gradient descent over imagined future trajectories. Its one-step replanning procedure samples $x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$8 rollouts under a learned stochastic policy, computes losses and gradients for each rollout, and forms a final action

$x^{t+1} = F(x^t; \theta, E^{(t)}, e^{(t)}),$9

with Boltzmann weights $F$0 (Nachkov et al., 14 Nov 2025). This hybrid of stochastic search and one-step gradient descent is described as a continuous-action, multi-agent MPC (Nachkov et al., 14 Nov 2025).

In calibration, differentiable epidemiological ABMs typically compare simulated time series to observations and optimize parameters by gradient descent or by variational methods. GradABM uses losses such as $F$1 and also parameterizes time-varying simulator parameters through a neural network $F$2, enabling end-to-end calibration from heterogeneous macro-data (Chopra et al., 2022). DeepABM formulates inverse ABM as gradient descent through an unrolled GNN using macro-observables such as daily new-case counts or hospitalizations (Chopra et al., 2021). DEpiABS calibrates 27 learnable parameters using the Normalised Deviation

$F$3

optimized in PyTorch with gradient descent from initial learning rate $F$4 annealed to $F$5 (Gao et al., 12 Feb 2026).

Bayesian calibration occupies a separate strand. Generalized variational inference is used to define a generalized posterior

$F$6

then approximate it with a normalizing flow $F$7 by minimizing a loss-plus-KL objective (Quera-Bofarull et al., 2023). A related framework calibrates a differentiable COVID-19 ABM by placing a prior on transmission parameters, using a robust scoring-rule loss on the simulated infection trajectory, and optimizing a Neural Spline Flow variational family (Quera-Bofarull et al., 2023). Automatic Differentiation of Agent-Based Models extends this line to Axtell’s model of firms, Sugarscape, and an SIR model, using hybrid AD and variational inference for efficient parameter calibration and sensitivity analysis (Quera-Bofarull et al., 3 Sep 2025).

In control, the large-scale traffic digital twin calibrates behavioral parameters $F$8 from detector counts by minimizing

$F$9

then optimizes dynamic tolls $f_\theta : \Theta \to X$0 via an analogous control loss (Makinoshima et al., 26 Mar 2026). Bike-sharing pricing directly optimizes a discount tensor $f_\theta : \Theta \to X$1 using autodiff through the full simulator, where the loss combines final inventory imbalance with total discount cost (Mitomi et al., 31 Jul 2025).

The broader significance is methodological: agent-based simulation becomes a differentiable inner loop inside planning, inference, or control pipelines rather than a standalone evaluation engine.

5. Computational performance and scaling

A major motivation for differentiable ABMs is computational scalability under calibration workloads that are prohibitive for gradient-free methods.

DeepABM reports that DeepABM-COVID can model 200 million interactions, over 100,000 agents across 180 time-steps, in 90 seconds on a GPU, whereas a pure-Python, object-oriented ABM using Mesa takes $f_\theta : \Theta \to X$2 for the same workload (Chopra et al., 2021). GradABM reports that 800,000 agents, 5 million edges, and $f_\theta : \Theta \to X$3 days require 4 seconds for a forward pass on a single V100 GPU, about 60 seconds on CPU, and that Mesa is roughly 300 times slower (Chopra et al., 2022). DEpiABS gives an empirical runtime fit on an NVIDIA RTX 4060,

$f_\theta : \Theta \to X$4

so that a 1,000-agent, 30-day forecast runs in approximately $f_\theta : \Theta \to X$5, described as 200×–250× faster than an unoptimised Mesa Python version (Gao et al., 12 Feb 2026).

The supply-chain model reports GPU versus CPU speed for SVI calibration at $f_\theta : \Theta \to X$6 firms: approximately $f_\theta : \Theta \to X$7 per epoch on an RTX 5090 versus approximately $f_\theta : \Theta \to X$8 on a Ryzen 9, a $f_\theta : \Theta \to X$9 speed-up (Hamid et al., 7 Nov 2025). The same paper states that gradient-based SVI needed approximately 300 simulator calls to reach the same in-sample loss that ABC achieved in approximately 30,000 calls (Hamid et al., 7 Nov 2025). The bike-sharing pricing model reports a more than 100-fold increase in convergence speed and a 73% to 78% reduction in loss relative to conventional methods in a 25-station, five-time-slot setting (Mitomi et al., 31 Jul 2025).

Large traffic digital twins push scale further. On the Chicago Sketch network with $\epsilon \sim q(\epsilon)$0 links, $\epsilon \sim q(\epsilon)$1 platoons, and $\epsilon \sim q(\epsilon)$2, the differentiable simulator runs one-hour nowcasting at 173 times real-time speed; calibration using the previous 30 minutes of traffic data takes $\epsilon \sim q(\epsilon)$3, a one-hour-ahead nowcast takes $\epsilon \sim q(\epsilon)$4, and gradient-guided control takes $\epsilon \sim q(\epsilon)$5 (Makinoshima et al., 26 Mar 2026). The same system simulates more than one million vehicles at 173 times real-time speed (Makinoshima et al., 26 Mar 2026).

These gains are closely tied to vectorization, sparse operations, JIT compilation, GPU kernels, and scan-style time integration (Chopra et al., 2021, Chopra et al., 2022, Hamid et al., 7 Nov 2025, Makinoshima et al., 26 Mar 2026). A plausible implication is that differentiability and scalability are often co-produced by the same tensorized reformulation: the changes that expose gradients also expose data parallelism.

6. Accuracy, empirical results, and benchmark behavior

Reported empirical results show that differentiable ABMs can improve planning accuracy, forecasting quality, or optimization efficiency, although the strength of the benefit depends on task type and estimator design.

For autonomous driving, DSS is reported to significantly improve tracking and path planning accuracy compared to sequence prediction, imitation learning, model-free RL, and other planning methods (Nachkov et al., 14 Nov 2025). For pure tracking, the critic is the mean squared waypoint error, while collision and offroad penalties are handled through frozen classifiers that make $\epsilon \sim q(\epsilon)$6 well-defined (Nachkov et al., 14 Nov 2025).

For multi-agent trajectory prediction, ITRA achieves $\epsilon \sim q(\epsilon)$7 and $\epsilon \sim q(\epsilon)$8 on the INTERACTION validation split, improving over ReCoG at $\epsilon \sim q(\epsilon)$9 (Scibior et al., 2021). The same work reports that future birdview feedback and the bicycle model are crucial, with ablations collapsing to deterministic outputs and exhibiting much higher ADE/FDE and near-zero MFD (Scibior et al., 2021).

For epidemic forecasting, GradABM reports mean $x = g(\epsilon;\theta)$0 std forecasting errors over five runs across ten Massachusetts counties. JDC-GradABM attains ND $x = g(\epsilon;\theta)$1, RMSE $x = g(\epsilon;\theta)$2, and MAE $x = g(\epsilon;\theta)$3 for COVID-19, and ND $x = g(\epsilon;\theta)$4, RMSE $x = g(\epsilon;\theta)$5, and MAE $x = g(\epsilon;\theta)$6 for influenza, outperforming ExpertSearch-ABM, SurrogateODE-ABM, DC-GradABM, and C-GradABM (Chopra et al., 2022). DEpiABS reports that, over five runs and ten Massachusetts counties, it reduces average Normalised Deviation from $x = g(\epsilon;\theta)$7 to $x = g(\epsilon;\theta)$8 on COVID-19 mortality data and from $x = g(\epsilon;\theta)$9 to $\nabla f$0 on influenza-like-illness data relative to the stated baseline, with RMSE $\nabla f$1 deaths/day versus $\nabla f$2 in one comparison (Gao et al., 12 Feb 2026).

For surrogate modeling, Graph Diffusion Networks report low micro-level Earth Mover’s Distance and macro-level symmetric MAPE below $\nabla f$3 in Schelling and below $\nabla f$4 in Predator-Prey, with out-of-sample rollouts 25 steps beyond the training window (Cozzi et al., 27 May 2025). The method is presented as preserving both individual-level patterns and emergent dynamics (Cozzi et al., 27 May 2025).

For optimization, the foundational differentiable ABM study finds that in traffic signal timing with 2,500 lights and 2,500 vehicles, gradient methods improve total progress by approximately 100 km after 100 batches, whereas DE, CNE, SA, and SPSA remain below 20 km (Andelfinger, 2021). With a 72-hour budget, Adam reaches approximately $\nabla f$5, while CNE reaches approximately $\nabla f$6 (Andelfinger, 2021). The same paper, however, reports a more mixed picture for epidemic-model calibration, where DE and CNE achieve about $\nabla f$7 error while gradient methods finish at about $\nabla f$8 (Andelfinger, 2021).

This mixed evidence is important. It shows that differentiability does not uniformly dominate all baselines for all tasks. Performance gains appear strongest in high-dimensional optimization and control, while calibration quality can still depend sensitively on estimator bias, gradient variance, loss choice, and simulator structure.

7. Limitations, calibration challenges, and methodological debates

Several papers emphasize that differentiable ABMs introduce new challenges rather than eliminating old ones.

One challenge is bias from surrogate gradients. Gumbel-Softmax and straight-through estimators preserve a pathwise derivative but are biased, and low variance is not guaranteed (Quera-Bofarull et al., 2023, Quera-Bofarull et al., 3 Sep 2025). In the SIR model examined in "Automatic Differentiation of Agent-Based Models" (Quera-Bofarull et al., 3 Sep 2025), only the unbiased SPA estimator tracks finite-difference gradients under high nonlinearity, while surrogate methods fail. This indicates that differentiability alone does not ensure faithful gradient information.

A second challenge is memory and time trade-offs between reverse-mode and forward-mode AD. Reverse-mode AD is efficient when outputs are few, but its memory can become prohibitive for long horizons and large populations (Quera-Bofarull et al., 2023, Quera-Bofarull et al., 3 Sep 2025). In GradABM-June, reverse-mode memory would rise to about 5 TB for 8 million agents and 300 days, whereas forward-mode requires about 18 GB (Quera-Bofarull et al., 2023). Hybrid FMAD–RMAD schemes are therefore introduced to combine memory-cheap simulator Jacobians with reverse-mode differentiation through normalizing flows (Quera-Bofarull et al., 2023, Quera-Bofarull et al., 3 Sep 2025).

A third challenge is gradient variance over long horizons. In the Brock–Hommes experiment, naïve pathwise gradients with horizon $\nabla f$9 do no better than a score-function estimator, while truncated horizons $\nabla L$0 or $\nabla L$1 reduce loss much faster (Quera-Bofarull et al., 2023). The same work argues that increasing $\nabla L$2 increases Monte Carlo gradient variance, making truncated backpropagation a bias-variance trade-off rather than a mere approximation (Quera-Bofarull et al., 2023).

A fourth challenge is fidelity versus stability in smoothing. The foundational traffic-and-epidemics study recommends tuning the smoothing parameter $\nabla L$3 so that the smoothed step remains close to the true branch but gradients do not vanish or explode (Andelfinger, 2021). It also notes that full differentiability of every branch and neighbor search is expensive, motivating selective disabling of AD on code regions that contribute little to objective sensitivity (Andelfinger, 2021). This suggests that practical differentiable ABMs are often partially differentiable systems rather than uniformly smooth reformulations.

A fifth limitation is structural scope. Graph Diffusion Networks assume that the interaction graph is known and that each ABM step is a single stochastic event, leaving latent graph inference and hierarchical decision sequences to future work (Cozzi et al., 27 May 2025). Surrogate-based approaches also raise a conceptual distinction between making the original simulator differentiable and learning a differentiable surrogate of it. The former preserves mechanistic structure more directly, while the latter may offer better gradient behavior or easier scaling.

Taken together, these debates indicate that differentiable agent-based simulation is best understood as a methodological program: recasting multi-agent simulation into autodiff-compatible computation while managing the distortions, biases, and systems constraints that this recasting introduces. The literature shows that this program can be effective at scales ranging from tens of agents to million-agent systems and parameter spaces exceeding 10,000 dimensions, but it also shows that calibration robustness, gradient fidelity, and memory efficiency remain active technical concerns (Makinoshima et al., 26 Mar 2026, Quera-Bofarull et al., 2023, Quera-Bofarull et al., 3 Sep 2025).