JL-GAT: Joint-Local Grounded Action Transfer

• JL-GAT is a decentralized sim-to-real transfer method that leverages local joint grounding to enable coordinated traffic signal control.
• The method employs forward and inverse models trained on both real and simulated data, allowing each agent to refine its actions based on its own and neighboring observations.
• Experiments in urban grid networks under adverse weather show that JL-GAT significantly reduces performance gaps by effectively capturing local inter-agent interactions.

Searching arXiv for the JL-GAT paper and closely related work on GAT and traffic signal control. JL-GAT, short for Joint-Local Grounded Action Transformation, is a sim-to-real transfer method for multi-agent traffic signal control that extends Grounded Action Transformation from single-agent reinforcement learning to decentralized multi-agent settings. It is designed for urban traffic networks in which intersections operate as coordinated agents, and it addresses the performance degradation that arises when policies trained in simulation are deployed under shifted real-world dynamics. The method balances scalability with enhanced grounding capability by allowing each agent to transform its action using not only its own local information but also information from neighboring agents, thereby occupying a middle ground between fully centralized and fully decentralized grounding schemes (Turnau et al., 21 Jul 2025).

1. Dec-POMDP formulation for multi-agent traffic signal control

Traffic signal control is formulated as a decentralized partially-observable Markov decision process. Let $N$ denote the set of intersections. The model includes a global state space $S$, with elements $s_t \in S$ encoding the joint traffic conditions across all intersections, although in practice $s_t$ is not fully observed. Instead, each agent $i$ receives a local observation $o_{i,t}$, such as vehicle counts on incoming and outgoing lanes. Each agent has an individual action space $A_i$; in the reported experiments, each $A_i$ corresponds to one of eight traffic-signal phases, encoded as a one-hot vector. The joint action is $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$. Transition dynamics are written as $P(s_{t+1}\mid s_t,a_t)$, and under partial observability these are equivalently understood as generating next observations $S$0 for each agent. The reward function $S$1 is the negative “pressure” across all intersections, namely the PressLight reward. Equivalently, each agent $S$2 receives $S$3 and seeks to maximize the discounted return

$S$4

Under this Dec-POMDP, agents learn decentralized policies $S$5 using independent DQN learners with experience replay (Turnau et al., 21 Jul 2025).

This formulation makes two structural properties explicit. First, each agent acts under partial observability rather than from a fully shared global state. Second, the transition of one intersection is coupled to neighboring intersections through vehicle flows, so strictly local action selection is operationally insufficient for sim-to-real adaptation. JL-GAT is built around this tension: it preserves decentralized policy learning while incorporating limited joint context where grounding is performed.

2. From single-agent GAT to the joint-local construction

Grounded Action Transformation was originally proposed for single-agent sim-to-real transfer. In that setting, one assumes access to real-world rollouts $S$6 consisting of tuples $S$7, together with a parameterized simulator $S$8 whose parameters $S$9 are to be tuned so that $s_t \in S$0, the true dynamics. GAT uses a two-stage mapping: a forward model $s_t \in S$1 trained on real trajectories, and an inverse model $s_t \in S$2 trained in simulation. The simulator-matching objective is

$s_t \in S$3

and the core transformation is

$s_t \in S$4

During on-line policy training, the raw policy action $s_t \in S$5 is replaced in the simulator by the grounded action $s_t \in S$6 (Turnau et al., 21 Jul 2025).

Directly generalizing this mechanism to multi-agent traffic control produces two limiting designs. A fully centralized GAT uses one global forward/inverse pair over the global state and joint action; it captures all interactions but does not scale. A fully decentralized GAT assigns each agent an individual forward/inverse pair based only on $s_t \in S$7; it scales, but ignores neighbor influence. JL-GAT is defined precisely as a compromise between these extremes. It retains decentralized per-agent grounding, but conditions each agent’s grounding on a local joint neighborhood. A plausible implication is that the method treats inter-intersection coupling as important but spatially bounded rather than globally dense.

3. Formal specification of JL-GAT

JL-GAT introduces a local-joint state and action representation for each agent. Given a sensing radius $s_t \in S$8 in Manhattan distance, define the neighborhood of agent $s_t \in S$9 as

$s_t$0

At time $s_t$1, agent $s_t$2 constructs

$s_t$3

The grounded transition model for agent $s_t$4 predicts the next individual observation $s_t$5 rather than the full global next state. With real-world trajectories $s_t$6, the JL-GAT objective is

$s_t$7

where $s_t$8 can be KL divergence or MSE for continuous observations (Turnau et al., 21 Jul 2025).

Each agent maintains a forward model

$s_t$9

trained by minimizing

$i$0

and an inverse model

$i$1

trained via categorical cross-entropy on simulated rollouts:

$i$2

The per-agent grounding transformation is therefore

$i$3

The significance of this construction is not merely architectural. By predicting the next local observation from a neighborhood-conditioned tuple $i$4 and then inverting that prediction into a grounded action, JL-GAT makes grounding sensitive to nearby coordinated behavior while avoiding a single monolithic model over the entire network. This suggests that the method operationalizes locality as the principal approximation to global coordination.

4. Training procedure and grounding schedules

The reported training procedure is decentralized. Each agent’s policy $i$5, forward model $i$6, inverse model $i$7, and buffers $i$8 and $i$9 are initialized first. The policy is then pre-trained in the simulator for $o_{i,t}$0 iterations to obtain a reasonable starting policy. For each epoch $o_{i,t}$1, the procedure may roll out $o_{i,t}$2 in the simulator and append transitions to $o_{i,t}$3, and may also roll out $o_{i,t}$4 in real or “real-like” SUMO and append trajectories to $o_{i,t}$5. The GAT models are updated per agent by fitting $o_{i,t}$6 on $o_{i,t}$7 and $o_{i,t}$8 on $o_{i,t}$9. During policy training with grounding, each agent observes $A_i$0, selects $A_i$1, gathers neighbors’ observations and actions to form $A_i$2, computes $A_i$3 and then $A_i$4, and replaces $A_i$5 by $A_i$6 according to a grounding pattern or grounding probability $A_i$7. The resulting transition $A_i$8 is stored in a replay buffer, after which $A_i$9 is updated via DQN or another RL update (Turnau et al., 21 Jul 2025).

Two grounding schedules are emphasized because simultaneous grounding can create self-inconsistency. Under pattern grounding, a subset of agents grounds according to a fixed pattern, such as alternating which subset grounds each epoch. Under probabilistic grounding, each agent grounds with probability $A_i$0. These schedules are introduced to mitigate what the authors term cascading invalidation: if multiple neighboring agents ground their actions simultaneously while assuming neighbors’ actions remain fixed, the transformed actions can invalidate one another. The code is publicly available at https://github.com/DaRL-LibSignal/JL-GAT/.

5. Sim-to-real gap, adverse weather, and benchmark results

The sim-to-real gap is attributed to unmodeled dynamics, including driver braking behavior, sensor noise, and weather effects. To stress-test transfer under adverse conditions, the real environment is instantiated as SUMO with weather-modified parameters, while the simulator used for training is CityFlow with default parameters. In this setup, “$A_i$1” is CityFlow with default parameters and “$A_i$2” is SUMO under adverse weather (Turnau et al., 21 Jul 2025).

Weather	Parameters
Rainy	$A_i$3, $A_i$4, $A_i$5, $A_i$6
Snowy	$A_i$7, $A_i$8, $A_i$9, $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$0

The core benchmarks use two grid topologies: a $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$1 tandem of signals and a $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$2 grid. Metrics are reported over six independent trials, each selecting the best epoch by lowest average travel time in $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$3, and include Average Travel Time (ATT), Queue length, Delay, Throughput (TP), and cumulative Reward. The sim-to-real gap is defined as $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$4, so smaller $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$5 or $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$6 is better, while larger negative $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$7 is better.

In the rainy $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$8 setting, Direct Transfer reports $$a_t=(a_{1,t},\dots,a_{N,t}) \in A \coloneqq \prod_i A_i$$9 with gap $P(s_{t+1}\mid s_t,a_t)$0, Queue gap $P(s_{t+1}\mid s_t,a_t)$1, Delay gap $P(s_{t+1}\mid s_t,a_t)$2, and TP gap $P(s_{t+1}\mid s_t,a_t)$3. Centralized GAT reduces the ATT gap to $P(s_{t+1}\mid s_t,a_t)$4, Decentralized GAT to $P(s_{t+1}\mid s_t,a_t)$5, JL-GAT with pattern grounding to $P(s_{t+1}\mid s_t,a_t)$6, and JL-GAT with probability $P(s_{t+1}\mid s_t,a_t)$7 to $P(s_{t+1}\mid s_t,a_t)$8. In the same setting, the Queue gap improves from $P(s_{t+1}\mid s_t,a_t)$9 to $S$00, and TP gap improves from $S$01 to $S$02. In the rainy $S$03 grid, Direct Transfer has ATT gap $S$04, Dec-GAT reduces it to $S$05, and JL-GAT with probability $S$06 reaches $S$07. Under snowy weather, the same qualitative ordering is reported: JL-GAT consistently yields the smallest sim-to-real gaps across all five metrics, outperforming both centralized and pure decentralized GAT variants.

The empirical pattern is therefore not simply that grounding helps, but that neighborhood-aware grounding helps more than either extreme. This suggests that local inter-agent coupling is one of the dominant error sources in sim-to-real transfer for MARL-based traffic signal control under the tested perturbations.

6. Ablation evidence, limitations, and prospective extensions

The ablation study isolates the contribution of neighbor information by removing neighbor observations and actions from the forward and inverse models. In the rainy $S$08 network, removing neighbor states from $S$09 substantially degrades performance, increasing the ATT gap to $S$10 versus $S$11 when they are included. Removing neighbor actions from either $S$12 or $S$13 produces ATT gaps near the ungrounded Direct Transfer baseline, approximately $S$14. The reported conclusion is that both neighbor states and neighbor actions must be included for JL-GAT’s success (Turnau et al., 21 Jul 2025).

These ablations address a likely misconception: the joint component is not an incidental embellishment added to an otherwise decentralized GAT pipeline. On the reported evidence, omitting neighbor information sharply weakens the grounding effect. In that sense, JL-GAT is better understood as a local-joint grounding method than as merely a decentralized one.

Two limitations are emphasized. The first is cascading invalidation, mitigated through pattern grounding or probabilistic grounding, both of which trade off flexibility for consistency. The second is the fixed sensing radius $S$15, which ignores interactions beyond the chosen neighborhood and may therefore omit relevant dependencies in more complex networks. The stated future directions include adaptive or learned grounding schedules rather than fixed patterns, cluster-based or graph-neural-network-driven neighborhood definitions, tighter integration with uncertainty quantification to decide when to ground, and extension to real robotic deployments beyond traffic signal control. A plausible implication is that JL-GAT establishes a template for neighborhood-conditioned action transformation in decentralized systems, but not yet a complete solution to long-range dependency modeling or consistency management under simultaneous multi-agent grounding.