DecompGAIL: Decomposed Multi-agent GAIL

• The paper introduces DecompGAIL, which decomposes adversarial realism into ego–map and ego–neighbor components to mitigate credit assignment instability in dense traffic scenes.
• It replaces a monolithic discriminator with a decomposed architecture and employs a social PPO objective that augments rewards using distance-weighted neighborhood influence.
• Empirical evaluations on the Waymo Open Motion Dataset demonstrate improved interactive realism and stability over traditional multi-agent GAIL formulations.

Searching arXiv for DecompGAIL and its primary multi-agent GAIL precursors. Decomposed Multi-agent GAIL (DecompGAIL) is a multi-agent adversarial imitation learning method for realistic traffic simulation that addresses a specific instability of decentralized GAIL in dense interactive scenes: a discriminator can penalize realistic ego behavior because of unrealistic behavior among the ego’s neighbors. The method, introduced in “DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learning” (Guo et al., 8 Oct 2025), frames traffic simulation as imitation learning in a Markov game and proposes to decompose realism into ego–map and ego–neighbor components, while intentionally excluding neighbor–neighbor, neighbor–map, and much of the higher-order entangled context from the discriminator. Relative to earlier multi-agent GAIL formulations such as MAGAIL (Song et al., 2018) and parameter-sharing driving models such as PS-GAIL (Bhattacharyya et al., 2020), DecompGAIL is distinguished by an explicit architectural decomposition of adversarial supervision and by a social PPO objective that augments each agent’s reward with distance-weighted neighborhood rewards.

1. Conceptual basis and problem formulation

DecompGAIL studies realistic traffic simulation from logged real-world driving data as a multi-agent imitation learning problem. A scene contains many traffic participants—vehicles, cyclists, pedestrians—interacting on an HD map over time, and the objective is to learn a simulator policy that can roll out all agents jointly and produce trajectories that look like real traffic (Guo et al., 8 Oct 2025). The environment is modeled as a Markov game with $N$ agents,

$$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$

where $s \in \mathcal{S}$ is the global traffic state, each agent $i$ has action space $\mathcal{A}_i$, $P$ is the transition function, $r_i$ is the reward for agent $i$, and $\gamma$ is the discount factor (Guo et al., 8 Oct 2025). Demonstrations are given by

$$\mathcal{D}_E=\{\tau_1,\ldots,\tau_D\}, \quad \tau=(s_1,\mathbf{a}_1,\ldots,s_T,\mathbf{a}_T).$$

The method adopts decentralized decision-making with parameter sharing: all agents use the same policy network, but each acts based on its own local context (Guo et al., 8 Oct 2025). This places DecompGAIL within the line of multi-agent GAIL methods that factorize the joint policy across agents. In MAGAIL, for example, the joint policy is written as

$$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$0

in a general Markov game formalism (Song et al., 2018). DecompGAIL retains this decentralized factorization but modifies how realism is scored and how credit is assigned.

The immediate motivation comes from the limitations of behavior cloning (BC) and standard GAIL in long-horizon interactive traffic. DecompGAIL first uses BC pretraining with objective

$$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$1

but emphasizes that BC suffers from covariate shift because the learned policy is trained under expert states yet rolled out under its own induced state distribution (Guo et al., 8 Oct 2025). In traffic simulation this is especially severe because the horizon is long, many agents interact simultaneously, small prediction errors alter future interactions, and those altered interactions create out-of-distribution states (Guo et al., 8 Oct 2025). This diagnosis is consistent with earlier driving-simulation work, which argued that transitioning from single-agent to multi-agent deployment reintroduces the covariate-shift problem even when GAIL addresses it in a fixed environment (Bhattacharyya et al., 2018, Bhattacharyya et al., 2020).

2. Standard GAIL in multi-agent traffic and the instability diagnosis

As background, DecompGAIL uses decentralized multi-agent GAIL with objective

$$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$2

The discriminator gives reward

$$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$3

This follows the standard GAIL paradigm in which the policy is trained to induce state-action occupancies that are indistinguishable from expert demonstrations, an interpretation developed explicitly for single-agent GAIL in driver modeling work (Bhattacharyya et al., 2020).

The central claim of DecompGAIL is that standard decentralized GAIL becomes unstable in traffic because the discriminator observes an entangled local scene containing ego history, nearby agents, and map context (Guo et al., 8 Oct 2025). Since nearby agents are also policy-generated and can behave unrealistically, the discriminator can punish the ego even when the ego itself behaves realistically. The paper names this failure mode irrelevant interaction misguidance: a discriminator penalizes an ego vehicle’s realistic behavior because it observes unrealistic interactions among the ego’s neighbors, even though those interactions are only weakly related to the ego’s own action (Guo et al., 8 Oct 2025).

To formalize the issue, the paper conceptually decomposes a standard local discriminator signal into four parts: $$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$4 Within this decomposition, the paper treats ego–map and ego–neighbor terms as relevant to ego credit assignment, while neighbor–neighbor, neighbor–map, and much of the higher-order entangled context are described as misleading or weakly relevant (Guo et al., 8 Oct 2025). The severity is said to grow with the number of input neighbors.

This diagnosis differs from earlier multi-agent GAIL critiques. MAGAIL emphasized multiple Nash equilibria and non-stationarity in general Markov games (Song et al., 2018), while PS-GAIL emphasized train–test interaction distribution shift when a single-agent driving policy is deployed across many vehicles (Bhattacharyya et al., 2018, Bhattacharyya et al., 2020). DecompGAIL narrows the instability to a discriminator-level credit-assignment failure caused by entangled local context (Guo et al., 8 Oct 2025).

3. Decomposition of realism and discriminator structure

The core idea of DecompGAIL is to decompose realism into components that are directly attributable to ego behavior (Guo et al., 8 Oct 2025). Specifically, the method separates realism into:

1. scene realism, corresponding to ego–map realism;
2. interaction realism, corresponding to ego–neighbor realism.

For each ego agent $$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$5 at time $$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$6, DecompGAIL computes a scene realism score $$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$7 based only on ego and map, and a set of pairwise interaction realism scores $$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$8 for each ego–neighbor pair (Guo et al., 8 Oct 2025). This is therefore an explicitly factored realism model rather than a monolithic local discriminator.

The naive PS-GAIL baseline discriminator in the paper evaluates ego using fused local scene features: $$\langle \mathcal{S}, \{\mathcal{A}_i\}_{i=1}^N, P, \{r_i\}_{i=1}^N, \gamma \rangle,$$9 Because all context is fused before scoring, this baseline can implicitly depend on all four conceptual terms $s \in \mathcal{S}$0 (Guo et al., 8 Oct 2025).

DecompGAIL replaces this with a decomposed discriminator. The scene branch is

$s \in \mathcal{S}$1

and thus captures whether ego behavior is realistic relative to the static scene (Guo et al., 8 Oct 2025). The interaction branch is pairwise: $s \in \mathcal{S}$2 Its inputs are the ego temporal feature, the neighbor temporal feature, and a relative positional encoding. Importantly, it does not receive all neighbors jointly (Guo et al., 8 Oct 2025).

The decomposed discriminator loss is

$s \in \mathcal{S}$3

The weights are distance-decayed: $s \in \mathcal{S}$4 with $s \in \mathcal{S}$5 and $s \in \mathcal{S}$6 (Guo et al., 8 Oct 2025).

The per-agent reward induced by this decomposition is

$s \in \mathcal{S}$7

The paper characterizes this as a factored and additive realism reward: scene reward plus weighted pairwise interaction rewards (Guo et al., 8 Oct 2025). The factorization is architectural and objective-level rather than theorem-driven. The paper explicitly states that the decomposition is an approximate causal or credit-assignment factorization enforced by architecture and input design rather than proved probabilistically (Guo et al., 8 Oct 2025).

A plausible implication is that DecompGAIL can be viewed as generalizing the shallow agent-wise factorization already present in MAGAIL. MAGAIL sums per-agent discriminator terms of the form $s \in \mathcal{S}$8 (Song et al., 2018), but it does not decompose state into ego–map and ego–neighbor factors. DecompGAIL therefore refines the unit of adversarial factorization from agents to ego-attributable realism factors (Guo et al., 8 Oct 2025).

4. Social PPO objective and policy optimization

DecompGAIL augments the decomposed ego reward with a neighborhood term to promote local collective realism. The paper defines the social reward as

$s \in \mathcal{S}$9

where $i$0 is distance-decayed analogously to $i$1,

$i$2

again with $i$3 (Guo et al., 8 Oct 2025). Nearby agents therefore influence each other’s training signal more strongly than distant agents.

The paper describes this as a social PPO objective. PPO is still run in an independent or decentralized way, but on socially augmented rewards rather than purely individual rewards (Guo et al., 8 Oct 2025). Training follows IPPO, advantages are estimated with GAE, and the value function is per agent (Guo et al., 8 Oct 2025). The motivation is that traffic realism is inherently interactive: an ego vehicle may increase its own realism score while inducing abrupt braking, collisions, or off-map behavior in nearby agents, and the social term counteracts that failure mode (Guo et al., 8 Oct 2025).

The algorithmic loop is an alternating adversarial procedure. The paper gives the following structure (Guo et al., 8 Oct 2025):

1. initialize policy from BC pretraining;
2. collect trajectories with $i$4 in the Markov game;
3. compute scene realism $i$5 and pairwise interaction realism $i$6;
4. update the discriminator with the BCE-style decomposed objective;
5. compute ego reward $i$7 and social reward $i$8;
6. estimate advantages and targets with GAE;
7. update $i$9 and $\mathcal{A}_i$0 using PPO with clipped surrogate and value loss.

The stability trick explicitly highlighted is the use of fresh rollouts at each training step followed by one discriminator update and one policy update (Guo et al., 8 Oct 2025). The paper states that this avoids stale adversarial data and helps stabilize training.

This optimization strategy contrasts with earlier PS-GAIL for highway driving, which used TRPO under a parameter-sharing decentralized policy with a shared critic and curriculum over agent count (Bhattacharyya et al., 2018, Bhattacharyya et al., 2020). DecompGAIL preserves parameter sharing and decentralized execution, but replaces the monolithic local realism signal with decomposed adversarial rewards and replaces the PS-TRPO training scheme with a social PPO formulation integrated into the SMART backbone (Guo et al., 8 Oct 2025).

5. Architecture, representations, and training pipeline

DecompGAIL is built on SMART, a token-based traffic model (Guo et al., 8 Oct 2025). The policy is tokenized and factorized: $\mathcal{A}_i$1 Each agent is predicted conditionally on shared scene context with a shared policy network (Guo et al., 8 Oct 2025).

The backbone uses a factorized Transformer with multi-head self-attention (MHSA) and multi-head cross-attention (MHCA). The map encoder computes encoded map tokens $\mathcal{A}_i$2. For each agent $\mathcal{A}_i$3 at time $\mathcal{A}_i$4, the motion encoder applies temporal attention,

$\mathcal{A}_i$5

map-agent attention,

$\mathcal{A}_i$6

and agent-agent attention,

$\mathcal{A}_i$7

After $\mathcal{A}_i$8 stacked layers, an MLP head predicts the next-token distribution (Guo et al., 8 Oct 2025).

The decomposed discriminator reuses intermediate backbone features. The scene discriminator is an MLP over $\mathcal{A}_i$9, and the interaction discriminator is an MLP over

$P$0

All MLPs use depth $P$1, width $P$2, and dropout $P$3 (Guo et al., 8 Oct 2025). The map encoder is shared between policy and discriminator and frozen during fine-tuning (Guo et al., 8 Oct 2025).

The training pipeline is reported as follows (Guo et al., 8 Oct 2025):

Stage	Description
BC pretraining	Train policy on expert trajectories for 32 epochs
Freeze encoder	Freeze shared map encoder during GAIL fine-tuning
Rollout	Collect fresh rollouts using current policy
Discriminator	Compute $P$4 and $P$5, then update BCE objective
RL update	Compute $P$6, $P$7, estimate GAE, update PPO/value
Fine-tuning	DecompGAIL fine-tuning for 2 epochs

Reported implementation details are: hardware $P$8 H800 80GB GPUs, total batch size $P$9, optimizer Adam, and weight decay $r_i$0 (Guo et al., 8 Oct 2025). Learning rates are $r_i$1 for policy pretraining, $r_i$2 for policy fine-tuning, and $r_i$3 for the discriminator (Guo et al., 8 Oct 2025). PPO hyperparameters are $r_i$4, clip $r_i$5, PPO epochs $r_i$6, PPO batch size $r_i$7, rollout length $r_i$8, GAE $r_i$9, and value loss weight $i$0 (Guo et al., 8 Oct 2025).

6. Empirical evaluation, ablations, and significance

Experiments use the Waymo Open Motion Dataset (WOMD), with 487k training scenes, 44k validation scenes, and 44k test scenes (Guo et al., 8 Oct 2025). Each scenario contains 1 second of history, 8 seconds of future, sampled at 10 Hz, with up to 128 traffic participants and HD maps (Guo et al., 8 Oct 2025). Evaluation follows prior work and uses 2% of the validation split, namely 880 of 44,097 validation scenarios (Guo et al., 8 Oct 2025). The primary benchmark target is the WOMD Sim Agents 2025 / WOSAC 2025 setup (Guo et al., 8 Oct 2025).

The official evaluation metrics include kinematic, interactive, and map-based weighted likelihoods, as well as a realism meta-metric and minADE (Guo et al., 8 Oct 2025). On the WOSAC 2025 leaderboard test split, the paper reports the following table (Guo et al., 8 Oct 2025):

Model	Metametric $i$1	Interactive $i$2
SMART-tiny-DecompGAIL	0.7864	0.8152
SMART-R1	0.7858	0.8110
SMART-tiny-RLFTSim	0.7857	0.8129
SMART-tiny	0.7814	0.8089

The reported main takeaways are best overall realism meta-metric, best interactive score, competitive kinematic and map-based scores, and worse minADE than some baselines, which the authors attribute to optimizing distributional realism rather than pointwise trajectory distance (Guo et al., 8 Oct 2025).

The stability results directly test the irrelevant interaction misguidance hypothesis. Comparing DecompGAIL against PS-GAIL with 5 nearest neighbors, 10 nearest neighbors, and all nearest neighbors within 60 m, the paper reports that PS-GAIL variance increases with more neighbors, mean discriminator score drops as neighbor count grows, and simulation realism degrades, whereas DecompGAIL maintains lower variance and discriminator scores near expected equilibrium $i$3, with validation realism improving steadily (Guo et al., 8 Oct 2025). This is presented as direct evidence that the instability worsens as neighborhood size increases.

The ablation study on the WOSAC 2% validation split further isolates the method’s components (Guo et al., 8 Oct 2025):

Model	Metametric $i$4	Interactive $i$5	Collision Likelihood $i$6
w/o DecompGAIL	0.7836	0.8204	0.9667
w/o scene realism	0.7801	0.8248	0.9794
w/o interact realism	0.7772	0.8132	0.9573
w/o neighborhood reward	0.7871	0.8258	0.9788
DecompGAIL	0.7889	0.8283	0.9837

The paper interprets these ablations as follows (Guo et al., 8 Oct 2025). Removing DecompGAIL fine-tuning hurts realism and collision likelihood, which supports the claim that BC alone suffers from rollout covariate shift. Removing scene realism strongly hurts the map-based metric, indicating that ego–map realism is important for road compliance. Removing interaction realism lowers the interactive metric and sharply lowers collision likelihood, which is described as strong evidence that explicit ego–neighbor modeling is essential. Replacing distance-decayed aggregation by mean aggregation degrades performance both for interaction realism and neighborhood reward. Removing the social reward lowers performance slightly, so the social PPO term contributes, though less dramatically than decomposition.

A plausible implication is that the decomposition primarily improves credit assignment, while the social term provides a secondary regularization of local collective behavior. This interpretation is aligned with the paper’s own emphasis that the main technical insight is not generic multi-agent non-stationarity, but discriminator structure that matches the causal relevance of ego-centered driving interactions (Guo et al., 8 Oct 2025).

7. Relation to predecessor methods, limitations, and open directions

DecompGAIL belongs to a progression of multi-agent adversarial imitation methods rather than standing in isolation. MAGAIL generalized GAIL to general Markov games, developed a multi-agent IRL formulation tied to Nash equilibrium, and introduced practical adversarial imitation algorithms under centralized, decentralized, and zero-sum reward-structure priors (Song et al., 2018). Its decentralized version already used a sum of per-agent discriminator terms,

$i$7

but this remained a shallow decomposition at the agent level (Song et al., 2018). DecompGAIL can therefore be understood as refining agent-wise adversarial factorization into scene and pairwise interaction terms that are more tightly aligned with ego credit assignment (Guo et al., 8 Oct 2025).

For driving specifically, PS-GAIL addressed the failure of single-agent GAIL under multi-agent deployment by training a shared decentralized policy across many interacting vehicles, using parameter sharing and curriculum learning (Bhattacharyya et al., 2018, Bhattacharyya et al., 2020). That line of work showed that imitation learning for driving must train under the same interactive multi-agent distribution in which it will be deployed, and that parameter sharing over homogeneous agents is an effective baseline (Bhattacharyya et al., 2018, Bhattacharyya et al., 2020). DecompGAIL inherits the parameter-sharing decentralized policy paradigm, but argues that PS-GAIL’s local discriminator remains vulnerable to entangled neighborhood artifacts and therefore unstable as neighbor count increases (Guo et al., 8 Oct 2025).

The paper does not dwell extensively on limitations, but several are made explicit or implicit (Guo et al., 8 Oct 2025). The decomposition intentionally drops higher-order interactions, so some genuine multi-agent coordination patterns involving more than two agents may be missed. The factorization is heuristic and architectural rather than theoretically proved optimal. Gains over the strongest baselines on the leaderboard are numerically modest, though consistent. MinADE is not improved, suggesting a focus on realism distributions rather than exact trajectory matching. The method is demonstrated specifically within a SMART tokenized backbone, and generality to substantially different architectures is described as plausible but not empirically established.

An open question raised by the method itself is whether richer decompositions can preserve the stability benefits of ego-attributable realism while recovering selected higher-order coordination structure. This suggests a natural research direction toward decomposed occupancy measures, interaction-factor discriminators, or graph-structured multi-agent adversarial objectives. Earlier driving work had already proposed explicitly modeling interactions in a centralized manner via Graph Neural Networks and coordination graphs as future work beyond PS-GAIL (Bhattacharyya et al., 2020). DecompGAIL’s results suggest that any such extension would need to preserve the causal alignment of reward with ego-attributable behavior in order to avoid reintroducing irrelevant interaction misguidance.

In summary, DecompGAIL is a multi-agent GAIL variant for traffic simulation whose central contribution is an architectural and objective-level decomposition of realism into ego–map and ego–neighbor components, coupled with a social PPO objective over distance-weighted local neighborhoods (Guo et al., 8 Oct 2025). Its importance lies less in changing the basic adversarial imitation-learning paradigm than in specifying which parts of local traffic context should and should not influence ego reward. Within the multi-agent GAIL lineage, it represents a move from agent-level factorization toward credit-assignment-aware realism decomposition.