PlayGen-MoG: Framework for Diverse Multi-Agent Play Generation via Mixture-of-Gaussians Trajectory Prediction

Abstract: Multi-agent trajectory generation in team sports requires models that capture both the diversity of possible plays and realistic spatial coordination between players on plays. Standard generative approaches such as Conditional Variational Autoencoders (CVAE) and diffusion models struggle with this task, exhibiting posterior collapse or convergence to the dataset mean. Moreover, most trajectory prediction methods operate in a forecasting regime that requires multiple frames of observed history, limiting their use for play design where only the initial formation is available. We present PlayGen-MoG, an extensible framework for formation-conditioned play generation that addresses these challenges through three design choices: 1/ a Mixture-of-Gaussians (MoG) output head with shared mixture weights across all agents, where a single set of weights selects a play scenario that couples all players' trajectories, 2/ relative spatial attention that encodes pairwise player positions and distances as learned attention biases, and 3/ non-autoregressive prediction of absolute displacements from the initial formation, eliminating cumulative error drift and removing the dependence on observed trajectory history, enabling realistic play generation from a single static formation alone. On American football tracking data, PlayGen-MoG achieves 1.68 yard ADE and 3.98 yard FDE while maintaining full utilization of all 8 mixture components with entropy of 2.06 out of 2.08, and qualitatively confirming diverse generation without mode collapse.

• The paper introduces a mixture-of-Gaussians framework that generates coordinated multi-agent trajectories non-autoregressively, ensuring both diversity and coherence.
• It incorporates novel formation encoding alongside relative spatial and bidirectional temporal attention to condition play generation on initial football formations.
• Empirical results on NFL tracking data demonstrate significant gains in accuracy and diversity over CVAE and diffusion-based approaches.

PlayGen-MoG: Mixture-of-Gaussians-Based Multi-Agent Trajectory Generation for Play Design

Problem Setting and Motivation

Multi-agent trajectory generation in team sports, particularly American football, presents a uniquely structured generative challenge: one must synthesize coordinated, physically plausible trajectories for all players, from only a single static initial formation, without the benefit of observed trajectory history. Existing approaches—such as CVAEs and diffusion models—struggle to jointly realize diversity and coherence. CVAE-based models typically collapse to a single mode, while diffusion approaches often yield incoherent motion, both of which are inadequate for practical play design, tactical analysis, and coaching.

PlayGen-MoG introduces a framework that directly addresses these challenges by leveraging a Mixture-of-Gaussians (MoG) head with shared mixture weights across all agents, introducing explicit coupling of player outcomes to distinct play scenarios. The model is explicitly formation-conditioned and operates in a non-autoregressive regime, predicting full trajectory displacements from the initial formation in a single forward pass.

Architectural Innovations

The PlayGen-MoG architecture is comprised of several interlocking components designed to achieve both diversity and realism in generative multi-agent settings:

• Formation Encoder: Utilizes a transformer layer with self-attention across all players, encoding spatial and role-based information into conditioning vectors that remain stable throughout trajectory prediction.
• Relative Spatial Attention: Augments standard attention with learned biases based on pairwise player positions and distances, enabling the model to internally specialize for various spatial relationships (e.g., proximity, downfield coverage).
• Bidirectional Temporal Attention: Rather than leveraging autoregressive or causal masking, the architecture allows information to flow freely across time, giving access to global frame-level context and facilitating coordinated, globally plausible trajectories.
• MoG Output Head: A single set of mixture weights, shared across agents and timesteps, determines the play scenario from which each agent’s trajectory is sampled (see Figure 1).

  Figure 1: Model architecture overview with formation encoding, relative spatial and temporal attention, and shared mixture-of-Gaussians output head for coordinated play generation.

This explicit play scenario mechanism ensures that sampled trajectories for all agents reflect a common, coordinated outcome, as opposed to degenerate averages or uncorrelated random draws.

Generation Dynamics and Training Strategies

The model is tasked with producing absolute displacements from the initial formation, thereby eliminating the compounding error drift endemic to frame-to-frame delta prediction. Each timestep's prediction is independent, benefitting from sinusoidal step embeddings and cross-timestep attention for temporal coherence.

Noteworthy aspects of the training procedure and objective include:

• Negative Log-Likelihood (NLL) of the MoG: The core criterion, treating the entire team and sequence jointly rather than as a collection of decoupled agents.
• Best-Component ADE Loss: Prioritizes accuracy of the most probable mixture component, sharpening spatial fidelity.
• Entropy Regularization: Maximizes the entropy of the mixture weights, explicitly guarding against mode collapse and ensuring all mixture components are active within the training regime.
• Temporal Smoothness Loss: Penalizes frame-to-frame discontinuities in means, forcing the model to learn physically plausible, continuous displacements.

At inference, trajectory generation for a given formation is controlled by sampling the mixture component from the time-averaged weights, guaranteeing coherent, realistic diversity aligned with actual football play concepts.

Empirical Findings

Experiments on NFL tracking data (2021–2022, 9,934 plays) demonstrate several key results. The non-autoregressive absolute-displacement model achieves substantial gains in both accuracy and diversity compared to autoregressive baselines and latent-variable diffusion methods:

Method	ADE (yds)	FDE (yds)	Entropy	APD (yds)
PlayGen-MoG (M=8)	1.68	3.98	2.06/2.08	1.13
CVAE	2.88	6.56	—	0.10
LED (diffusion)	1.34	3.14	—	—

PlayGen-MoG uniquely maintains full mixture utilization, with entropy near the theoretical maximum for $M=8$, and produces visibly distinct, coordinated play scenarios, as confirmed both quantitatively and qualitatively.

Figure 2: Generation of three distinct, realistic play concepts per formation; samples are diverse and coordinated.

Importantly, the APD metric (average pairwise diversity across samples for a fixed formation) demonstrates that increased mixture cardinality correlates strongly with meaningful diversity, particularly for receiver and quarterback roles. APD increases monotonically with mixture size, confirming that each component codes for a distinct play variant.

Figure 3: Qualitative comparison—CVAE shows posterior collapse, LED generates incoherent trajectories, while PlayGen-MoG yields diverse, realistic plays.

Further, PlayGen-MoG’s ADE remains stable across horizon lengths, and diversity naturally unfolds over time as routes diverge (see Figure 4).

Figure 4: Progressive formation of routed play concepts across time horizons, with trajectories separating as routes develop.

Implications and Future Directions

The PlayGen-MoG framework introduces a robust and computationally efficient paradigmatic shift for multi-agent generative modeling in structured settings:

• Practical Impact: Direct applicability to play design and tactical analytics in American football, supporting rapid exploration of play concepts from canonical personnel groupings solely from the initial static alignment.
• Scalability: The shared mixture-head design avoids quadratic complexity associated with full cross-agent covariance models, thus remaining tractable even as team size or trajectory length increases.
• Versatility: While currently demonstrated on offensive football data, the architecture is sport-agnostic in principle, with extensions possible for set-plays in other structured sports.
• Extension Paths: Conditional generation on text labels or defensive alignments, as well as joint modeling of both offensive and defensive agents, could enable rich counterfactual “what-if” queries in competitive analytics.

The findings that continuous latent spaces are insufficient for discrete play distribution, and that eliminating autoregressive error propagation is critical for long-range coherence, both have broader ramifications for trajectory generation tasks across other multi-agent or robotic domains.

Conclusion

PlayGen-MoG establishes a new model class for diverse, coordinated multi-agent generative modeling under minimal conditioning. The explicit mixture mechanism, relative spatial relation modeling, and non-autoregressive displacement prediction together deliver both high-fidelity and high-diversity trajectory samples, overcoming core pathologies of prior approaches. The framework’s modularity and empirical rigor provide a strong foundation for both immediate deployment in sports analytics and further research in multi-agent generative modeling.