MultiPark: Multi-Agent Parking Prediction

• MultiPark is a multi-agent automated parking formulation that jointly models high-level ego maneuver selection and surrounding agent responses.
• It integrates discrete intention tokenization, conditional joint trajectory prediction, and safety-guided denoising to refine predictions in unstructured parking scenes.
• The framework enables what-if reasoning and counterfactual knowledge distillation, supporting planners in evaluating multiple ego options under safety constraints.

MultiPark denotes a multi-agent automated parking formulation in which high-level ego maneuver selection and the joint future motions of surrounding agents are modeled as a coupled prediction problem rather than as separate tasks. In the setting aligned with "ParkDiffusion++: Ego Intention Conditioned Joint Multi-Agent Trajectory Prediction for Automated Parking using Diffusion Models," MultiPark consists of ego intention prediction, ego-intention-conditioned joint trajectory prediction, and what-if reasoning over alternative maneuvers, with the goal of supporting upstream planning in dense, unstructured parking scenes (Wei et al., 24 Feb 2026).

1. Operational setting

In the MultiPark formulation, the ego vehicle is agent $0$ with past trajectory $\mathbf{x}_{0,1:T_p}$, where each state $\mathbf{x}_{i,t}\in\mathbb{R}^D$ includes position, heading, velocity, and acceleration. The scene also contains $N-1$ surrounding agents, including vehicles and pedestrians, with states $\mathbf{x}_{i,1:T_p}$ for $i=1,\dots,N-1$, agent category $c_i$, and size $\mathbf{s}_i$. The environment is represented in an ego-centric frame over a short horizon with $T_p = 10$ past steps and $T_f = 10$ future steps at $\mathbf{x}_{0,1:T_p}$0 s per step, corresponding to a $\mathbf{x}_{0,1:T_p}$1 s horizon (Wei et al., 24 Feb 2026).

The map representation is vectorized and includes parking slots, drivable boundaries as soft constraints $\mathbf{x}_{0,1:T_p}$2, and static obstacles as hard constraints $\mathbf{x}_{0,1:T_p}$3. Typical interactions in these scenes include vehicles searching for slots, stopping, reversing, and turning sharply; yielding to oncoming vehicles in narrow aisles; avoiding pedestrians crossing between cars; and negotiating with vehicles entering or exiting slots.

The paper explicitly maps this problem to a "MultiPark"-style framework. In that interpretation, ego intention prediction provides the high-level decision of where or how the ego will maneuver, while ego-conditioned joint trajectory prediction models, for each candidate ego intention $\mathbf{x}_{0,1:T_p}$4, the conditional distribution

$\mathbf{x}_{0,1:T_p}$5

where $\mathbf{x}_{0,1:T_p}$6 denotes the joint future of all agents. This arrangement enables what-if reasoning for planning: for each candidate ego maneuver, the system predicts how surrounding agents respond.

2. Multi-agent difficulty in parking environments

Automated parking differs materially from lane-following highway or urban driving because the geometry is unstructured, free space is limited, and interactions occur at close range. The paper identifies weak or absent lane semantics, arbitrary travel angles, and even opposite-direction motion in aisles as characteristic features of parking scenes (Wei et al., 24 Feb 2026).

The interaction regime is also unusually dense. Vehicles may wait behind others that are maneuvering into or out of slots, temporarily block one another at very low speed, or negotiate passage through narrow aisles. Pedestrians can appear between parked cars, and vehicles can be partially hidden by obstacles. These factors increase the salience of occlusion, local uncertainty, and mixed-agent interaction.

Intent complexity is central. In parking, the ego may continue searching, turn into a particular slot, pass another vehicle, or wait and yield. Because these choices affect how other agents decelerate, stop, deviate, or proceed, multi-agent modeling becomes necessary at the scene level. The paper’s core claim is that modeling ego intention and others’ reactions independently is inadequate because it can break social consistency or miss the coupling between maneuver choice and scene response.

Within this framing, MultiPark is not merely trajectory forecasting in a parking lot. It is a conditional, interactive, scene-level prediction problem in which the planner must compare multiple ego options and assess their downstream social and safety consequences.

3. Ego intention tokenization

The MultiPark formulation in ParkDiffusion++ begins with a discrete ego intention tokenizer trained from agent histories and vectorized map polylines. The scene input is

$\mathbf{x}_{0,1:T_p}$7

together with scene context $\mathbf{x}_{0,1:T_p}$8, which includes agent attributes and the two map streams $\mathbf{x}_{0,1:T_p}$9 and $\mathbf{x}_{i,t}\in\mathbb{R}^D$0 (Wei et al., 24 Feb 2026).

Scene encoding proceeds by applying temporal $\mathbf{x}_{i,t}\in\mathbb{R}^D$1D convolution and a GRU to each agent history, followed by a Transformer over agents for interaction, yielding an agent feature $\mathbf{x}_{i,t}\in\mathbb{R}^D$2. A type-aware modulation produces $\mathbf{x}_{i,t}\in\mathbb{R}^D$3. The ego feature $\mathbf{x}_{i,t}\in\mathbb{R}^D$4 cross-attends separately to the soft and hard map streams, and pooled outputs produce a global map summary $\mathbf{x}_{i,t}\in\mathbb{R}^D$5. Social context is defined by max-pooling over non-ego agent features,

$\mathbf{x}_{i,t}\in\mathbb{R}^D$6

These components are fused into a holistic scene context vector

$\mathbf{x}_{i,t}\in\mathbb{R}^D$7

The tokenizer uses learned mode embeddings

$\mathbf{x}_{i,t}\in\mathbb{R}^D$8

For each mode $\mathbf{x}_{i,t}\in\mathbb{R}^D$9, the model forms

$N-1$0

then predicts a continuous endpoint token

$N-1$1

and a mode logit

$N-1$2

The logits define a categorical intention distribution

$N-1$3

This tokenization maps continuous scene observations into a small set of discrete intention indices, each associated with a learned endpoint and a probability. In experiments, $N-1$4 gives the best trade-off for DLP, and the tokens roughly correspond to distinct maneuver classes such as continuing in the aisle or turning into a left or right slot.

Training uses a winner-takes-all loss. The winning token is the token whose predicted endpoint is closest to the ground-truth final ego position. Stage 1 combines SmoothL1 endpoint regression, cross-entropy mode selection, and a diversity regularizer: $N-1$5 Stage 2 freezes the tokenizer and uses the ground-truth token for supervised training and non-ground-truth tokens for counterfactual training.

4. Ego-intention-conditioned joint trajectory prediction

The second stage of the MultiPark pipeline predicts joint multi-agent futures conditioned on a chosen ego intention token. The encoder reuses the Stage-1 scene representation and produces per-agent features $N-1$6, the social feature $N-1$7, and the map summary $N-1$8 (Wei et al., 24 Feb 2026).

Conditioning is performed by embedding the selected endpoint token $N-1$9 into $\mathbf{x}_{i,1:T_p}$0 and modulating each agent representation with FiLM. The base feature is

$\mathbf{x}_{i,1:T_p}$1

and the conditioned latent is

$\mathbf{x}_{i,1:T_p}$2

where $\mathbf{x}_{i,1:T_p}$3 and $\mathbf{x}_{i,1:T_p}$4 are small MLPs. This mechanism injects the same ego intention into every agent representation so that non-ego predictions become explicitly dependent on the ego’s chosen maneuver.

To emphasize agents most likely to be affected by the ego’s path, the model introduces an exposure scalar $\mathbf{x}_{i,1:T_p}$5. Let $\mathbf{x}_{i,1:T_p}$6 be the line segment from the ego’s last position to the intention endpoint $\mathbf{x}_{i,1:T_p}$7, and let $\mathbf{x}_{i,1:T_p}$8 be the last position of agent $\mathbf{x}_{i,1:T_p}$9. The paper defines

$i=1,\dots,N-1$0

with learnable scalars $i=1,\dots,N-1$1, $i=1,\dots,N-1$2, $i=1,\dots,N-1$3, and $i=1,\dots,N-1$4. A small gating network then uses $i=1,\dots,N-1$5 to modulate $i=1,\dots,N-1$6, reducing sensitivity to geometrically irrelevant agents.

The decoder generates $i=1,\dots,N-1$7 future trajectories per agent,

$i=1,\dots,N-1$8

together with scores. Because exhaustive joint composition would require evaluating $i=1,\dots,N-1$9 combinations, the model first retains the top-$c_i$0 marginals for each agent, then uses beam search to assemble $c_i$1 joint candidates

$c_i$2

A scene selector scores these assembled joint scenes using pooled scene features and a softmax over scene logits.

The paper defines “socially consistent” predictions as coherent scene-level realizations in which all agents’ trajectories are mutually compatible: no impossible collisions, no contradictory occupancy of the same space at the same time, and interaction patterns such as yielding, waiting, and following realized at the joint-scene level. In the MultiPark interpretation, this scene selector is the mechanism that converts per-agent marginals into a feasible interactive future under a shared ego plan.

5. Safety-guided denoising and geometric constraints

A central component of the MultiPark formulation is the safety-guided denoiser (SGD), which refines predicted joint scenes toward collision-free, map-compliant, and smooth trajectories. The denoiser is pretrained as a Leapfrog-style diffusion denoiser $c_i$3. Given a clean trajectory $c_i$4, Gaussian noise $c_i$5, and noise scale $c_i$6, the noisy sample is

$c_i$7

and the denoiser is trained with

$c_i$8

This is described as implicitly learning the score $c_i$9 of the trajectory distribution (Wei et al., 24 Feb 2026).

During Stage 2, the denoiser is frozen and used as a refinement operator. Refinement first projects the trajectory toward the learned data manifold,

$\mathbf{s}_i$0

then applies geometric guidance,

$\mathbf{s}_i$1

for $\mathbf{s}_i$2 refinement steps.

The differentiable potential $\mathbf{s}_i$3 is a weighted sum of five terms: agent-agent overlap $\mathbf{s}_i$4, obstacle clearance $\mathbf{s}_i$5, ego path-tube $\mathbf{s}_i$6, ego endpoint anchoring $\mathbf{s}_i$7, and motion smoothness $\mathbf{s}_i$8. The total potential is

$\mathbf{s}_i$9

These handcrafted geometric potentials impose explicit physical and map-based constraints on predicted joint scenes.

In Stage 2 supervised training, the student decoder predicts $T_p = 10$0, the denoiser refines it to $T_p = 10$1, and the loss combines a supervised term, a consistency term against the stop-gradient refined prediction, and a differentiable collision penalty: $T_p = 10$2

The paper states that, in inference, the selected intention token is fed into $T_p = 10$3 to directly output joint trajectories, and refinement via the denoiser is primarily a training-time tool. Quantitatively, the denoiser is associated with a strong reduction in overlap rate: on DLP, OR drops from $T_p = 10$4 for ParkDiffusion$T_p = 10$5 to $T_p = 10$6 after adding SGD, and to $T_p = 10$7 with full ParkDiffusion++.

6. Counterfactual knowledge distillation

Counterfactual knowledge distillation (CKD) addresses a structural supervision problem in MultiPark: datasets provide labels only for the realized ego trajectory, not for alternative ego intentions that were available but not taken. CKD uses an EMA teacher decoder together with the frozen safety-guided denoiser to generate pseudo-target joint futures for counterfactual ego intentions, and trains the student decoder to match them (Wei et al., 24 Feb 2026).

The teacher decoder $T_p = 10$8 is updated by

$T_p = 10$9

For a counterfactual token $T_f = 10$0 with $T_f = 10$1, the teacher produces

$T_f = 10$2

while the student predicts

$T_f = 10$3

The distillation loss is

$T_f = 10$4

The full Stage-2 loss is

$T_f = 10$5

where $T_f = 10$6. Best results are achieved at $T_f = 10$7.

Within the MultiPark interpretation, CKD is the mechanism that endows the predictor with what-if capability despite the absence of explicit counterfactual labels. It allows the model to learn how other agents would react if the ego chose a different slot, route, or waiting behavior. The paper’s formulation suggests a broader principle: safety-refined pseudo-targets can serve as a substitute supervisory signal for unobserved interactive futures.

7. Evaluation, qualitative behavior, and research outlook

The empirical evaluation uses two datasets. Dragon Lake Parking (DLP) is a real-world parking-lot dataset containing dense, unstructured parking scenes with vehicles maneuvering in aisles, entering and exiting slots, and pedestrians moving through the lot. The training and validation split is $T_f = 10$8 and $T_f = 10$9 samples, respectively. Intersections Drone (inD) is a drone-based dataset of German intersections, evaluated on the Bendplatz and Frankenburg locations with $\mathbf{x}_{0,1:T_p}$00 training samples and $\mathbf{x}_{0,1:T_p}$01 validation samples. Although inD is not a parking dataset, it is used for complementary testing of dense, low-speed interactive behavior with vehicles and pedestrians (Wei et al., 24 Feb 2026).

The evaluation protocol reports oracle and final metrics over $\mathbf{x}_{0,1:T_p}$02 predicted joint scenes. Oracle metrics use the best scene among the candidates and include minADE, minFDE, minMR, minOR, and mAP. Final metrics evaluate the top-1 selected scene and include f-ADE, f-FDE, f-MR, f-OR, and f-mAP. MR is a scene-level miss rate with an FDE threshold of $\mathbf{x}_{0,1:T_p}$03 m for all valid agents. OR counts a scene as overlapping if any vehicle-vehicle, vehicle-pedestrian, or vehicle-obstacle collision occurs. mAP is defined through a precision-recall criterion over joint candidates, with correctness again determined by all valid agents having FDE $\mathbf{x}_{0,1:T_p}$04 m.

On DLP, ParkDiffusion++ achieves oracle metrics of minADE $\mathbf{x}_{0,1:T_p}$05 m, minFDE $\mathbf{x}_{0,1:T_p}$06 m, minMR $\mathbf{x}_{0,1:T_p}$07, minOR $\mathbf{x}_{0,1:T_p}$08, and mAP $\mathbf{x}_{0,1:T_p}$09, all reported as best. Its final metrics are f-ADE $\mathbf{x}_{0,1:T_p}$10 m, f-FDE $\mathbf{x}_{0,1:T_p}$11 m, f-MR $\mathbf{x}_{0,1:T_p}$12, f-OR $\mathbf{x}_{0,1:T_p}$13, and f-mAP $\mathbf{x}_{0,1:T_p}$14, with f-ADE second-best and the others best. On inD, the model attains oracle minADE $\mathbf{x}_{0,1:T_p}$15 m, minFDE $\mathbf{x}_{0,1:T_p}$16 m, minMR $\mathbf{x}_{0,1:T_p}$17, minOR $\mathbf{x}_{0,1:T_p}$18, and mAP $\mathbf{x}_{0,1:T_p}$19; the corresponding final metrics are f-ADE $\mathbf{x}_{0,1:T_p}$20 m, f-FDE $\mathbf{x}_{0,1:T_p}$21 m, f-MR $\mathbf{x}_{0,1:T_p}$22, f-OR $\mathbf{x}_{0,1:T_p}$23, and f-mAP $\mathbf{x}_{0,1:T_p}$24. The baselines are WIMP, SceneTransformer, ScePT, MotionLM, DTPP, and ParkDiffusion$\mathbf{x}_{0,1:T_p}$25, all using the same vectorized-map frontend.

Ablation results show a monotonic improvement across ParkDiffusion$\mathbf{x}_{0,1:T_p}$26, $\mathbf{x}_{0,1:T_p}$27Joint Selector, $\mathbf{x}_{0,1:T_p}$28Safety-Guided Denoiser, and $\mathbf{x}_{0,1:T_p}$29CKD. On DLP, oracle minADE improves from $\mathbf{x}_{0,1:T_p}$30 to $\mathbf{x}_{0,1:T_p}$31 to $\mathbf{x}_{0,1:T_p}$32 to $\mathbf{x}_{0,1:T_p}$33; minFDE improves from $\mathbf{x}_{0,1:T_p}$34 to $\mathbf{x}_{0,1:T_p}$35 to $\mathbf{x}_{0,1:T_p}$36 to $\mathbf{x}_{0,1:T_p}$37; and OR decreases from $\mathbf{x}_{0,1:T_p}$38 to $\mathbf{x}_{0,1:T_p}$39 to $\mathbf{x}_{0,1:T_p}$40 to $\mathbf{x}_{0,1:T_p}$41. Tokenizer ablations indicate that $\mathbf{x}_{0,1:T_p}$42 is preferable to $\mathbf{x}_{0,1:T_p}$43, $\mathbf{x}_{0,1:T_p}$44, or $\mathbf{x}_{0,1:T_p}$45, and that Stage-1 ranking is the best source of counterfactual tokens. CKD ablations show that $\mathbf{x}_{0,1:T_p}$46 is optimal and that teacher variants improve in the order None $\mathbf{x}_{0,1:T_p}$47 EMA only $\mathbf{x}_{0,1:T_p}$48 EMA $\mathbf{x}_{0,1:T_p}$49 denoiser (unguided) $\mathbf{x}_{0,1:T_p}$50 EMA $\mathbf{x}_{0,1:T_p}$51 SGD.

Qualitative visualizations in Figure 1 show that when the ego uses the most likely intention from the tokenizer, the predicted joint trajectories align with ground truth for all agents; when the ego is assigned alternative predicted intentions, the model generates different non-ego behaviors, including vehicles yielding or deviating and pedestrians adjusting motion. The authors note that the model is somewhat conservative, with vehicles and pedestrians sometimes yielding or slowing more than necessary. This is consistent with the strong safety potentials and collision penalties used in training.

For a MultiPark system, the paper positions ParkDiffusion++ as a core prediction module between perception and planning. Stage 1 yields discrete intention tokens, endpoints, and intention probabilities that can serve as a candidate goal set for a planner. Stage 2 yields joint scene samples and scene scores for each candidate intention, enabling multi-intention, multi-scene simulation and downstream risk evaluation. The paper also identifies several limitations: reliance on handcrafted geometric potentials, the absence of joint optimization with planning and control, the lack of closed-loop evaluation, conservative behavior induced by safety penalties, and unresolved questions about scalability to very large parking structures, significantly different layouts, complex multi-level garages, and richer agent taxonomies. A plausible implication is that a full MultiPark benchmark or system would require explicit integration of planning, closed-loop simulation, and a more adaptive trade-off between safety and efficiency.