Dino-Diffusion Parking: Modular Zero-Shot Planner

• Dino-Diffusion Parking (DDP) is a modular, vision-only autonomous parking system that integrates a frozen DINOv2 backbone, BEV representation, diffusion-based planning in SE(2), and a Stanley controller for trajectory tracking.
• DDP employs a goal-conditioned BEV intermediate representation with dynamic target relabeling to mitigate the compounding errors common in end-to-end autonomous parking approaches.
• Empirical results demonstrate that DDP delivers over 90% parking success under diverse conditions, highlighting its effective zero-shot generalization across varying domains.

Searching arXiv for the cited DDP paper and a few directly related papers for accurate citation support. Dino-Diffusion Parking (DDP) is a modular, vision-only autonomous parking system designed explicitly for cross-domain generalization. It combines a frozen visual foundation model, specifically DINOv2, for perception, a BEV-based intermediate representation with target relabeling, a diffusion model for trajectory planning in $SE(2)$, and a classical Stanley controller for trajectory tracking. In the reported formulation, the system is trained in CARLA at regular setting and transferred in a zero-shot fashion to more adversarial settings, including weather and lighting shifts and a 3D Gaussian splatting environment reconstructed from a real parking lot, without additional labeled data or fine-tuning (Wu et al., 23 Oct 2025).

1. Definition and problem setting

DDP was introduced to address a specific weakness of recent end-to-end autonomous parking systems: strong in-domain performance paired with substantial degradation under distribution shifts. The motivating shifts include different weather and lighting, changes in visual appearance such as textures and parked vehicle types, and sim-to-real or sim-to-3DGS transfer. In parking, this sensitivity is particularly consequential because motion is low speed but highly constrained, and small perception or planning errors can quickly lead to collisions or failure in tight spaces (Wu et al., 23 Oct 2025).

The central design objective is zero-shot cross-domain generalization for parking without collecting additional data in the target domains. The method defines “domain-agnostic” in operational rather than purely representational terms: the perception features and planning behavior are intended not to depend on specific appearance statistics of the training domain, and the same model trained only in CARLA with a default “sunny-noon” configuration is deployed zero-shot under other CARLA weather and lighting regimes and in a 3D Gaussian Splatting environment reconstructed from a real parking lot (Wu et al., 23 Oct 2025).

This formulation stands in contrast to conventional end-to-end systems that map raw images directly to control or waypoints and are typically trained and evaluated in a single domain. The paper attributes part of their fragility to behavior-cloned end-to-end policies suffering from compounding error: small deviations move the vehicle into unseen states where the policy becomes unreliable (Wu et al., 23 Oct 2025). A plausible implication is that DDP should be read not merely as a new planner, but as a decomposition strategy for mitigating coupled perception-policy overfitting.

2. Architectural composition

DDP is organized as a modular, cascaded pipeline with four stages: perception, target fusion with relabeling during training, diffusion-based local planning, and classical control. The input is multi-view RGB camera imagery from a surround-view CARLA parking setup. The perception backbone is DINOv2 ViT-B/14 (“distilled”, approximately 86M parameters), used in frozen form. Its outputs are a multi-view BEV feature map $F_b$ in ego coordinates and a BEV segmentation map $F_s$ encoding target slot, occupied slots, and free space (Wu et al., 23 Oct 2025, Oquab et al., 2023).

The target pose $P_{tgt}$, corresponding to the user-specified parking slot pose, is not visible in the images. DDP therefore fuses $P_{tgt}$ into the BEV representation via cross-attention and FiLM conditioning. During training, hindsight relabeling perturbs $P_{tgt}$ and edits the segmentation label, augmenting the training set with alternative target slots. The planner then consumes the fused BEV embedding $F_b$, segmentation $F_s$, and target pose to generate an ego-centric $SE(2)$ trajectory

$\tau = \{x_i, y_i, \theta_i\}_{i=1}^{N_w}.$

Trajectory execution is handled by a Stanley controller that minimizes heading and cross-track errors, with throttle and brake governed by simple rules (Wu et al., 23 Oct 2025).

A defining feature of the system is decoupling at two interfaces. First, gradients do not flow from the diffusion planner back into the BEV and visual foundation model perception module; the perception network is trained with supervised losses and the planner is subsequently trained on frozen BEV features. Second, planning is performed in trajectory space while low-level execution is delegated to the controller. The stated intent is to prevent perception from overfitting to specific action distributions and to allow the planner to focus on geometrically valid paths rather than control-level tracking (Wu et al., 23 Oct 2025).

3. Perception, BEV generation, and target conditioning

The perception backbone is the frozen DINOv2 visual foundation model, formalized as

$F_b$0

where $F_b$1 is the stack of $F_b$2 camera images and $F_b$3 is the feature for the $F_b$4-th patch of camera $F_b$5. These features are post-processed and paired with a learned depth prediction head: $F_b$6 They are then projected into bird’s-eye view by a Lift-Splat-Shoot-style transform using camera intrinsics and extrinsics: $F_b$7 The resulting BEV feature map is the basis for both semantic segmentation and goal-conditioned planning (Wu et al., 23 Oct 2025, Philion et al., 2020).

The target-conditioning procedure has two parts. During training, if $F_b$8, the target pose is perturbed,

$F_b$9

and the segmentation label is relabeled accordingly,

$F_s$0

This hindsight relabeling is explicitly intended to prevent memorization of frequently used slots or expert parking patterns. The perturbed or original target is then fused into the BEV via cross-attention and FiLM: $F_s$1

$F_s$2

A segmentation head $F_s$3 finally produces

$F_s$4

with labels for target slot, occupied slots, and background (Wu et al., 23 Oct 2025).

The reported robustness analysis assigns substantial importance to this perception design. DDP’s features show an approximately 70% reduction in domain gap at the post-backbone stage relative to an EfficientNet-based baseline, alongside approximately 36% smaller pre-BEV gap, approximately 6% smaller depth gap, and approximately 56% smaller BEV gap under the reported RMSE feature-distance analysis. UMAP visualizations are described as showing DINOv2 features clustering consistently across domains, whereas EfficientNet features scatter under domain shifts (Wu et al., 23 Oct 2025). This suggests that the system’s cross-domain behavior is not attributed solely to diffusion planning; it depends materially on representation stability induced by the frozen visual foundation model.

4. Diffusion planning in $F_s$5

DDP formulates local parking motion planning as conditional diffusion over waypoint trajectories in $F_s$6. Each waypoint is $F_s$7 in ego-centric coordinates, and a trajectory segment is written as

$F_s$8

Expert demonstrations are segmented into local sub-trajectories of fixed length and downsampled every $F_s$9 timesteps to obtain exactly $P_{tgt}$0 waypoints per plan. These sub-trajectories, of dimension $P_{tgt}$1, become the clean data $P_{tgt}$2 for the diffusion model (Wu et al., 23 Oct 2025).

The forward process follows DDPM-style Gaussian diffusion: $P_{tgt}$3

$P_{tgt}$4

The reverse model $P_{tgt}$5 predicts the noise conditioned on the noisy trajectory, diffusion timestep, BEV feature map, and segmentation map. The training objective is the standard DDPM noise-prediction loss: $P_{tgt}$6 Architecturally, the planner is described as a U-Net-like network that takes a representation of $P_{tgt}$7, a timestep embedding, and spatial BEV features $P_{tgt}$8 and $P_{tgt}$9, implementing a conditional distribution $P_{tgt}$0 (Wu et al., 23 Oct 2025, Ho et al., 2020).

At inference, the process starts from Gaussian noise,

$P_{tgt}$1

and iteratively denoises to obtain $P_{tgt}$2, a sequence of $P_{tgt}$3 waypoints that is consistent with the conditional scene and target slot. The paper motivates this choice by arguing that diffusion can model multimodal trajectory distributions and reduce compounding error because it predicts entire sequences rather than step-by-step controls. It also emphasizes that inclusion of $P_{tgt}$4 is critical in parking, where final orientation alignment with the slot is essential (Wu et al., 23 Oct 2025).

A common misconception is to equate DDP with a diffusion controller. That is inaccurate. Diffusion is used for local planning in trajectory space, while control is explicitly delegated to a Stanley controller. The planner is therefore generative over paths, not directly over throttle and steering commands (Wu et al., 23 Oct 2025).

5. Training protocol and empirical performance

Training is fully offline on a single-domain dataset collected in CARLA Town04. The dataset consists of 800 expert demonstration trajectories gathered in a default “sunny-noon” weather setting, following the protocol of E2E Parking (E2EAPA) but with automated BEV segmentation labels. Each trajectory includes multi-view RGB images, depth images, BEV segmentation ground truth, vehicle state, and control commands (Wu et al., 23 Oct 2025).

On the perception side, the segmentation head is trained against ground-truth segmentation maps, with the paper indicating standard pixel-wise cross-entropy over the BEV grid. The depth head is described as likely trained with regression to ground-truth depth, though the paper focuses mainly on segmentation and BEV robustness. On the planning side, the diffusion planner is trained with the $P_{tgt}$5 noise prediction objective above, which the paper characterizes as behavior cloning in trajectory space (Wu et al., 23 Oct 2025).

The reported evaluation spans three CARLA conditions: Same, Mild domain shift, and Large domain shift. Each condition uses Town04 with 16 parking slots per side and 6 initial poses per slot. Metrics include Target Success Rate (TSR), Target Failure Rate (TFR), Non-Target Success Rate (NTSR), Non-Target Failure Rate (NTFR), collision rate, timeout rate, Average Position Error, Average Orientation Error, Average Parking Time, and Average Inference Time (Wu et al., 23 Oct 2025).

Method	Same TSR	Mild TSR	Large TSR
CAA-Policy	$P_{tgt}$6	$P_{tgt}$7	$P_{tgt}$8
CAA with frozen DINO	$P_{tgt}$9	$P_{tgt}$0	$P_{tgt}$1
DDP-emb+seg	$P_{tgt}$2	$P_{tgt}$3	$P_{tgt}$4
DDP-seg	$P_{tgt}$5	$P_{tgt}$6	$P_{tgt}$7

These results support two conclusions stated in the paper. First, DDP consistently achieves a parking success rate above 90% across all tested out-of-distribution scenarios. Second, the segmentation-only variant, DDP-seg, attains 100% success across Same, Mild, and Large conditions, though with slightly worse orientation precision, reported as approximately $P_{tgt}$8 to $P_{tgt}$9, compared with approximately $F_b$0 for DDP-emb+seg. DDP-emb+seg reports Average Position Error of approximately $F_b$1 to $F_b$2 m, while DDP-seg reports approximately $F_b$3 to $F_b$4 m. Collision and timeout rates are described as essentially zero for the DDP variants (Wu et al., 23 Oct 2025).

The ablation studies isolate three system-level claims. DINOv2 improves robustness of perception and downstream TSR under domain shifts; freezing DINOv2 is better than end-to-end fine-tuning to control; and replacing a transformer control head with diffusion plus Stanley improves TSR, with an example improvement of $F_b$5 in large shift, while also improving APE and AOE (Wu et al., 23 Oct 2025).

6. Sim-to-real relevance, related directions, and limitations

To probe sim-to-real transfer, the authors reconstruct a real parking lot as a 3D Gaussian Splatting environment using thousands of real images and a hierarchical 3DGS model. The same multi-camera rig is then used to render views from arbitrary viewpoints for closed-loop parking tests analogous to CARLA. DDP, trained only in CARLA’s sunny-noon domain, is deployed in this 3DGS world without fine-tuning. The reported qualitative outcome is that perception produces a reasonable BEV segmentation, the diffusion planner often succeeds in parking, and non-trivial zero-shot sim-to-3DGS transfer is observed (Wu et al., 23 Oct 2025).

The paper also identifies the main failure sources in 3DGS: residual noise or artifacts in segmentation, and layout patterns that deviate significantly from CARLA scenes. This is important for interpreting the system’s scope. The 3DGS result is presented as promising sim-to-real transfer, not as full real-vehicle validation (Wu et al., 23 Oct 2025).

Several limitations are either explicit or directly implied. The environment is static, with no dynamic obstacles considered. Training uses expert trajectories from a single town and layout, namely CARLA Town04. The system performs no online adaptation, no DAGGER-style aggregation, and no reinforcement-learning fine-tuning. Diffusion is computationally acceptable for parking but may require acceleration for broader driving or multi-agent planning. Average inference time is reported at approximately $F_b$6 to $F_b$7 s per step for DDP, versus approximately $F_b$8 to $F_b$9 s for baselines, corresponding to roughly 8–9 Hz, which the paper deems acceptable for parking tasks (Wu et al., 23 Oct 2025).

Within the broader diffusion-for-parking landscape, DDP occupies a distinct position. Unlike ParkDiffusion++, which targets ego-intention-conditioned joint multi-agent trajectory prediction for automated parking and emphasizes what-if reasoning under agent interaction, DDP is formulated for static autonomous parking with zero-shot cross-domain robustness (Wei et al., 24 Feb 2026). Unlike DRIP, which refines RL prior action distributions in confined-space parking, DDP diffuses over $F_s$0 waypoint trajectories conditioned on frozen BEV perception rather than over low-level actions (Jiang et al., 15 Oct 2025). A plausible implication is that these works define complementary axes of development: DDP emphasizes domain-agnostic visual generalization and modularity, ParkDiffusion++ emphasizes multi-agent reactive prediction, and DRIP emphasizes prior-guided action refinement in highly constrained spaces.

The principal significance of DDP lies in the specific synthesis it demonstrates: a frozen foundation-model perception stack, an explicitly goal-conditioned BEV representation with relabeling, diffusion-based planning in $F_s$1, and analytical low-level control. The paper’s future directions—video world models, human demonstrations in 3DGS environments, and real-vehicle deployment—indicate that the method is best understood as a robust modular parking pipeline whose current evidence is strongest in cross-domain simulation and simulation-to-3DGS transfer, rather than as a fully solved real-world parking system (Wu et al., 23 Oct 2025).