Multi-View Diffusion Trajectories (MDT)

• Multi-View Diffusion Trajectories (MDT) are defined as dynamic trajectories and 3D boxes that act as explicit geometric anchors in multi-view diffusion systems.
• MDT enables risk-controllable driving scenarios by integrating risk-conditioned motion generation with geometry-aware masking and localized preference optimization.
• Empirical evaluations show that incorporating MDT improves metrics like FID, MV-SSIM, and Depth AbsRel, demonstrating enhanced cross-view consistency and motion realism.

Searching arXiv for the primary paper and closely related MDT-relevant works. Multi-View Diffusion Trajectories (MDT) denotes a family of multi-view diffusion constructions in which cross-view coherence is organized around an explicit trajectory-like object. In the driving-scenario generator "Risk-Controllable Multi-View Diffusion for Driving Scenario Generation," MDT refers specifically to physically-informed, risk-conditioned dynamic trajectories and 3D boxes that act as explicit geometric anchors for a multi-view diffusion video generator, converting “risk” from an after-the-fact label into a time-resolved control signal (Lin et al., 12 Mar 2026). In other strands of the literature, the same expression denotes coordinated denoising trajectories across multiple views, or, in the language of multi-view geometry, time-inhomogeneous products of view-specific random-walk operators that induce diffusion distances and embeddings (Debaussart-Joniec et al., 1 Dec 2025, Lindenbaum et al., 2015). The term therefore spans both generative diffusion modeling and spectral diffusion geometry, with the common theme that view interaction is not treated as an auxiliary constraint but as part of the diffusion dynamics itself.

1. Core meaning and scope of MDT

In RiskMV-DPO, MDT addresses two coupled problems: controllable synthesis of long-tail risky scenarios, and robust multi-view geometry under outdoor dynamics, where standard diffusion models drift across views and fail under occlusion or fast motion. MDT is defined there as the pairing of dynamic trajectories and 3D boxes synthesized under a target risk profile, together with multi-view diffusion conditioned on those motion controls. By anchoring synthesis on physically grounded trajectories, the system enforces metric-level cross-view consistency and provides a direct mechanism to generate scenes at specified risk levels, rather than relying on sampling-and-filtering or post-hoc risk labeling (Lin et al., 12 Mar 2026).

A recurrent source of ambiguity is that the literature uses “diffusion” in two technical senses. In recent generative work, diffusion denotes denoising processes used for video, image, or multi-view synthesis. In earlier and more theoretical multi-view geometry work, diffusion denotes random-walk propagation on kernels or Markov operators. "Multi-View Diffusion Maps" constructs a cross-view Markov chain whose steps are constrained to hop between different views, and the sequences of states visited by that chain are described as multi-view diffusion trajectories in the trajectory-level interpretation of the method (Lindenbaum et al., 2015). "Multi-view diffusion geometry using intertwined diffusion trajectories" makes this interpretation explicit by defining MDTs as inhomogeneous diffusion processes that iteratively combine random walk operators of multiple data views (Debaussart-Joniec et al., 1 Dec 2025).

Several recent generative papers do not use the term MDT as formal nomenclature, but their technical summaries describe them as realizing MDT-like capabilities. "Cavia" conditions generation on per-frame camera trajectories and jointly denoises a 4D feature volume spanning space, time, and view (Xu et al., 2024). "ViewMorpher3D" jointly enhances packets of rendered views using a shared denoiser with full 3D self-attention and geometry-aware conditioning (Zanjani et al., 12 Jan 2026). "Coupled Diffusion Sampling for Training-Free Multi-View Image Editing" explicitly evolves two synchronized diffusion trajectories, one governed by a 2D editing prior and one by a multi-view prior, and couples them throughout denoising (Alzayer et al., 16 Oct 2025). This suggests that MDT functions both as a paper-specific construct and as a broader organizing concept for multi-view diffusion systems.

2. MDT in risk-controllable driving scenario generation

RiskMV-DPO implements MDT through four tightly coupled components. First, a risk-control module generates future trajectories and 3D boxes for all agents from multi-view history, an HD map, optional text context, and a user-specified target risk $r^*$:

$$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$

Second, a diffusion backbone, MagicDrive-STDiT3, renders future multi-view frames conditioned on those motion controls:

$$\hat{\mathbf{I}}_{T+1:T+H} = f(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{U}).$$

Third, a geometry-appearance alignment module injects compact 3D geometry priors into the diffusion backbone. Fourth, a region-aware direct preference optimization strategy with motion-aware masking concentrates learning on dynamic, risk-dominant regions (Lin et al., 12 Mar 2026).

Within this formulation, MDT is not merely a conditioning variable. The trajectories and 3D boxes are explicit geometric anchors that determine where and how agents move, and the diffusion model uses them as structured motion signals. The paper emphasizes that this shifts world models from passive environment prediction to proactive, risk-controllable synthesis. In that sense, MDT provides both a control interface and a geometric stabilization mechanism.

The conditioning pathway is multimodal. View tokens are fused with motion cues derived from trajectories and boxes, while scenario context can include HD maps and optional text. The geometry-appearance alignment module extracts compact geometry priors from DGGT/VGGT features on the first multi-view frame, compresses them into learnable geometric tokens, and aligns them to the diffusion backbone’s appearance features during training. The RA-DPO stage then uses localized preference learning to improve motion realism without disturbing the static background. A plausible implication is that MDT in this framework is best understood as a system-level design in which motion synthesis, geometric conditioning, and preference optimization are co-designed rather than appended sequentially.

3. Physically grounded risk modeling and trajectory synthesis

A defining feature of MDT in RiskMV-DPO is that risk is computed per frame and per agent using a direction-aware, field-inspired formulation. For ego and agent $i$ at time $t$, relative displacement is

$\mathbf{r}_i^t = \mathbf{p}_i^t - \mathbf{p}_e^t,$

with approaching cues derived from dot products between velocities and the relative displacement. These cues are mapped to a discrete interaction weight $\omega_i^t$ that distinguishes bi-directional closing, agent-approach, ego-approach, and away. Severity is further scaled by an agent-type coefficient $\mu_i = \mathrm{TypeCoeff}(\mathrm{cls}_i)$, while longitudinal amplification and lateral attenuation modulate the contribution according to closing speed and non-colliding motion. The per-agent risk contribution is

$$R_i^t = K\,C \cdot \frac{\omega_i^t\,\mu_i\,\alpha_i^t\,\beta_i^t}{\lVert \mathbf{r}_i^t \rVert_2 + \epsilon},$$

and frame-level risk is aggregated by either sum or max across agents (Lin et al., 12 Mar 2026).

Risk-conditioned motion generation is implemented through FiLM-based modulation of BEV or latent features:

$$\mathbf{e}_r = h(\mathbf{r}^*), \quad (\boldsymbol{\gamma}_r,\boldsymbol{\beta}_r) = \psi(\mathbf{e}_r), \quad \tilde{\mathbf{F}} = \boldsymbol{\gamma}_r \odot \mathbf{F} + \boldsymbol{\beta}_r.$$

The motion generator produces $$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$0 motion hypotheses, each assigned a scalar risk by the same functional used for frame-level evaluation. Training uses min-max risk objectives so that the minimum-risk hypothesis matches $$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$1 and the maximum-risk hypothesis matches $$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$2. The resulting objective is

$$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$3

The paper also lists representative risk terms that can be integrated into physically grounded synthesis, including time-to-collision, an aggregated trajectory-level risk score, and discrete vehicle-dynamics constraints. It states, however, that the experiments use the direction-aware formulation above. This distinction is important: MDT is not defined by a single universal risk equation, but in RiskMV-DPO it is instantiated through a specific physically grounded risk functional whose output directly conditions motion generation.

4. Geometric anchoring, alignment, and localized preference optimization

MDT supplies geometric anchoring in two ways. First, trajectories and 3D boxes are encoded as motion tokens and injected into the diffusion backbone, thereby anchoring agent placement and animation across views and time. Second, the system samples 3D points along trajectories, projects them into each camera using known intrinsics and extrinsics,

$$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$4

and rasterizes the projected points into geometry-consistent multi-view masks. These masks are fused with motion magnitudes from temporal differencing so that corruption and preference learning are concentrated on dynamic regions impacted by the trajectories (Lin et al., 12 Mar 2026).

The geometry-appearance alignment module injects 3D priors through cosine-similarity alignment between geometric tokens and pooled appearance features. Geometric tokens are extracted from VGGT features on the first multi-view frame, projected into the diffusion latent dimension, and stochastically dropped for classifier-free guidance. The alignment loss is

$$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$5

where $$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$6 and $$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$7 are normalized geometric and appearance features. The paper notes that multi-view consistency is enforced via geometry-aware masking and alignment rather than an explicit reprojection loss.

Region-aware DPO constructs localized preference pairs by progressively corrupting only masked regions. Motion-aware masking is defined from frame differences, fused with the geometric masks, and used to create weakly and strongly corrupted reconstructions within the same spatial support. The RA-DPO objective is applied with a noise-adaptive weight, and a supervised fine-tuning term on masked regions complements it. The total objective combines supervised masked reconstruction, region-aware DPO, and geometry-appearance alignment:

$$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$8

A common misconception is that multi-view consistency in such systems is achieved solely by cross-view attention or camera conditioning. In RiskMV-DPO, consistency is instead distributed across explicit motion controls, geometry-aware masking, compact 3D priors, and localized preference optimization. The design therefore treats geometry as an active training signal rather than only a conditioning label.

5. Related MDT formulations in generative and geometric literature

The generative literature contains several MDT-adjacent constructions. "Cavia" formulates multi-view generation by jointly denoising the entire 4D set of features over space, time, view, and channels, with cross-frame 3D attention for temporal consistency and cross-view 3D attention for view consistency. Camera control is provided by per-pixel Plücker coordinates derived from user-specified intrinsics and extrinsics, and multiple distinct camera paths for the same scene are generated in one joint diffusion process (Xu et al., 2024). In this setting, MDT-like behavior is associated primarily with camera trajectories and shared denoising across views.

"ViewMorpher3D" occupies a different regime. It is a post-process enhancer for low-fidelity 3D Gaussian Splatting renderings rather than a from-scratch scene generator. It conditions on coordinate maps, Plücker ray embeddings, and a binary reference/target mask, stacks all packet views into a joint latent set, and applies full 3D self-attention over flattened spatial dimensions across all views. The paper describes this as realizing the essence of MDT through a coordinated multi-view update driven by shared pose and geometry conditioning, even though the model is distilled into a single-step SD-Turbo mapping (Zanjani et al., 12 Jan 2026). This is a useful counterexample to the assumption that MDT necessarily implies a long iterative denoising chain.

A third generative interpretation appears in "Coupled Diffusion Sampling for Training-Free Multi-View Image Editing." There, MDT describes two synchronized latent trajectories: a per-view trajectory following a 2D editing prior and a multi-view trajectory following a pre-trained multi-view diffusion prior. The two are coupled in predicted-clean-latent space by the quadratic energy

$$\mathbf{U} = g(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{r}^{*}).$$9

whose gradient softly pulls the trajectories together at each timestep. The result is an implicit 3D regularization scheme that avoids explicit NeRF or Gaussian Splatting optimization while improving multi-view consistency (Alzayer et al., 16 Oct 2025).

The theoretical literature uses MDT in yet another sense. "Multi-view diffusion geometry using intertwined diffusion trajectories" defines MDTs as time-inhomogeneous Markov processes built from products or convex blends of view-specific random-walk operators,

$$\hat{\mathbf{I}}_{T+1:T+H} = f(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{U}).$$0

with $$\hat{\mathbf{I}}_{T+1:T+H} = f(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{U}).$$1 either chosen from a discrete set of view-specific operators or formed as convex combinations of them. The paper derives trajectory-dependent diffusion distances,

$$\hat{\mathbf{I}}_{T+1:T+H} = f(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{U}).$$2

and corresponding embeddings via singular value decomposition, and it places alternating diffusion, integrated diffusion, and constant convex fusion within the same operator space (Debaussart-Joniec et al., 1 Dec 2025). "MultiView Diffusion Maps" provides an antecedent by constructing a cross-view block kernel with zero diagonal blocks, so that single-step transitions are forced to hop between views; its diffusion distances compare endpoint distributions of such cross-view trajectories (Lindenbaum et al., 2015).

These usages show that MDT is not a standardized synonym for one architecture. In generative papers it usually refers to synchronized or conditioned denoising across views. In spectral geometry it refers to multi-view random-walk dynamics and the induced geometry. The shared conceptual core is intertwined view interaction over a diffusion trajectory.

6. Empirical behavior, limitations, and future directions

RiskMV-DPO reports experiments on nuScenes with 700 train scenes and 150 evaluation scenes, using resolution $$\hat{\mathbf{I}}_{T+1:T+H} = f(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{U}).$$3 per camera and 16-frame clips. The model is initialized from MagicDriveV2 and trained for 20k steps with AdamW, learning rate $$\hat{\mathbf{I}}_{T+1:T+H} = f(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{U}).$$4, BF16, EMA decay $$\hat{\mathbf{I}}_{T+1:T+H} = f(\mathbf{I}_{1:T}, \mathbf{M}, \mathbf{y}, \mathbf{U}).$$5, and approximately 58M trainable parameters on top of a backbone of approximately 2.5B parameters. Relative to MagicDriveV2, it reduces FID from 20.91 to 15.70, reduces FVD from 94.84 to 87.65, and raises 3D detection mAP from 18.17 to 30.50. Against DriveDreamer-2 and Panacea, it achieves the best FID of 15.70 (Lin et al., 12 Mar 2026).

Ablation studies are used to isolate the contribution of MDT-associated components. Motion-aware masks and 3D multi-view masks steadily improve MV-SSIM and Depth AbsRel. Adding VGGT-GAM notably reduces Depth AbsRel, and enabling alignment achieves the best overall quality, with FID 15.70, MV-SSIM 0.856, and Depth AbsRel 0.204. The interpretation provided in the paper is that MDT conditioning stabilizes geometry by explicitly anchoring agent motion across cameras, while RA-DPO concentrates learning on motion-dominant regions critical for risk.

The limitations reported for RiskMV-DPO are structural rather than incidental. Performance depends on the quality of risk-conditioned trajectories and 3D boxes. Some complex, underexplored risk patterns remain challenging. Extreme occlusions or rare camera configurations can still degrade consistency. Related systems exhibit analogous constraints: Cavia reports limited ability to generate very large object motion and limited generalization to varying intrinsics (Xu et al., 2024); ViewMorpher3D is sensitive to pose and C-map errors (Zanjani et al., 12 Jan 2026); coupled diffusion sampling increases compute and memory because both the 2D editor and the multi-view model must run during inference (Alzayer et al., 16 Oct 2025).

Future directions are stated explicitly in the driving-scenario setting: enrich risk models with additional terms such as explicit TTC, minimum distance, deceleration severity, and conflict probability; integrate closed-loop simulation where MDT interacts with policy agents; and scale to more complex multi-agent interactions and longer horizons (Lin et al., 12 Mar 2026). In the operator-based MDT framework, future work includes continuous-time MDTs, adaptive or data-driven schedules, and integration with deep encoders (Debaussart-Joniec et al., 1 Dec 2025). Taken together, these directions indicate that MDT is evolving from a narrow multi-view consistency technique into a broader framework for controllable, geometry-aware, and task-directed diffusion across views.