DD-MDN: Diffusion Dual Mixture Density Network

• DD-MDN is a diffusion-based probabilistic model for multi-agent human trajectory forecasting that combines a few-shot denoising diffusion backbone with dual Gaussian-mixture representations.
• It fuses temporal, spatial, and social inputs using LSTM, CNN, and attention mechanisms to generate ranked multimodal trajectory hypotheses with calibrated uncertainty.
• The dual mixture density architecture enables per-timestep calibration and coherent full-path predictions, yielding superior accuracy and robustness in diverse benchmarks.

DD-MDN, short for Diffusion-based Dual Mixture Density Network, is a probabilistic model for human trajectory forecasting that combines a few-shot denoising diffusion backbone with a dual Gaussian-mixture representation and an end-to-end hypothesis generator. It was introduced to address a specific combination of objectives that are often treated separately in trajectory prediction: high positional accuracy, calibrated uncertainty, confidence-ranked multimodal forecasts, and robustness when only very short observation windows are available. In its formulation, DD-MDN learns self-calibrated residence areas and probability-ranked anchor paths, from which diverse trajectory hypotheses are derived without predefined anchors, waypoints, or endpoint proposals (Hetzel et al., 11 Feb 2026).

1. Problem formulation and forecasting target

DD-MDN is defined for multi-agent human trajectory forecasting. Let $A$ be the number of agents, $T_{\mathrm{hist}}$ the observation length, and $T_{\mathrm{fut}}$ the prediction horizon. For each agent $a$, the observed trajectory is

$$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$

and the ground-truth future is

$$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$

Over all agents,

$$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$

The model also uses an occupancy grid $\mathbf{G}$ encoding environmental context. Its output for each agent is a set of $K$ future trajectory hypotheses and associated probabilities,

$$\dot{\mathcal{Y}}_a=\{\dot{\mathbf{Y}}_a^k\}_{k=1}^K, \qquad \dot{\mathcal{P}}_a=\{\dot{P}_a^k\}_{k=1}^K,$$

so that the learned predictor is

$T_{\mathrm{hist}}$0

The paper’s stated motivation is that downstream systems such as path planners and collision-avoidance modules require not only multiple plausible futures, but also reliable probabilities and calibrated uncertainty, especially under short observation periods (Hetzel et al., 11 Feb 2026).

2. Encoding pipeline and diffusion backbone

The architecture has three stages: encoding, probabilistic modeling, and deterministic hypothesis generation. The encoding stage fuses temporal, spatial, and social context. Observed trajectories are processed by an LSTM followed by self-attention to produce $T_{\mathrm{hist}}$1. Spatial context is represented by a single-layer occupancy grid centered on the target agent within a 10 m radius; two CoordConv channels align image coordinates and world coordinates, and a small CNN with self-attention maps this input to $T_{\mathrm{hist}}$2. Social interactions are modeled by Social-Out-Way Attention, described as a Transformer variant with exit gates that suppress irrelevant neighbors without manually fixed distance thresholds, producing $T_{\mathrm{hist}}$3. The diffusion backbone conditions on

$T_{\mathrm{hist}}$4

while the final hypothesis generator uses temporal and social features (Hetzel et al., 11 Feb 2026).

The probabilistic core is a shared Transformer-based diffusion backbone with three probabilistic heads. Unlike trajectory diffusion models that denoise future trajectories directly, DD-MDN denoises a latent parameter state from which Gaussian mixture parameters are decoded. Starting from $T_{\mathrm{hist}}$5, the recurrence is

$T_{\mathrm{hist}}$6

for diffusion steps $T_{\mathrm{hist}}$7. After $T_{\mathrm{hist}}$8 iterations, the final latent state is decoded into shared Gaussian means and covariances together with two distinct sets of mixture weights. The paper describes this as a few-shot denoising diffusion backbone, emphasizing that the final system produces a small number of ranked hypotheses rather than a very large diffusion sample set (Hetzel et al., 11 Feb 2026).

3. Dual mixture density formulation

The defining technical feature of DD-MDN is its dual mixture density network. It learns two linked Gaussian-mixture views of the future trajectory.

The first view is a per-timestep Gaussian mixture, written

$T_{\mathrm{hist}}$9

Here $T_{\mathrm{fut}}$0 is the mean position at future time $T_{\mathrm{fut}}$1, $T_{\mathrm{fut}}$2 is the covariance, and $T_{\mathrm{fut}}$3 is the mixture weight. This head is used to model future residence areas, namely the calibrated positional uncertainty at each future step (Hetzel et al., 11 Feb 2026).

The second view is a trajectory-space Gaussian mixture over the whole future path. For each mode $T_{\mathrm{fut}}$4, the means are stacked across time,

$T_{\mathrm{fut}}$5

and the covariance is block-diagonal,

$T_{\mathrm{fut}}$6

The resulting mixture is

$T_{\mathrm{fut}}$7

The means $T_{\mathrm{fut}}$8 define anchor trajectories, and the weights $T_{\mathrm{fut}}$9 rank these modes. The paper stresses that both probabilistic views share the same Gaussian means and covariances; what differs is the assignment and interpretation of the mixture weights (Hetzel et al., 11 Feb 2026).

This split has a specific functional rationale. A per-timestep mixture can represent calibrated marginals at each future instant, but it does not by itself enforce temporal coherence across future steps. A trajectory-space mixture can represent coherent full paths, but it is not by itself the mechanism used for per-step calibration. DD-MDN therefore uses $a$0 for calibrated positional uncertainty and $a$1 for coherent anchor-path structure. The model also applies annealed mode pruning. With epoch-dependent threshold

$a$2

the gated and renormalized anchor weights are

$a$3

so that only active modes with $a$4 remain (Hetzel et al., 11 Feb 2026).

4. Likelihood training, self-calibration, and hypothesis generation

DD-MDN is trained without ground-truth mixture parameters. Instead, the diffusion backbone and mixture heads are optimized indirectly through exact likelihood terms. The probabilistic loss is

$a$5

with $a$6 and $a$7. According to the paper, this weighting is important because both heads share the same means and covariances: if the anchor term dominates, per-timestep calibration deteriorates; if it is too weak, the temporal binding needed for coherent anchor trajectories is lost (Hetzel et al., 11 Feb 2026).

The paper’s calibration argument is based on negative log-likelihood. For a single Gaussian at time $a$8,

$a$9

For a Gaussian mixture,

$$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$0

The paper treats NLL as a strictly proper scoring rule: means far from the data receive weak responsibility, diffuse covariances are penalized because they lower density at the ground truth, and softmax-normalized weights concentrate on components that explain the data better. This is the basis of the model’s stated uncertainty self-calibration (Hetzel et al., 11 Feb 2026).

Final trajectory hypotheses are generated around the learned anchor paths. For each active anchor mode $$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$1,

$$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$2

where $$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$3 is the mean anchor trajectory, $$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$4 is an overall variance scale derived from the RMS of the diagonal entries of $$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$5, and $$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$6 is a set of normalized unit-variance residual trajectories predicted by an MLP decoder conditioned on covariance-derived scale features together with temporal and social context. Each active mode contributes at least one hypothesis. The remaining $$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$7 hypotheses are distributed proportionally to the anchor weights by a Hamilton largest-remainder rule,

$$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$8

This guarantees exactly $$\mathbf{X}_a^{\mathrm{in}} = \left\{\mathbf{x}_{tp}^a \in \mathbb{R}^2 \mid tp=1,\dots,T_{\mathrm{hist}}\right\},$$9 outputs and makes the number of hypotheses per anchor proportional to the learned mode confidence (Hetzel et al., 11 Feb 2026).

The hypothesis generator is trained with

$$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$0

The paper assigns distinct roles to these terms: $$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$1 pulls all hypotheses toward the ground truth, $$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$2 emphasizes the best hypothesis via annealed Winner-Takes-It-All, and $$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$3 constrains generated hypotheses to remain inside a desired confidence region of $$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$4 (Hetzel et al., 11 Feb 2026).

5. Benchmarks, quantitative results, and operating profile

The main benchmark protocol uses 8 observed frames, 12 predicted frames, and 2.5 Hz sampling rate, corresponding to 3.2 s of observation and 4.8 s of prediction. The paper also evaluates a momentary observation regime with only 2 observed frames. Performance is reported on ETH/UCY, SDD, inD, and IMPTC using Best-of-$$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$5 ADE ($$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$6) and Best-of-$$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$7 FDE ($$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$8). Calibration is evaluated with $$\mathbf{Y}_a^{\mathrm{gt}} = \left\{\mathbf{x}_{tf}^a \in \mathbb{R}^2 \mid tf=T_{\mathrm{hist}}+1,\dots,T_{\mathrm{hist}}+T_{\mathrm{fut}}\right\}.$$9 and $$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$0, where 100% denotes perfect calibration, and Q-Q plots are used to visualize reliability across timesteps (Hetzel et al., 11 Feb 2026).

Dataset / setting	DD-MDN result	Unit
ETH/UCY, 8 observed frames	$$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$1	m
ETH/UCY, 2 observed frames	$$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$2	m
SDD, 8 observed frames	$$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$3	pixels
SDD, 2 observed frames	$$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$4	pixels
inD, 8 observed frames	$$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$5	m
inD, 2 observed frames	$$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$6	m

On ETH/UCY, the paper reports $$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$7 m, identified as the best overall average in the reported table. Under momentary observation on the same benchmark, DD-MDN reaches $$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$8 m, compared with $$\mathcal{X}^{\mathrm{in}}=\{\mathbf{X}_a^{\mathrm{in}}\}_{a=1}^A, \qquad \mathcal{Y}^{\mathrm{gt}}=\{\mathbf{Y}_a^{\mathrm{gt}}\}_{a=1}^A.$$9 for MID, $\mathbf{G}$0 for EigenTraj, and $\mathbf{G}$1 for SingularTrajectory. On inD, DD-MDN reports $\mathbf{G}$2 m with 8 observed frames and $\mathbf{G}$3 with 2 observed frames; the paper states that the latter corresponds to roughly 46% and 38% improvement in ADE and FDE over prior state of the art (Hetzel et al., 11 Feb 2026).

Calibration results are central to the model’s identity. On ETH, DD-MDN reports $\mathbf{G}$4 and $\mathbf{G}$5; on SDD, $\mathbf{G}$6 and $\mathbf{G}$7; on inD, $\mathbf{G}$8 and $\mathbf{G}$9. On the larger full-dataset evaluations, reliability approaches near-perfect values: IMPTC$K$0 reports $K$1, inD$K$2 reports $K$3, and SDD$K$4 reports $K$5 for $K$6. The paper also reports an ablation in which the diffusion backbone outperforms linear and MLP backbones in calibration, especially in $K$7, which it interprets as evidence that iterative denoising better regularizes the structured parameter manifold (Hetzel et al., 11 Feb 2026).

The implementation profile is comparatively compact. Reported raw weights are 4.5 MB, compared with 60.0 MB for LED and 60.2 MB for MoFlow. Inference latency is 15.5 ms at batch size $K$8 and 21.9 ms at batch size $K$9. Reported memory use is 818.7 MB in FP32 at $$\dot{\mathcal{Y}}_a=\{\dot{\mathbf{Y}}_a^k\}_{k=1}^K, \qquad \dot{\mathcal{P}}_a=\{\dot{P}_a^k\}_{k=1}^K,$$0, including fixed CUDA context. Training uses AdamW with a cosine schedule on NVIDIA RTX 4090 hardware and an AMD Ryzen 9950X CPU, and the paper reports averages over 3 training runs because of the probabilistic nature of the method (Hetzel et al., 11 Feb 2026).

6. Nomenclature, broader MDN context, and limitations

The label DD-MDN refers specifically to the human trajectory forecasting model described above. By contrast, the acronym MDN is used differently elsewhere in the literature. In robotics and imitation learning, MDN denotes a Mixture Density Network attached to recurrent controllers to model multimodal action distributions (Rahmatizadeh et al., 2016). In visual odometry, MDN has been used as a probabilistic output layer on top of CNN–RNN backbones for mixture-based relative pose estimation and confidence prediction (Kaygusuz et al., 2021). Other uses of the same three letters include Marginal Debiased Network for fair visual recognition (Wang et al., 2024), Metadata Normalization (Lu et al., 2021), and Momentum DeltaNet for delta linear attention (Huang et al., 7 May 2026). This suggests that DD-MDN belongs to a broader probabilistic-regression lineage centered on multimodal density estimation, while remaining a distinct trajectory-forecasting construction defined by its dual mixture representation and diffusion backbone.

Several limitations are explicit. The paper states that in crowded scenes, forecasts can become underconfident and broad. Its trajectory-space covariance is block-diagonal, so cross-time off-diagonal covariance is not modeled explicitly within each mode. The method description also indicates that some loss details and calibration formulas are described verbally rather than fully specified. More broadly, the paper positions DD-MDN as a response to a benchmark culture dominated by best-of-$$\dot{\mathcal{Y}}_a=\{\dot{\mathbf{Y}}_a^k\}_{k=1}^K, \qquad \dot{\mathcal{P}}_a=\{\dot{P}_a^k\}_{k=1}^K,$$1 geometric accuracy. Its main contribution is therefore not merely generating multiple futures, but coupling multimodal forecasting to calibrated positional uncertainty, anchor-level probability ranking, and short-observation robustness in a single end-to-end framework (Hetzel et al., 11 Feb 2026).