Normalizing Trajectory Models (NTM)

• NTMs are machine learning models that use normalizing flows to map simple base distributions to complex, multi-modal trajectory distributions with exact likelihood evaluation.
• They integrate autoregressive and energy-based paradigms with flexible conditioning and neural architectures to incorporate physical constraints.
• NTMs have demonstrated significant improvements in applications such as autonomous driving, motion forecasting, and dynamic optimal transport through efficient sampling and reduced planning costs.

Normalizing Trajectory Models (NTM) are a class of machine learning models that leverage normalizing flows to parameterize probability distributions over trajectories—sequences of actions or states evolving over time. NTMs provide an explicit, invertible mapping between simple base distributions and complex, multi-modal trajectory distributions, supporting exact likelihood evaluation, flexible conditioning, and efficient sampling. Applications span autonomous driving, motion forecasting, dynamic optimal transport, trajectory planning, density estimation, and generative modeling of high-dimensional data sequences. The NTM framework generalizes both autoregressive and energy-based paradigms, allowing integration of neural architectures, physical constraints, and domain-specific normalization strategies.

1. Mathematical Foundations and Formulation

NTMs are built upon the change-of-variables formula for bijective mappings, extending normalizing flows to domains where the objects of interest are full trajectories. Let $z \sim p_Z(z)$ denote a base latent, typically $z \sim \mathcal{N}(0, I)$. An invertible map $f_\theta(z; c)$—possibly conditioned on a context $c$ such as scene or history—transforms $z$ to the trajectory space $u$: $u = f_\theta(z; c)$. The resulting density is given by:

$$p_\theta(u|c) = p_Z(f_\theta^{-1}(u; c)) \cdot |\det \partial_u f_\theta^{-1}(u; c)|$$

Such a transformation underlies conditional normalizing flows for behavior modeling (Schöller et al., 2021), trajectory planning (Agarwal et al., 2020, Rabenstein et al., 2024), and dynamic density estimation (Gu et al., 8 May 2026).

For continuous-time modeling, NTMs employ continuous normalizing flows (CNFs), parameterizing the ODE:

$\frac{dx(t)}{dt} = f_\theta(x(t), t; c)$

with density evolution governed by the instantaneous Jacobian trace (Tong et al., 2020, Kosieradzki et al., 24 Jan 2025). In optimal transport and mean-field games, the NTM provides a physically grounded model of agent flows in phase space, parameterizing the transport map as a composition of flows across discretized time steps (Huang et al., 2022).

2. Architectural Designs and Conditioning Mechanisms

NTMs encompass a variety of architectures, tailored to both discrete and continuous sequential domains.

• Blockwise Flow Structures: Many NTMs use stacks of invertible, parameterized transformation blocks, such as RealNVP affine couplings, neural autoregressive flows, or rational-quadratic splines (Schöller et al., 2021, Mészáros et al., 2023, Gu et al., 8 May 2026). Architectural variations may alternate conditioning directions, stack permutation layers for expressivity, and employ batch normalization to stabilize training (Kosieradzki et al., 24 Jan 2025).
• Causal and Contextual Encoding: Conditioning on external information (scene, history, target, time) is achieved via learned encoders. GRUs, LSTMs, or neural controlled differential equations produce fixed-size embeddings of observations, which inform the flow parameters at each coupling layer (Kosieradzki et al., 24 Jan 2025, Mészáros et al., 2023).
• Hybrid Structures: Several NTM instantiations integrate normalizing flows within a larger pipeline (e.g., by first encoding raw state sequences into a compact latent space or learning scene adaptors with FiLM-style layers) (Agarwal et al., 2020, Mészáros et al., 2023).
• Parallel Predictors and Shallow Invertible Blocks: In generative modeling (e.g., image generation), NTMs combine shallow invertible per-step flows with deep parallel predictors (Transformers) across the entire trajectory, aligning exact likelihood computation with high sample quality in few sampling steps (Gu et al., 8 May 2026).

3. Training Objectives, Loss Functions, and Regularization

NTMs are trained via exact or approximate maximum likelihood, with objectives adapted for context:

• Negative Log-Likelihood (NLL): Core NTMs minimize the NLL of trajectories under the model, either directly (via maximum likelihood estimation) or in combination with behavior cloning or imitation (Agarwal et al., 2020, Schöller et al., 2021).
• Energy-Based and Reverse KL Objectives: When planning with respect to a cost manifold, NTMs minimize the reverse KL divergence $KL[p_\theta(u|x) \Vert p_E(u|x)]$, where the energy-based distribution takes Boltzmann form $z \sim \mathcal{N}(0, I)$0. The resulting loss combines log-density with the planner cost (Agarwal et al., 2020).
• Trajectory Regularization and Transport Cost: NTMs used for mean-field games or high-dimensional OT incorporate additional kinetic energy (transport) regularization $z \sim \mathcal{N}(0, I)$1, controlling the trajectory’s Lipschitz constant and guiding solutions towards physically plausible flows (Huang et al., 2022, Tong et al., 2020).
• Auxiliary Constraints: Domain knowledge and stability are injected via density regularization (e.g., noise injection, scaling augmentation), velocity alignment (with observed physical fields), or explicit domain normalization (Frenét coordinate transforms) (Schöller et al., 2021, Ye et al., 2023).

4. Inference, Sampling Strategies, and Planning Integration

NTMs enable diverse inference modalities beyond sample generation:

• Trajectory Sampling: Sampling involves drawing latent variables, transforming them via the flow, and selecting (or weighting) resulting trajectories according to planning cost, likelihood, or other metrics. In model predictive control, NTM-derived distributions efficiently populate the control space for trajectory optimization, yielding substantial reductions in average cost versus independent Gaussian samplers (Agarwal et al., 2020, Rabenstein et al., 2024).
• Likelihood Ranking and Oracle Selection: Models like FloMo report both sample quality (minADE/FDE) and negative log-likelihood, enabling evaluators to correlate likelihood with physical error metrics and prioritize high-probability trajectories for downstream planning (Schöller et al., 2021).
• Self-Distillation and Denoising: In generative NTM variants, exact trajectory likelihoods facilitate self-distillation, where a lightweight denoiser network is trained to refine or reconstruct the model’s sample trajectories, significantly boosting inference efficiency in low-step generation settings (Gu et al., 8 May 2026).
• Occupancy and Marginal Density Computation: Marginal NTM formulations facilitate direct estimation of per-time-step occupancy probabilities, supporting continuous occupancy grid fusion for motion forecasting (Kosieradzki et al., 24 Jan 2025).

5. Empirical Results and Performance Benchmarks

NTMs have been systematically evaluated across synthetic and real-world domains:

• Autonomous Vehicle Planning: In sequential planning tasks, NTM-based samplers reduce expected planning cost by 20–57% over basic or heuristic alternatives while preserving real-time suitability (Agarwal et al., 2020, Rabenstein et al., 2024).
• Motion Forecasting: On ETH/UCY (pedestrian), nuScenes, rounD (vehicle), and Stanford Drone datasets, models such as FloMo and TrajFlow achieve or surpass state-of-the-art sample-based and likelihood-based performance, e.g., minADE=0.22 and minFDE=0.37 for FloMo (Schöller et al., 2021), minADE=0.19 and minFDE=0.38 for CDE-CNF TrajFlow (Kosieradzki et al., 24 Jan 2025). Table summaries appear in the original works.
• Generalizability and Domain Shift: Domain normalization via Frenét transforms reduces error degradation in cross-domain generalization benchmarks by up to a factor of two, with LaneGCN minADE shift from +7% to +1.55% (Ye et al., 2023).
• Generative Image Modeling: In text-to-image synthesis, NTM achieves competitive or superior compositional accuracy to diffusion and flow baselines in four steps, maintaining exact model likelihoods (e.g., GenEval=0.82 at 4 steps) (Gu et al., 8 May 2026).
• Dynamic Optimal Transport and Mean-Field Games: NTM-based architectures match Eulerian PDE solvers in transport cost, remain tractable in high dimensions (d=100), and empirically control Lipschitz and terminal divergence metrics (Huang et al., 2022, Tong et al., 2020).

6. Extensions, Theoretical Insights, and Limitations

• Marginal vs. Joint Trajectory Modeling: Marginal distributions over future positions enable continuous sampling and superior long-horizon accuracy compared to joint modeling, particularly in contexts where trajectories are highly stochastic or the marginal is unimodal given history (Kosieradzki et al., 24 Jan 2025).
• Plug-and-Play Generalization Layers: Domain normalization strategies (e.g., Frenét+, (Ye et al., 2023)) transform the problem geometry, enhance invariance to scene specifics, and seamlessly integrate with arbitrary backbone architectures without changing loss functions or hyperparameters.
• Limitations: Two-stage training pipelines (e.g., autoencoder+flow), potential information loss in latent compressions, and assumed access to high-fidelity map or feature information introduce practical constraints. NTM expressivity at very low step count (e.g., T=1) can be limited unless the flow depth is significantly increased (Gu et al., 8 May 2026).
• Theoretical Underpinnings: The connection between normalizing flow training and mean-field game objectives unifies kinetic energy, interaction, and terminal-matching losses into a single variational framework, with regularization controlling global solution behavior and generalization (Huang et al., 2022).

7. Applications and Impact

NTMs have been applied to autonomous integration of expert and cost-based planning (Agarwal et al., 2020), sample-efficient model predictive control (Rabenstein et al., 2024), generalizable trajectory prediction (Schöller et al., 2021, Mészáros et al., 2023, Kosieradzki et al., 24 Jan 2025), robust distributional motion forecasting with domain adaptation (Ye et al., 2023), dynamic cellular fate inference (Tong et al., 2020), high-dimensional mean-field equilibration (Huang et al., 2022), and expressive generative modeling at minimal inference steps (Gu et al., 8 May 2026). This versatility illustrates the capacity of the NTM framework to bridge exact likelihood, complex temporal structure, and domain-specific constraints, with rigorous evaluation standards grounded in likelihood and sample error metrics across domains.