---
title: Prediction-Driven Motion Planning
url: https://www.emergentmind.com/topics/prediction-driven-motion-planning
type: topic
---

# Prediction-Driven Motion Planning

Prediction-driven motion planning denotes a family of methods in which forecasts of future trajectories, occupancies, maps, or latent interaction states are treated as first-class planning variables rather than as auxiliary perception outputs. In these systems, the planner selects an ego action sequence while explicitly reasoning over predicted future evolution of other agents or of the environment, and in stronger formulations the predictor itself is conditioned on ego intent or embedded inside the planner. The resulting design space spans planning-aware trajectory prediction for autonomous driving, uncertainty-aware model predictive control for mobile robots, human–robot co-optimization with differentiable predictors, map prediction in unknown environments, and even motion planning cast as video prediction [2207.10422] [2204.02392] [1910.08184].

## 1. Conceptual foundations

A common formalization is to optimize an ego control sequence $u$ against a forecast distribution over future scene realizations $Y$:
$$
\min_{u}\; \mathbb{E}_{Y \sim p(Y \mid X, u)}[J(u,Y)]
\quad \text{s.t. dynamics and safety constraints.}
$$
Here $X$ denotes the observed scene history and context, $p(Y \mid X,u)$ is a planning-conditioned predictor, and $J$ aggregates progress, safety, comfort, legality, or task costs. The defining distinction is the dependence of prediction on the planned ego action. In decoupled stacks, prediction is often modeled as $p(Y\mid X)$; prediction-driven formulations instead treat future motion as interactive and planning-dependent.

This distinction is explicit in autonomous driving. "Deep Interactive Motion Prediction and Planning" embeds a learned interactive multi-agent policy inside a game-theoretic MPC and teacher-forces the ego planned state sequence into the predictor, so that surrounding agents react to the candidate ego plan during optimization [2204.02392]. "Differentiable Integrated Motion Prediction and Planning" uses a differentiable nonlinear optimizer that takes predicted trajectories of surrounding agents as input and jointly learns the planning cost weights, so the prediction module becomes planning-centric rather than merely forecast-centric [2207.10422]. "Planning by Simulation: Motion Planning with Learning-based Parallel Scenario Prediction for Autonomous Driving" argues that some methods overlook the significant influence of the ego vehicle’s planning on the possible trajectories of other agents, and proposes Planning by Simulation with learning-based parallel scenario prediction, where predictions are deduced iteratively based on Monte Carlo Tree Search [2411.09887].

Prediction-driven planning is not limited to interaction conditioning. Goal and route information can also enter prediction. "Prediction-Driven Motion Planning: Route Integration Strategies in Attention-Based Prediction Models" extends an attention-based joint predictor with route polylines and a goal token, thereby addressing the mismatch between goal-conditioned planning and traditional prediction models that ignore navigation intent [2512.03756]. This suggests that, in integrated systems, the forecast object is not just “what others will do,” but “what futures remain compatible with the ego’s route, goal, and control authority.”

## 2. What is predicted

The predicted object in prediction-driven planning varies substantially across subfields. In many robotic and driving systems, the predictor outputs multimodal trajectories or occupancy sets. "Future-Oriented Navigation" uses a one-shot multimodal energy-based predictor that produces a stack of per-time-step probability maps $P_t$ for $t=1,\ldots,N$, with $N=20$ steps, corresponding to $4\,\mathrm{s}$ ahead at sampling time $0.2\,\mathrm{s}$; these maps are then clustered into modes and fitted with elliptical Gaussian occupancies [2505.00237]. CogDrive likewise decodes multiple trajectory modes with probabilities and per-step covariances, using a multimodal Gaussian formulation and explicit topological interaction modes such as yielding, neutral, and aggressive behavior [2512.02777].

A second representation is set-valued prediction. "Robust Predictive Motion Planning by Learning Obstacle Uncertainty" learns an intended control set $\hat U_t^s$ for each obstacle by solving a linear program, then propagates forward reachable sets $\hat R^s_{i|t}$ and position occupancies $\hat O^s_{i|t}$ over the prediction horizon [2403.06222]. "A Hamilton-Jacobi Reachability-Based Framework for Predicting and Analyzing Human Motion for Safe Planning" goes further by augmenting the human state with a belief over model parameters and computing Belief-Augmented Forward Reachable Sets through a Hamilton–Jacobi PDE, with allowable action sets
$$
U(z)=\{u_H : P(u_H \mid z)\ge \delta\},
$$
thereby yielding prediction sets that are continuous in state and time and explicitly robust to prior misspecification [1910.13369].

A third representation emphasizes calibrated uncertainty. "Adaptive Conformal Prediction for Motion Planning among Dynamic Agents" constructs online multistep uncertainty radii $C^\tau_{t+1}$ from delayed residuals
$$
R_t^\tau := \|Y_t - \hat y^\tau_{t-\tau}\|,
$$
and uses an adaptive recursion
$$
\delta^\tau_{t+1} := \delta^\tau_t + \gamma(\delta - e^\tau_t)
$$
to maintain horizon-specific uncertainty sets with probabilistic coverage under distribution shift [2212.00278].

Prediction-driven planning also includes non-trajectory forecasts. "Map-Predictive Motion Planning in Unknown Environments" predicts occupancies of unobserved map cells with a Conditional Neural Process and then plans all the way to the goal without heuristic frontier selection [1910.08184]. "Planning Robot Motion using Deep Visual Prediction" predicts up to 10 future egocentric frames from monocular video using PROM-Net and passes those predictions to an MPC, while "Robot Motion Planning as Video Prediction" recasts path generation itself as next-frame prediction over map, robot-state, and goal channels [1906.10182] [2208.11287]. In these cases, the predicted object is the future sensor stream or workspace occupancy rather than a list of agent trajectories.

## 3. How prediction enters the planner

The most common coupling mechanism is MPC with prediction-derived costs or constraints. In "Future-Oriented Navigation", predicted ellipses are used as hard avoidance constraints in the near term and as soft costs over the full horizon, with $N=20$, $N_{\mathrm{crit}}=5$, and a solve time budget of $0.1\,\mathrm{s}$ per cycle [2505.00237]. In "Adaptive Conformal Prediction for Motion Planning among Dynamic Agents", the MPC enforces a Lipschitz-tightened safety constraint
$$
c(x_{t+\tau}, \hat y_t^\tau) \ge L C^\tau_{t+1},
$$
which converts distribution-free conformal uncertainty radii into robust planning constraints [2212.00278]. In "Robust Predictive Motion Planning by Learning Obstacle Uncertainty", reachable sets derived from the learned control sets are converted into polytope separation constraints with slack variables, allowing the ego trajectory to remain outside obstacle occupancies over the horizon [2403.06222].

A second coupling mechanism is closed-loop interactive rollout inside a search or game formulation. "Deep Interactive Motion Prediction and Planning" uses an Interactive Multi-Agent Policy within an iterative leader-follower or iterative best-response MPC, and optimizes ego sequences with the Cross-Entropy Method while the predictor simulates other agents’ best responses [2204.02392]. "Planning by Simulation" uses Monte Carlo Tree Search to balance and prune unreasonable actions and scenarios, explicitly exploring future interactions encoded within the prediction network [2411.09887]. In both cases, prediction is not an exogenous input but part of the search semantics.

A third mechanism is end-to-end differentiable optimization. DIPP defines the planning problem as a sum of squared residuals for speed tracking, comfort, road adherence, traffic-light compliance, and safety, then solves it with a differentiable Gauss–Newton least-squares optimizer. Because the optimizer is unrolled during training, gradients from planning losses propagate back into both the predictor and the cost weights [2207.10422]. This is a stronger form of prediction-driven planning: the prediction model is trained by the downstream planning objective rather than by open-loop forecast metrics alone.

Sampling-based and feedback planners also exploit predictive structure. "Adaptive Dual-Headway Unicycle Pose Control and Motion Prediction for Optimal Sampling-Based Feedback Motion Planning" derives an explicit convex bound on the future closed-loop trajectory,
$$
x(\tau) \in \mathrm{conv}\{x(t),x_h(t),x_{t_g}(t),x_g\},
$$
and uses this shrinking convex set for constant-time safety verification of candidate edges [2412.10350]. Prediction here is not a learned multimodal future but a feedback motion envelope, yet it serves the same planning function: candidate motions are accepted or rejected based on predicted future occupancy.

## 4. Representative domains and empirical evidence

Autonomous driving has supplied some of the clearest evidence that open-loop prediction accuracy is not sufficient for safe planning. "P4P: Conflict-Aware Motion Prediction for Planning in Autonomous Driving" reports that DenseTNT achieves minFDE $3.65\,\mathrm{m}$ and minADE $1.74\,\mathrm{m}$, but only Top-6 conflict recall $22.11\%$ and a closed-loop collision rate of $12.49\%$. By contrast, P4P records minFDE $15.67\,\mathrm{m}$ and minADE $5.87\,\mathrm{m}$, yet Top-6 conflict recall $97.07\%$ and collision rate $1.39\%$ [2211.01634]. The critical planning signal is therefore conflict identification and relation inference, not merely pointwise displacement error.

Integrated driving systems show similar effects. DIPP attains open-loop prediction ADE/FDE of $0.740/1.814$, and in closed-loop log-replay reports collision rate $5\%$, off-route rate $0\%$, and progress $77.57\,\mathrm{m}$, improving over the separated planning-plus-prediction pipeline, which records collision rate $7\%$ and progress $76.28\,\mathrm{m}$ [2207.10422]. In warehouse-style navigation, "Future-Oriented Navigation" reports success rates of $96\%$ in Scenario 1, $91\%$ in Scenario 2, $77\%$ in Scenario 3, and $90\%$ in Scenario 4, together with inference latency of about $3\,\mathrm{ms}$ per object and typical MPC solve times between $0.008\,\mathrm{s}$ and $0.019\,\mathrm{s}$ [2505.00237].

Human-aware robotic planning provides a second major application area. "HMPO: Human Motion Prediction in Occluded Environments for Safe Motion Planning" couples an occlusion-aware CNN+LSTM predictor to an optimizer that inflates human capsules according to predicted visibility confidence. On Occlusion MoCap, HMPO reduces 3 s joint prediction error from $51.6\,(17.7)\,\mathrm{cm}$ for Tracking+EKF to $31.8\,(6.9)\,\mathrm{cm}$, a $38\%$ reduction, and the qualitative planning results show that the planner first reduces occlusion and then completes the task with collision-free trajectories [2006.00424]. "Planning Coordinated Human-Robot Motions with Neural Network Full-Body Prediction Models" goes further by introducing latent modifiers to a differentiable recurrent human predictor and jointly optimizing robot controls and human prediction adjustments; on collision-avoidance tasks it reports success rate $78\%$ for the joint method, compared with $58\%$ for robot_avoids and $36\%$ for human_avoids [2210.13317].

Prediction-driven planning also appears in compact and fully learned robotic systems. PROM-Net predicts the next 10 future frames in an unsupervised manner, uses about $6$ million trainable parameters and a model size of about $5$ Megabytes, and is intended to feed predicted frames and latent states into MPC for dynamic obstacle avoidance [1906.10182]. STP-Net reformulates motion planning itself as video prediction and reports success rates from $99.8\%$ to $100\%$, while achieving at least $5\times$, $2.6\times$, and $1.8\times$ faster speed with lower path cost on 2D Random Forest, 2D Maze, and 3D Random Forest environments, respectively [2208.11287]. "Map-Predictive Motion Planning in Unknown Environments" uses learned occupancy prediction in unobserved space, achieving about $0.5\,\mathrm{s}$ per planning iteration, roughly $11\times$ faster than the comparison method, while maintaining time-efficient safe navigation [1910.08184].

## 5. Trade-offs, misconceptions, and failure modes

A persistent misconception is that better open-loop trajectory metrics automatically imply better planning. P4P directly contradicts this: low minADE/minFDE predictors can still miss the conflicts that determine downstream collision risk [2211.01634]. A plausible implication is that evaluation protocols for prediction-driven planning must include planning-centric metrics such as conflict recall, closed-loop success, or safety margins, not only forecast displacement.

A second trade-off is conservatism versus feasibility. In "Robust Predictive Motion Planning by Learning Obstacle Uncertainty", worst-case RMPC and the proposed learned-set method both achieve $100\%$ collision-free rate in Monte Carlo trials, but RMPC has completion rate $73\%$ while the proposed method reaches $100\%$; DMPC, which assumes zero uncertainty, has completion rate $100\%$ but only $76\%$ collision-free rate [2403.06222]. Prediction-driven planning therefore lives between optimistic forecasting and worst-case reachability, and the representation of uncertainty largely determines where a system falls on that spectrum.

A third issue is uncertainty calibration. Adaptive conformal prediction addresses this by producing distribution-free, adaptive uncertainty sets with average coverage and an average closed-loop safety guarantee under recursive feasibility, rather than relying on fixed heuristic margins [2212.00278]. By contrast, "Future-Oriented Navigation" shows that loss design itself affects planning behavior: ENLL yields concentrated occupancy and higher success rates than BCE or KLD, whereas BCE and KLD overly inflate occupancy and hurt feasibility; grouping predicted obstacles is further used to mitigate the Freezing Robot Problem [2505.00237].

Prediction-driven systems also inherit modality-specific failure modes. PROM-Net’s predictions blur with horizon under mean squared error training, although motion direction is preserved [1906.10182]. Interactive planners depend on the fidelity of the behavioral model they embed; this suggests that planning-aware conditioning can improve interaction realism, but only insofar as the predictor remains calibrated under the actions the planner explores. In route-conditioned attention models, "SceneMotion-A1" improves open-loop planning score while navigation loss does not, indicating that route conditioning and closed-loop stability are distinct issues rather than interchangeable ones [2512.03756].

## 6. Unification and current directions

Recent work increasingly treats prediction, planning, and simulation as variations of the same motion-modeling problem. "UniMotion" uses a decoder-only Transformer with dedicated interaction modes and joint training across simulation, prediction, and planning. After fine-tuning, it reports prediction minADE $0.5718$, minFDE $1.1643$, mAP $0.4534$, planning error $0.083/0.591/2.246$ at $1/3/5\,\mathrm{s}$, and collision $1.565$, while also achieving a Realism Meta Metric of $0.7851$ in simulation [2602.00566]. This indicates a shift from modular pipelines toward shared motion backbones with task-specific decoding and masking.

A parallel direction is richer conditioning. Route integration in attention-based predictors improves both prediction and planning utility: SceneMotion-A1 reports average mAP $0.3751$, minADE $0.4262\,\mathrm{m}$, minFDE $1.0526\,\mathrm{m}$, and open-loop score $83.59$, compared with a SceneMotion baseline open-loop score of $82.87$ [2512.03756]. CogDrive adds cognitive interaction modes and a two-phase emergency trajectory tree with a short-term root branch that is safe across all modes and long-term branches that remain available under low-probability switching behaviors; it reports on Argoverse 2 b-minFDE $1.833\,\mathrm{m}$, minFDE $1.209\,\mathrm{m}$, miss rate $0.120$, and minADE $0.803\,\mathrm{m}$, as well as minJointFDE $0.914\,\mathrm{m}$ and minJointADE $0.301\,\mathrm{m}$ on INTERACTION [2512.02777].

These developments suggest that the field is converging on a few recurrent principles. Prediction must be action-aware or goal-aware when interaction matters. Uncertainty must be represented in forms planners can consume, whether as modes, ellipses, reachable sets, conformal radii, or convex feedback envelopes. Evaluation must be closed-loop and planning-centric. And, increasingly, the predictive model is no longer a detachable upstream module: it is part of the planning algorithm’s state, objective, or solver itself.

Source: https://www.emergentmind.com/topics/prediction-driven-motion-planning