MPCC: Multimodal Planning with Complex Constraints

• MPCC is a planning framework that synthesizes feasible trajectories under tightly coupled temporal, geometric, dynamic, and perceptual constraints.
• It employs manifold projection, sequence-structured MPC, scenario trees, and complementarity-based formulations to handle multimodality effectively.
• Practical implementations in robotics and autonomous driving demonstrate significant runtime improvements, enhanced feasibility, and robust collision avoidance.

Searching arXiv for the cited papers and closely related work on MPCC to ground the article. Multimodal Planning with Complex Constraints denotes a family of planning and control problems in which feasible behavior must be synthesized under tightly coupled temporal, geometric, dynamic, logical, contact, or resource constraints, while simultaneously reasoning over multiple modes, branches, manifolds, or perceptual inputs. In robotics and control, the term covers formulations based on constraint manifolds, sequence- or graph-structured task constraints, scenario trees, and complementarity-constrained hybrid dynamics; in recent multimodal-agent benchmarks, it denotes planning from images, screenshots, or web interfaces under dense cross-modal constraints such as budget, timing, and spatial feasibility (Qureshi et al., 2020, Toussaint et al., 2022, Manganaris et al., 19 Mar 2026, Wang et al., 9 Feb 2026). Across these usages, the central technical problem is not merely choosing an action sequence, but preserving global feasibility when each local decision changes the admissible set of subsequent states, modes, or plans.

1. Formal scope and canonical problem structures

A common formalization models feasible behavior as a trajectory constrained to lie on one or more manifolds. In CoMPNet, the robot configuration space is $C \subset \mathbb{R}^d$, constraints are given by $h(q)=0$ and $g(q)\le 0$, the constraint manifold is

$M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$

and multimodal planning seeks a continuous path $\tau:[0,1]\to C_{\mathrm{free}}$ such that $\tau(t)\in \bigcup_i M_i$, with mode-dependent feasibility tolerances when the path lies in mode $i$ (Qureshi et al., 2020). This formulation captures disjoint or weakly connected feasible sets induced by grasping, upright transport, articulated-object manipulation, or other task-dependent kinematic relations.

A second canonical form represents multimodality as a sequence or partial order of constraint activations. Sequence-of-Constraints MPC assumes a fixed Task-and-Motion Planning skeleton with phases $k=1,\dots,K$, waypoint constraints $\hat{\phi}_k(x)\le 0$, and running constraints $\bar{\phi}_k(x,\dot{x})\le 0$, and solves

$h(q)=0$0

subject to phase ordering, waypoint satisfaction, and phasewise running feasibility (Toussaint et al., 2022). GoC-MPC generalizes this total order to a directed acyclic graph $h(q)=0$1, where node constraints are enforced at times $h(q)=0$2 and edge constraints over intervals $h(q)=0$3, with additional binary assignment variables $h(q)=0$4 for dynamic multi-agent allocation (Manganaris et al., 19 Mar 2026).

A third form arises in tightly coupled symbolic-planning benchmarks. In WorldTravel, activities share temporal variables $h(q)=0$5 and $h(q)=0$6, and feasibility is defined by constraints such as

$h(q)=0$7

so that a single booking decision can determine downstream feasibility across an entire itinerary (Wang et al., 9 Feb 2026). This same coupling pattern reappears in calendar and meeting planning benchmarks where budget, concurrency, and travel constraints must be satisfied jointly (Ji et al., 31 Jul 2025).

2. Constraint encodings and the role of multimodality

The literature employs several distinct constraint encodings. In complementarity-based formulations, mode switching is represented directly by orthogonality conditions such as

$h(q)=0$8

or, in planar stick-slip manipulation,

$h(q)=0$9

under nonnegativity constraints on friction-cone edge variables and tangential motion rates (Moura et al., 2021). In dynamic locomotion with unscheduled contact, unilateral contact is encoded by

$g(q)\le 0$0

so that contact activation and deactivation emerge from the solve rather than from a prescribed schedule (Mastalli et al., 2019).

Scenario-tree methods encode multimodality through branching rather than complementarity. Branch MPC and stochastic MPC for driving represent the future of uncontrolled agents by a finite set of modes or policies, enforce shared-prefix non-anticipativity before a branching time, and optimize branchwise trajectories subject to collision-avoidance and road constraints (Chen et al., 2021, Nair et al., 2023). Reachability-based Branch MPC reduces a large prediction set to driving corridors, then imposes branch-dependent time-varying corridor constraints while preserving all prediction modes through pruning and clustering by intersection (Bouzidi et al., 4 Feb 2025).

A different encoding appears in contouring-control formulations, where constraints are expressed in a path-aligned coordinate frame through contouring and lag errors, lateral corridor bounds, and progress variables. These formulations are central in autonomous racing and articulated-vehicle planning, although in that literature “MPCC” denotes Model Predictive Contouring Control rather than complementarity (Li et al., 6 Feb 2025, Aertssen et al., 27 Apr 2026).

These encodings serve different computational regimes. Complementarity offers direct access to hybrid mode logic but inherits non-smoothness and degeneracy. Skeleton- or graph-based formulations trade generality for online solvability by fixing or partially fixing the discrete structure. Scenario trees externalize discrete uncertainty into branches, while manifold methods avoid discrete variables by projecting continuous plans back onto constraint sets. This suggests that “multimodality” is not a single object: depending on the formulation, it may mean multiple contact modes, multiple future agent behaviors, multiple constraint manifolds, or multiple feasible plan hypotheses.

3. Planning architectures: projection, decomposition, batching, and receding-horizon repair

CoMPNet is representative of learned constrained motion planning on manifolds. It combines a task encoder, an environment encoder, and a stochastic planning network $g(q)\le 0$1, then projects generated states onto the active manifold with a Jacobian-based step

$g(q)\le 0$2

inside a bidirectional planner akin to BiRRT (Qureshi et al., 2020). The method was evaluated on Baxter manipulation tasks with TSR-encoded constraints and reported mean runtimes such as $g(q)\le 0$3 s versus $g(q)\le 0$4 s for CBiRRT on bartender J/F/S tasks, while success rates across constrained manipulation tasks were reported as around $g(q)\le 0$5 for all compared methods.

Sequence-of-Constraints MPC addresses the execution gap between TAMP and reactive control by decomposing each MPC cycle into three nonlinear programs: waypoint optimization, timing optimization over phase durations $g(q)\le 0$6 and waypoint velocities $g(q)\le 0$7, and a short receding-horizon path optimizer over $g(q)\le 0$8 s (Toussaint et al., 2022). The timing subproblem uses cubic interpolation with per-segment cost

$g(q)\le 0$9

and the controller adds phase backtracking when expected waypoint transitions or running constraints fail. Reported cycle time was approximately $M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$0 ms on a standard CPU, with fewer than $M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$1 Newton steps per subproblem in steady state, and in 20 trials on randomized waypoints total transition time was $M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$2 s versus $M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$3 s for a sequence of independent 1-phase controllers.

GoC-MPC extends this decomposition to partially ordered, multi-agent tasks. Each cycle solves a waypoint-and-assignment MINLP, an inter-agent timing QP over cubic splines, and a short-horizon tracking QP, while using graph-based progression and backtracking to reinsert violated nodes into the remaining subgraph (Manganaris et al., 19 Mar 2026). The assignment matrix is row-stochastic, constraints are defined over tracked 3D keypoints, and the method runs from visual observations alone. In the reported static setting, GoC-MPC achieved 10/10 success on Block-Stacking with Max $M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$4 s, Avg $M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$5 s, and Length $M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$6 m, compared with ReKep at 7/10 success, Max $M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$7 s, Avg $M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$8 s, and Length $M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},$9 m.

Batch non-holonomic trajectory optimization offers a different architecture for multimodal MPC. Instead of one local search, it runs several goal-directed trajectory optimizations in parallel, reusing common KKT systems across modalities and scoring the results by a meta cost (Adajania et al., 2021). The resulting batch optimizer reported trajectories with lower meta cost and up to $\tau:[0,1]\to C_{\mathrm{free}}$0 faster runtime than competing baselines, with mean $\tau:[0,1]\to C_{\mathrm{free}}$1 s per MPC cycle at batch size $\tau:[0,1]\to C_{\mathrm{free}}$2.

A more recent synthesis combines learning-based multimodal generation with optimization-based refinement. Unified generation-refinement planning uses reward-guided conditional flow matching to produce multiple candidate trajectories, then refines them with MPPI; the optimal MPPI trajectory warm-starts the next CFM generation, while CFM samples provide informed priors for MPPI (Mizuta et al., 2 Aug 2025). On UCY and SDD, the combined method reported $\tau:[0,1]\to C_{\mathrm{free}}$3 collision rate and terminal goal-reaching error $\tau:[0,1]\to C_{\mathrm{free}}$4 m and $\tau:[0,1]\to C_{\mathrm{free}}$5 m, respectively, with average planning time $\tau:[0,1]\to C_{\mathrm{free}}$6 s.

4. Autonomous driving and contouring-control variants

Driving is a major domain in which multimodality is induced by uncertain surrounding agents. In the stochastic-MPC line, target-vehicle futures are modeled by Gaussian mixtures or linear time-varying mode-conditioned predictors, and collision avoidance is enforced through multimodal chance constraints. One formulation jointly optimizes over affine feedback policies and per-mode risk allocations $\tau:[0,1]\to C_{\mathrm{free}}$7, using the aggregate condition

$\tau:[0,1]\to C_{\mathrm{free}}$8

to distribute risk across modes (Nair et al., 2023). In unprotected-left-turn experiments, reported feasibility rose from $\tau:[0,1]\to C_{\mathrm{free}}$9 for open loop and $\tau(t)\in \bigcup_i M_i$0 for fixed risk to $\tau(t)\in \bigcup_i M_i$1 for the proposed joint policy-and-risk optimization; in lane change, feasibility rose to $\tau(t)\in \bigcup_i M_i$2, with solve time $\tau(t)\in \bigcup_i M_i$3 ms.

At traffic intersections, a related SMPC formulation exploited GMM structure to design a feedback policy class amenable to SOCP reformulation, and reported $\tau(t)\in \bigcup_i M_i$4 feasibility for the proposed method across three simulated intersection scenarios, compared with lower feasibility for open-loop baselines; runtimes were approximately $\tau(t)\in \bigcup_i M_i$5–$\tau(t)\in \bigcup_i M_i$6 ms for the proposed method and $\tau(t)\in \bigcup_i M_i$7–$\tau(t)\in \bigcup_i M_i$8 ms for an open-loop ablation (Nair et al., 2021).

Branch MPC addresses interaction by maintaining multiple executable futures that share a common prefix before branching. In simulation and quadruped experiments, coherent risk measures such as CVaR acted as a tuning knob between performance and robustness, yielding overtake, lane-change, merging, and passing behaviors that the paper describes as human-like (Chen et al., 2021). Reachability-based Branch MPC refines this idea by replacing explicit obstacle constraints with branchwise driving corridors whose number is reduced by Jaccard-based clustering. In Monte Carlo experiments with 100 random merges and 3 traffic participants, the reported result was $\tau(t)\in \bigcup_i M_i$9 success and $i$0 collision, with real-time runtime below $i$1 ms even in intersection scenes with up to 12 traffic participants (Bouzidi et al., 4 Feb 2025).

In racing and heavy-vehicle control, contouring-control variants treat multimodality less as external uncertainty and more as trade-offs among progress, curvature, and multiple vehicle anchor points. Curvature-integrated MPCC augments the standard contouring objective with a curvature-derived reference velocity profile

$i$2

and on a 1:10 F1TENTH platform reported lap-time reduction of $i$3–$i$4 and mean-speed increase of $i$5–$i$6 relative to baselines (Li et al., 6 Feb 2025). For tractor-semitrailers, multi-anchor MPCC introduces separate progress states and contouring errors for tractor front axle, tractor rear axle, and semitrailer axle, together with explicit corridor constraints for each anchor. The reported implementation used $i$7, $i$8 s, and computation time of approximately $i$9 ms per iteration, and demonstrated successful navigation of a representative logistics scenario in both forward and reverse direction (Aertssen et al., 27 Apr 2026).

5. Complementarity-based MPCC: contact, hybrid modes, and solver technology

In optimization and contact mechanics, MPCC conventionally means Mathematical Programs with Complementarity Constraints. The defining structure is the complementarity pair

$k=1,\dots,K$0

or its equivalent orthogonality form $k=1,\dots,K$1 under nonnegativity (Monteiro et al., 2015, Wang et al., 27 Mar 2025). This strand is directly relevant to multimodal planning because contact/no-contact, stick/slip, and activate/deactivate decisions are hybrid mode switches expressed without discrete variables.

For planar non-prehensile manipulation, trajectory optimization with complementarity encodes sticking and sliding via auxiliary variables $k=1,\dots,K$2 and tangential motion rates $k=1,\dots,K$3, with MPCC constraints

$k=1,\dots,K$4

The reported controller used $k=1,\dots,K$5, $k=1,\dots,K$6 s, and a 50 Hz loop; compared with a mixed-integer alternative, it achieved better tracking and more consistent computation times, while the planner converged faster and scaled better with horizon and obstacle presence (Moura et al., 2021).

In dynamic legged motion without scheduled contacts, both a centroidal-dynamics MPCC and a full-body MPCC were used to discover contact sequences automatically. Contact was encoded by unilateral complementarity $k=1,\dots,K$7, and the reported hierarchical approach showed median time reduction of $k=1,\dots,K$8 and average time reduction of $k=1,\dots,K$9 on step-jumping tasks relative to single-shot full-body optimization, while also discovering countermovement and transient foothold strategies (Mastalli et al., 2019).

Because complementarity violates standard constraint qualifications, numerical treatment is a central topic. One regularization strategy solves a sequence of NLPs such as

$\hat{\phi}_k(x)\le 0$0

with $\hat{\phi}_k(x)\le 0$1, inside an SQP framework (Monteiro et al., 2015). On 95 MacMPEC problems, the reported failure rates were 3 for Reg, 1 for Reg-one, 54 for Reg-eq, and 29 for Reg-eq-one, establishing the superiority of inequality relaxations over equality relaxations in that implementation. A more recent distributed approach, ALADIN-$\hat{\phi}_k(x)\le 0$2, applies an $\hat{\phi}_k(x)\le 0$3-Exact Penalty-Barrier reformulation,

$\hat{\phi}_k(x)\le 0$4

then splits the resulting problem into distributed subproblems coordinated by an ALADIN QP (Wang et al., 27 Mar 2025). The reported result was fast convergence without a globalization strategy while maintaining high precision.

A recurring misconception is that complementarity is the only mathematically proper way to represent multimodality. The literature does not support that claim. Complementarity is indispensable when contact forces, gap functions, or friction-edge activations must be modeled explicitly, but sequence-structured, graph-structured, and branch-structured methods often avoid complementarity precisely to retain real-time solvability (Toussaint et al., 2022, Manganaris et al., 19 Mar 2026).

6. Benchmarks, terminology, and open directions

Recent benchmarks have extended MPCC from robotics and control into multimodal large-model evaluation. WorldTravel defines a benchmark of 150 real-world travel scenarios across 5 cities, with an average of 15+ interdependent constraints and a multimodal environment of 2,003 rendered webpages returned by 8 screenshot-based APIs (Wang et al., 9 Feb 2026). On this benchmark, GPT-5.2 achieved $\hat{\phi}_k(x)\le 0$5 feasibility in text-only settings and $\hat{\phi}_k(x)\le 0$6 in multimodal settings, and feasibility collapsed from $\hat{\phi}_k(x)\le 0$7 at 6–7 hard constraints to $\hat{\phi}_k(x)\le 0$8 at 10–11 hard constraints. The MPCC benchmark for multimodal LLMs contains 2,700 tasks and 6,300 images across Flight Planning, Calendar Planning, and Meeting Planning, and reported that closed-source models achieved only $\hat{\phi}_k(x)\le 0$9 feasible plans while open-source models averaged below $\bar{\phi}_k(x,\dot{x})\le 0$0 (Ji et al., 31 Jul 2025). MARBLE targets multimodal spatial reasoning and planning through M-Portal and M-Cube; all 12 advanced models obtained near-random performance on M-Portal and $\bar{\phi}_k(x,\dot{x})\le 0$1 accuracy on M-Cube, while only simplified subtasks exceeded random baselines (Jiang et al., 28 Jun 2025).

The abbreviation itself is heterogeneous:

Usage of “MPCC”	Typical meaning	Representative domain
Multimodal Planning with Complex Constraints	Multimodal-input or multi-constraint planning under tightly coupled feasibility conditions	Travel, calendar, meeting, spatial-reasoning benchmarks
Mathematical Programs with Complementarity Constraints	Hybrid dynamics and mode switching via orthogonality or complementarity	Contact-rich manipulation, locomotion, numerical optimization
Model Predictive Contouring Control	Path-following MPC with contouring/lag errors and corridor constraints	Racing, autonomous driving, articulated commercial vehicles

This terminological ambiguity is not merely lexical. It reflects three distinct technical emphases: multimodal perception and long-horizon reasoning, hybrid optimization with complementarity, and contouring-based predictive control. A plausible implication is that cross-community comparisons can become misleading unless the underlying mathematical object is specified explicitly.

Several open directions recur across the corpus. WorldTravel and MARBLE argue for tighter integration of high-fidelity perception with long-horizon reasoning, because perception and planning remain independent bottlenecks (Wang et al., 9 Feb 2026, Jiang et al., 28 Jun 2025). GoC-MPC and SecMPC point toward richer online integration of perception with partial orders, disturbance recovery, and assignment changes (Manganaris et al., 19 Mar 2026, Toussaint et al., 2022). The generation–refinement framework suggests one concrete synthesis: learned multimodal proposal generation paired with optimization-based refinement and safety enforcement (Mizuta et al., 2 Aug 2025). A plausible implication is that future MPCC systems will increasingly combine four elements that today often appear separately: structured representations of modes, differentiable or symbolic feasibility checkers, uncertainty-aware receding-horizon optimization, and multimodal perceptual grounding.