---
title: 'MPCC: Multimodal Planning with Complex Constraints'
url: https://www.emergentmind.com/topics/multimodal-planning-with-complex-constraints-mpcc
type: topic
---

# MPCC: Multimodal Planning with Complex Constraints

Searching arXiv for the cited papers and closely related work on MPCC to ground the article.
Multimodal Planning with Complex Constraints denotes a family of planning and control problems in which feasible behavior must be synthesized under tightly coupled temporal, geometric, dynamic, logical, contact, or resource constraints, while simultaneously reasoning over multiple modes, branches, manifolds, or perceptual inputs. In robotics and control, the term covers formulations based on constraint manifolds, sequence- or graph-structured task constraints, scenario trees, and complementarity-constrained hybrid dynamics; in recent multimodal-agent benchmarks, it denotes planning from images, screenshots, or web interfaces under dense cross-modal constraints such as budget, timing, and spatial feasibility [2008.03787] [2203.05390] [2603.18400] [2602.08367]. Across these usages, the central technical problem is not merely choosing an action sequence, but preserving global feasibility when each local decision changes the admissible set of subsequent states, modes, or plans.

## 1. Formal scope and canonical problem structures

A common formalization models feasible behavior as a trajectory constrained to lie on one or more manifolds. In CoMPNet, the robot configuration space is \(C \subset \mathbb{R}^d\), constraints are given by \(h(q)=0\) and \(g(q)\le 0\), the constraint manifold is
\[
M=\{q\in C \mid h(q)=0,\; g(q)\le 0\},
\]
and multimodal planning seeks a continuous path \(\tau:[0,1]\to C_{\mathrm{free}}\) such that \(\tau(t)\in \bigcup_i M_i\), with mode-dependent feasibility tolerances when the path lies in mode \(i\) [2008.03787]. This formulation captures disjoint or weakly connected feasible sets induced by grasping, upright transport, articulated-object manipulation, or other task-dependent kinematic relations.

A second canonical form represents multimodality as a sequence or partial order of constraint activations. Sequence-of-Constraints MPC assumes a fixed Task-and-Motion Planning skeleton with phases \(k=1,\dots,K\), waypoint constraints \(\hat{\phi}_k(x)\le 0\), and running constraints \(\bar{\phi}_k(x,\dot{x})\le 0\), and solves
\[
\min_{\xi,t_{1:K}} \; t_K + \int_0^{t_K} c(\xi,\dot{\xi},\ddot{\xi})\,dt
\]
subject to phase ordering, waypoint satisfaction, and phasewise running feasibility [2203.05390]. GoC-MPC generalizes this total order to a directed acyclic graph \(\mathcal{G}=(V,E,\Phi_V,\Phi_E)\), where node constraints are enforced at times \(t_v\) and edge constraints over intervals \([t_a,t_b]\), with additional binary assignment variables \(A\in\{0,1\}^{K\times M}\) for dynamic multi-agent allocation [2603.18400].

A third form arises in tightly coupled symbolic-planning benchmarks. In WorldTravel, activities share temporal variables \(s_i\) and \(f_i\), and feasibility is defined by constraints such as
\[
s_j \ge f_i + \tau_{ij} + \delta_j,\qquad
s_i \in \mathcal{R}_i,\qquad
[s_i,f_i]\subseteq \mathcal{W}_i,\qquad
f_i-s_i \ge p_i,
\]
so that a single booking decision can determine downstream feasibility across an entire itinerary [2602.08367]. This same coupling pattern reappears in calendar and meeting planning benchmarks where budget, concurrency, and travel constraints must be satisfied jointly [2507.23382].

## 2. Constraint encodings and the role of multimodality

The literature employs several distinct constraint encodings. In complementarity-based formulations, mode switching is represented directly by orthogonality conditions such as
\[
0 \le \lambda \perp g(x) \ge 0,
\]
or, in planar stick-slip manipulation,
\[
\lambda_v^\top \dot{\phi}_v + \varepsilon = 0
\]
under nonnegativity constraints on friction-cone edge variables and tangential motion rates [2109.13145]. In dynamic locomotion with unscheduled contact, unilateral contact is encoded by
\[
0 \le f_{n,i} \perp \phi_i(q) \ge 0,
\]
so that contact activation and deactivation emerge from the solve rather than from a prescribed schedule [1904.04600].

Scenario-tree methods encode multimodality through branching rather than complementarity. Branch MPC and stochastic MPC for driving represent the future of uncontrolled agents by a finite set of modes or policies, enforce shared-prefix non-anticipativity before a branching time, and optimize branchwise trajectories subject to collision-avoidance and road constraints [2109.05128] [2310.20561]. Reachability-based Branch MPC reduces a large prediction set to driving corridors, then imposes branch-dependent time-varying corridor constraints while preserving all prediction modes through pruning and clustering by intersection [2502.02550].

A different encoding appears in contouring-control formulations, where constraints are expressed in a path-aligned coordinate frame through contouring and lag errors, lateral corridor bounds, and progress variables. These formulations are central in autonomous racing and articulated-vehicle planning, although in that literature “MPCC” denotes Model Predictive Contouring Control rather than complementarity [2502.03695] [2604.24064].

These encodings serve different computational regimes. Complementarity offers direct access to hybrid mode logic but inherits non-smoothness and degeneracy. Skeleton- or graph-based formulations trade generality for online solvability by fixing or partially fixing the discrete structure. Scenario trees externalize discrete uncertainty into branches, while manifold methods avoid discrete variables by projecting continuous plans back onto constraint sets. This suggests that “multimodality” is not a single object: depending on the formulation, it may mean multiple contact modes, multiple future agent behaviors, multiple constraint manifolds, or multiple feasible plan hypotheses.

## 3. Planning architectures: projection, decomposition, batching, and receding-horizon repair

CoMPNet is representative of learned constrained motion planning on manifolds. It combines a task encoder, an environment encoder, and a stochastic planning network \( \hat q = \mathrm{PNet}(Z_c,Z_o,q_t,q_T)\), then projects generated states onto the active manifold with a Jacobian-based step
\[
q \leftarrow \hat q - J_F(\hat q)^+ F(\hat q),
\]
inside a bidirectional planner akin to BiRRT [2008.03787]. The method was evaluated on Baxter manipulation tasks with TSR-encoded constraints and reported mean runtimes such as \(4.92 \pm 2.42\) s versus \(54.81 \pm 25.82\) s for CBiRRT on bartender J/F/S tasks, while success rates across constrained manipulation tasks were reported as around \(90\%\) for all compared methods.

Sequence-of-Constraints MPC addresses the execution gap between TAMP and reactive control by decomposing each MPC cycle into three nonlinear programs: waypoint optimization, timing optimization over phase durations \(\tau_{1:K}\) and waypoint velocities \(v_k\), and a short receding-horizon path optimizer over \(H\approx 1\) s [2203.05390]. The timing subproblem uses cubic interpolation with per-segment cost
\[
\psi=\frac{12}{\tau_k^3}\|D_k\|^2+\frac{1}{\tau_k}\|V_k\|^2,
\]
and the controller adds phase backtracking when expected waypoint transitions or running constraints fail. Reported cycle time was approximately \(30\) ms on a standard CPU, with fewer than \(5\) Newton steps per subproblem in steady state, and in 20 trials on randomized waypoints total transition time was \(11.2\pm0.6\) s versus \(14.1\pm0.6\) s for a sequence of independent 1-phase controllers.

GoC-MPC extends this decomposition to partially ordered, multi-agent tasks. Each cycle solves a waypoint-and-assignment MINLP, an inter-agent timing QP over cubic splines, and a short-horizon tracking QP, while using graph-based progression and backtracking to reinsert violated nodes into the remaining subgraph [2603.18400]. The assignment matrix is row-stochastic, constraints are defined over tracked 3D keypoints, and the method runs from visual observations alone. In the reported static setting, GoC-MPC achieved 10/10 success on Block-Stacking with Max \(0.520 \pm 0.311\) s, Avg \(0.108 \pm 0.012\) s, and Length \(2.54 \pm 0.216\) m, compared with ReKep at 7/10 success, Max \(8.38 \pm 0.062\) s, Avg \(7.11 \pm 0.186\) s, and Length \(4.75 \pm 2.59\) m.

Batch non-holonomic trajectory optimization offers a different architecture for multimodal MPC. Instead of one local search, it runs several goal-directed trajectory optimizations in parallel, reusing common KKT systems across modalities and scoring the results by a meta cost [2109.10392]. The resulting batch optimizer reported trajectories with lower meta cost and up to \(6\times\) faster runtime than competing baselines, with mean \(0.07\) s per MPC cycle at batch size \(l=11\).

A more recent synthesis combines learning-based multimodal generation with optimization-based refinement. Unified generation-refinement planning uses reward-guided conditional flow matching to produce multiple candidate trajectories, then refines them with MPPI; the optimal MPPI trajectory warm-starts the next CFM generation, while CFM samples provide informed priors for MPPI [2508.01192]. On UCY and SDD, the combined method reported \(0.00\%\) collision rate and terminal goal-reaching error \(0.10\pm0.03\) m and \(0.10\pm0.04\) m, respectively, with average planning time \(0.076\) s.

## 4. Autonomous driving and contouring-control variants

Driving is a major domain in which multimodality is induced by uncertain surrounding agents. In the stochastic-MPC line, target-vehicle futures are modeled by Gaussian mixtures or linear time-varying mode-conditioned predictors, and collision avoidance is enforced through multimodal chance constraints. One formulation jointly optimizes over affine feedback policies and per-mode risk allocations \(r_{t,j}\), using the aggregate condition
\[
\sum_{j=1}^{J} p_{t,j} r_{t,j} \ge 1-\epsilon
\]
to distribute risk across modes [2310.20561]. In unprotected-left-turn experiments, reported feasibility rose from \(81.14\%\) for open loop and \(98.37\%\) for fixed risk to \(99.88\%\) for the proposed joint policy-and-risk optimization; in lane change, feasibility rose to \(98.76\%\), with solve time \(397.19\) ms.

At traffic intersections, a related SMPC formulation exploited GMM structure to design a feedback policy class amenable to SOCP reformulation, and reported \(100\%\) feasibility for the proposed method across three simulated intersection scenarios, compared with lower feasibility for open-loop baselines; runtimes were approximately \(61\)–\(64\) ms for the proposed method and \(1.6\)–\(1.7\) ms for an open-loop ablation [2109.09792].

Branch MPC addresses interaction by maintaining multiple executable futures that share a common prefix before branching. In simulation and quadruped experiments, coherent risk measures such as CVaR acted as a tuning knob between performance and robustness, yielding overtake, lane-change, merging, and passing behaviors that the paper describes as human-like [2109.05128]. Reachability-based Branch MPC refines this idea by replacing explicit obstacle constraints with branchwise driving corridors whose number is reduced by Jaccard-based clustering. In Monte Carlo experiments with 100 random merges and 3 traffic participants, the reported result was \(96\%\) success and \(0\%\) collision, with real-time runtime below \(100\) ms even in intersection scenes with up to 12 traffic participants [2502.02550].

In racing and heavy-vehicle control, contouring-control variants treat multimodality less as external uncertainty and more as trade-offs among progress, curvature, and multiple vehicle anchor points. Curvature-integrated MPCC augments the standard contouring objective with a curvature-derived reference velocity profile
\[
g(K_i^n)=\exp(-\alpha (K_i^n)^2),\qquad
v_{\mathrm{ref}}(s_i)=(1-\beta_i)v_+ + \beta_i \bar v,
\]
and on a 1:10 F1TENTH platform reported lap-time reduction of \(11.4\%\)–\(12.5\%\) and mean-speed increase of \(15.2\%\)–\(17.0\%\) relative to baselines [2502.03695]. For tractor-semitrailers, multi-anchor MPCC introduces separate progress states and contouring errors for tractor front axle, tractor rear axle, and semitrailer axle, together with explicit corridor constraints for each anchor. The reported implementation used \(N=75\), \(\Delta t=0.2\) s, and computation time of approximately \(100\) ms per iteration, and demonstrated successful navigation of a representative logistics scenario in both forward and reverse direction [2604.24064].

## 5. Complementarity-based MPCC: contact, hybrid modes, and solver technology

In optimization and contact mechanics, MPCC conventionally means Mathematical Programs with Complementarity Constraints. The defining structure is the complementarity pair
\[
0 \le y \perp c(x) \ge 0,
\]
or its equivalent orthogonality form \(G(x)^\top H(x)=0\) under nonnegativity [1510.05925] [2503.21502]. This strand is directly relevant to multimodal planning because contact/no-contact, stick/slip, and activate/deactivate decisions are hybrid mode switches expressed without discrete variables.

For planar non-prehensile manipulation, trajectory optimization with complementarity encodes sticking and sliding via auxiliary variables \(\lambda_\pm=\mu f_n \pm f_t\) and tangential motion rates \(\dot\phi_{C+},\dot\phi_{C-}\), with MPCC constraints
\[
\dot{\phi}_{C+}\ge 0,\;
\dot{\phi}_{C-}\ge 0,\;
\lambda_+\ge 0,\;
\lambda_-\ge 0,\;
\lambda_v^\top \dot{\phi}_v + \varepsilon = 0.
\]
The reported controller used \(N=25\), \(\Delta t=1/25\) s, and a 50 Hz loop; compared with a mixed-integer alternative, it achieved better tracking and more consistent computation times, while the planner converged faster and scaled better with horizon and obstacle presence [2109.13145].

In dynamic legged motion without scheduled contacts, both a centroidal-dynamics MPCC and a full-body MPCC were used to discover contact sequences automatically. Contact was encoded by unilateral complementarity \(0 \le f_{n,i} \perp \phi_i(q)\ge 0\), and the reported hierarchical approach showed median time reduction of \(48.29\%\) and average time reduction of \(30.46\%\) on step-jumping tasks relative to single-shot full-body optimization, while also discovering countermovement and transient foothold strategies [1904.04600].

Because complementarity violates standard constraint qualifications, numerical treatment is a central topic. One regularization strategy solves a sequence of NLPs such as
\[
x_{1j}x_{2j}\le t
\quad\text{or}\quad
x_1^\top x_2 \le t,
\]
with \(t\downarrow 0\), inside an SQP framework [1510.05925]. On 95 MacMPEC problems, the reported failure rates were 3 for Reg, 1 for Reg-one, 54 for Reg-eq, and 29 for Reg-eq-one, establishing the superiority of inequality relaxations over equality relaxations in that implementation. A more recent distributed approach, ALADIN-\(\beta\), applies an \(\ell_1\)-Exact Penalty-Barrier reformulation,
\[
\min_{x,p,n}\; f(x)-\mu\sum_i \ln(-x^{(i)})+\rho (p+n)^\top e
\quad
\text{s.t.}\quad
g(x)-p+n=0,\; p\ge 0,\; n\ge 0,
\]
then splits the resulting problem into distributed subproblems coordinated by an ALADIN QP [2503.21502]. The reported result was fast convergence without a globalization strategy while maintaining high precision.

A recurring misconception is that complementarity is the only mathematically proper way to represent multimodality. The literature does not support that claim. Complementarity is indispensable when contact forces, gap functions, or friction-edge activations must be modeled explicitly, but sequence-structured, graph-structured, and branch-structured methods often avoid complementarity precisely to retain real-time solvability [2203.05390] [2603.18400].

## 6. Benchmarks, terminology, and open directions

Recent benchmarks have extended MPCC from robotics and control into multimodal large-model evaluation. WorldTravel defines a benchmark of 150 real-world travel scenarios across 5 cities, with an average of 15+ interdependent constraints and a multimodal environment of 2,003 rendered webpages returned by 8 screenshot-based APIs [2602.08367]. On this benchmark, GPT-5.2 achieved \(32.67\%\) feasibility in text-only settings and \(19.33\%\) in multimodal settings, and feasibility collapsed from \(36.3\%\) at 6–7 hard constraints to \(3.4\%\) at 10–11 hard constraints. The MPCC benchmark for multimodal large language models contains 2,700 tasks and 6,300 images across Flight Planning, Calendar Planning, and Meeting Planning, and reported that closed-source models achieved only \(21.3\%\) feasible plans while open-source models averaged below \(11\%\) [2507.23382]. MARBLE targets multimodal spatial reasoning and planning through M-Portal and M-Cube; all 12 advanced models obtained near-random performance on M-Portal and \(0\%\) accuracy on M-Cube, while only simplified subtasks exceeded random baselines [2506.22992].

The abbreviation itself is heterogeneous:

| Usage of “MPCC” | Typical meaning | Representative domain |
|---|---|---|
| Multimodal Planning with Complex Constraints | Multimodal-input or multi-constraint planning under tightly coupled feasibility conditions | Travel, calendar, meeting, spatial-reasoning benchmarks |
| Mathematical Programs with Complementarity Constraints | Hybrid dynamics and mode switching via orthogonality or complementarity | Contact-rich manipulation, locomotion, numerical optimization |
| Model Predictive Contouring Control | Path-following MPC with contouring/lag errors and corridor constraints | Racing, autonomous driving, articulated commercial vehicles |

This terminological ambiguity is not merely lexical. It reflects three distinct technical emphases: multimodal perception and long-horizon reasoning, hybrid optimization with complementarity, and contouring-based predictive control. A plausible implication is that cross-community comparisons can become misleading unless the underlying mathematical object is specified explicitly.

Several open directions recur across the corpus. WorldTravel and MARBLE argue for tighter integration of high-fidelity perception with long-horizon reasoning, because perception and planning remain independent bottlenecks [2602.08367] [2506.22992]. GoC-MPC and SecMPC point toward richer online integration of perception with partial orders, disturbance recovery, and assignment changes [2603.18400] [2203.05390]. The generation–refinement framework suggests one concrete synthesis: learned multimodal proposal generation paired with optimization-based refinement and safety enforcement [2508.01192]. A plausible implication is that future MPCC systems will increasingly combine four elements that today often appear separately: structured representations of modes, differentiable or symbolic feasibility checkers, uncertainty-aware receding-horizon optimization, and multimodal perceptual grounding.

Source: https://www.emergentmind.com/topics/multimodal-planning-with-complex-constraints-mpcc