Multi-Modal Path Planning

• Multi-Modal Path Planning is defined as the integration of continuous variables and discrete modes to generate feasible, optimal trajectories under realistic dynamic constraints.
• Techniques include sampling-based hybrid planners, grid-based searches, and optimization methods that address mode-switch costs, uncertainty, and complex dynamics.
• Applications span robotic locomotion, multi-robot manipulation, and multimodal transportation, highlighting challenges in scalability and real-time adaptability.

A multi-modal path planning problem arises when an agent (or team of agents) must reason over a state space comprising both continuous variables (e.g., geometric or kinematic states) and discrete modes that encode qualitatively distinct system behaviors or task states. This class encompasses robotic systems with hybrid locomotion capabilities (e.g., walking, flying, rolling, swimming), collaborative multi-robot manipulation tasks with discrete grasp/handover events, as well as high-level routing across multimodal transportation networks and probabilistic prediction/planning frameworks. Multi-modal path planning methodologies seek to efficiently generate feasible, often optimal or near-optimal, trajectories that incorporate discrete mode switches and continuous actions under realistic dynamic constraints and uncertainty.

1. Formal Definition and Theoretical Frameworks

The multi-modal path planning problem is formalized by coupling a continuous configuration space with a discrete mode space, leading to a joint hybrid or product state space. For multi-robot task sequences, as in Hartmann et al. (Hartmann et al., 5 Mar 2025), the planning state is $q=(q_R, m)$ with $q_R \in Q_{r_1} \times \dots \times Q_{r_N}$ (robot joint configurations) and $m$ a discrete mode encoding task assignments and object attachments. Valid trajectories must satisfy $s(t) \in \mathcal{M}$, the set of mode sequences compatible with a partial-order dependency graph $G=(S,E)$ of primitive tasks, and respect mode-specific collision constraints.

In hybrid locomotion or mobility systems, the state is $x=(c, m)$ where $c$ is the continuous robot state (e.g., position, velocity) and $m$ denotes the motion mode (e.g., walking, flying, crawling). Mode transitions are governed by guard conditions and reset maps, yielding a formal hybrid automaton structure (Suh et al., 2019, Rajput, 2023, Sihite et al., 2022). Edge costs within each mode correspond to optimal transport or energy-optimal trajectory costs, whereas inter-modal transitions are associated with explicit transition costs reflecting time, energy, or mechanical configuration penalties.

Beyond robotics, dynamic multimodal stochastic shortest paths (DMSSP) generalize this to settings where the agent reasons over discrete modes, continuous states, and uncertain exogenous context, formulating the problem as an MDP with state space $S = \mathcal{D} \times \mathcal{C}$, action set including both intra-modal and mode-switch actions, and both stochastic and deterministic transitions (Choudhury et al., 2019).

2. State-Space Construction and Mode Encoding

State-space construction in modern multi-modal planners universally integrates discrete modes into a single search state, either as

• direct product spaces (e.g., $(q, m)$), or
• hybrid graphs/sampled representations comprising both continuous states and modes as vertices $q_R \in Q_{r_1} \times \dots \times Q_{r_N}$0 (Hartmann et al., 5 Mar 2025, Suh et al., 2019, Bao et al., 7 Sep 2025, Sihite et al., 2022).

In robotics, modes correspond either to high-level semantic actions (e.g., grasp, handover), types of locomotion (e.g., walk, fly), or discrete robot assignments on task graphs. Edge feasibility and cost are mode-dependent: within-mode actions follow the kinematics/dynamics and constraints of the current mode, while mode-switch edges enforce mechanical feasibility and often invoke system-specific reset or morphing routines (e.g., flight/ground transitions in a morphogenetic UAV).

In the context of multi-robot task and motion planning, task modes are encoded by a dependency graph $q_R \in Q_{r_1} \times \dots \times Q_{r_N}$1; mode transitions correspond to the completion of primitive tasks (e.g., object attachment), and the planner must respect the partial ordering imposed by $q_R \in Q_{r_1} \times \dots \times Q_{r_N}$2 (Hartmann et al., 5 Mar 2025). For multimodal transportation, mode-annotated graphs merge road networks, time-expanded transit graphs, and transfer/link nodes; transitions are allowed only at interface nodes (e.g., train-to-bus stations), with each edge labeled by its permitted travel mode(s) (Tischner, 2018).

3. Algorithmic Approaches and Planning Paradigms

Contemporary algorithmic approaches fall into several broad families:

Sampling-Based Hybrid Planners

• Multi-modal RRT*/PRM*: These index all modes into the search state, allowing mode switches to occur at transition samples (with mode hopping driven by task constraints or workspace structure). The planners retain probabilistic completeness and asymptotic optimality, provided sampling covers all modes (Hartmann et al., 5 Mar 2025).
• Hybrid A*: For mobile robots with multiple kinematic behaviors (e.g., independently steerable wheels), Hybrid A* is extended by adding a discrete mode index to the node state and designing heuristics that incorporate mode-switch penalties and mode-dependent Reeds-Shepp curves (Bao et al., 7 Sep 2025).

Grid/Graph-Based Search

• A* or Dijkstra search on PRMs or uniform grids where nodes encode mode, with mode-dependent cost maps and feasibility checks (Sihite et al., 2022, Rajput, 2023).
• For multi-modal route planning in transit/road networks, Access-Node Routing (ANR) combines optimized engines for individual modal sub-networks and small-scale spatial joins (Tischner, 2018).

Optimization and Approximate Dynamic Programming

• Joint optimization is decomposed into discrete mode-switch planning and local continuous optimal control within each mode (Suh et al., 2019). Edge costs in hybrid graphs are learned offline from batches of trajectory optimizations and approximated for rapid online graph search.
• Receding-horizon or model-predictive control (MPC) paradigms incorporating multi-modal prediction distributions (e.g., mixtures of stochastic future trajectories for other agents in autonomous driving) are solved via stochastic programming over feedback policies and adaptive risk allocation (Nair et al., 2023).

Hierarchical and Reactive Control

• Adaptive-dimensionality and multi-representation search: planning is first performed in low-dimensional, mode-specific spaces and then lifted (as needed) to high-dimensional full-body/plant representations to verify full feasibility (e.g., humanoid mobility with biped, crawl, climb representations, plus full joint graph) (Dornbush et al., 2018).
• Hierarchical interleaving combines discrete high-level sequencing (tasks, locomotion modes) with closed-loop local planning or value-iteration over continuous states under uncertainty (Choudhury et al., 2019).
• Multi-modal Model Predictive Path Integral (MPPI) control blends candidate controls corresponding to different high-level plans or behaviors (e.g., brake, accelerate, evasive-steer), weighting them according to simulated trajectory cost, enabling real-time adaptation to disturbances and complex environments (Zhang et al., 2023, Bertipaglia et al., 29 Aug 2025).

Perception-Driven and AI-Augmented Hybrid Planners

• Integration of language/vision models for semantic waypoint selection and spatial grounding, as in MMP-A* (Ha et al., 5 Jan 2026), leverages LLM-generated high-level guides refined by vision-LLMs to filter out spatially infeasible waypoints, embedding this information into heuristic search.
• Task-conditioned modality routing with LLMs and compact tokenization supports path planning robust to sensor degradations, enabling planning and scene captioning under challenging perceptual conditions (Wang et al., 28 Feb 2026).

4. Representative Application Domains and Benchmarks

Multi-modal path planning spans diverse domains, with distinct methodological emphases and benchmarking:

Multi-Robot Manipulation and Industrial Assembly

• Tasks involve multiple robots (arms, mobile bases) performing collaborative manipulation, handovers, and sequential assembly, requiring planners to optimize over both geometric arrangements and discrete task modes with dependencies. Hartmann et al. (Hartmann et al., 5 Mar 2025) provide a benchmark with 21 base scenarios encompassing extensive variation in dimensionality, robot heterogeneity, and goal horizon.

Legged-Aerial and Hybrid Robotic Locomotion

• Path planning for morphogenetic UAVs or hybrid robots, such as the M4 robot (Rajput, 2023), combines ground traversability estimation (via CNNs on 2.5D elevation maps), explicit energy modeling for each mobility mode, and A* search over the combined grid-mode space.
• Energy-aware multi-modal planners for flying-driving or amphibious robots leverage approximate dynamic programming, define edge costs via offline trajectory optimization, and obtain significant energy savings over naive, single-modal baselines (Sihite et al., 2022, Suh et al., 2019).
• MorphoMove (Mustafa et al., 2024) applies a hybrid A* planner to select between walking and flight, using MPC-based step-length control for precise ground trajectory following.

Multi-Agent and Multi-Goal Problems

• Composite multi-modal planners for multi-goal, multi-robot tasks require concurrent search over all robot configurations and shared object modes (e.g., attached, grasped, stowed), with mode transitions encoding collaborative actions (Hartmann et al., 5 Mar 2025).

Autonomous Vehicles and Transportation

• Multi-modal routing over road and transit networks demands specialized NN search structures (cover trees) for stop access, heuristic-accelerated shortest path solvers (ALT), efficient timetable scanning (CSA), and careful modeling of transition costs (Tischner, 2018).
• Integrated prediction and planning with route intention points (as in Int2Planner (Chen et al., 22 Jan 2025)) samples plausible short-term end-goals along a reference route, permitting joint optimization of multi-modal trajectories in urban traffic involving agent interactions.
• Risk-optimized stochastic predictive control accommodates multi-modal uncertainty in other vehicles' motions (mixtures over maneuvers), ensuring constraint satisfaction at real-time frequencies (Nair et al., 2023).

Perception-Conditioned and LLM-Coupled Planning

• Wild-Drive (Wang et al., 28 Feb 2026) exemplifies a pipeline where LLMs, coupled with task-conditioned modality routing, yield robust captioning and trajectory planning even with missing or degraded sensor modalities, using compact, locality-aware tokenization and a dedicated planning token for structured input to a lightweight GRU decoder.

5. Evaluation Metrics and Empirical Performance

Benchmarking of multi-modal path planners is domain specific, with common metrics including:

• Solution cost (trajectory length, makespan, energy consumption, or a composite metric sensitive to mode-switching penalties) (Hartmann et al., 5 Mar 2025, Suh et al., 2019, Rajput, 2023).
• Initial solution time and cost convergence rate (especially for sampling-based planners) (Hartmann et al., 5 Mar 2025).
• Success rate (feasibility/completeness), robustness under uncertainty or exogenous disturbances, and constraint violations (Dornbush et al., 2018, Choudhury et al., 2019, Nair et al., 2023).
• Root-mean-square and maximum tracking error for path execution, notably when path following is decoupled from global planning (Mustafa et al., 2024).
• Computational resource usage (CPU/GPU time per solve, memory footprint), scalability to environment/robot dimensionality, and performance under hardware or inference latency.
• Scenario-specific metrics: final displacement/minimum average error for path prediction under observation/model uncertainty (Wang et al., 28 Feb 2026), energy use compared to all-flight baselines (Sihite et al., 2022), operational and storage ratios relative to classical A* (Ha et al., 5 Jan 2026).

Empirical results across benchmark domains show that multi-modal planners can find feasible, often near-optimal solutions to high-dimensional and combinatorial problems in industrial manipulation, hybrid mobility, and urban navigation, outperforming single-modal or heuristically decoupled baselines both in solution quality and efficiency (Hartmann et al., 5 Mar 2025, Suh et al., 2019, Bao et al., 7 Sep 2025, Tischner, 2018). Notably, real-world and high-fidelity simulated deployments report reliability, energy efficiency, and robustness to uncertainty comparable to (or exceeding) systems built atop separate single-mode planning stacks (Sihite et al., 2022, Rajput, 2023, Nair et al., 2023).

6. Insights, Limitations, and Directions for Future Research

Unified formulation in composite (continuous × discrete) spaces—either via explicit product graphs or adaptive-dimensionality search—enables algorithmic completeness and optimality in principle (Hartmann et al., 5 Mar 2025, Dornbush et al., 2018, Suh et al., 2019). However, as the number of robots, modes, and subgoals increases, the curse of dimensionality and combinatorics constrains scalability. Approaches that exploit problem structure (e.g., subdimensional search, conflict-based search, receding-horizon decoupling, or hierarchical abstraction) are actively investigated but still face trade-offs in completeness and computation (Hartmann et al., 5 Mar 2025, Choudhury et al., 2019).

Sampling and learning-based methods (informed or experience-based sampling, offline cost learning, and AI-guided waypoint generation) show promise in accelerating convergence, especially in high-dimensional or unstructured environments (Hartmann et al., 5 Mar 2025, Ha et al., 5 Jan 2026, Wang et al., 28 Feb 2026). The integration of multimodal prediction and perception modules, as well as real-time feedback policy adaptation under uncertainty, are proving essential in dynamic, human-populated, or semantically rich domains (Nair et al., 2023, Ha et al., 5 Jan 2026).

Common limitations include scalability to very high-dimensional problems (e.g., $q_R \in Q_{r_1} \times \dots \times Q_{r_N}$3 DOF, $q_R \in Q_{r_1} \times \dots \times Q_{r_N}$4 task horizons (Hartmann et al., 5 Mar 2025)), reliance on high-quality mode transition sampling or precomputed cost maps, and, in learning-based pipelines, robustness to systematic model bias or mode-misspecification (Ha et al., 5 Jan 2026, Wang et al., 28 Feb 2026). Proposed future directions include hierarchical and parallelized planning, deeper learning-driven mode transition discovery, accelerated collision checking, and tighter unification of perception and planning to support real-world autonomy at scale.

---

References:

• "A Benchmark for Optimal Multi-Modal Multi-Robot Multi-Goal Path Planning with Given Robot Assignment" (Hartmann et al., 5 Mar 2025)
• "Hybrid Planning for Dynamic Multimodal Stochastic Shortest Paths" (Choudhury et al., 2019)
• "Energy-Efficient Motion Planning for Multi-Modal Hybrid Locomotion" (Suh et al., 2019)
• "Hybrid A* Path Planning with Multi-Modal Motion Extension for Four-Wheel Steering Mobile Robots" (Bao et al., 7 Sep 2025)
• "MorphoMove: Bi-Modal Path Planner with MPC-based Path Follower for Multi-Limb Morphogenetic UAV" (Mustafa et al., 2024)
• "Towards Autonomous Multi-Modal Mobility Morphobot (M4) Robot: Traversability Estimation and 3D Path Planning" (Rajput, 2023)
• "Efficient Path Planning and Tracking for Multi-Modal Legged-Aerial Locomotion Using Integrated Probabilistic Road Maps (PRM) and Reference Governors (RG)" (Sihite et al., 2022)
• "A Single-Planner Approach to Multi-Modal Humanoid Mobility" (Dornbush et al., 2018)
• "Int2Planner: An Intention-based Multi-modal Motion Planner for Integrated Prediction and Planning" (Chen et al., 22 Jan 2025)
• "Predictive Control for Autonomous Driving with Uncertain, Multi-modal Predictions" (Nair et al., 2023)
• "MMP-A*: Multimodal Perception Enhanced Incremental Heuristic Search on Path Planning" (Ha et al., 5 Jan 2026)
• "Wild-Drive: Off-Road Scene Captioning and Path Planning via Robust Multi-modal Routing and Efficient LLM" (Wang et al., 28 Feb 2026)
• "Multi-Modal MPPI and Active Inference for Reactive Task and Motion Planning" (Zhang et al., 2023)
• "Multi-Modal Model Predictive Path Integral Control for Collision Avoidance" (Bertipaglia et al., 29 Aug 2025)
• "Multi-Modal Route Planning in Road and Transit Networks" (Tischner, 2018)
• "Multi-Step Gaussian Process Propagation for Adaptive Path Planning" (Beaudin et al., 21 Apr 2026)