MOSU: Multimodal Outdoor Navigation

• MOSU is a multimodal outdoor navigation system that integrates GPS/QGIS routing with LiDAR, RGB imagery, and Vision–Language Models to enable robust and socially aware robot autonomy.
• The system uses a hierarchical planning architecture by decoupling global route planning from local trajectory generation with methods like CVAE for candidate sampling and DWA for motion control.
• Empirical evaluations on the GND dataset show a 10% improvement in terrain traversability while maintaining efficient navigation distance, demonstrating its practical benefits in complex on-road environments.

Searching arXiv for papers on "MOSU" and related variants to ground the article. MOSU is an autonomous long-range navigation system for mobile robots that enhances global navigation through multimodal perception and on-road scene understanding. It integrates geometric, semantic, and contextual information, combining GPS and QGIS map-based routing for high-level global path planning with multi-modal trajectory generation for local navigation refinement. For local trajectory generation, it leverages LiDAR-based geometric data for obstacle avoidance, image-based semantic segmentation for traversability assessment, and Vision–LLMs (VLMs) to capture social context and enable adherence to social norms in complex environments. In real-world on-road environments and on the GND dataset, MOSU achieves a 10% improvement in traversability on navigable terrains while maintaining a comparable navigation distance to existing global navigation methods (Liang et al., 7 Jul 2025).

1. Definition and operational scope

MOSU is presented as a multimodal outdoor navigation stack for long-range robot autonomy. Its central design premise is that outdoor navigation cannot be reduced to geometric obstacle avoidance alone: scene understanding must also incorporate semantic traversability and contextual or social acceptability. The system therefore combines three local perceptual channels—LiDAR, RGB imagery, and VLM-mediated contextual reasoning—with a separate global routing stage based on GPS coordinates and QGIS services (Liang et al., 7 Jul 2025).

The architecture is explicitly hierarchical. GPS/QGIS routing produces an ordered list of high-level subgoals $\{g_0,g_1,\dots,g_M\}$ spaced approximately $50$ m apart, while the local planner runs in a loop at approximately $5$–$10$ Hz and generates short-horizon trajectories toward the current subgoal. This suggests that MOSU is not a monolithic end-to-end policy, but a layered navigation system in which long-range route structure and short-range motion feasibility are optimized by different modules.

A recurrent misconception in multimodal navigation discussions is that language-conditioned models replace conventional planning. In MOSU, the VLM does not replace routing, trajectory generation, or low-level control. Rather, it contributes a social compliance score $r(\tau_n)$ and a velocity mode, while trajectory sampling is handled by a CVAE and motion execution is handled by Dynamic Window Approach (DWA).

2. Hierarchical planning architecture

The system is organized into four stages: GPS/QGIS routing, multi-modal trajectory generation, score fusion and trajectory selection, and motion and velocity control (Liang et al., 7 Jul 2025).

At the routing level, the inputs are start and goal GPS coordinates, and the output is a sequence of high-level subgoals. Although MOSU relies on a public routing service rather than constructing its own waypoint graph, the routing problem is formalized over a graph $G=(V,E)$, where vertices are GPS-indexed map waypoints and edges represent navigable roads or paths with nonnegative costs $c_{ij}$. A standard Dijkstra or A* formulation is given as

$$\min_{P=(v_0,\dots,v_K)}\sum_{k=0}^{K-1} c_{v_k v_{k+1}} \quad \text{s.t. } v_0=s,\; v_K=g.$$

For A*, the usual augmented objective is

$f(v)=g(v)+h(v),$

where $g(v)$ is the cost from $50$0 to $50$1. After computing the route, MOSU downsamples or re-samples the resulting waypoints at approximately $50$2 m intervals to form the GPS subgoals $50$3.

This global-local decomposition has a clear systems consequence: route feasibility at the map level and route compliance at the scene level are decoupled. A plausible implication is that MOSU can preserve long-range route consistency while adapting local motion to transient obstacles, sidewalk structure, and socially constrained settings.

3. Multi-modal trajectory generation and scene understanding

The local planner synthesizes candidate trajectories $50$4 by combining geometric, semantic, and contextual signals (Liang et al., 7 Jul 2025).

For geometric reasoning, MOSU uses LiDAR-based obstacle avoidance. Raw 3D point clouds $50$5 are voxel-filtered and ground-segmented. Candidate trajectories are generated with a CVAE. The paper gives

$50$6

followed by sampling $50$7 and decoding to obtain $50$8. Collision checking is defined through

$50$9

and geometric confidence is written as

$5$0

or equivalently in terms of predicted latent variance.

For semantic reasoning, MOSU applies Mask2Former to RGB images. The model outputs per-pixel class probabilities

$5$1

for classes such as road, sidewalk, and vegetation. The segmentation objective is the standard cross-entropy loss

$5$2

Traversability for a trajectory $5$3 is computed by projecting its waypoints into the image, collecting the associated pixels $5$4, and averaging whether the predicted class belongs to $5$5:

$5$6

For contextual reasoning, MOSU uses a VLM. The image embedding is denoted $5$7, the prompt embedding is $5$8, and the context score is expressed as

$5$9

where $10$0 is a binary mask overlaying $10$1 on the image. In practice, the VLM outputs a ranking position $10$2, which is converted to

$10$3

The VLM also outputs a velocity mode that is later passed to DWA. This module is therefore not merely descriptive; it directly influences both path choice and execution aggressiveness.

4. Score fusion, trajectory selection, and control

MOSU fuses four scores for each candidate trajectory: geometric confidence $10$4, semantic traversability $10$5, VLM ranking $10$6, and distance-to-GPS-subgoal $10$7 (Liang et al., 7 Jul 2025). The selection rule is

$10$8

The weights $10$9 can be set by cross-validation on held-out traversability data or learned end-to-end by back-propagating a differentiable surrogate of the selection objective.

Once the best trajectory $r(\tau_n)$0 is selected, MOSU feeds $r(\tau_n)$1 together with the VLM’s velocity mode into Dynamic Window Approach. DWA then outputs velocity commands $r(\tau_n)$2. The operational loop is: transform the current GPS subgoal into the robot frame; read LiDAR, RGB, and odometry history; generate candidate trajectories with the CVAE; compute semantic traversability from Mask2Former outputs; query the VLM every $r(\tau_n)$3 loops or asynchronously; compute the distance score; fuse scores; select $r(\tau_n)$4; execute through DWA; and update the subgoal index when the robot comes within $r(\tau_n)$5 of the current subgoal.

This control structure clarifies the division of labor among modules. Geometric feasibility is enforced before execution, semantic segmentation estimates whether the robot is on road or sidewalk, the VLM injects social-context priors, and DWA remains the final motion-level controller. A plausible implication is that MOSU is designed for robustness under heterogeneous failure modes rather than for maximal architectural simplicity.

5. Training objectives and evaluation protocol

MOSU is evaluated on the GND (Global Navigation Dataset), described as covering 10 campuses with multimodal LiDAR and RGB data, per-pixel traversability labels, and ground-truth optimal paths (Liang et al., 7 Jul 2025).

The CVAE is trained with a loss consisting of a reconstruction term and a KL regularizer:

$r(\tau_n)$6

When joint finetuning is used, the total objective is

$r(\tau_n)$7

The VLM is used off-the-shelf, with Gemini given as an example, and is only prompt-tuned if needed.

The reported evaluation metrics are traversability, distance-to-target, success rate, and path length or efficiency. Traversability is defined as

$r(\tau_n)$8

Success rate is the fraction of runs that reach the GPS goal within a time limit, and path length or efficiency is the ratio of actual path length to straight-line distance.

In real-world on-road evaluations and on GND, MOSU achieves approximately $r(\tau_n)$9 traversability, described as $G=(V,E)$0 over the best prior; approximately $G=(V,E)$1 distance-to-target, described as on par with the state of the art; a comparable success rate; and runtime of approximately $G=(V,E)$2 s per loop, dominated by the VLM. The runtime characterization is important because it identifies the contextual module, rather than the geometric or semantic modules, as the primary latency bottleneck.

6. Interpretation, limitations, and terminology

MOSU’s stated contribution is to couple GPS/QGIS routing with LiDAR-, vision-, and language-based local reasoning in order to obtain more robust, socially aware, and higher-traversability long-range navigation than prior mapless methods (Liang et al., 7 Jul 2025). The empirical result emphasized in the abstract is improved traversability on navigable terrains without sacrificing navigation distance, which situates the method in a trade-off regime where route progress and local safety are jointly optimized rather than treated as competing objectives.

At the same time, the formulation makes several design choices that delimit its scope. MOSU relies on a public routing service rather than building its own waypoint graph. The VLM is off-the-shelf and only prompt-tuned if needed. Runtime is dominated by the VLM, at approximately $G=(V,E)$3 s per loop. These facts indicate that the system’s performance depends not only on perception quality and trajectory sampling, but also on the latency and ranking fidelity of an external multimodal model. This suggests that deployment constraints may be influenced by VLM query frequency, asynchronous scheduling, and prompt design, even though the paper’s core navigation framework is otherwise modular.

The acronym should also be distinguished from similarly named works in adjacent areas. “MoSu” refers to “Most Replayed Multimodal Video Summarization,” a large-scale trimodal video summarization benchmark with visual, text, and audio features (Kim et al., 1 Mar 2026), while “MoSo” denotes “moving-one-sample-out,” a first-order data-pruning approach based on leave-one-out empirical risk approximation (Tan et al., 2023). Those works are unrelated in task formulation, modality semantics, and evaluation regime. In the robotics literature represented here, MOSU specifically denotes the autonomous long-range robot navigation system based on multi-modal scene understanding.