TopoMaskV2: YOLOP and Lyapunov Navigation

• TopoMaskV2 is an integrated system combining YOLOP perception with a Lyapunov-based controller for map-free, stable autonomous navigation.
• It employs 2D-to-3D projection and cubic polynomial fitting to reconstruct smooth lane centerlines from segmentation outputs in real time.
• The Lyapunov controller guarantees asymptotic error convergence and enhanced performance under nonholonomic dynamics, validated on embedded hardware.

Integrated YOLOP perception and Lyapunov-based control refers to a unified framework in which a deep-learning multi-task perception module based on YOLOP is coupled with a provably stable, Lyapunov-based tracking controller, thereby enabling real-time, robust, and map-free navigation for nonholonomic differential-drive autonomous robots. The approach features an end-to-end system that reconstructs geometric lane centerlines from visual segmentations and drives tracking convergence via rigorous Lyapunov stability design, verified in real-world experiments on embedded hardware (Chen, 1 Dec 2025).

1. Multi-Task YOLOP Perception Module

The perception component utilizes YOLOP (You Only Look Once for Panoptic driving Perception), which extends a CSPDarknet-like YOLOv5 backbone with multiple heads sharing a common encoder. The typical heads include: (1) a detection head for objectness, class probabilities, and bounding boxes via anchor-based, multi-scale feature maps; (2) a lane-line segmentation head producing a two-channel, pixel-wise map; and optionally (3) a drivable-area head for semantic road segmentation. Training is performed on datasets such as BDD100K with the overall loss

$$\mathcal{L} = \mathcal{L}_\text{det} + \mathcal{L}_\text{da-seg} + \mathcal{L}_\text{ll-seg}$$

where $\mathcal{L}_\text{det}$ combines binary cross-entropy, multi-class cross-entropy, and CIoU bounding box regression, while the lane-line and drivable-area segmentation heads utilize a combination of pixel-wise cross-entropy and IoU losses, providing strong supervision for thin lane structures.

At inference, YOLOP processes an input image $I \in \mathbb{R}^{H \times W \times 3}$, applies letterboxing and normalization, and outputs segmentation masks and bounding boxes after postprocessing steps such as thresholding and non-max suppression. Lane boundaries are thus converted to coherent pixel coordinates for subsequent geometric recovery (Chen, 1 Dec 2025).

2. 2D-to-3D Projection, Arc-Length Resampling, and Lane Centerline Estimation

The raw 2D lane mask outputs are reprojected onto the ground plane under the flat-ground assumption or via stereo/depth cues. Given calibration matrix $K$,

$$K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} = \lambda \begin{bmatrix} X_c \ Y_c \ Z_c \end{bmatrix},\quad \text{with}~Z_c = 0$$

and a spatial transform $T_{\rm veh\leftarrow cam} \in SE(3)$ yields the full trajectory of lane-polyline points $\{P_i\}$ in vehicle coordinates.

These points are generally unevenly spaced due to discrete segmentation, so cumulative arc-length $S_i$ is computed along the lane polyline. Uniformly sampled positions $L_k$ at interval $\Delta s$ are interpolated as $\mathcal{L}_\text{det}$0, ensuring near-equal point spacing. The resulting $\mathcal{L}_\text{det}$1 are robustly fit with a cubic polynomial

$\mathcal{L}_\text{det}$2

via a Vandermonde matrix and QR decomposition with column pivoting: $\mathcal{L}_\text{det}$3 The lane centerline is then computed by pointwise averaging of left and right lane fits for a smooth, vehicle-aligned geometric target. This geometric abstraction from raw perception is foundational for reliable path tracking without reliance on HD maps or GPS (Chen, 1 Dec 2025).

3. Lyapunov-Based Path Tracking Controller

The tracking controller is derived directly from Lyapunov stability principles, operating on the error between the robot's current pose $\mathcal{L}_\text{det}$4 and a time-varying target $\mathcal{L}_\text{det}$5 extracted from the centerline. Errors are encoded as

$\mathcal{L}_\text{det}$6

The robot's nonholonomic kinematics yield the system

$\mathcal{L}_\text{det}$7

with the error dynamics

$\mathcal{L}_\text{det}$8

A composite Lyapunov function

$\mathcal{L}_\text{det}$9

is used. Control laws are selected such that

$I \in \mathbb{R}^{H \times W \times 3}$0

These choices ensure negative semi-definite time derivatives of all Lyapunov components, guaranteeing asymptotic convergence: $I \in \mathbb{R}^{H \times W \times 3}$1, $I \in \mathbb{R}^{H \times W \times 3}$2, $I \in \mathbb{R}^{H \times W \times 3}$3 as $I \in \mathbb{R}^{H \times W \times 3}$4.

Gain selection $I \in \mathbb{R}^{H \times W \times 3}$5 is used to tune performance, e.g., on Jetson AGX Orin: $I \in \mathbb{R}^{H \times W \times 3}$6, $I \in \mathbb{R}^{H \times W \times 3}$7, $I \in \mathbb{R}^{H \times W \times 3}$8, $I \in \mathbb{R}^{H \times W \times 3}$9, ensuring all error trajectories are bounded and actuator limits are observed across diverse tracks (Chen, 1 Dec 2025).

4. End-to-End Embedded Perception–Control Pipeline

The system operates on a closed-loop pipeline as follows:

1. Stereo images (e.g., from ZED camera) are acquired at 20–30 Hz, processed by YOLOP at $K$025 Hz.
2. Lane masks are thresholded and clustered for left/right boundaries.
3. Clustered pixels $K$1 are projected to $K$2 in the vehicle frame.
4. Outliers are filtered, arc-length resampling to $K$3 spacing is performed.
5. Robust cubic fitting via QR returns lane curves, centerline is computed.
6. Multiple lookahead targets (2–3 sampled at 2–3 m ahead) are extracted from the centerline.
7. The Lyapunov controller computes linear $K$4 and angular $K$5 velocities for the robot’s differential drive.

This pipeline achieves an end-to-end latency of 30–40 ms (YOLOP: 25 ms, geometric projection and fitting: ~5 ms, control: ≪1 ms), supporting control loops at $K$620 Hz or higher on Jetson AGX Orin hardware (Chen, 1 Dec 2025).

5. Experimental Results and Performance Metrics

Empirical evaluation on embedded platforms was performed on Formula-One–style track courses, comparing the Lyapunov-based controller to conventional benchmarks, for both preset and perception-centric scenarios at $K$7 m/s and $K$8 m/s. Key outcomes (at $K$9 m/s, vision-based control) include:

Metric	Lyapunov-based	Benchmark
Lateral MAE [m]	0.51	0.61
Orientation MAE [rad]	0.15	0.23
RMSE linear speed [m/s]	0.091	0.13
Accum. orientation change [rad]	20.36	26.02
Completion time [s]	206.54	207.00

The system maintained closed-loop stability and smoothness under perceptual degradation (partial lane markings, zebra crossings), with the controller reliably guided by multi-meter lookahead targets, preventing loss of track lock. At higher speeds ($$K^{-1} \begin{bmatrix} u \ v \ 1 \end{bmatrix} = \lambda \begin{bmatrix} X_c \ Y_c \ Z_c \end{bmatrix},\quad \text{with}~Z_c = 0$$0 m/s), all stability and accuracy metrics continued to surpass baseline, with minor increases in error but zero collisions and trajectory smoothness preserved.

6. Context within Perception–Based Control Paradigms

The described architecture exemplifies a fully integrated perception–control loop in which semantic vision outputs are transformed into geometric path primitives via principled 2D–3D modeling and polynomial fitting. The Lyapunov controller operates directly on these geometric references, in contrast to approaches requiring HD prior maps or explicit global state estimation.

Related works extend this paradigm by incorporating learned observation–space certificates, such as Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs), directly into the perception–control stack. For example, in the LOCUS controller (Dawson et al., 2022), neural networks trained on YOLOP-derived features yield CBF and CLF guarantees for safety and liveness, with switching hybrid controllers optimizing both trajectory pursuit and constraint satisfaction in real-time via QPs. However, the classic integrated YOLOP and Lyapunov-based approach (Chen, 1 Dec 2025) achieves robust, theoretically guaranteed convergence using fully analytic control design, thereby serving as a reference for direct provably stable perception-based navigation pipelines.

7. Limitations and Prospects

The framework eliminates the need for global maps and tolerates partial or noisy segmentation. Nonetheless, limitations arise from the flat-ground assumption (or depth sensor fidelity), the discrete sampling of lookahead points, and possible cumulative errors in the projection and fitting pipeline. The Lyapunov-based controller attains bounded and asymptotically convergent tracking under the modeled error dynamics, but extreme perception failures or rapid topological changes remain challenging.

Integration with hybrid CBF/CLF–based architectures, further data-driven adaptation, and real-time adaptation of gains under environmental uncertainty represent promising directions for extending and generalizing the current framework. The demonstrated architecture provides a high-confidence pathway toward certifiable, embedded, real-time mobile robot navigation based entirely on visual semantic perception and analytically grounded feedback control (Chen, 1 Dec 2025, Dawson et al., 2022).