OR-C2P: Off-Road Caption-to-Plan Benchmark

• The paper introduces OR-C2P, a benchmark integrating structured off-road scene captioning with ego-frame trajectory planning to address adverse sensing conditions.
• Key methodology employs coupled tasks with closed-set captioning and planning via LiDAR-camera fusion, demonstrating robust performance metrics like BLEU and BERTScore.
• The design leverages modality-routing through MoRo-Former to handle single-modality failures, ensuring safety-centric, interpretable outputs for off-road navigation.

The OR-C2P Benchmark is a multimodal off-road benchmark introduced with "Wild-Drive: Off-Road Scene Captioning and Path Planning via Robust Multi-modal Routing and Efficient LLM" (Wang et al., 28 Feb 2026). OR-C2P stands for Off-Road Caption-to-Plan and is designed to jointly train and evaluate two coupled tasks: structured off-road scene captioning and downstream path planning. Its defining emphasis is robustness to naturally degraded sensing and explicit single-modality failures in unstructured environments such as rain, fog, snow, and darkness, where urban driving priors—lane markings, traffic rules, and stable sensor quality—do not reliably apply (Wang et al., 28 Feb 2026).

1. Definition, scope, and terminological boundaries

In (Wang et al., 28 Feb 2026), OR-C2P denotes a benchmark for structured, task-oriented off-road scene captioning together with downstream path planning, built to evaluate robustness under adverse environmental conditions and modality degradation. The benchmark couples interpretable language outputs with a concrete local trajectory prediction task rather than treating captioning as an isolated vision-language problem (Wang et al., 28 Feb 2026).

The acronym is not globally unique. A different 2026 paper, "Concept-to-Pixel: Prompt-Free Universal Medical Image Segmentation," concerns a universal medical segmentation framework called C2P and evaluates it on a unified medical benchmark, but that paper itself is not the off-road OR-C2P benchmark discussed here (Chen et al., 18 Mar 2026). The OCT registration paper "Non-rigid Point Cloud Registration for Middle Ear Diagnostics with Endoscopic Optical Coherence Tomography" explicitly states that it does not explicitly define or name a benchmark called “OR-C2P” (Liu et al., 2023). A separate MaxSAT benchmark on weighted AND/OR graphs is also described under the OR-C2P shorthand in the provided literature, but it targets identification of security-critical cyber-physical components rather than multimodal driving (Barrère et al., 2019). Within current off-road multimodal driving literature, the benchmark name refers specifically to the Off-Road Caption-to-Plan setting of (Wang et al., 28 Feb 2026).

2. Coupled task design and structured outputs

OR-C2P is organized around two coupled outputs: a structured caption and an ego-frame future trajectory. The captioning side uses concise, closed-set templates generated from five task heads, with canonical labels intended to improve robustness and reproducibility for efficient LLMs (Wang et al., 28 Feb 2026).

• Weather and illumination: Weather $\in \{\text{sunny, cloudy, rainy, snowy, foggy}\}$ and illumination $$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$.
• Drivable area: Availability $\in \{\text{clear, partially\_blocked, blocked}\}$ and direction $$\in \{\text{front, front\_left, front\_right, left, right}\}$$.
• Terrain traversability: Terrain type $$\in \{\text{dirt, gravel, grass, mud, sand, snow, rock}\}$$; the paper lists these as examples, while the dataset summary states 10 terrain types. Difficulty $\in \{\text{easy, moderate, hard, impassable}\}$.
• Obstacle detection: Category $$\in \{\text{vehicle, pedestrian, animal, rock, tree, pole, building, unknown}\}$$, with direction $$\in \{\text{front, front\_left, front\_right, left, right}\}$$ and distance $\in \{\text{near, mid, far}\}$.
• Driving suggestion: Action $$\in \{\text{go\_straight, turn\_left, turn\_right, stop}\}$$.

The discrete driving suggestion is derived by clustering future 10 s ego-frame trajectories with K-means:

$$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$0

where $$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$1 is the vectorized future trajectory of sample $$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$2 in the ego frame (Wang et al., 28 Feb 2026).

Path planning is supervised using trajectories built by registering LiDAR with KISS-ICP and then smoothing with B-spline interpolation. Future targets are defined at $$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$3 seconds from the current timestamp. The model-side planning interface uses a special planning token whose embedding conditions a GRU decoder:

$$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$4

where $$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$5 is the learned embedding of the planning token (Wang et al., 28 Feb 2026).

This task design ties semantics directly to control-relevant outputs. A plausible implication is that OR-C2P evaluates not only descriptive fidelity but also whether compact language abstractions remain aligned with local navigation behavior under sensing degradation.

3. Dataset composition, sensing modalities, and annotation pipeline

OR-C2P is built upon ORAD-3D and curates aligned LiDAR-camera samples with caption and planning supervision. The benchmark uses synchronized monocular RGB and LiDAR as its core modalities. RGB resolution is 1280×720, and the LiDAR-camera pairs are calibrated and temporally aligned; the paper reports 57K aligned LiDAR–camera pairs and also gives a table total of 57,808 frames (Wang et al., 28 Feb 2026).

Split	Sequences	Frames
Train	100	39,727
Val	15	5,717
Test	29	12,164

The table reports 144 sequences / 57,808 frames overall, while the narrative elsewhere mentions 145 sequences and “57K” pairs (Wang et al., 28 Feb 2026). The benchmark also reports 19,527 frames with obstacles, 5 weather types, 4 illumination types, and 10 terrain types. Collection spans China from spring to winter, with each sequence covering roughly 100 meters of off-road terrain, including forest- and trail-like areas, mud, snow, gravel, grass, rock, and low-light or night scenes (Wang et al., 28 Feb 2026).

GNSS ground truth from the underlying ORAD-3D is available and used as reference, while KISS-ICP is additionally used to derive smooth pose sequences for trajectory supervision (Wang et al., 28 Feb 2026).

The annotation pipeline is structured rather than free-form. Qwen2.5-VL-72B is used to produce initial supervision through concise fixed-vocabulary Q&A templates, after which outputs are post-processed into canonical labels. Weather and terrain traversability are generated as camera-based pseudo-labels. Drivable area and obstacle detection are obtained by image-first inference, then refined with calibrated LiDAR point clouds through clustering and filtering. The paper reports random human verification for quality assurance (Wang et al., 28 Feb 2026).

4. Robustness protocol and evaluation methodology

A central design goal of OR-C2P is evaluation under naturally degraded sensing rather than purely synthetic corruption regimes. The benchmark includes rain, snow, fog, and low-light/night scenes collected from real-world off-road environments. It does not define artificial corruption severity levels (Wang et al., 28 Feb 2026).

Missing-modality robustness is explicitly built into the protocol. During training, inputs are randomly set to camera-only, LiDAR-only, or dual-modality with equal probability:

• Camera-only: probability $$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$6
• LiDAR-only: probability $$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$7
• Dual-modality: probability $$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$8

This is intended to induce robust routing and fusion under single-modality failures (Wang et al., 28 Feb 2026). Additional sensing such as RADAR, thermal, event cameras, and IMU is not part of the benchmark as described.

Captioning is evaluated with BLEU-1, BLEU-2, BLEU-4, and BERTScore, reported in the paper as BERT-P. The BLEU definition is

$$\in \{\text{bright\_light, daylight, twilight, darkness}\}$$9

with brevity penalty

$\in \{\text{clear, partially\_blocked, blocked}\}$0

where $\in \{\text{clear, partially\_blocked, blocked}\}$1 is clipped $\in \{\text{clear, partially\_blocked, blocked}\}$2-gram precision, $\in \{\text{clear, partially\_blocked, blocked}\}$3 is candidate length, and $\in \{\text{clear, partially\_blocked, blocked}\}$4 is reference length (Wang et al., 28 Feb 2026).

For BERTScore, the paper reports precision, while the standard formulation is

$\in \{\text{clear, partially\_blocked, blocked}\}$5

$\in \{\text{clear, partially\_blocked, blocked}\}$6

$\in \{\text{clear, partially\_blocked, blocked}\}$7

Planning is evaluated with FDE and minADE. The paper defines FDE at the final horizon as

$\in \{\text{clear, partially\_blocked, blocked}\}$8

and gives the multimodal minADE metric as

$\in \{\text{clear, partially\_blocked, blocked}\}$9

Evaluation is carried out on the provided train/val/test split, with results reported on the test set. Random seeds, repeated runs, and confidence intervals are not reported; the published values are single-run means (Wang et al., 28 Feb 2026).

5. Baselines and reported benchmark results

OR-C2P is evaluated against several baseline families. Captioning baselines include MiniGPT-4, LLaVA-1.5, Instruct-BLIP, LLaMA-AdapterV2, LiDAR-LLM, BEV-LLM-1B, and BEV-LLM-8B. Planning baselines include BEV-LLM-GRU, TopoPath, CoverNet, MTP, and MultiPath (Wang et al., 28 Feb 2026).

For captioning, the benchmark reports that Wild-Drive-3B attains the strongest scores among the listed methods, with BLEU-1 71.72, BLEU-2 60.79, BLEU-4 49.26, and BERT-P 98.13 (Wang et al., 28 Feb 2026).

Method	BLEU-4	BERT-P
Wild-Drive-3B	49.26	98.13
Wild-Drive-0.5B	47.21	95.79
BEV-LLM-8B	47.93	88.17
LiDAR-LLM	46.11	87.49
Instruct-BLIP	10.29	—

The reported trend is that robust multimodal fusion and routing improve both lexical and semantic captioning metrics relative to larger but fixed-fusion LLM systems (Wang et al., 28 Feb 2026).

Planning results show a more differentiated picture. Wild-Drive outperforms the LLM-based planning baselines and several classic multimodal predictors, but TopoPath remains the strongest dedicated planner with lower error (Wang et al., 28 Feb 2026).

Method	FDE	minADE
TopoPath	0.92	0.43
Wild-Drive	1.09	0.66
BEV-LLM-GRU	1.33	0.97
MTP	1.59	0.84
MultiPath	1.56	0.92
CoverNet	2.47	1.31

This benchmark outcome is important because it prevents a common overgeneralization: OR-C2P does not show that an LLM-centric system is universally best at planning. Rather, it shows that the benchmark can discriminate between caption quality, LLM-based multimodal robustness, and specialized planning performance.

6. Reference architecture, interpretive significance, and limitations

OR-C2P is closely tied to the Wild-Drive reference system, and the benchmark’s structure makes its intended algorithmic stresses explicit. Wild-Drive uses a Modality-Routing Transformer (MoRo-Former) with $$\in \{\text{front, front\_left, front\_right, left, right}\}$$0 task groups and $$\in \{\text{front, front\_left, front\_right, left, right}\}$$1 queries per task, yielding 320 learnable queries total, fewer than BEV-LLM (512) and LiDAR-LLM (576). RGB is encoded with frozen DINOv3 ViT-S/16, LiDAR BEV with VoxelNet, and the LLM backbone is Qwen2.5-0.5B-Instruct or Qwen2.5-3B-Instruct with LoRA, rank = alpha = 16 (Wang et al., 28 Feb 2026).

The routing mechanism predicts branch assignment over LiDAR-only, camera-only, and dual-modality experts; hard routing is applied at inference. The multimodal output is then used both for structured text generation and for the GRU-based trajectory head. This coupling indicates that OR-C2P is not merely a dataset of paired captions and trajectories; it is a benchmark for modality-aware caption-to-plan transfer (Wang et al., 28 Feb 2026).

Ablations reinforce that interpretation. Full Wild-Drive achieves BLEU-1/BLEU-4/BERT-P = 71.72 / 49.26 / 98.13. Replacing MoRo-Former with Q-Former over DINOv3 only yields 63.41 / 44.59 / 86.24; over VoxelNet only, 60.39 / 41.72 / 83.51; and over DINOv3+VoxelNet, 66.79 / 45.92 / 92.74. The paper summarizes the effect as modality-routing and token compression outperforming uniform query bridging by approximately 5.4% on BLEU/BERTScore (Wang et al., 28 Feb 2026). Qualitative analysis further reports that queries route preferentially toward LiDAR at night or in low light, whereas routing shifts toward camera or fusion when LiDAR becomes sparse or corrupted.

The benchmark also has explicit scope limits. It focuses on camera–LiDAR sensing and does not include RADAR, thermal, event cameras, or IMU. It uses naturally degraded sensing and missing-modality settings, but does not define synthetic corruption severities. The paper states that the code and benchmark will be publicly available, yet license and detailed download instructions are not specified, and evaluation scripts are not explicitly detailed (Wang et al., 28 Feb 2026).

Relative to urban benchmarks, OR-C2P emphasizes closed-set, safety-centric captions tied to planning, ego-frame short-horizon trajectory prediction, and robust multimodal routing rather than fixed fusion (Wang et al., 28 Feb 2026). This makes it a focused benchmark for interpretable off-road perception-to-planning rather than a general-purpose autonomous driving benchmark.