WildCross Dataset
- WildCross is a comprehensive benchmark dataset for evaluating robotic perception in unstructured natural environments, with a focus on visual place recognition (VPR), LiDAR place recognition (LPR), cross-modal place recognition (CMPR), and metric depth estimation.
- The dataset contains over 476,000 RGB frames, semi-dense metric-depth and surface-normal annotations, accurate 6DoF poses, and synchronized dense LiDAR submaps, collected across approximately 33 km of forest traversals in two distinct natural environments, Venman and Karawatha, spanning 14 months of variation.
- Results with WildCross indicate that geometric structure is more robust than visual features under natural environment variations, making LiDAR place recognition effective, but VPR particularly challenging, especially during reverse traversals and significant temporal changes.
WildCross is a large-scale cross-modal benchmark for robotic perception in unstructured natural environments, targeting visual place recognition (VPR), LiDAR place recognition (LPR), cross-modal place recognition (CMPR), and metric depth estimation. It contains more than 476,000 sequential RGB frames, semi-dense metric-depth and surface-normal annotations, accurate 6DoF poses, and synchronized dense LiDAR submaps collected across forest traversals. The benchmark is designed to evaluate robustness under repetitive vegetation, irregular terrain, occlusion, viewpoint change, reverse traversal, illumination variation, long-term environmental change, and the domain shift between urban-trained models and natural scenes (Knights et al., 2 Mar 2026).
1. Purpose and scope
WildCross addresses the limitations of robotics datasets dominated by structured urban or indoor environments. Natural environments provide fewer rigid landmarks, greater depth variation, dense vegetation, narrow trails, repeated visual patterns, and substantial viewpoint-dependent occlusion. These properties complicate both place correspondence and monocular geometric prediction.
The benchmark supports four principal tasks:
- Visual place recognition: retrieving a previously observed place from an RGB query.
- LiDAR place recognition: retrieving a corresponding LiDAR submap from a LiDAR query.
- Cross-modal place recognition: retrieving a LiDAR representation using an RGB query.
- Metric depth estimation: predicting metric depth from an RGB image.
The benchmark is derived from the Wild-Places LiDAR place-recognition dataset and extends it by regenerating sequential RGB frames with accurate camera poses, synchronizing them with LiDAR submaps, and producing semi-dense depth and surface-normal annotations through visibility-aware map projection. Its stated applications include localization, loop closure, relocalization, SLAM, multimodal scene understanding, geometric reconstruction, and field robotics.
A central benchmark property is the coexistence of spatially corresponding but visually dissimilar observations. Reverse traversals can place the same physical location in opposing viewing directions, while long-term revisits alter vegetation, illumination, and occlusion. Consequently, geometric correspondence is often more persistent than RGB appearance, but it is also difficult to infer from a single image.
2. Dataset composition and sensing
WildCross contains eight traversals across two natural environments, Venman and Karawatha, for approximately 33 km of trajectory. The data span 14 months of traversal variation and include more than 476,000 RGB frames and approximately 63,300 LiDAR submaps.
| Environment | Traversals | Distance | RGB/depth frames | LiDAR submaps |
|---|---|---|---|---|
| Venman | V-01–V-04 | 12.68 km | 176.3K | 23.5K |
| Karawatha | K-01–K-04 | 20.24 km | 299.7K | 39.8K |
| Total | 8 | 33 km | 476K | 63.3K |
Sequence 01 is a reference traversal, sequence 02 approximately reverses the route of sequence 01, sequence 03 follows an alternate extended route, and sequence 04 approximately repeats the reference route. The reverse traversals are particularly challenging because a forward-facing camera may observe little overlapping appearance when the same physical trail is viewed in the opposite direction.
RGB images are captured by a forward-facing camera at 15 Hz and rectified using calibrated lens-distortion parameters. LiDAR data are processed with a LiDAR-inertial SLAM system and accumulated into dense global point clouds. Submaps are formed from points within a 30 m radius of the sensor-payload position and within a 1 s temporal window, sampled every 0.5 s along the global trajectory.
Each RGB frame is associated with a synchronized 6DoF pose comprising a translation and unit-quaternion orientation. Poses are interpolated to RGB timestamps using spherical linear interpolation for orientation and linear interpolation for translation. Camera poses are obtained using calibrated camera–sensor extrinsics, while global maps from different sessions are aligned with ICP.
3. Depth and surface-normal annotation
WildCross provides semi-dense metric depth rather than dense depth. For a valid pixel , the depth corresponds to the distance from the camera center to the nearest visible surface along the camera ray. Pixels without sufficiently supported projected geometry remain unlabeled.
The annotation pipeline estimates local surface normals from point-cloud neighborhoods. For each point, neighboring points within 0.5 m are used to construct a covariance matrix; the eigenvector associated with the smallest eigenvalue is selected as the local surface normal and oriented toward the observation location. Normals are transformed into the camera coordinate frame and rasterized together with depth.
Projected points are filtered if they lie outside image bounds, behind the camera, have normals oriented away from the camera, or are occluded by another surface. Occlusion is handled using generalized hidden point removal (GHPR). For a point , spherical reflection is defined by
with exponential inversion
The reflected point cloud is processed through its convex hull; points on the hull are treated as visible. This procedure reduces erroneous depth behind foreground vegetation and produces cleaner annotations than naïve projection of all map points.
The resulting normal annotation is a three-channel image in the camera coordinate frame. The original benchmark supplies normals as an additional modality, although the principal reported experiments focus on place recognition and metric depth rather than a separate normal-estimation task.
The annotation process has limitations. Depth is map-derived rather than directly measured at every image pixel, and its accuracy depends on SLAM alignment, local normal estimation, point-cloud density, and the GHPR visibility approximation. Thin foliage, noisy point clouds, and multilayer vegetation can remain difficult.
4. Benchmark protocols and evaluation metrics
For VPR, positive training pairs are defined by camera-pose distance within 5 m and bearing difference within ; negative pairs are more than 50 m apart. At evaluation, a retrieved image is correct when its pose lies within 25 m of the query. Results are reported as Recall@1 and Recall@5.
The principal VPR organization uses four cross-sequence folds. In fold , sequence from both environments is held out, while the remaining sequences provide training data and, for inter-sequence evaluation, the database.
| Fold | Held-out sequences | LiDAR train/test | Camera train/test |
|---|---|---|---|
| 1 | V-01, K-01 | 49.8K / 13.5K | 374.6K / 101.4K |
| 2 | V-02, K-02 | 48.7K / 14.6K | 366.3K / 109.7K |
| 3 | V-03, K-03 | 39.7K / 23.6K | 298.0K / 177.9K |
| 4 | V-04, K-04 | 51.8K / 11.5K | 389.1K / 86.9K |
Intra-sequence evaluation retrieves within a held-out traversal, while inter-sequence evaluation matches held-out queries against training traversals from the corresponding environment. The depth experiment uses a separate split: V-01 and K-01 for testing, K-02 for validation, and the remaining sequences for training.
For LPR, positive pairs have centers within 3 m and negative pairs are separated by more than 20 m. A retrieved submap is correct within a 3 m pose threshold. For CMPR, RGB queries retrieve LiDAR submaps, with evaluation performed inter-sequence and a 25 m positive spatial threshold.
Metric depth is evaluated only at valid annotated pixels using threshold accuracy , absolute relative error, and root mean square error:
and
0
5. Baseline performance and principal findings
Visual place recognition
The evaluated VPR methods are NetVLAD, MixVPR, SALAD, and BoQ. Zero-shot models are applied without WildCross training, while fine-tuned models are trained on the WildCross training folds.
| Method | Setting | Intra R@1 | Intra R@5 | Inter R@1 | Inter R@5 |
|---|---|---|---|---|---|
| NetVLAD | Zero-shot | 39.28 | 48.89 | 21.00 | 36.84 |
| MixVPR | Zero-shot | 46.85 | 53.96 | 44.92 | 52.86 |
| SALAD | Zero-shot | 45.85 | 52.24 | 49.38 | 57.32 |
| BoQ | Zero-shot | 46.15 | 52.24 | 53.76 | 61.48 |
| NetVLAD | Fine-tuned | 58.77 | 63.30 | 55.63 | 59.96 |
| MixVPR | Fine-tuned | 58.71 | 63.05 | 57.77 | 62.19 |
| SALAD | Fine-tuned | 61.76 | 66.14 | 61.41 | 65.86 |
| BoQ | Fine-tuned | 63.17 | 67.91 | 61.87 | 66.19 |
Fine-tuning improves recognition substantially, but reverse traversals remain difficult. The strongest fine-tuned BoQ performance is far below the more than 90% Recall@1 reported for BoQ on urban benchmarks such as Pittsburgh and MSLS. This indicates a substantial natural-environment domain gap rather than merely a lack of model capacity.
LiDAR place recognition
LiDAR methods substantially outperform RGB methods, particularly for intra-sequence recognition.
| Method | Intra R@1 | Intra R@5 | Inter R@1 | Inter R@5 |
|---|---|---|---|---|
| MinkLoc3Dv2 | 81.98 | 96.64 | 84.62 | 95.61 |
| LoGG3D-Net | 84.04 | 97.10 | 78.24 | 90.11 |
| HOTFormerLoc | 85.56 | 93.92 | 84.85 | 95.36 |
The stronger LPR performance suggests that geometric structure is more stable than RGB appearance under illumination, vegetation, and temporal variation. However, inter-sequence recognition remains difficult, particularly in Karawatha, where vegetation, occlusion, and traversal differences alter the observed geometry.
Cross-modal place recognition
CMPR is substantially harder than either unimodal task. LIP-Loc is evaluated with ResNet-50, DINOv2-S, and DINOv3-S visual backbones.
| Method | Venman R@1 | Venman R@5 | Karawatha R@1 | Karawatha R@5 | Average R@1 | Average R@5 |
|---|---|---|---|---|---|---|
| LIP-Loc, ResNet-50 | 40.16 | 54.45 | 34.25 | 48.91 | 37.20 | 51.68 |
| LIP-Loc, DINOv2-S | 52.55 | 62.71 | 45.26 | 57.40 | 48.90 | 60.06 |
| LIP-Loc, DINOv3-S | 56.54 | 63.19 | 48.16 | 57.06 | 52.35 | 60.12 |
Stronger visual pretraining improves CMPR, but the image-to-LiDAR modality gap remains large. RGB encodes color, texture, and appearance, whereas LiDAR encodes spatial geometry and occupancy. Their correspondence is further complicated by viewpoint changes and vegetation occlusion.
Metric depth estimation
DepthAnythingV2 performs poorly zero-shot but improves substantially after WildCross fine-tuning.
| Setting | Backbone | 1 | AbsRel | RMSE |
|---|---|---|---|---|
| Zero-shot | ViT-S | 0.284 | 0.558 | 7.651 |
| Zero-shot | ViT-B | 0.222 | 0.769 | 7.915 |
| Zero-shot | ViT-L | 0.074 | 1.478 | 13.734 |
| Fine-tuned | ViT-S | 0.746 | 0.172 | 3.412 |
| Fine-tuned | ViT-B | 0.766 | 0.167 | 3.289 |
| Fine-tuned | ViT-L | 0.789 | 0.157 | 3.150 |
The zero-shot results expose a strong urban-to-natural domain gap. Larger models are not necessarily better without in-domain adaptation; ViT-L performs worst among the zero-shot models. After fine-tuning, larger models perform better.
The companion analysis evaluates pseudo-ground-truth depth from Depth Anything V3, RANSAC-rescaled predictions, and PriorDA. PriorDA, which incorporates sparse metric depth, achieves 2, AbsRel 3, and RMSE 4 when its generated depth is evaluated against valid WildCross depth. Fine-tuning with measured semi-dense WildCross depth remains the strongest direct supervision for numerical accuracy, while pseudo-ground-truth methods can preserve more fine-grained vegetation detail (Hall et al., 10 Jun 2026).
6. Geometry-aware methods and research significance
WildCross has motivated methods that explicitly introduce geometric information into visual place recognition. One depth-aware distillation approach injects a learned depth embedding into a frozen DINOv2/SALAD representation. Its student descriptor is
5
where 6 is the visual encoder, 7 is a learnable depth pathway, and 8 controls depth strength. The training objective combines triplet retrieval loss with cosine alignment to the appearance-only teacher. On WildCross, this method improves average inter-sequence Recall@1 from 49.38% to 62.69% and Recall@5 from 57.32% to 69.68% relative to a frozen RGB baseline; it also slightly exceeds an RGB-only fine-tuned baseline (Nedov et al., 11 Jun 2026).
A later geometry-conditioned formulation uses a Geometric Foundation Model, specifically Depth Anything V2 adapted with WildCross LiDAR supervision, to project depth into the token space of a DINOv2 Vision Foundation Model. Instead of applying a single scalar depth weight, it uses a channel-wise gate:
9
where 0 is the appearance-token sequence and 1 is the projected depth-token sequence. A two-stage procedure first aligns the geometry-conditioned descriptor with the appearance teacher and then fine-tunes selected visual transformer blocks and the SALAD aggregation head.
This method obtains average inter-sequence performance of 66.37% Recall@1 and 72.49% Recall@5, compared with 61.41% and 65.86% for the matched appearance-only SALAD baseline. Average intra-sequence performance is 65.34% and 70.36%, compared with 61.76% and 66.14% for SALAD. The largest gains occur under reverse traversal and long-term appearance variation, although performance is not uniformly better on every sequence (Nedov et al., 23 Sep 2026).
These results support several interpretations:
- Geometry is complementary to appearance: depth does not replace RGB; selective fusion is more effective than indiscriminate concatenation.
- Natural-environment VPR is structurally difficult: repeated foliage and weak landmarks create perceptual aliasing that urban-trained descriptors do not reliably resolve.
- Reverse traversal is a distinctive failure mode: geographically close images may have little visual overlap when viewed in opposite directions.
- Cross-modal supervision is valuable: LiDAR and semi-dense depth provide geometric information unavailable from RGB alone.
- Adaptation is necessary: zero-shot models trained on urban or synthetic environments generalize poorly to forest scenes.
- Sequential data remain underused: the benchmark supplies trajectories, poses, and synchronized sensing, but many reported baselines operate on frame-level descriptors without temporal aggregation.
WildCross also has limitations. It covers two natural environments near Brisbane and uses a forward-facing camera configuration. Depth is semi-dense and map-derived, while surface normals are supplied but not fully benchmarked. The benchmark does not encompass all forest types, ecosystems, weather conditions, geographic regions, robot platforms, or sensor configurations. Reported baseline implementation details are not always complete in the papers, and some aggregate values differ slightly between prose and displayed tables.
Despite these constraints, WildCross provides an unusually integrated evaluation setting: sequential RGB, metric geometric annotations, surface normals, accurate poses, and dense LiDAR submaps are aligned across repeated natural-environment traversals. Its results establish a performance hierarchy in which LiDAR place recognition is strongest, RGB place recognition is substantially more difficult, cross-modal retrieval is harder still, and monocular metric depth requires significant natural-environment adaptation. The benchmark therefore functions both as a dataset and as a stress test for multimodal robotic perception under appearance, viewpoint, geometry, and domain shifts.