Papers
Topics
Authors
Recent
Search
2000 character limit reached

WildCross Dataset

Updated 29 September 2026
  • WildCross is a comprehensive benchmark dataset for evaluating robotic perception in unstructured natural environments, with a focus on visual place recognition (VPR), LiDAR place recognition (LPR), cross-modal place recognition (CMPR), and metric depth estimation.
  • The dataset contains over 476,000 RGB frames, semi-dense metric-depth and surface-normal annotations, accurate 6DoF poses, and synchronized dense LiDAR submaps, collected across approximately 33 km of forest traversals in two distinct natural environments, Venman and Karawatha, spanning 14 months of variation.
  • Results with WildCross indicate that geometric structure is more robust than visual features under natural environment variations, making LiDAR place recognition effective, but VPR particularly challenging, especially during reverse traversals and significant temporal changes.

WildCross is a large-scale cross-modal benchmark for robotic perception in unstructured natural environments, targeting visual place recognition (VPR), LiDAR place recognition (LPR), cross-modal place recognition (CMPR), and metric depth estimation. It contains more than 476,000 sequential RGB frames, semi-dense metric-depth and surface-normal annotations, accurate 6DoF poses, and synchronized dense LiDAR submaps collected across forest traversals. The benchmark is designed to evaluate robustness under repetitive vegetation, irregular terrain, occlusion, viewpoint change, reverse traversal, illumination variation, long-term environmental change, and the domain shift between urban-trained models and natural scenes (Knights et al., 2 Mar 2026).

1. Purpose and scope

WildCross addresses the limitations of robotics datasets dominated by structured urban or indoor environments. Natural environments provide fewer rigid landmarks, greater depth variation, dense vegetation, narrow trails, repeated visual patterns, and substantial viewpoint-dependent occlusion. These properties complicate both place correspondence and monocular geometric prediction.

The benchmark supports four principal tasks:

  1. Visual place recognition: retrieving a previously observed place from an RGB query.
  2. LiDAR place recognition: retrieving a corresponding LiDAR submap from a LiDAR query.
  3. Cross-modal place recognition: retrieving a LiDAR representation using an RGB query.
  4. Metric depth estimation: predicting metric depth from an RGB image.

The benchmark is derived from the Wild-Places LiDAR place-recognition dataset and extends it by regenerating sequential RGB frames with accurate camera poses, synchronizing them with LiDAR submaps, and producing semi-dense depth and surface-normal annotations through visibility-aware map projection. Its stated applications include localization, loop closure, relocalization, SLAM, multimodal scene understanding, geometric reconstruction, and field robotics.

A central benchmark property is the coexistence of spatially corresponding but visually dissimilar observations. Reverse traversals can place the same physical location in opposing viewing directions, while long-term revisits alter vegetation, illumination, and occlusion. Consequently, geometric correspondence is often more persistent than RGB appearance, but it is also difficult to infer from a single image.

2. Dataset composition and sensing

WildCross contains eight traversals across two natural environments, Venman and Karawatha, for approximately 33 km of trajectory. The data span 14 months of traversal variation and include more than 476,000 RGB frames and approximately 63,300 LiDAR submaps.

Environment Traversals Distance RGB/depth frames LiDAR submaps
Venman V-01–V-04 12.68 km 176.3K 23.5K
Karawatha K-01–K-04 20.24 km 299.7K 39.8K
Total 8 33 km 476K 63.3K

Sequence 01 is a reference traversal, sequence 02 approximately reverses the route of sequence 01, sequence 03 follows an alternate extended route, and sequence 04 approximately repeats the reference route. The reverse traversals are particularly challenging because a forward-facing camera may observe little overlapping appearance when the same physical trail is viewed in the opposite direction.

RGB images are captured by a forward-facing camera at 15 Hz and rectified using calibrated lens-distortion parameters. LiDAR data are processed with a LiDAR-inertial SLAM system and accumulated into dense global point clouds. Submaps are formed from points within a 30 m radius of the sensor-payload position and within a 1 s temporal window, sampled every 0.5 s along the global trajectory.

Each RGB frame is associated with a synchronized 6DoF pose comprising a translation and unit-quaternion orientation. Poses are interpolated to RGB timestamps using spherical linear interpolation for orientation and linear interpolation for translation. Camera poses are obtained using calibrated camera–sensor extrinsics, while global maps from different sessions are aligned with ICP.

3. Depth and surface-normal annotation

WildCross provides semi-dense metric depth rather than dense depth. For a valid pixel (u,v)(u,v), the depth corresponds to the distance from the camera center to the nearest visible surface along the camera ray. Pixels without sufficiently supported projected geometry remain unlabeled.

The annotation pipeline estimates local surface normals from point-cloud neighborhoods. For each point, neighboring points within 0.5 m are used to construct a covariance matrix; the eigenvector associated with the smallest eigenvalue is selected as the local surface normal and oriented toward the observation location. Normals are transformed into the camera coordinate frame and rasterized together with depth.

Projected points are filtered if they lie outside image bounds, behind the camera, have normals oriented away from the camera, or are occluded by another surface. Occlusion is handled using generalized hidden point removal (GHPR). For a point pp, spherical reflection is defined by

F(p;γ)=p∥p∥2−1f(∥p∥2;γ),F(p;\gamma)=p\|p\|_2^{-1}f(\|p\|_2;\gamma),

with exponential inversion

f(d;γ)=dγ,γ<0.f(d;\gamma)=d^\gamma,\qquad \gamma<0.

The reflected point cloud is processed through its convex hull; points on the hull are treated as visible. This procedure reduces erroneous depth behind foreground vegetation and produces cleaner annotations than naïve projection of all map points.

The resulting normal annotation is a three-channel image in the camera coordinate frame. The original benchmark supplies normals as an additional modality, although the principal reported experiments focus on place recognition and metric depth rather than a separate normal-estimation task.

The annotation process has limitations. Depth is map-derived rather than directly measured at every image pixel, and its accuracy depends on SLAM alignment, local normal estimation, point-cloud density, and the GHPR visibility approximation. Thin foliage, noisy point clouds, and multilayer vegetation can remain difficult.

4. Benchmark protocols and evaluation metrics

For VPR, positive training pairs are defined by camera-pose distance within 5 m and bearing difference within 15∘15^\circ; negative pairs are more than 50 m apart. At evaluation, a retrieved image is correct when its pose lies within 25 m of the query. Results are reported as Recall@1 and Recall@5.

The principal VPR organization uses four cross-sequence folds. In fold ii, sequence ii from both environments is held out, while the remaining sequences provide training data and, for inter-sequence evaluation, the database.

Fold Held-out sequences LiDAR train/test Camera train/test
1 V-01, K-01 49.8K / 13.5K 374.6K / 101.4K
2 V-02, K-02 48.7K / 14.6K 366.3K / 109.7K
3 V-03, K-03 39.7K / 23.6K 298.0K / 177.9K
4 V-04, K-04 51.8K / 11.5K 389.1K / 86.9K

Intra-sequence evaluation retrieves within a held-out traversal, while inter-sequence evaluation matches held-out queries against training traversals from the corresponding environment. The depth experiment uses a separate split: V-01 and K-01 for testing, K-02 for validation, and the remaining sequences for training.

For LPR, positive pairs have centers within 3 m and negative pairs are separated by more than 20 m. A retrieved submap is correct within a 3 m pose threshold. For CMPR, RGB queries retrieve LiDAR submaps, with evaluation performed inter-sequence and a 25 m positive spatial threshold.

Metric depth is evaluated only at valid annotated pixels using threshold accuracy δ1\delta_1, absolute relative error, and root mean square error:

δ1=1∣Ω∣∑p∈Ω1[max⁡(d^pdp,dpd^p)<1.25],\delta_1=\frac{1}{|\Omega|}\sum_{p\in\Omega} \mathbf{1}\left[ \max\left(\frac{\hat d_p}{d_p},\frac{d_p}{\hat d_p}\right)<1.25 \right],

AbsRel=1∣Ω∣∑p∈Ω∣d^p−dp∣dp,\mathrm{AbsRel}= \frac{1}{|\Omega|} \sum_{p\in\Omega} \frac{|\hat d_p-d_p|}{d_p},

and

pp0

5. Baseline performance and principal findings

Visual place recognition

The evaluated VPR methods are NetVLAD, MixVPR, SALAD, and BoQ. Zero-shot models are applied without WildCross training, while fine-tuned models are trained on the WildCross training folds.

Method Setting Intra R@1 Intra R@5 Inter R@1 Inter R@5
NetVLAD Zero-shot 39.28 48.89 21.00 36.84
MixVPR Zero-shot 46.85 53.96 44.92 52.86
SALAD Zero-shot 45.85 52.24 49.38 57.32
BoQ Zero-shot 46.15 52.24 53.76 61.48
NetVLAD Fine-tuned 58.77 63.30 55.63 59.96
MixVPR Fine-tuned 58.71 63.05 57.77 62.19
SALAD Fine-tuned 61.76 66.14 61.41 65.86
BoQ Fine-tuned 63.17 67.91 61.87 66.19

Fine-tuning improves recognition substantially, but reverse traversals remain difficult. The strongest fine-tuned BoQ performance is far below the more than 90% Recall@1 reported for BoQ on urban benchmarks such as Pittsburgh and MSLS. This indicates a substantial natural-environment domain gap rather than merely a lack of model capacity.

LiDAR place recognition

LiDAR methods substantially outperform RGB methods, particularly for intra-sequence recognition.

Method Intra R@1 Intra R@5 Inter R@1 Inter R@5
MinkLoc3Dv2 81.98 96.64 84.62 95.61
LoGG3D-Net 84.04 97.10 78.24 90.11
HOTFormerLoc 85.56 93.92 84.85 95.36

The stronger LPR performance suggests that geometric structure is more stable than RGB appearance under illumination, vegetation, and temporal variation. However, inter-sequence recognition remains difficult, particularly in Karawatha, where vegetation, occlusion, and traversal differences alter the observed geometry.

Cross-modal place recognition

CMPR is substantially harder than either unimodal task. LIP-Loc is evaluated with ResNet-50, DINOv2-S, and DINOv3-S visual backbones.

Method Venman R@1 Venman R@5 Karawatha R@1 Karawatha R@5 Average R@1 Average R@5
LIP-Loc, ResNet-50 40.16 54.45 34.25 48.91 37.20 51.68
LIP-Loc, DINOv2-S 52.55 62.71 45.26 57.40 48.90 60.06
LIP-Loc, DINOv3-S 56.54 63.19 48.16 57.06 52.35 60.12

Stronger visual pretraining improves CMPR, but the image-to-LiDAR modality gap remains large. RGB encodes color, texture, and appearance, whereas LiDAR encodes spatial geometry and occupancy. Their correspondence is further complicated by viewpoint changes and vegetation occlusion.

Metric depth estimation

DepthAnythingV2 performs poorly zero-shot but improves substantially after WildCross fine-tuning.

Setting Backbone pp1 AbsRel RMSE
Zero-shot ViT-S 0.284 0.558 7.651
Zero-shot ViT-B 0.222 0.769 7.915
Zero-shot ViT-L 0.074 1.478 13.734
Fine-tuned ViT-S 0.746 0.172 3.412
Fine-tuned ViT-B 0.766 0.167 3.289
Fine-tuned ViT-L 0.789 0.157 3.150

The zero-shot results expose a strong urban-to-natural domain gap. Larger models are not necessarily better without in-domain adaptation; ViT-L performs worst among the zero-shot models. After fine-tuning, larger models perform better.

The companion analysis evaluates pseudo-ground-truth depth from Depth Anything V3, RANSAC-rescaled predictions, and PriorDA. PriorDA, which incorporates sparse metric depth, achieves pp2, AbsRel pp3, and RMSE pp4 when its generated depth is evaluated against valid WildCross depth. Fine-tuning with measured semi-dense WildCross depth remains the strongest direct supervision for numerical accuracy, while pseudo-ground-truth methods can preserve more fine-grained vegetation detail (Hall et al., 10 Jun 2026).

6. Geometry-aware methods and research significance

WildCross has motivated methods that explicitly introduce geometric information into visual place recognition. One depth-aware distillation approach injects a learned depth embedding into a frozen DINOv2/SALAD representation. Its student descriptor is

pp5

where pp6 is the visual encoder, pp7 is a learnable depth pathway, and pp8 controls depth strength. The training objective combines triplet retrieval loss with cosine alignment to the appearance-only teacher. On WildCross, this method improves average inter-sequence Recall@1 from 49.38% to 62.69% and Recall@5 from 57.32% to 69.68% relative to a frozen RGB baseline; it also slightly exceeds an RGB-only fine-tuned baseline (Nedov et al., 11 Jun 2026).

A later geometry-conditioned formulation uses a Geometric Foundation Model, specifically Depth Anything V2 adapted with WildCross LiDAR supervision, to project depth into the token space of a DINOv2 Vision Foundation Model. Instead of applying a single scalar depth weight, it uses a channel-wise gate:

pp9

where F(p;γ)=p∥p∥2−1f(∥p∥2;γ),F(p;\gamma)=p\|p\|_2^{-1}f(\|p\|_2;\gamma),0 is the appearance-token sequence and F(p;γ)=p∥p∥2−1f(∥p∥2;γ),F(p;\gamma)=p\|p\|_2^{-1}f(\|p\|_2;\gamma),1 is the projected depth-token sequence. A two-stage procedure first aligns the geometry-conditioned descriptor with the appearance teacher and then fine-tunes selected visual transformer blocks and the SALAD aggregation head.

This method obtains average inter-sequence performance of 66.37% Recall@1 and 72.49% Recall@5, compared with 61.41% and 65.86% for the matched appearance-only SALAD baseline. Average intra-sequence performance is 65.34% and 70.36%, compared with 61.76% and 66.14% for SALAD. The largest gains occur under reverse traversal and long-term appearance variation, although performance is not uniformly better on every sequence (Nedov et al., 23 Sep 2026).

These results support several interpretations:

  • Geometry is complementary to appearance: depth does not replace RGB; selective fusion is more effective than indiscriminate concatenation.
  • Natural-environment VPR is structurally difficult: repeated foliage and weak landmarks create perceptual aliasing that urban-trained descriptors do not reliably resolve.
  • Reverse traversal is a distinctive failure mode: geographically close images may have little visual overlap when viewed in opposite directions.
  • Cross-modal supervision is valuable: LiDAR and semi-dense depth provide geometric information unavailable from RGB alone.
  • Adaptation is necessary: zero-shot models trained on urban or synthetic environments generalize poorly to forest scenes.
  • Sequential data remain underused: the benchmark supplies trajectories, poses, and synchronized sensing, but many reported baselines operate on frame-level descriptors without temporal aggregation.

WildCross also has limitations. It covers two natural environments near Brisbane and uses a forward-facing camera configuration. Depth is semi-dense and map-derived, while surface normals are supplied but not fully benchmarked. The benchmark does not encompass all forest types, ecosystems, weather conditions, geographic regions, robot platforms, or sensor configurations. Reported baseline implementation details are not always complete in the papers, and some aggregate values differ slightly between prose and displayed tables.

Despite these constraints, WildCross provides an unusually integrated evaluation setting: sequential RGB, metric geometric annotations, surface normals, accurate poses, and dense LiDAR submaps are aligned across repeated natural-environment traversals. Its results establish a performance hierarchy in which LiDAR place recognition is strongest, RGB place recognition is substantially more difficult, cross-modal retrieval is harder still, and monocular metric depth requires significant natural-environment adaptation. The benchmark therefore functions both as a dataset and as a stress test for multimodal robotic perception under appearance, viewpoint, geometry, and domain shifts.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to WildCross.