AURORA-KITTI: Robust Depth Completion Benchmark

• AURORA-KITTI is a multi-modal, multi-weather benchmark featuring 82K+ paired RGB-LiDAR data to evaluate robust depth completion and denoising in adverse conditions.
• It provides weather-corrupted inputs with paired clean references and dense metric ground truth, enabling controlled evaluation of sensor fusion under real-world degradation.
• The associated DDCD baseline leverages distillation from clean-image depth priors to inject structural geometric information, achieving state-of-the-art performance.

Searching arXiv for the target paper and closely related works mentioned in the provided data. AURORA-KITTI is a large-scale multi-modal, multi-weather benchmark for robust depth completion in the wild, introduced together with the formulation of Depth Completion and Denoising (DCD) as a unified task. It is designed for RGB-LiDAR depth perception under adverse weather, where both camera imagery and LiDAR measurements are corrupted by weather-induced effects. The benchmark contains over 82K weather-consistent RGBL pairs with metric depth ground truth, spanning multiple weather types, three severity levels, day and night scenes, paired clean references, lens occlusion conditions, and textual descriptions. In the same work, DDCD is introduced as a distillation-based baseline that uses depth foundation models to inject clean structural priors into in-the-wild DCD training, achieving state-of-the-art performance on AURORA-KITTI and the real-world DENSE dataset while maintaining efficiency (Wang et al., 16 Mar 2026).

1. Definition and scope

AURORA-KITTI is presented as the first large-scale, physics-consistent, multi-weather RGB-LiDAR benchmark specifically designed for the DCD task. Its stated purpose is to enable robust evaluation and training of depth perception models under adverse weather, with an emphasis on physically matched degradation across sensing modalities (Wang et al., 16 Mar 2026).

The benchmark extends the conventional depth completion setting by assuming that both the RGB observation and the sparse depth input are weather-corrupted. Rather than only densifying a sparse depth signal, DCD jointly targets depth reconstruction and denoising. In the notation used for the task, conventional depth completion is written as

$D_c = f(I_c, S_c),$

where $I_c$ and $S_c$ denote clean RGB and sparse depth, respectively. The adverse-weather extension is written as

$\hat{D}_c = f^{w}(I_w, S_w),$

where $I_w = W_{rgb}(I_c; \theta)$ and $S_w = W_{lidar}(S_c; \theta)$ are corrupted by weather processes, while supervision remains the clean dense target $D_c$ (Wang et al., 16 Mar 2026).

This formulation makes the benchmark relevant not only to depth completion in the narrow sense, but also to robust sensor fusion, denoising-aware geometric reconstruction, and adverse-weather 3D scene understanding. A plausible implication is that AURORA-KITTI serves simultaneously as a dataset, an evaluation protocol, and an operational definition of robustness for RGB-LiDAR depth estimation under weather shift.

2. Dataset composition and annotation structure

AURORA-KITTI contains 82,177 RGB-LiDAR pairs. The splits are approximately 70,500 for training, 5,800 for validation, and 5,900 for test, with each split covering all scene and weather variations (Wang et al., 16 Mar 2026).

Its core modalities are RGB images, LiDAR point clouds, and sparse depth maps. RGB images are simulated using a physically-based rendering pipeline and combined with GAN-based style transfer for appearance shifts such as nighttime and wet road conditions. LiDAR point clouds are generated with physics-based degradation that includes particle-induced backscatter, attenuation, spurious/fake returns, and occlusions. Sparse depth maps are then produced by projecting corrupted LiDAR point clouds onto the image plane using accurate camera-LiDAR calibration (Wang et al., 16 Mar 2026).

The weather and lighting coverage is explicitly structured. Weather types include clear, fog, rain, and snow, and all are paired with day/night variants, yielding compound conditions such as night-rain and night-snow. Each weather category has three physically-meaningful severity levels. The examples given are rain rates of 10/100/200 mm/hr and fog attenuation coefficients of 0.01/0.1/0.2 m$^{-1}$. The dataset also models lens occlusion, including adherent raindrops and snowflakes that induce local blur and chromatic effects (Wang et al., 16 Mar 2026).

The annotation schema is unusually rich for depth completion. Each sample includes dense ground truth depth, weather metadata, paired clean references, and textual descriptions. The dense ground truth is clean, metric-scale, and pixel-aligned. Weather metadata includes weather type, severity, time of day, scene context, and lens condition. The paired clean reference provides a clear-weather version of each adverse sample, enabling controlled comparison and distillation. The textual descriptions summarize scene and sensor attributes per frame (Wang et al., 16 Mar 2026).

Component	Description
Scale	82,177 RGB-LiDAR pairs
Splits	~70,500 train, ~5,800 val, ~5,900 test
Weather	Clear, fog, rain, snow; all with day/night variants
Severity	Three physically-meaningful levels per weather
Additional factors	Lens occlusion, paired clean references, textual descriptions

The combination of paired clean references and dense metric supervision is central. The paper explicitly notes that obtaining paired, aligned, clean depth ground truth with adverse-weather inputs is infeasible in the real world; the synthetic paired construction is therefore what makes controlled, scalable DCD training and evaluation possible (Wang et al., 16 Mar 2026).

3. Physics-consistent generation process

A defining property of AURORA-KITTI is cross-modal consistency: RGB and LiDAR degradations are matched by shared parameters so that weather corruption is coherent across sensing channels. This is intended to support realistic evaluation of RGB-LiDAR fusion models, which otherwise risk being assessed on modality combinations that do not correspond to the same physical conditions (Wang et al., 16 Mar 2026).

The camera degradation pipeline is described as a multi-layer image synthesis process involving scene appearance, volumetric particles, and explicitly modeled lens occlusions. On the LiDAR side, adverse-weather simulation is based on established models, including LISA for rain and Hahner et al. for snow/fog. The paper characterizes the resulting LiDAR corruption as range-dependent and non-uniform, consistent with the physical mechanisms of scattering, attenuation, and false returns (Wang et al., 16 Mar 2026).

This design matters because adverse weather degrades the two modalities differently. RGB images exhibit appearance shifts, blur, streaking, and occlusion-like effects, while LiDAR exhibits missing returns, fake returns, and depth-dependent measurement distortion. By coupling both degradations through shared weather parameters, AURORA-KITTI avoids the simplified assumption that adverse weather is a purely image-side nuisance or a purely point-cloud-side nuisance. This suggests that the benchmark is optimized for studying failure modes of fusion architectures rather than for isolated single-modality robustness.

4. The DCD task and evaluation protocol

The DCD task generalizes standard depth completion by requiring models to recover a dense, accurate, and clean depth map from corrupted sparse RGB-LiDAR inputs. The two explicit subproblems are densification from LiDAR sparsity and suppression of weather-induced noise and artifacts in both RGB and LiDAR (Wang et al., 16 Mar 2026).

Benchmarking uses four lower-is-better metrics:

• RMSE (mm): Root Mean Squared Error for depth
• MAE (mm): Mean Absolute Error
• iRMSE (1/km): Root Mean Squared Error of inverse depth
• iMAE (1/km): Mean Absolute Error of inverse depth

All models are evaluated in the DCD setting, meaning that the input is $(I_w, S_w)$ and the target is the clean depth $D_c$ (Wang et al., 16 Mar 2026).

Two principal evaluation regimes are defined. In zero-shot evaluation, models trained on clear-weather data are tested directly on AURORA-KITTI to measure robustness under distribution shift. In in-domain retraining, models are retrained or fine-tuned on the AURORA-KITTI training split without architecture changes, isolating the effect of exposure to robust weather-aware data. A third axis, cross-dataset generalization, evaluates models trained on AURORA-KITTI on the real-world DENSE benchmark (Wang et al., 16 Mar 2026).

This protocol separates at least three factors that are often conflated in adverse-weather depth estimation: base architecture quality, robustness from training distribution, and transfer to real scenes. The paper’s later ablations place particular emphasis on this separation.

5. DDCD baseline and distillation mechanism

The proposed baseline, DDCD, is a distillation-based method for DCD. Its stated motivation is that depth foundation models encode strong geometric priors in clean conditions but degrade in severe weather, whereas sparse LiDAR is comparatively weather-resilient but lacks density and is itself corrupted. DDCD is therefore designed to fuse adverse RGB-LiDAR inputs with clean priors while explicitly supporting denoising (Wang et al., 16 Mar 2026).

The architecture has two paths. The student network is a depth completion model that takes $I_c$0 as input. The teacher foundation model is a pretrained, frozen depth estimator, with examples including Depth Anything v2/v3, operating on the paired clean RGB image $I_c$1. The teacher provides dense geometric priors (Wang et al., 16 Mar 2026).

Teacher prediction is denoted

$I_c$2

Because teacher outputs may be metric or relative, the paper introduces GT-aligned normalisation. For a disparity-like teacher, the aligned form is

$I_c$3

while for a metric-depth teacher it is

$I_c$4

The parameters $I_c$5 are estimated by least-squares fitting on valid pixels (Wang et al., 16 Mar 2026).

The distillation is applied in a multi-scale Scale-and-Shift Invariant (SSI) Distillation form. For each scale $I_c$6,

$I_c$7

and the aligned teacher prior is

$I_c$8

The corresponding SSI loss is reported as

$I_c$9

To regularize local consistency, the method adds Residual Gradient Regularization:

$S_c$0

where $S_c$1. A supervised $S_c$2 loss against $S_c$3 enforces metric correctness. The total objective is given as

$S_c$4

The teacher is frozen during training, while the student is trained on adverse $S_c$5 using both clean depth supervision and the distillation losses derived from the teacher’s prediction on paired clean RGB (Wang et al., 16 Mar 2026).

The underlying rationale is not that the teacher solves adverse-weather depth estimation directly, but that clean-image predictions can act as structural priors when aligned appropriately. This suggests a separation between geometric prior acquisition and adverse-weather inference: the teacher supplies geometry under clean conditions, and the student learns to recover it from corrupted multimodal inputs.

6. Experimental findings and benchmark significance

On AURORA-KITTI, DDCD is reported to achieve state-of-the-art results. The reported RMSE is 1799.55 mm, the MAE is 566.31 mm, and iRMSE/iMAE are best across all compared methods. The method is also reported as efficient at 48.6 ms per frame, in contrast to the diffusion-based Marigold-DC at 18s/frame (Wang et al., 16 Mar 2026).

Qualitatively, DDCD is described as preserving object boundaries and scene structure while remaining robust to artifacts such as rain streaks, occlusions, and fog. By contrast, RGB-only approaches are described as having unstable scale, and naïve fusion approaches are described as propagating noise (Wang et al., 16 Mar 2026).

The paper also emphasizes robustness under severe and real adverse weather. Foundation models alone are said to perform well in clean or simulated adverse conditions, but to weaken under real, severe, or previously unseen weather, including on the DENSE dataset. Failures include unstable scale and structure, particularly at long ranges or under snow and fog. DDCD is reported to preserve both structure and metric accuracy more effectively than all baselines, especially in severe weather and cross-domain evaluation (Wang et al., 16 Mar 2026).

A central ablation result concerns the relative importance of data and architecture. Retraining standard models such as BP-Net and Sparse2Dense with AURORA-KITTI yields much larger gains—reported as up to 57.9% improvement—than architectural modifications or even the clean-prior distillation itself. Distillation remains complementary, improving accuracy particularly at near/mid range and on inverse-error metrics, but the paper’s conclusion is that robustness mainly derives from realistic, large-scale, weather-aware data (Wang et al., 16 Mar 2026).

The choice of teacher also matters. Distillation using Depth Anything v2 is reported to provide better cross-domain robustness than v3, while v3 is slightly better for in-domain synthetic evaluation. The stated implication is that stronger in-domain performance of a foundation model does not necessarily translate into better severe-weather robustness (Wang et al., 16 Mar 2026).

7. Research implications and applications

AURORA-KITTI is positioned as a benchmark for reliable outdoor and autonomous driving perception under adverse weather. The paper also identifies applications in robotics navigation in all weather/light conditions, safety-critical 3D scene understanding, and the development of cross-modal, weather-adaptive, or foundation-prior-based depth fusion architectures (Wang et al., 16 Mar 2026).

Its contribution is not limited to benchmarking. Because the dataset is physically consistent and richly annotated, it supports attribute-aware evaluation across weather, scene, lighting, and sensor factors. This enables controlled analysis of specific failure modes such as long-range instability, night-weather combinations, or lens occlusion effects (Wang et al., 16 Mar 2026).

A common misconception in robust depth perception is that adverse-weather robustness is primarily an architectural problem. The reported ablations challenge that view directly: the paper finds that weather-aware, physically consistent data contributes more to robustness than architectural modifications alone (Wang et al., 16 Mar 2026). Another plausible implication is that future progress in adverse-weather RGB-LiDAR fusion may depend as much on benchmark design and simulation fidelity as on novel fusion operators.

Within the broader depth estimation landscape, AURORA-KITTI formalizes a setting in which dense prediction, denoising, sensor corruption modeling, and sim-to-real transfer are inseparable. Its paired clean references and metric supervision make it particularly suitable for studying how foundation-model priors can be imported into adverse-weather geometric inference without assuming that the foundation model itself is robust to weather.