---
title: 'AURORA-KITTI: Robust Depth Completion Benchmark'
url: https://www.emergentmind.com/topics/aurora-kitti
type: topic
---

# AURORA-KITTI: Robust Depth Completion Benchmark

Searching arXiv for the target paper and closely related works mentioned in the provided data.
AURORA-KITTI is a large-scale multi-modal, multi-weather benchmark for robust depth completion in the wild, introduced together with the formulation of Depth Completion and Denoising (DCD) as a unified task. It is designed for RGB-LiDAR depth perception under adverse weather, where both camera imagery and LiDAR measurements are corrupted by weather-induced effects. The benchmark contains over *82K* weather-consistent RGBL pairs with metric depth ground truth, spanning multiple weather types, three severity levels, day and night scenes, paired clean references, lens occlusion conditions, and textual descriptions. In the same work, DDCD is introduced as a distillation-based baseline that uses depth foundation models to inject clean structural priors into in-the-wild DCD training, achieving state-of-the-art performance on AURORA-KITTI and the real-world DENSE dataset while maintaining efficiency [2603.14701].

## 1. Definition and scope

AURORA-KITTI is presented as the first large-scale, physics-consistent, multi-weather RGB-LiDAR benchmark specifically designed for the DCD task. Its stated purpose is to enable robust evaluation and training of depth perception models under adverse weather, with an emphasis on physically matched degradation across sensing modalities [2603.14701].

The benchmark extends the conventional depth completion setting by assuming that both the RGB observation and the sparse depth input are weather-corrupted. Rather than only densifying a sparse depth signal, DCD jointly targets depth reconstruction and denoising. In the notation used for the task, conventional depth completion is written as
$$
D_c = f(I_c, S_c),
$$
where \(I_c\) and \(S_c\) denote clean RGB and sparse depth, respectively. The adverse-weather extension is written as
$$
\hat{D}_c = f^{w}(I_w, S_w),
$$
where \(I_w = W_{rgb}(I_c; \theta)\) and \(S_w = W_{lidar}(S_c; \theta)\) are corrupted by weather processes, while supervision remains the clean dense target \(D_c\) [2603.14701].

This formulation makes the benchmark relevant not only to depth completion in the narrow sense, but also to robust sensor fusion, denoising-aware geometric reconstruction, and adverse-weather 3D scene understanding. A plausible implication is that AURORA-KITTI serves simultaneously as a dataset, an evaluation protocol, and an operational definition of robustness for RGB-LiDAR depth estimation under weather shift.

## 2. Dataset composition and annotation structure

AURORA-KITTI contains **82,177 RGB-LiDAR pairs**. The splits are approximately **70,500** for training, **5,800** for validation, and **5,900** for test, with each split covering all scene and weather variations [2603.14701].

Its core modalities are RGB images, LiDAR point clouds, and sparse depth maps. RGB images are simulated using a physically-based rendering pipeline and combined with GAN-based style transfer for appearance shifts such as nighttime and wet road conditions. LiDAR point clouds are generated with physics-based degradation that includes **particle-induced backscatter**, **attenuation**, **spurious/fake returns**, and **occlusions**. Sparse depth maps are then produced by projecting corrupted LiDAR point clouds onto the image plane using accurate camera-LiDAR calibration [2603.14701].

The weather and lighting coverage is explicitly structured. Weather types include **clear, fog, rain, and snow**, and all are paired with **day/night variants**, yielding compound conditions such as night-rain and night-snow. Each weather category has **three physically-meaningful severity levels**. The examples given are rain rates of **10/100/200 mm/hr** and fog attenuation coefficients of **0.01/0.1/0.2 m\(^{-1}\)**. The dataset also models **lens occlusion**, including adherent raindrops and snowflakes that induce local blur and chromatic effects [2603.14701].

The annotation schema is unusually rich for depth completion. Each sample includes **dense ground truth depth**, **weather metadata**, **paired clean references**, and **textual descriptions**. The dense ground truth is clean, metric-scale, and pixel-aligned. Weather metadata includes weather type, severity, time of day, scene context, and lens condition. The paired clean reference provides a clear-weather version of each adverse sample, enabling controlled comparison and distillation. The textual descriptions summarize scene and sensor attributes per frame [2603.14701].

| Component | Description |
|---|---|
| Scale | 82,177 RGB-LiDAR pairs |
| Splits | ~70,500 train, ~5,800 val, ~5,900 test |
| Weather | Clear, fog, rain, snow; all with day/night variants |
| Severity | Three physically-meaningful levels per weather |
| Additional factors | Lens occlusion, paired clean references, textual descriptions |

The combination of paired clean references and dense metric supervision is central. The paper explicitly notes that obtaining paired, aligned, clean depth ground truth with adverse-weather inputs is infeasible in the real world; the synthetic paired construction is therefore what makes controlled, scalable DCD training and evaluation possible [2603.14701].

## 3. Physics-consistent generation process

A defining property of AURORA-KITTI is **cross-modal consistency**: RGB and LiDAR degradations are matched by shared parameters so that weather corruption is coherent across sensing channels. This is intended to support realistic evaluation of RGB-LiDAR fusion models, which otherwise risk being assessed on modality combinations that do not correspond to the same physical conditions [2603.14701].

The camera degradation pipeline is described as a **multi-layer image synthesis** process involving scene appearance, volumetric particles, and explicitly modeled lens occlusions. On the LiDAR side, adverse-weather simulation is based on established models, including **LISA for rain** and **Hahner et al. for snow/fog**. The paper characterizes the resulting LiDAR corruption as **range-dependent** and **non-uniform**, consistent with the physical mechanisms of scattering, attenuation, and false returns [2603.14701].

This design matters because adverse weather degrades the two modalities differently. RGB images exhibit appearance shifts, blur, streaking, and occlusion-like effects, while LiDAR exhibits missing returns, fake returns, and depth-dependent measurement distortion. By coupling both degradations through shared weather parameters, AURORA-KITTI avoids the simplified assumption that adverse weather is a purely image-side nuisance or a purely point-cloud-side nuisance. This suggests that the benchmark is optimized for studying failure modes of fusion architectures rather than for isolated single-modality robustness.

## 4. The DCD task and evaluation protocol

The DCD task generalizes standard depth completion by requiring models to recover a **dense, accurate, and clean** depth map from **corrupted sparse RGB-LiDAR inputs**. The two explicit subproblems are densification from LiDAR sparsity and suppression of weather-induced noise and artifacts in both RGB and LiDAR [2603.14701].

Benchmarking uses four lower-is-better metrics:

- **RMSE (mm)**: Root Mean Squared Error for depth
- **MAE (mm)**: Mean Absolute Error
- **iRMSE (1/km)**: Root Mean Squared Error of inverse depth
- **iMAE (1/km)**: Mean Absolute Error of inverse depth

All models are evaluated in the DCD setting, meaning that the input is \((I_w, S_w)\) and the target is the clean depth \(D_c\) [2603.14701].

Two principal evaluation regimes are defined. In **zero-shot evaluation**, models trained on clear-weather data are tested directly on AURORA-KITTI to measure robustness under distribution shift. In **in-domain retraining**, models are retrained or fine-tuned on the AURORA-KITTI training split without architecture changes, isolating the effect of exposure to robust weather-aware data. A third axis, **cross-dataset generalization**, evaluates models trained on AURORA-KITTI on the real-world **DENSE** benchmark [2603.14701].

This protocol separates at least three factors that are often conflated in adverse-weather depth estimation: base architecture quality, robustness from training distribution, and transfer to real scenes. The paper’s later ablations place particular emphasis on this separation.

## 5. DDCD baseline and distillation mechanism

The proposed baseline, **DDCD**, is a distillation-based method for DCD. Its stated motivation is that depth foundation models encode strong geometric priors in clean conditions but degrade in severe weather, whereas sparse LiDAR is comparatively weather-resilient but lacks density and is itself corrupted. DDCD is therefore designed to fuse adverse RGB-LiDAR inputs with clean priors while explicitly supporting denoising [2603.14701].

The architecture has two paths. The **student network** is a depth completion model that takes \((I_w, S_w)\) as input. The **teacher foundation model** is a pretrained, frozen depth estimator, with examples including **Depth Anything v2/v3**, operating on the paired clean RGB image \(I_c\). The teacher provides dense geometric priors [2603.14701].

Teacher prediction is denoted
$$
P = f_d(I_c).
$$
Because teacher outputs may be metric or relative, the paper introduces **GT-aligned normalisation**. For a disparity-like teacher, the aligned form is
$$
D'(x) = \frac{1}{aP(x) + b},
$$
while for a metric-depth teacher it is
$$
D'(x) = aP(x) + b.
$$
The parameters \((a,b)\) are estimated by least-squares fitting on valid pixels [2603.14701].

The distillation is applied in a **multi-scale Scale-and-Shift Invariant (SSI) Distillation** form. For each scale \(l\),
$$
(a^*, b^*) = \underset{a, b}{\arg\min} \; |W^{(l)} \odot \left(D_s^{(l)} - (a D_t'^{(l)} + b)\right)|^2,
$$
and the aligned teacher prior is
$$
\hat{D}_t^{(l)} = a^* D_t^{(l)} + b^*.
$$
The corresponding SSI loss is reported as
$$
\mathcal{L}_{\text{ssi} = \sum_{l=0}^{L-1} w_l |W^{(l)} \odot (D_s^{(l)} - \hat{D}_t^{(l)})|_1.
$$

To regularize local consistency, the method adds **Residual Gradient Regularization**:
$$
\mathcal{L}_{\text{grad} = \sum_{l=0}^{L-1} w_l \left( |\nabla_x R^{(l)}|_1 + |\nabla_y R^{(l)}|_1 \right),
$$
where \(R^{(l)} = D_s^{(l)} - \hat{D}_t^{(l)}\). A supervised \(L_1\) loss against \(D_c\) enforces metric correctness. The total objective is given as
$$
\mathcal{L} = \mathcal{L}_{\text{sup} + \lambda_{\text{ssi} \mathcal{L}_{\text{ssi} + \lambda_{\text{grad} \mathcal{L}_{\text{grad}.
$$
The teacher is frozen during training, while the student is trained on adverse \((I_w, S_w)\) using both clean depth supervision and the distillation losses derived from the teacher’s prediction on paired clean RGB [2603.14701].

The underlying rationale is not that the teacher solves adverse-weather depth estimation directly, but that clean-image predictions can act as structural priors when aligned appropriately. This suggests a separation between geometric prior acquisition and adverse-weather inference: the teacher supplies geometry under clean conditions, and the student learns to recover it from corrupted multimodal inputs.

## 6. Experimental findings and benchmark significance

On AURORA-KITTI, DDCD is reported to achieve state-of-the-art results. The reported **RMSE is 1799.55 mm**, the **MAE is 566.31 mm**, and **iRMSE/iMAE** are best across all compared methods. The method is also reported as efficient at **48.6 ms per frame**, in contrast to the diffusion-based **Marigold-DC** at **18s/frame** [2603.14701].

Qualitatively, DDCD is described as preserving object boundaries and scene structure while remaining robust to artifacts such as rain streaks, occlusions, and fog. By contrast, RGB-only approaches are described as having unstable scale, and naïve fusion approaches are described as propagating noise [2603.14701].

The paper also emphasizes robustness under severe and real adverse weather. Foundation models alone are said to perform well in clean or simulated adverse conditions, but to weaken under real, severe, or previously unseen weather, including on the **DENSE** dataset. Failures include unstable scale and structure, particularly at long ranges or under snow and fog. DDCD is reported to preserve both structure and metric accuracy more effectively than all baselines, especially in severe weather and cross-domain evaluation [2603.14701].

A central ablation result concerns the relative importance of data and architecture. Retraining standard models such as **BP-Net** and **Sparse2Dense** with AURORA-KITTI yields much larger gains—reported as **up to 57.9% improvement**—than architectural modifications or even the clean-prior distillation itself. Distillation remains complementary, improving accuracy particularly at near/mid range and on inverse-error metrics, but the paper’s conclusion is that robustness mainly derives from realistic, large-scale, weather-aware data [2603.14701].

The choice of teacher also matters. Distillation using **Depth Anything v2** is reported to provide better cross-domain robustness than **v3**, while **v3** is slightly better for in-domain synthetic evaluation. The stated implication is that stronger in-domain performance of a foundation model does not necessarily translate into better severe-weather robustness [2603.14701].

## 7. Research implications and applications

AURORA-KITTI is positioned as a benchmark for **reliable outdoor and autonomous driving perception** under adverse weather. The paper also identifies applications in **robotics navigation in all weather/light conditions**, **safety-critical 3D scene understanding**, and the development of **cross-modal, weather-adaptive, or foundation-prior-based depth fusion architectures** [2603.14701].

Its contribution is not limited to benchmarking. Because the dataset is physically consistent and richly annotated, it supports **attribute-aware evaluation** across weather, scene, lighting, and sensor factors. This enables controlled analysis of specific failure modes such as long-range instability, night-weather combinations, or lens occlusion effects [2603.14701].

A common misconception in robust depth perception is that adverse-weather robustness is primarily an architectural problem. The reported ablations challenge that view directly: the paper finds that weather-aware, physically consistent data contributes more to robustness than architectural modifications alone [2603.14701]. Another plausible implication is that future progress in adverse-weather RGB-LiDAR fusion may depend as much on benchmark design and simulation fidelity as on novel fusion operators.

Within the broader depth estimation landscape, AURORA-KITTI formalizes a setting in which dense prediction, denoising, sensor corruption modeling, and sim-to-real transfer are inseparable. Its paired clean references and metric supervision make it particularly suitable for studying how foundation-model priors can be imported into adverse-weather geometric inference without assuming that the foundation model itself is robust to weather.

Source: https://www.emergentmind.com/topics/aurora-kitti