ReaLiTy and LADS: A Unified Framework and Dataset Suite for LiDAR Adaptation Across Sensors and Adverse Weather Conditions

Abstract: Reliable LiDAR perception requires robustness across sensors, environments, and adverse weather. However, existing datasets rarely provide physically consistent observations of the same scene under varying sensor configurations and weather conditions, limiting systematic analysis of domain shifts. This work presents ReaLiTy, a unified physics-informed framework that transforms LiDAR data to match target sensor specifications and weather conditions. The framework integrates physically grounded cues with a learning-based module to generate realistic intensity patterns, while a physics-based weather model introduces consistent geometric and radiometric degradations. Building on this framework, we introduce the LiDAR Adaptation Dataset Suite (LADS), a collection of physically consistent, transformation-ready point clouds with one-to-one correspondence to original datasets. Experiments demonstrate improved cross-domain consistency and realistic weather effects. ReaLiTy and LADS provide a reproducible foundation for studying LiDAR adaptation and simulation-driven perception in intelligent transportation systems.

• The paper proposes ReaLiTy, a unified framework that enables physically consistent LiDAR transformation across sensor and weather domains.
• It leverages a physics-informed cycle-consistent GAN and state-of-the-art weather augmentation to accurately adapt both geometric and intensity features.
• LADS, the accompanying dataset suite, provides paired, transformation-consistent benchmarks that enhance the evaluation of LiDAR perception models.

ReaLiTy and LADS: A Unified and Physically-Informed Framework for LiDAR Sensor and Weather Domain Adaptation

Introduction

Accurate and robust LiDAR perception is paramount to the deployment of autonomous vehicles in real-world, diverse, and challenging operating conditions. However, systematic study and benchmarking of LiDAR’s performance across varying sensors and under adverse weather conditions have been hampered by the absence of physically consistent datasets and comprehensive simulation frameworks. The paper “ReaLiTy and LADS: A Unified Framework and Dataset Suite for LiDAR Adaptation Across Sensors and Adverse Weather Conditions” (2604.10213) addresses these limitations by proposing ReaLiTy—a unified physics-informed framework for LiDAR transformation—and LADS, a paired dataset suite supporting controlled evaluation across sensor and weather domains. This essay provides a technical synthesis of the problem addressed, the architecture and methodology of ReaLiTy, dataset construction in LADS, empirical results, and theoretical and practical implications for the field.

Unified Framework: ReaLiTy

Motivation and Problem Setting

LiDAR observations are inherently variable: sensor hardware differences, environmental interactions (e.g., material reflectivity, angle of incidence), and atmospheric phenomena (e.g., attenuation, scattering) all induce significant changes in both geometry and radiometry. The challenge lies in learning or simulating a transformation $$\mathcal{F}: \mathcal{D}_s \rightarrow \mathcal{D}_t$$ that maps LiDAR point clouds from a source domain (sensor distribution or weather condition) to a target domain, while maintaining physical and statistical plausibility. Existing approaches are fragmented—either grounded in rigid physical simulation or driven by data—but rarely achieve unified cross-domain, cross-condition transformation with physical consistency.

Architecture Overview

The ReaLiTy architecture is modular and unifies both sensor and weather adaptation under shared physical principles:

• Sensor Adaptation Branch transforms intensity patterns according to target sensor characteristics, preserving geometry.
• Weather Adaptation Branch introduces both geometric and radiometric distortions informed by physics-based modeling and refines intensity via a learning-driven module.

  Figure 1: Overview of the ReaLiTy framework modular design showing unified sensor and weather adaptation driven by shared physical cues such as range, incidence angle, and material reflectance.

A shared preprocessing stage extracts a stack of physics-informative features (range, incidence angle, reflectance), which guide both branches, ensuring that domain adaptation is physically plausible in terms of signal structure.

Sensor Adaptation

A physics-informed cycle-consistent GAN (PICGAN) architecture is leveraged for unpaired sensor-to-sensor domain translation. Two generators and two discriminators establish bidirectional mappings and exploit cycle consistency, while adversarial and physics-based losses constrain the learning to respect real-world sensor intensity statistics.

Weather Adaptation

For adverse-weather simulation, a state-of-the-art physics-based augmentation module (incorporating Mie scattering, extinction, backscatter probabilities) alters the geometry, simulating real effects of rain and snow. This weather-augmented geometry is then input to a physics-informed generator, which performs intensity mapping, optionally conditioned for multi-style adaptation (rain, snow). Consistency constraints ensure that each weather effect is distinct yet physically valid across scenarios.

Unified Loss Functions

The objective combines adversarial, reconstruction (cycle-consistency), and physics-guided losses. For example, the physics consistency loss explicitly enforces that predicted intensities align with a reference physical model, parametrized for both sensor calibration (clear) and adverse weather (via Beer-Lambert law).

LADS: LiDAR Adaptation Dataset Suite

Dataset Construction

LADS systematically addresses the absence of paired, physically consistent benchmarks by generating transformation-ready point clouds with frame-wise correspondence to the original datasets. Each LADS frame is derived using the ReaLiTy framework, ensuring that:

• The underlying geometry and calibration structures of the source datasets (e.g., SemanticKITTI, nuScenes, Voxelscape) are retained.
• Transformations are provided for both sensor adaptation (intensity translation) and weather adaptation (snow, rain), with strict one-to-one sample mapping for fair evaluation and integration.

LADS currently supports snow and rain variants across major benchmarks, includes multi-sensor alignment, and is designed to be extendable to additional domains and conditions.

Seamless Integration

Dataset generation preserves file structure, annotations, calibration, and indices, enabling immediate reuse in existing machine learning and robotics pipelines without conversion or re-labeling overhead.

Methodological Innovations

The combination of physically accurate geometric simulation and adversarial learning for intensity domain transfer represents a notable synthesis:

• Physics-Informed Feature Stack: Range, incidence, and reflectance as mandatory conditioning features tightly couple data-driven modeling with signal theory, ensuring both realism and transferability.
• Multi-Stage Adaptation: Separation of geometry and intensity transformations enables precise modeling of phenomena that affect only radiometry (sensor shifts) and those that affect both radiometry and geometry (weather).
• Unified Training: All branches are jointly optimized, and architecture is fully modular and config-driven—facilitating new domain extensions and reproducible research.

Empirical Results

Qualitative Assessment

Qualitative comparisons validate the physical fidelity of the generated transformations.

Figure 2: Results of ReaLiTy trained on SemanticKITTI showing physics-based simulation, PICGAN-adapted intensity, and real sensor data, highlighting visually plausible sensor-domain alignment.

Figure 3: Analogous qualitative results for nuScenes, illustrating the adaptability of the framework across diverse sensor targets.

Figure 4: Snow weather adaptation—KITTI-clear data, ReaLiTy-generated KITTI-snow, and real CADC snow data—demonstrates the transfer of snow-specific radiometric and geometric degradation.

Figure 5: Rain weather adaptation—nuScenes-clear, ReaLiTy rain-transformed, and Boreas real rain—emphasize realistic simulation of precipitation effects on LiDAR perception.

Across both benchmark and external validation datasets (e.g., CADC, Boreas), adapted point clouds closely mirror true sensor or weather domain signatures, including attenuation, point dropouts, and backscatter.

Quantitative Analysis: 3D Object Detection Robustness

Benchmarking on PV-RCNN for 3D detection reveals that naively training on clear-weather data yields severe performance collapse under snow/rain evaluation, whereas:

• Models augmented with ReaLiTy-generated snow/rain data exhibit substantial improvements in recall and precision.
• Combining synthetic (ReaLiTy) and real adverse-weather (CADC) data further boosts detection robustness, outperforming clear+real and synthetic-only regimes.

Notably, ReaLiTy-generated data significantly narrows the sim-to-real gap, with best results achieved via a combination of synthetic and real-weather supervision, supporting the value of high-fidelity, paired transformations for perception model generalization.

Implications and Future Directions

Practical

The provision of LADS as a paired, physically-grounded dataset with modular sensor/weather transformation tools sets a new standard for reproducibility and benchmarking in LiDAR simulation and adaptation research. These resources are immediately useful for:

• Controlled domain adaptation studies;
• Sensor-in-the-loop simulation for AV development and safety validation;
• Data-efficient learning, enabling rare-event/weather robustness with minimal real-world adverse data collection.

Theoretical

The explicit incorporation of shared physical modalities as learning anchors and the disentanglement of geometry and intensity adaptation offer a robust paradigm for further work in domain-invariant feature learning and physics-aware sim2real transfer.

Future Directions

• Expansion of LADS: Inclusion of additional sensor models, finer-grained weather conditions, and more diverse domains.
• Generative Extensions: Integration with diffusion-based or transformer-based generative models for further realism and scalability.
• Multi-Modal Sensing: Unification of LiDAR, radar, and camera adaptation/simulation for holistic AV perception modeling.
• Downstream Applications: Evaluation in semantic segmentation, SLAM, and robust localization under adverse conditions.

Conclusion

ReaLiTy and LADS jointly address the absence of unified, physically-informed frameworks for LiDAR adaptation across sensors and adverse-weather domains. By synthesizing domain knowledge with learning-based approaches and releasing paired, transformation-consistent datasets, this work enables both systematic evaluation and substantive advances in LiDAR realism and autonomous perception robustness. The modular, extensible design points to a broader future in simulation-driven AI, where domain shifts are closed by principled, physics-guided generative modeling.