ShotcreteDepth: A Bi-modal Dataset for Robust Robotic Depth Perception in Shotcrete Construction Environments

Abstract: We introduce ShotcreteDepth, a bi-modal dataset from the construction domain that captures both an active shotcreting process and general construction environments. The dataset comprises stereo RGB imagery and LiDAR point clouds acquired under harsh real-world conditions, including high turbidity and poor illumination. Such conditions adversely affect sensor measurements, leading to incomplete and noisy observations that pose significant challenges for perception systems in autonomous applications. Alongside the dataset, we release a lightweight annotation tool designed for time-efficient labeling of LiDAR point clouds. ShotcreteDepth consists of 11,252 temporally synchronized data samples, of which 220 are annotated for evaluation purposes. The dataset supports research in stereo matching, depth completion, and depth estimation under conditions that closely reflect the operational complexities found in industrial settings. Project repository: https://github.com/dtu-pas/shotcrete-depth

• The paper presents a novel bi-modal dataset capturing synchronized stereo RGB and LiDAR data in challenging, dust-laden shotcrete construction sites.
• The methodology features custom dust-proof sensor housings, precise hardware synchronization, and a dual-view annotation tool for effective noise reduction.
• Key benchmarks demonstrate superior accuracy with neural stereo and monocular depth estimation models, achieving metrics such as MAE: 0.311–0.337 m and REL: 0.133.

ShotcreteDepth: A Bi-modal Dataset for Robust Robotic Depth Perception in Shotcrete Construction Environments

Motivation and Context

Robotic automation in niche construction tasks such as shotcrete application is impeded by the hostile, turbid, and poorly illuminated environments typically encountered in tunnels and mines. Depth perception systems for autonomous operation must contend with occluded, noisy, and incomplete sensor data—especially due to dust-laden air—and yet research in this specific domain suffers from critical data shortages. The "ShotcreteDepth" dataset (2606.23152) directly addresses these limitations by providing synchronized stereo RGB and LiDAR modalities, specifically captured under real shotcreting conditions, including variable illumination and high particulate concentration.

Dataset Design and Acquisition

ShotcreteDepth comprises 11,252 synchronized data samples (RGB, disparity, LiDAR), with 220 annotated samples designated for evaluation. The acquisition platform utilizes a Roboception rc_visard 160c stereo camera and a Velodyne PUCK LiDAR, both installed in custom 3D-printed dust-proof housing to mitigate sensor contamination

Figure 1: 3D printed, dust-proof sensor housing integrating the stereo camera and LiDAR for robust data capture in high-turbidity environments.

Temporal synchronization is maintained by hardware clocks and GPS, while data is collected at modality-specific frequencies (RGB 25Hz, disparity 3Hz, LiDAR 10Hz). Intrinsic and extrinsic calibrations are performed using manufacturer tools and MATLAB frameworks. The dataset spans a variety of operational states, capturing before, during, and after shotcreting, as well as general construction context.

Annotation Tool and Dust Exclusion

A lightweight, dual-view annotation tool is released with the dataset to facilitate efficient labeling and dust cloud removal from point cloud ground truth

Figure 2: Annotation tool for efficient manual exclusion of dust clouds from LiDAR point clouds; includes 3D and projected RGB overlay views.

The tool allows granular labeling of individual points ("kept" or "removed"), both manually and algorithmically. The removal process is tailored to exclude occlusions and unreliable measurements, maintaining valid geometry for evaluation.

Sensor Modality Characteristics and Turbidity Impact

Analysis reveals that LiDAR is affected more severely by airborne shotcrete particles than stereo cameras, suffering from:

• Frequent missing returns, sometimes complete "blindness" in dense dust
• Increased noise in point clouds during active shotcreting
• Dust clouds visible in LiDAR yet mostly translucent in RGB imagery

This highlights the complementary strengths of stereo and LiDAR, recommending sensor fusion as an optimal approach for robust perception.

Baseline Evaluations: Stereo Matching, Depth Completion, Depth Estimation

Baselines across three task modalities were evaluated on ShotcreteDepth to elucidate dataset characteristics.

Stereo Matching

Three state-of-the-art neural stereo networks (RAFT-Stereo, FoundationStereo, Stereo Anywhere) and the embedded SGM implementation are compared

Figure 3: Comparison of disparity maps: deep models yield more complete, edge-preserving depth in occluded/turbid regions vs SGM.

The deep models demonstrate full coverage (100%) and superior accuracy (MAE: 0.311–0.337 m, RMSE: 0.729–0.795 m) versus SGM (MAE: 1.467 m, RMSE: 2.207 m), but at substantially greater computational expense.

Depth Completion

Three modern depth completion methods (Marigold-DC, Marigold-SSD, VPP4DC) are evaluated with sparse depth input. Marigold-DC (ensemble) achieves lowest MAE (0.325 m) and RMSE (0.793 m); however, Marigold-SSD offers competitive performance with orders of magnitude faster runtime

Figure 4: Visualization of depth completion results—ensemble methods achieve superior completeness and boundary accuracy.

Boundary metrics (PDBE) further differentiate methods, emphasizing the importance of edge refinement in sparse ground-truth scenarios.

Depth Estimation

Three zero-shot metric depth estimation models (DepthAnything v3, Marigold-E2E, MoGe-2) are benchmarked. DepthAnything v3 delivers the most accurate predictions (REL: 0.133, $\delta_1$: 0.834), confirming the value of ViT-based monocular foundational models

Figure 5: Qualitative comparison of monocular depth estimation models operating under shotcreting conditions.

Affine alignment of predictions is performed using sampled sparse depth for fair metric evaluation.

Numerical Results and Claims

• Neural stereo matching methods achieve full coverage and superior accuracy (MAE: 0.311–0.337 m) relative to the embedded SGM baseline (MAE: 1.467 m).
• Depth completion is feasible with sparse, noisy input; Marigold-DC (ensemble) attains lowest MAE (0.325 m) but requires significant runtime, while Marigold-SSD offers a practical tradeoff.
• Monocular depth estimation, especially DepthAnything v3, is robust under turbidity (REL: 0.133, $\delta_1$: 0.834), supporting zero-shot transfer even in niche environments.

Practical and Theoretical Implications

ShotcreteDepth provides a pivotal resource for developing depth perception systems in extreme environments, enabling benchmarking in conditions characterized by sparse, unreliable, and occluded data. The dataset reveals that high-turbidity environments are tractable for deep learning-based depth reasoning, provided sensor fusion and redundancy are exploited. Practically, this paves the way for reducing human exposure, minimizing material waste, and achieving regulatory compliance in automated shotcrete application.

From a theoretical standpoint, the results stimulate research in domain adaptation, sensor fusion, robust edge detection, and efficient annotation. The challenge of large model runtimes versus embedded deployment is highlighted, calling for future work in model compression and real-time optimization.

Future Directions in AI and Robotic Perception

Advancements will likely focus on:

• Sensor fusion algorithms leveraging stereo and LiDAR for optimal coverage and precision
• Adaptation and transfer of foundation models to highly adverse domains
• Automated annotation and ground-truth generation using active perception and self-supervision
• Hardware-software co-design to minimize runtime constraints and enable deployment in embedded platforms

Domain-specific, real-world datasets such as ShotcreteDepth will continue to catalyze robust and reliable perception methods, with implications for other high-risk, high-turbidity industrial sectors.

Conclusion

ShotcreteDepth sets a new benchmark for robust depth perception research in construction environments with challenging visibility and sensor reliability constraints. The dataset, associated annotation tools, and comprehensive baseline evaluations document both the obstacles and the practical solutions for advancing autonomous robotic systems in the shotcreting domain. The findings support the claim that careful sensor integration, modern deep models, and targeted annotation strategies enable feasible, accurate, and efficient depth perception for industrial automation applications in extreme conditions.