StereoCarla: Synthetic Stereo Dataset for Driving

• StereoCarla is a synthetic stereo dataset that diversifies camera geometry with five baseline distances and multiple viewing angles to boost cross-domain generalization.
• It comprises 693,435 high-resolution (1600×900) stereo samples featuring left/right RGB images and dense disparity maps captured across various simulated town layouts.
• Experimental results demonstrate that models trained on StereoCarla outperform those trained on 11 existing datasets on benchmarks like KITTI, Middlebury, and ETH3D.

StereoCarla is a high-fidelity synthetic stereo dataset for autonomous driving built on the CARLA simulator and designed to improve the cross-domain generalization of stereo matching models. It provides left/right stereo RGB images and dense disparity maps at a resolution of 1600 × 900, with a reported total of 693,435 stereo samples. Its defining characteristic is controlled diversity in stereo geometry and scene conditions: the dataset varies baseline, viewpoint, sensor placement, lighting, weather, road geometry, and town layout, and the associated paper reports that models trained on StereoCarla outperform models trained on 11 existing stereo datasets in average cross-domain generalization across KITTI2012, KITTI2015, Middlebury, and ETH3D (Guo et al., 16 Sep 2025).

1. Concept and motivation

StereoCarla was introduced in response to a recurrent limitation in stereo matching research: strong benchmark performance does not necessarily translate into robust cross-domain behavior. The motivating argument is that many existing stereo datasets are constrained by narrow camera geometry, limited environmental diversity, weak cross-domain coverage, or by synthetic imagery that is “too indoor-like or not realistic enough” for autonomous driving, while real datasets are expensive and less controllable (Guo et al., 16 Sep 2025).

Within the CARLA ecosystem, this design objective distinguishes StereoCarla from earlier CARLA-derived resources. KITTI-CARLA is a synthetic KITTI-like driving dataset generated in CARLA v0.9.10 with KITTI-style LiDAR and two color cameras, intended for semantic segmentation, odometry, and transfer learning / domain adaptation to real KITTI; however, it is explicitly not presented as a stereo-disparity benchmark with stereo-specific evaluation (Deschaud, 2021). StereoCarla, by contrast, is centered on stereo supervision and generalization, not on KITTI imitation alone (Guo et al., 16 Sep 2025).

A plausible implication is that StereoCarla should be understood not merely as another CARLA rendering corpus, but as a stereo-specific dataset whose main variable is camera geometry itself. That emphasis is consistent with the paper’s conclusion that models trained on a more diverse stereo distribution learn representations that transfer more effectively across heterogeneous benchmarks (Guo et al., 16 Sep 2025).

2. Dataset composition and stereo geometry

StereoCarla contains Left/right stereo RGB images, Dense disparity maps, and a High resolution of 1600 × 900, with 693,435 stereo samples collected across multiple CARLA towns (Guo et al., 16 Sep 2025). The paper reports the following town distribution:

Town	Samples
Town01	81,815
Town02	87,535
Town03	87,180
Town04	87,445
Town05	87,490
Town06	87,465
Town07	87,115
Town10	87,390

The dataset’s central methodological feature is systematic variation of stereo rig geometry. StereoCarla uses five baseline distances: 10 cm, 54 cm, 100 cm, 200 cm, and 300 cm. The paper identifies this as a major distinguishing feature and states that the range is broader than in most existing datasets, with the intent of helping models handle different stereo rigs or mounting configurations in deployment (Guo et al., 16 Sep 2025).

Viewpoint diversity is also explicit. StereoCarla varies Horizontal viewing angles of 0°, 5°, 15°, and 30°, and it includes Elevated viewpoints in which the camera pair is placed at 10 m height with 0° horizontal view and −30° downward pitch (Guo et al., 16 Sep 2025). In addition, the paper highlights variation in sensor height, pitch, roll, and horizontal viewing angle, so the dataset is not restricted to a fixed canonical rectified automotive rig.

The paper further reports the following camera-related statistics: Focal length: 1385.6 px, Disparity range: 0–3318, and Average/median disparity: 80.8 / 31 (Guo et al., 16 Sep 2025). These values indicate substantial disparity variability across near and far structures. This suggests that StereoCarla is engineered to expose stereo networks to both conventional driving baselines and out-of-distribution camera placements that are still physically plausible within CARLA.

3. Environmental and scene diversity

StereoCarla augments geometric diversity with environmental diversity. The paper lists the following weather and illumination conditions: clear, cloudy, foggy, humid, night, storm, and sunset. It also emphasizes lighting changes and road/scene variations (Guo et al., 16 Sep 2025).

Town diversity is used to induce variation in road layouts, lane structures, urban geometry, and scene composition, so the dataset varies not only in appearance but also in scene geometry and driving context (Guo et al., 16 Sep 2025). The resulting training distribution therefore spans changes in rig configuration and changes in the scene manifold simultaneously.

Relative to other CARLA-based data resources, StereoCarla occupies a specific niche. SCaRL provides 140,000 synchronized and aligned data frames with 6× RGB cameras, 6× semantic/instance segmentation cameras, 6× depth cameras, 6× lidar, and 6× radar, emphasizing synchronized multi-modal sensing, coherent lidar, and MIMO radar rather than stereo disparity as the central supervision (Ramesh et al., 2024). Car-STAGE is an automated CARLA framework that supports multiple rigidly attached cameras, synchronized collection, visibility-aware annotations, and efficient memory-mapped storage, but it is described as a general high-dimensional sensing pipeline rather than a stereo-specialized dataset generator (Almutairi et al., 5 Mar 2025). StereoCarla therefore differs from multimodal CARLA datasets by making stereo generalization, rather than modality breadth or collection automation, the primary target (Guo et al., 16 Sep 2025).

4. Training setup, protocol, and metrics

The paper’s experiments are based on OpenStereo, using NMRF-Stereo with Swin Transformer backbone as the baseline model. Training uses AdamW, batch size 16, and augmentations consisting of random crop to 352 × 640, color jitter, and random erasing. The reported training schedule uses 31,250 iterations for single-dataset training and 81,000 iterations for multi-dataset training. The learning rate is 1e-3 with OneCycleLR when training on SceneFlow and 5e-4 for fine-tuning on other datasets (Guo et al., 16 Sep 2025).

StereoCarla is evaluated in both in-domain evaluation and cross-domain generalization evaluation. For StereoCarla itself, the paper uses EPE (End-Point Error) for disparity and Abs Rel and δ < 1.25 for depth evaluation (Guo et al., 16 Sep 2025).

Cross-domain evaluation uses four standard benchmarks: KITTI2012 with 194 stereo pairs, KITTI2015 with 200 stereo pairs, Middlebury with 15 image pairs, evaluated at half resolution, and ETH3D with 27 grayscale stereo pairs. The metrics are D1-all on KITTI, Bad 2.0 on Middlebury, and Bad 1.0 on ETH3D. The paper explicitly defines these as the percentage of pixels with disparity error > 3 px, > 2 px, and > 1 px, respectively (Guo et al., 16 Sep 2025).

This protocol is significant because it tests transfer not only to automotive benchmarks but also to datasets with markedly different scene statistics. The paper specifically notes that Middlebury and ETH3D are very different from driving scenes, which makes them useful probes of whether a training set induces broadly generalizable stereo priors rather than narrow automotive specialization (Guo et al., 16 Sep 2025).

5. Reported generalization performance

The paper’s central result is that StereoCarla achieves the best average cross-domain performance among the compared training datasets. In the zero-shot evaluation table, SceneFlow → StereoCarla reports K12: 4.11, K15: 4.87, Midd: 9.12, E3D: 3.17, and Mean: 5.32. The baseline trained on SceneFlow alone reports Mean: 13.99, while other listed alternatives include SceneFlow → TartanAir: 7.02, SceneFlow → CREStereo: 8.42, SceneFlow → Spring: 8.96, SceneFlow → Sintel: 9.46, and SceneFlow → VirtualKITTI2: 25.99 (Guo et al., 16 Sep 2025).

Two aspects of this table are especially important. First, the reported Mean: 5.32 is the best among the listed single-dataset fine-tuning settings. Second, the dataset is described as especially strong on “hard transfer” targets, with Middlebury: 9.12 and ETH3D: 3.17, despite those benchmarks being unlike autonomous-driving imagery (Guo et al., 16 Sep 2025). This suggests that deliberate variation in baseline, angle, and scene conditions can improve not only automotive transfer but also broader stereo robustness.

The multi-dataset training results reinforce the same claim. The best overall mixture is MIX 9, with K12: 3.51, K15: 4.04, Midd: 6.36, E3D: 2.96, and Mean: 4.22 (Guo et al., 16 Sep 2025). The paper also reports mixtures with and without StereoCarla:

Setting	Mean
MIX 7 (w/o SC)	7.48
MIX 7	4.37
MIX 8 (w/o SC)	6.77
MIX 8	4.39
MIX 9 (w/o SC)	5.93
MIX 9	4.22

The associated interpretation in the paper is that removing StereoCarla causes a sharp degradation, especially on ETH3D and Middlebury, indicating that StereoCarla contributes complementary diversity not supplied by the other datasets (Guo et al., 16 Sep 2025). In that sense, StereoCarla is presented not only as a standalone training set but also as a strong component of mixed-dataset stereo curricula.

6. Ablations, misconceptions, and ecosystem context

The ablation studies isolate which aspects of StereoCarla matter most. For baseline diversity, the paper reports that training on a single baseline overfits to that geometry; a very short baseline (10 cm) gives low in-domain error but poor transfer to larger baselines, while a very large baseline (300 cm) also generalizes poorly to short baselines. The best overall baseline setting is All baselines, with In-domain mean EPE: 1.69 and Cross-domain mean: 5.32 (Guo et al., 16 Sep 2025).

For camera-angle diversity, the paper evaluates Normal, Roll 5°, Roll 15°, Roll 30°, Pitch 0°, Pitch −30°, and All angles. Single-angle models perform best on their own angle but generalize poorly to other orientations, whereas All angles gives the best overall result, with In-domain mean EPE: 1.46 and Cross-domain mean: 5.32 (Guo et al., 16 Sep 2025). For weather diversity, training without weather variation yields mean 5.45, while training with all weather conditions yields mean 5.32, indicating a smaller but consistent gain (Guo et al., 16 Sep 2025).

These results address two common misconceptions. The first is that StereoCarla is simply another fixed-rig CARLA dataset. The paper’s own ablations show that its contribution depends on broad rig variation, not merely on rendering more frames. The second is that its value is limited to automotive benchmarks. The reported transfer to Middlebury and ETH3D indicates otherwise (Guo et al., 16 Sep 2025).

Within the broader CARLA stereo ecosystem, StereoCarla is complementary to other resources rather than interchangeable with them. KITTI-CARLA offers KITTI-like two-camera geometry and ground-truth poses but focuses on semantic segmentation, odometry, and sim-to-real transfer rather than disparity-centric evaluation (Deschaud, 2021). Car-STAGE supplies a GUI-driven, synchronized multi-sensor collection framework with multiple rigidly attached cameras and visibility-aware annotation, but no dedicated stereo mode (Almutairi et al., 5 Mar 2025). StereoGeo addresses pattern-free stereo calibration and is trained in part on 12,220 stereo pairs from CARLA, with baseline sampled in $[0.20, 0.70]$ m, roll and pitch sampled in $[\pm 0.5^\circ]$, and vertical FoV sampled in $[55^\circ, 75^\circ]$; this is relevant because StereoCarla’s explicit rig diversity could plausibly support calibration-oriented experiments as well, although that use is not claimed in the StereoCarla paper itself (Meddour et al., 12 Jun 2026).

More broadly, StereoCarla aligns with several technical directions in stereo research. The speed–accuracy trade-off remains a central issue in autonomous-driving stereo systems (Fan et al., 2020). Methods such as FFCA-Net target decoder-side efficiency for stereo image compression using stereo priors (Xia et al., 2023), L3C-Stereo exploits disparity-based warping for lossless stereo compression (Huang et al., 2021), and CogStereo emphasizes cross-domain robustness in occlusions and weak textures through cognition-guided refinement (Fang et al., 25 Oct 2025). A plausible implication is that StereoCarla’s value is not limited to benchmarking a single model family: its diverse camera geometry and environmental variation make it relevant wherever generalizable stereo representations are the core objective.