---
title: 'StereoCarla: Synthetic Stereo Dataset for Driving'
url: https://www.emergentmind.com/topics/stereocarla
type: topic
---

# StereoCarla: Synthetic Stereo Dataset for Driving

StereoCarla is a high-fidelity synthetic stereo dataset for autonomous driving built on the CARLA simulator and designed to improve the cross-domain generalization of stereo matching models. It provides left/right stereo RGB images and dense disparity maps at a resolution of `1600 × 900`, with a reported total of **693,435 stereo samples**. Its defining characteristic is controlled diversity in stereo geometry and scene conditions: the dataset varies baseline, viewpoint, sensor placement, lighting, weather, road geometry, and town layout, and the associated paper reports that models trained on StereoCarla outperform models trained on **11 existing stereo datasets** in average cross-domain generalization across **KITTI2012**, **KITTI2015**, **Middlebury**, and **ETH3D** [2509.12683].

## 1. Concept and motivation

StereoCarla was introduced in response to a recurrent limitation in stereo matching research: strong benchmark performance does not necessarily translate into robust cross-domain behavior. The motivating argument is that many existing stereo datasets are constrained by **narrow camera geometry**, **limited environmental diversity**, **weak cross-domain coverage**, or by synthetic imagery that is “too indoor-like or not realistic enough” for autonomous driving, while real datasets are expensive and less controllable [2509.12683].

Within the CARLA ecosystem, this design objective distinguishes StereoCarla from earlier CARLA-derived resources. **KITTI-CARLA** is a synthetic KITTI-like driving dataset generated in **CARLA v0.9.10** with KITTI-style LiDAR and two color cameras, intended for **semantic segmentation**, **odometry**, and **transfer learning / domain adaptation** to real KITTI; however, it is explicitly not presented as a stereo-disparity benchmark with stereo-specific evaluation [2109.00892]. StereoCarla, by contrast, is centered on stereo supervision and generalization, not on KITTI imitation alone [2509.12683].

A plausible implication is that StereoCarla should be understood not merely as another CARLA rendering corpus, but as a stereo-specific dataset whose main variable is camera geometry itself. That emphasis is consistent with the paper’s conclusion that models trained on a more diverse stereo distribution learn representations that transfer more effectively across heterogeneous benchmarks [2509.12683].

## 2. Dataset composition and stereo geometry

StereoCarla contains **Left/right stereo RGB images**, **Dense disparity maps**, and a **High resolution** of `1600 × 900`, with **693,435 stereo samples** collected across multiple CARLA towns [2509.12683]. The paper reports the following town distribution:

| Town | Samples |
|---|---:|
| Town01 | 81,815 |
| Town02 | 87,535 |
| Town03 | 87,180 |
| Town04 | 87,445 |
| Town05 | 87,490 |
| Town06 | 87,465 |
| Town07 | 87,115 |
| Town10 | 87,390 |

The dataset’s central methodological feature is systematic variation of stereo rig geometry. StereoCarla uses **five baseline distances**: **10 cm**, **54 cm**, **100 cm**, **200 cm**, and **300 cm**. The paper identifies this as a major distinguishing feature and states that the range is broader than in most existing datasets, with the intent of helping models handle different stereo rigs or mounting configurations in deployment [2509.12683].

Viewpoint diversity is also explicit. StereoCarla varies **Horizontal viewing angles** of **0°**, **5°**, **15°**, and **30°**, and it includes **Elevated viewpoints** in which the camera pair is placed at **10 m height** with **0° horizontal view** and **−30° downward pitch** [2509.12683]. In addition, the paper highlights variation in **sensor height**, **pitch**, **roll**, and **horizontal viewing angle**, so the dataset is not restricted to a fixed canonical rectified automotive rig.

The paper further reports the following camera-related statistics: **Focal length: 1385.6 px**, **Disparity range: 0–3318**, and **Average/median disparity: 80.8 / 31** [2509.12683]. These values indicate substantial disparity variability across near and far structures. This suggests that StereoCarla is engineered to expose stereo networks to both conventional driving baselines and out-of-distribution camera placements that are still physically plausible within CARLA.

## 3. Environmental and scene diversity

StereoCarla augments geometric diversity with environmental diversity. The paper lists the following weather and illumination conditions: **clear**, **cloudy**, **foggy**, **humid**, **night**, **storm**, and **sunset**. It also emphasizes **lighting changes** and **road/scene variations** [2509.12683].

Town diversity is used to induce variation in **road layouts**, **lane structures**, **urban geometry**, and **scene composition**, so the dataset varies not only in appearance but also in **scene geometry and driving context** [2509.12683]. The resulting training distribution therefore spans changes in rig configuration and changes in the scene manifold simultaneously.

Relative to other CARLA-based data resources, StereoCarla occupies a specific niche. **SCaRL** provides **140,000 synchronized and aligned data frames** with **6× RGB cameras**, **6× semantic/instance segmentation cameras**, **6× depth cameras**, **6× lidar**, and **6× radar**, emphasizing synchronized multi-modal sensing, coherent lidar, and MIMO radar rather than stereo disparity as the central supervision [2405.17030]. **Car-STAGE** is an automated CARLA framework that supports multiple rigidly attached cameras, synchronized collection, visibility-aware annotations, and efficient memory-mapped storage, but it is described as a general high-dimensional sensing pipeline rather than a stereo-specialized dataset generator [2503.03100]. StereoCarla therefore differs from multimodal CARLA datasets by making stereo generalization, rather than modality breadth or collection automation, the primary target [2509.12683].

## 4. Training setup, protocol, and metrics

The paper’s experiments are based on **OpenStereo**, using **NMRF-Stereo with Swin Transformer backbone** as the baseline model. Training uses **AdamW**, **batch size 16**, and augmentations consisting of **random crop to `352 × 640`**, **color jitter**, and **random erasing**. The reported training schedule uses **31,250** iterations for single-dataset training and **81,000** iterations for multi-dataset training. The learning rate is **1e-3** with **OneCycleLR** when training on SceneFlow and **5e-4** for fine-tuning on other datasets [2509.12683].

StereoCarla is evaluated in both **in-domain evaluation** and **cross-domain generalization evaluation**. For StereoCarla itself, the paper uses **EPE (End-Point Error)** for disparity and **Abs Rel** and **δ < 1.25** for depth evaluation [2509.12683].

Cross-domain evaluation uses four standard benchmarks: **KITTI2012** with **194 stereo pairs**, **KITTI2015** with **200 stereo pairs**, **Middlebury** with **15 image pairs, evaluated at half resolution**, and **ETH3D** with **27 grayscale stereo pairs**. The metrics are **D1-all** on KITTI, **Bad 2.0** on Middlebury, and **Bad 1.0** on ETH3D. The paper explicitly defines these as the percentage of pixels with disparity error **> 3 px**, **> 2 px**, and **> 1 px**, respectively [2509.12683].

This protocol is significant because it tests transfer not only to automotive benchmarks but also to datasets with markedly different scene statistics. The paper specifically notes that **Middlebury** and **ETH3D** are very different from driving scenes, which makes them useful probes of whether a training set induces broadly generalizable stereo priors rather than narrow automotive specialization [2509.12683].

## 5. Reported generalization performance

The paper’s central result is that StereoCarla achieves the best average cross-domain performance among the compared training datasets. In the zero-shot evaluation table, **SceneFlow → StereoCarla** reports **K12: 4.11**, **K15: 4.87**, **Midd: 9.12**, **E3D: 3.17**, and **Mean: 5.32**. The baseline trained on **SceneFlow** alone reports **Mean: 13.99**, while other listed alternatives include **SceneFlow → TartanAir: 7.02**, **SceneFlow → CREStereo: 8.42**, **SceneFlow → Spring: 8.96**, **SceneFlow → Sintel: 9.46**, and **SceneFlow → VirtualKITTI2: 25.99** [2509.12683].

Two aspects of this table are especially important. First, the reported **Mean: 5.32** is the best among the listed single-dataset fine-tuning settings. Second, the dataset is described as especially strong on “hard transfer” targets, with **Middlebury: 9.12** and **ETH3D: 3.17**, despite those benchmarks being unlike autonomous-driving imagery [2509.12683]. This suggests that deliberate variation in baseline, angle, and scene conditions can improve not only automotive transfer but also broader stereo robustness.

The multi-dataset training results reinforce the same claim. The best overall mixture is **MIX 9**, with **K12: 3.51**, **K15: 4.04**, **Midd: 6.36**, **E3D: 2.96**, and **Mean: 4.22** [2509.12683]. The paper also reports mixtures with and without StereoCarla:

| Setting | Mean |
|---|---:|
| MIX 7 (w/o SC) | 7.48 |
| MIX 7 | 4.37 |
| MIX 8 (w/o SC) | 6.77 |
| MIX 8 | 4.39 |
| MIX 9 (w/o SC) | 5.93 |
| MIX 9 | 4.22 |

The associated interpretation in the paper is that removing StereoCarla causes a sharp degradation, especially on **ETH3D** and **Middlebury**, indicating that StereoCarla contributes complementary diversity not supplied by the other datasets [2509.12683]. In that sense, StereoCarla is presented not only as a standalone training set but also as a strong component of mixed-dataset stereo curricula.

## 6. Ablations, misconceptions, and ecosystem context

The ablation studies isolate which aspects of StereoCarla matter most. For baseline diversity, the paper reports that training on a single baseline overfits to that geometry; a **very short baseline (10 cm)** gives low in-domain error but poor transfer to larger baselines, while a **very large baseline (300 cm)** also generalizes poorly to short baselines. The best overall baseline setting is **All baselines**, with **In-domain mean EPE: 1.69** and **Cross-domain mean: 5.32** [2509.12683].

For camera-angle diversity, the paper evaluates **Normal**, **Roll 5°**, **Roll 15°**, **Roll 30°**, **Pitch 0°**, **Pitch −30°**, and **All angles**. Single-angle models perform best on their own angle but generalize poorly to other orientations, whereas **All angles** gives the best overall result, with **In-domain mean EPE: 1.46** and **Cross-domain mean: 5.32** [2509.12683]. For weather diversity, training **without weather variation** yields **mean 5.45**, while training **with all weather conditions** yields **mean 5.32**, indicating a smaller but consistent gain [2509.12683].

These results address two common misconceptions. The first is that StereoCarla is simply another fixed-rig CARLA dataset. The paper’s own ablations show that its contribution depends on broad rig variation, not merely on rendering more frames. The second is that its value is limited to automotive benchmarks. The reported transfer to **Middlebury** and **ETH3D** indicates otherwise [2509.12683].

Within the broader CARLA stereo ecosystem, StereoCarla is complementary to other resources rather than interchangeable with them. **KITTI-CARLA** offers KITTI-like two-camera geometry and ground-truth poses but focuses on **semantic segmentation**, **odometry**, and sim-to-real transfer rather than disparity-centric evaluation [2109.00892]. **Car-STAGE** supplies a GUI-driven, synchronized multi-sensor collection framework with multiple rigidly attached cameras and visibility-aware annotation, but no dedicated stereo mode [2503.03100]. **StereoGeo** addresses pattern-free stereo calibration and is trained in part on **12,220 stereo pairs from CARLA**, with baseline sampled in **\([0.20, 0.70]\) m**, roll and pitch sampled in **\([\pm 0.5^\circ]\)**, and vertical FoV sampled in **\([55^\circ, 75^\circ]\)**; this is relevant because StereoCarla’s explicit rig diversity could plausibly support calibration-oriented experiments as well, although that use is not claimed in the StereoCarla paper itself [2606.14619].

More broadly, StereoCarla aligns with several technical directions in stereo research. The speed–accuracy trade-off remains a central issue in autonomous-driving stereo systems [2012.03194]. Methods such as **FFCA-Net** target decoder-side efficiency for stereo image compression using stereo priors [2312.16963], **L3C-Stereo** exploits disparity-based warping for lossless stereo compression [2108.09422], and **CogStereo** emphasizes cross-domain robustness in occlusions and weak textures through cognition-guided refinement [2510.22119]. A plausible implication is that StereoCarla’s value is not limited to benchmarking a single model family: its diverse camera geometry and environmental variation make it relevant wherever generalizable stereo representations are the core objective.

Source: https://www.emergentmind.com/topics/stereocarla