---
title: 'SYNTHIA: Synthetic Urban Scene Benchmark'
url: https://www.emergentmind.com/topics/synthia
type: topic
---

# SYNTHIA: Synthetic Urban Scene Benchmark

to=arxiv_search.search  彩神争霸苹果_json
{"query":"SYNTHIA dataset semantic segmentation domain adaptation", "max_results": 10}
to=arxiv.search  天天中彩票的_json
{"query":"SYNTHIA dataset semantic segmentation domain adaptation", "max_results": 10}
SYNTHIA is a synthetic urban-scene benchmark used extensively in computer vision as a labeled source domain for semantic segmentation, domain adaptation, video understanding, and related scene-perception tasks. In the literature summarized here, it appears most often as a driving-simulator dataset of urban scenes that provides synthetic RGB images, pixel-wise semantic labels, and, in several settings, depth / z-buffer information under varied weather and illumination conditions. Derivative benchmarks such as SYNTHIA-PANO and Omni-SYNTHIA extend it to panoramic and omnidirectional settings, while a separate group of recent papers reuses the same name for unrelated systems in microscopy, design generation, persona modeling, and solar physics [1810.03756] [1909.00532] [1907.12849] [2507.14922].

## 1. Dataset composition and benchmark conventions

In the driving-vision literature, SYNTHIA is described as a synthetic dataset of urban scenes rendered in a 3D environment, and more specifically as a driving-simulator dataset of urban scenes with synthetic RGB images, pixel-wise semantic segmentation labels, depth / z-buffer information, and varied weather and illumination conditions. A commonly used subset is **SYNTHIA-RAND-CITYSCAPES**, reported as containing **9,400 images** at resolution **\(1280 \times 760\)**, with labels consistent with Cityscapes; this is the standard source-domain configuration in much of the synthetic-to-real adaptation literature [2006.06570].

Benchmark protocols differ materially across papers. In SYNTHIA \(\rightarrow\) Cityscapes adaptation, performance is commonly reported as **mIoU@16** and **mIoU@13**, because not all methods evaluate the same subset of classes, and one translation-based paper further notes that SYNTHIA lacks annotations for **terrain, truck, and train**, so evaluation is performed on the **16 common classes** [2111.03522]. A latent-diffusion UDA paper describes SYNTHIA as **9,400 photo-realistic frames** at **\(1280 \times 960\)** with **13 classes**, which suggests that different papers may be referring to different benchmark conventions or internal preprocessing choices rather than to a single immutable specification [2412.16859].

The dataset is also used in sequence form. One point-cloud segmentation study describes the original SYNTHIA data as **stereo RGBD images generated by 4 cameras located on the top of a moving car**, and preprocesses **6 video sequences** spanning **9 different weather environments** into dynamic 3D point cloud sequences [2012.10860]. A recurrent video-segmentation paper notes that SYNTHIA contains **over 200,000 images with varying weather and seasons**, but uses only a portion of the **Highway** sequence in **summer conditions**, again illustrating that most published results rely on task-specific subsets rather than on the entire corpus [1611.05435].

## 2. SYNTHIA in synthetic-to-real semantic segmentation

The dominant use of SYNTHIA is as a labeled synthetic source domain for unsupervised or semi-supervised transfer to real urban-scene datasets such as Cityscapes, Mapillary Vistas, and BDD. The typical setup is to train on labeled SYNTHIA images while treating the target-domain training split as unlabeled during adaptation and evaluating on the target validation split. The rationale is consistent across papers: SYNTHIA offers large-scale dense supervision “for free,” but models trained only on synthetic appearance statistics overfit and transfer poorly to real images because the source-to-target gap is substantial [2006.06570] [2111.03522].

A recurrent theme is that SYNTHIA is not merely a convenient source of labels; it is a stress test for adaptation methods. One semantically consistent translation paper explicitly calls SYNTHIA a harder adaptation source than GTA5 because the domain gap is wider, with more varied viewpoints, fewer annotated images, and some missing classes [2111.03522]. Another self-supervised adaptation paper emphasizes that SYNTHIA differs from Cityscapes not only in appearance but also in **viewpoint statistics** and scene geometry, which makes pseudo-label quality especially fragile [2206.09683]. This suggests that strong performance on SYNTHIA \(\rightarrow\) Cityscapes is often interpreted as evidence of robustness to both texture and geometric shift.

The reported results are benchmark-specific and use different backbones, loss functions, and class protocols, so they are not directly interchangeable. Even so, the trajectory of published results is informative.

| Benchmark setting | Reported result | Paper |
|---|---|---|
| SYNTHIA \(\rightarrow\) Cityscapes, SPIGAN | **34.7 mIoU** at \(320 \times 640\); **SPIGAN-no-PI: 31.1** | [1810.03756] |
| SYNTHIA \(\rightarrow\) Cityscapes, FCN\(_{adv}\)+RPT\(^3\)+MS | **51.7% mIoU@16**, **59.5% mIoU@13** | [2006.06570] |
| SYNTHIA \(\rightarrow\) Cityscapes, semantically consistent I2I + SSL | **42.7% \(mIoU^{16}\)**, **49.8% \(mIoU^{13}\)** | [2111.03522] |
| SYNTHIA \(\rightarrow\) Cityscapes, DRSL+ | **46.7 mIoU**, **53.2 mIoU\*** | [2206.09683] |
| SYNTHIA \(\rightarrow\) Cityscapes, CICLD / ICCLD | **67.2 mIoU** | [2412.16859] |

When a small amount of target supervision is allowed, SYNTHIA also serves as the source domain in semi-supervised domain adaptation. A 2024 SSDA study reports **64.5 / 73.9** with **50 labels** and **67.2 / 75.7** with **100 labels** on **Synthia\(\rightarrow\)Cityscapes**, where the paired numbers correspond to **16 / 13 classes**; the fully supervised references are **68.9 / 73.1** [2411.18728]. A plausible implication is that, in this benchmark family, a small labeled target subset can close much of the residual gap that classical UDA leaves open.

## 3. Privileged information, cross-task transfer, and auxiliary supervision

One of the most influential reinterpretations of SYNTHIA is as a source not only of labels but also of simulator-internal signals. In **SPIGAN**, SYNTHIA supplies synthetic triplets \(X_s=\{(x_s,y_s,z_s)\}\), where \(y_s\) is semantic ground truth and \(z_s\) is **depth from the simulator’s z-buffer**. The framework jointly learns a generator \(G\), discriminator \(D\), task network \(T\), and privileged network \(P\), with the objective
$$
\min_{\theta_G,\theta_T,\theta_P}\max_{\theta_D} \alpha \mathcal{L}_{\text{GAN}} + \beta \mathcal{L}_T + \gamma \mathcal{L}_P + \delta \mathcal{L}_{\text{perc}}
$$
and weights \(\alpha = 1\), \(\beta = 0.5\), \(\gamma = 0.1\), \(\delta = 0.33\). The paper frames this as **Learning Using Privileged Information (LUPI)** for sim-to-real adaptation, with SYNTHIA acting simultaneously as appearance source, label source, and privileged-information source [1810.03756].

A related but distinct line of work augments semantic adaptation with additional derived signals. In an edge-aware entropy-based adversarial framework, SYNTHIA is again the labeled source domain, but now provides **semantic labels** \(y\), **depth labels** \(z\), and **edge labels** \(e\), where the edge supervision is extracted from the semantic ground-truth maps using **Canny edge detection**. The key design choice is to concatenate edge probabilities with entropy maps before domain discrimination; on the reported benchmarks this yields **46.1 mIoU** for SYNTHIA \(\rightarrow\) Cityscapes **16 classes**, **72.0** for the **7-class** Cityscapes protocol, and **69.6** for the **7-class** SYNTHIA \(\rightarrow\) Mapillary protocol [2310.07149]. This directly addresses a common misconception that synthetic-to-real adaptation is only about aligning texture or feature distributions: boundary precision is treated here as a first-class adaptation signal.

SYNTHIA has also been used to learn relationships *between* tasks. In **Across Tasks and Domains Transfer (AT/DT)**, it is the fully supervised source domain \(A\) for **semantic segmentation** and **monocular depth estimation**. The framework trains task networks \(N_j(x)=D_j(E_j(x))\) and then learns a transfer mapping \(G_{1\rightarrow2}\colon E_1(x)\mapsto E_2(x)\) on SYNTHIA before applying it to partially supervised target domains such as Cityscapes, KITTI, and Carla. In the reported **SYNTHIA \(\rightarrow\) Cityscapes** **depth \(\rightarrow\) semantics** setting, **mIoU** rises from **7.72** to **23.24** and **accuracy** from **28.49** to **64.03** [1904.04744]. Here SYNTHIA is valuable not just because it is synthetic, but because it is fully supervised for multiple dense tasks.

## 4. Temporal, 3D, panoramic, and omnidirectional extensions

SYNTHIA has repeatedly been used to study whether scene understanding improves when the input departs from the single forward-facing frame. In video semantic segmentation, recurrent and multi-stream architectures exploit the temporal continuity of driving scenes. On **SYNTHIA-CVPR'16**, **MSFCN-3** reports **94.38** mean IoU for the **Highway** scenario and **88.89** for the **City** scenario, compared with **85.42** and **73.88** for the single-frame FCN baseline; the same paper reports that temporal models especially improve thin or weak classes such as **Sidewalk** and **Pole** [1901.02511]. An earlier recurrent-FCN study on the **Highway summer** sequence reports **FC-VGG: 0.755** mean class IoU and **RFC-VGG: 0.812**, a **5.7% absolute improvement**, with especially large gains for **Car** and **Pedestrian** [1611.05435].

The dataset is equally important in dynamic 3D point-cloud segmentation. In **ASTA3DCNN**, SYNTHIA is converted from RGBD video into point-cloud sequences by constructing, for each frame, a **\(50m \times 50m \times 50m\)** cube centered on the moving car and then applying **Farthest Point Sampling** to retain **8,192 points** per frame. The reported split uses **19,888** training frames, **815** validation frames, and **1,886** test frames. The segmentation network organizes each core point around **4 virtual anchors** in a **regular tetrahedron**, and the anchor features are aggregated by
$$
f'_i=\sigma\!\left(\sum_{j = 1}^4\omega_{ij}e_i^j+b_j\right).
$$
On SYNTHIA, the paper reports **84.26 mIoU** for **2-frame** input and **84.77 mIoU** for **3-frame** input, while also noting a persistent failure case: the **Bicycle** class remains at **0.00 IoU** for all methods in the main table [2012.10860].

Panoramic and spherical derivatives extend SYNTHIA’s geometric coverage. **SYNTHIA-PANO** is built by stitching the **left, forward, right, and backward** views after cylindrical projection, yielding panoramic images of size **(3340, 760)** with **360-degree FoV**. The paper reports that training with panoramic images is beneficial, and that **180-degree FoV** is the best trade-off: performance improves from **90** to **180 degrees**, but degrades at **360 degrees** because of the extreme aspect ratio and receptive-field limitations of ICNet [1909.00532]. The same paper also describes an internal inconsistency in the distortion discussion: one sentence says that when focal length is shorter than **625**, \(M_{orin}\) performs better in some regimes, while a later conclusion says that when focal length is shorter than **625**, \(M_{pano180}\) always performs better. The stated overall conclusion is nevertheless that panoramic training improves anti-distortion robustness [1909.00532].

For omnidirectional segmentation, **Omni-SYNTHIA** samples SYNTHIA onto an **icosahedron mesh** with
$$
f_r = 20 \cdot 4^r,\qquad n_r = 2 + 10 \cdot 4^r.
$$
At **\(r=6\)** the proposed **HexUNet** reports **43.6 mIoU** and **52.2 mAcc**, outperforming **UNet** and **UGSCNN**; at **\(r=7\)** it reaches **48.3 mIoU** and **57.1 mAcc** [1907.12849]. The paper emphasizes that the filters are **orientation-aware** rather than rotation invariant, because in upright road scenes **sky** tends to be “up” and **road** tends to be “down.” Even here, small objects remain difficult: **pedestrian** and **sign** are weak, and **all methods fail for cyclist** [1907.12849].

## 5. Domain generalization and panoptic adaptation

Although SYNTHIA is most often discussed in UDA, it is also a strong testbed for **domain generalization**. A texture-randomization framework treats **SYNTHIA-RANDCITYSCAPES** as one of two synthetic source domains and reports **39.7 mIoU** for **SYNTHIA \(\rightarrow\) Cityscapes**, **35.3** for **SYNTHIA \(\rightarrow\) BDDS**, and **36.4** for **SYNTHIA \(\rightarrow\) Mapillary** with **ResNet-101**. The method uses **Global Texture Randomization (GTR)**, **Local Texture Randomization (LTR)**, and a consistency term **CGL**, and the authors explicitly compare against domain-adaptation methods despite using **no target-domain data** during training [2108.02376]. The central claim is that aggressive appearance diversification forces the model away from texture shortcuts and toward shape, layout, and context.

A second domain-generalization line exploits the fact that SYNTHIA contains multiple weather and environment shifts. **ASH+** trains on **Highway-Dawn**, **Highway-Fog**, or **Highway-Spring**, and evaluates on unseen conditions across **New York-like** and **Old European Town** environments. The paper reports **36.30** for **Highway (Dawn)**, **34.54** for **Highway (Fog)**, and **41.39** for **Highway (Spring)**, and identifies the best stylization-strength setting on SYNTHIA as **\((\sigma_1,\sigma_2)=(0,1)\)** [2304.09347]. This is presented as evidence that hallucination strength should be dataset- and pixel-dependent rather than globally fixed.

SYNTHIA has also become a benchmark for **panoptic segmentation UDA**. In **EDAPS**, it is the synthetic source domain for **SYNTHIA \(\rightarrow\) Cityscapes** and **SYNTHIA \(\rightarrow\) Mapillary Vistas**, with **9,400 synthetic images with panoptic labels**. The supplementary description follows **CVRN** and uses **11 stuff classes and 8 thing classes**. The proposed shared **MiT-B5** transformer encoder with task-specific decoders reports **mSQ 72.7, mRQ 53.6, mPQ 41.2** on **SYNTHIA \(\rightarrow\) Cityscapes** and **mSQ 71.7, mRQ 46.1, mPQ 36.6** on **SYNTHIA \(\rightarrow\) Mapillary Vistas** [2304.14291]. The gains over prior work are large, but the failure modes are also explicit: on Cityscapes, **fence** and **bike** remain very weak, and on Mapillary the **fence** class remains at **0 PQ** [2304.14291]. This reinforces a broader pattern visible across SYNTHIA-based research: thin structures, small objects, and rare classes remain difficult even with synthetic dense supervision.

## 6. Reuse of the name outside the driving benchmark

The label **SYNTHIA / Synthia / SynthIA** is no longer unique to the driving dataset. In **QuPAINT**, **Synthia** is a **physics-based synthetic data generator** for optical microscopy patches of two-dimensional quantum material flakes. It uses the **Transfer Matrix Method (TMM)**, CIE 1931 color rendering, a white-balance-aware module, and a substrate-aware module, and the paper states that it generates **50,000 samples per material**, for a total of **400,000 synthetic samples** in the training set [2602.17478].

In generative design, **SYNTHIA** denotes a framework for **novel concept design with affordance composition**. Its ontology is reported as spanning **30 superordinates, 590 concepts, 1172 parts, and 686 affordances**, and the abstract summarizes human-evaluation gains of **25.1%** in **novelty** and **14.7%** in **functional coherence** over prior methods [2502.17793]. In persona-driven language modeling, **SYNTHIA** is a synthetic persona dataset with **30,000 backstories** generated from **10,000 real BlueSky users** across **three temporal windows**—**10%**, **50%**, and **100%** of each user’s latest two-year posting history [2507.14922]. In solar physics, **SynthIA** means **Synthetic Inversion Approximation**, a U-Net-based system that maps **25-channel** SDO/HMI observation stacks to Hinode-like magnetic inversion products; the paper states that the model contains **over 10 million trainable parameters** and produces the **SynodeP** data product [2108.12421].

Across the cited vision papers, however, **SYNTHIA** continues to denote a synthetic urban-scene source domain and its derivatives. The recurring technical significance of that benchmark is not simply that it is synthetic, but that it exposes several distinct research problems at once: dense supervision without manual labeling, simulator-internal privileged information, severe synthetic-to-real domain shift, multi-view and temporal structure, and controlled variations in weather, illumination, and geometry.

Source: https://www.emergentmind.com/topics/synthia