SynVS-All: Synthetic Vehicle Search Benchmark

• SynVS-All is a synthetic dataset designed for end-to-end vision-based vehicle search with integrated detection and re-identification.
• It leverages Unreal Engine to simulate multi-camera urban scenes under day, dawn, rain, and night conditions with precise bounding box annotations.
• The benchmark reveals that environmental diversity, particularly adverse conditions, significantly impacts mAP performance, underscoring system robustness challenges.

Searching arXiv for the primary paper and closely related benchmarks to ground the article in the cited literature. SynVS-All is a synthetic dataset for vision-based vehicle search introduced in "CLIPVehicle: A Unified Framework for Vision-based Vehicle Search" (Wang et al., 6 Aug 2025). Within that benchmark, it serves as the all-weather synthetic counterpart to SynVS-Day and complements the real-world CityFlowVS set. Its design targets end-to-end vehicle search, in which detection and vehicle re-identification are evaluated jointly rather than as a pipeline that first stores pre-detected patches and then applies a separate Re-ID model. In the reported benchmark, SynVS-All is the largest and most diverse synthetic vehicle-search dataset, with multi-camera identities, exact synthetic bounding-box annotations, disjoint train/test towns, and evaluation under day, dawn, rain, and night conditions (Wang et al., 6 Aug 2025).

1. Conceptual role in vehicle search

The dataset was introduced in the context of a broader problem statement: existing vehicle-search methods typically pre-detect and store all vehicle patches from surveillance videos and then run vehicle re-identification, which is described as resource-intensive and not very practical (Wang et al., 6 Aug 2025). SynVS-All was therefore constructed as part of a benchmark for joint detection and re-identification, alongside CityFlowVS and SynVS-Day.

In that benchmark, "end-to-end" vehicle search means that, given a query crop $q$ and gallery frames $\{x_k\}$, a model must produce detected boxes together with ID-score similarity to the query. This framing is important because it evaluates the full operational problem rather than an isolated Re-ID subtask. A plausible implication is that SynVS-All is intended not simply as a synthetic Re-ID corpus, but as a benchmark for integrated search systems under realistic multi-camera surveillance conditions.

2. Synthetic generation pipeline

SynVS-All is derived from the "Synthehicle" synthetic multi-camera multi-target tracking dataset. The source environment is rendered in Unreal Engine across five virtual towns, Town01–Town05, using a variety of pre-built crossroads and highway scenes (Wang et al., 6 Aug 2025).

The scene design includes 340 virtual cameras mounted around intersections and roadways, covering 64 distinct scene setups. Weather and illumination conditions span day, dawn, rain, and night; SynVS-All includes all four conditions, whereas SynVS-Day retains only the daytime frames. The vehicle fleet is described as a large pool of high-fidelity 3D vehicle models spanning sedans, vans, SUVs, trucks, and related categories, with randomized colors and minor styling variations. Vehicle motion follows realistic traffic flows and turning maneuvers (Wang et al., 6 Aug 2025).

The sampling procedure removes temporal redundancy and enforces multi-camera relevance. Every 5th frame is first extracted. Frames without any vehicles are then discarded, as are vehicles seen in only one camera. This preprocessing step is aligned with the search objective, since single-camera-only instances do not support true cross-camera retrieval.

3. Annotation model and identity curation

The annotation format is explicitly matched to the real-world CityFlowVS benchmark. For each frame $x_t$, SynVS-All stores bounding boxes and integer identities in the form

$y_t = \{(b_{tj}, c_{tj})\}_{j=1\ldots N^t}.$

Here, $b_{tj} \in \mathbb{R}^4$ denotes the bounding box for each visible vehicle, and $c_{tj} \in \{1,\ldots,C\}$ is an integer vehicle identity consistent across cameras and scenes (Wang et al., 6 Aug 2025).

No explicit orientation or attribute tags are provided. However, color and vehicle type are implicit in the synthetic textures. Annotation quality control is unusually strong because the ground truth is synthetic and therefore exact; no manual relabeling is required. Pedestrian annotations are dropped, and any identity that appears in only one town or camera is removed to ensure a genuine multi-camera search setting (Wang et al., 6 Aug 2025).

This annotation scheme is significant because it couples exact localization with persistent cross-camera identity labels. In practical terms, SynVS-All is structured for detection-and-matching rather than for standalone object detection or standalone Re-ID.

4. Corpus scale, splits, and relation to companion datasets

SynVS-All contains 2,300 IDs, 69,918 frames, 585,125 boxes, and 4,038 queries in total. The average number of vehicles per frame is reported as $585\,125 \div 69\,918 \simeq 8.37$ (Wang et al., 6 Aug 2025).

The train/test partition is town-disjoint and identity-disjoint. Town01–03 are used for training, while Town04–05 are used for testing. For testing, one query image per vehicle per camera is sampled, yielding 4,038 queries (Wang et al., 6 Aug 2025).

Split	IDs	Frames / Boxes / Queries
Train	1,457	44,273 / 388,087 / –
Test	843	25,645 / 197,038 / 4,038
Total	2,300	69,918 / 585,125 / 4,038

Relative to the companion datasets in the same benchmark, CityFlowVS has 666 IDs, 18,111 frames, and 50,629 boxes, and is restricted to daytime urban intersections. SynVS-Day has 810 IDs, 17,912 frames, and 145,398 boxes, and is limited to daytime scenes in Town01–05. SynVS-All includes dawn, rain, and night in addition to day, producing much greater appearance variation. It is described as having roughly 3× more vehicles and frames than SynVS-Day and more than 4× more than CityFlowVS (Wang et al., 6 Aug 2025).

These differences matter because they alter both the data regime and the difficulty regime. SynVS-All is not merely larger than SynVS-Day; it systematically broadens the illumination and weather manifold.

5. Evaluation protocol and benchmark behavior

The benchmark evaluates end-to-end search using mean Average Precision and Cumulative Matching Characteristics Top-1. A detected box is counted as a true positive if

$$\mathrm{IoU}(\mathrm{box}, \mathrm{gt\_box\_sameID}) > 0.5.$$

For each query $q$, a ranked list of gallery detections is produced. If $AP(q)$ denotes the area under its precision–recall curve, then

$\{x_k\}$0

CMC@1, or Top-1, is defined as the fraction of queries whose correct match appears at rank 1 (Wang et al., 6 Aug 2025).

Within this protocol, CLIPVehicle achieves 24.6 %mAP on SynVS-All, compared with 32.4 %mAP on SynVS-Day and 14.1 %mAP on CityFlowVS (Wang et al., 6 Aug 2025). The reported interpretation is that SynVS-All is harder than SynVS-Day because of its more diverse illumination and weather conditions. Per-scene analysis further shows that even the best model drops from 32.6 %mAP in daytime to 21.6 %mAP at night, identifying night scenes as the steepest degradation regime (Wang et al., 6 Aug 2025).

A common misconception is that exact synthetic ground truth necessarily yields an easy benchmark. The reported results do not support that view. Instead, the night-time degradation indicates that annotation precision does not eliminate the appearance challenges induced by low light and adverse weather. This suggests that SynVS-All is primarily difficult because of cross-condition visual discrimination rather than label noise.

6. Relation to CLIPVehicle and benchmark significance

SynVS-All was introduced together with CLIPVehicle, a unified framework containing a dual-granularity semantic-region alignment module for leveraging vision-LLMs in vehicle discrimination modeling and a multi-level vehicle identification learning strategy spanning global, instance, and feature levels (Wang et al., 6 Aug 2025). Although the principal ablations in the paper focus on CityFlowVS, they are reported to validate both the dual-granularity semantic alignment and the multi-level identification learning components, and these improvements are said to extend even to the most challenging SynVS-All setting.

In the benchmark as summarized, SynVS-All occupies a specific methodological role. It provides synthetic data with perfect multi-camera IDs and bounding boxes, while remaining substantially harder than the daytime-only synthetic subset. That combination makes it useful for studying robustness to environmental variation under a unified detection-plus-Re-ID protocol. The data support the view that SynVS-All is not merely a scale-up of SynVS-Day, but a harder benchmark defined by weather and illumination diversity, disjoint-town generalization, and multi-camera identity persistence (Wang et al., 6 Aug 2025).