CityFlowVS: End-to-End Vehicle Search

• CityFlowVS is a benchmark for end-to-end vision-based vehicle search that jointly evaluates detection and identification in raw surveillance frames.
• It is built on the CityFlowV2 dataset, using scenario-based sampling to inherit real-world vehicle bounding boxes and identity labels across multi-camera urban scenes.
• The accompanying CLIPVehicle framework leverages dual-granularity semantic alignment and multi-level identification learning to address detection uncertainty and identity discrimination.

CityFlowVS is a real-world benchmark for end-to-end vision-based vehicle search, defined as the joint problem of detecting vehicles in raw surveillance frames and retrieving instances that match a query vehicle image. It was introduced together with the CLIPVehicle framework to address a gap between conventional vehicle detection datasets, which localize vehicles, and crop-based vehicle re-identification datasets, which assume that gallery vehicles have already been perfectly detected and cropped. In CityFlowVS, the gallery consists of raw frames rather than pre-extracted vehicle patches, so localization and identity discrimination are evaluated simultaneously (Wang et al., 6 Aug 2025).

1. Position within the CityFlow benchmark lineage

CityFlowVS belongs to a broader research lineage shaped by the CityFlow family of urban traffic benchmarks. The earlier CityFlow benchmark was introduced for multi-target multi-camera vehicle tracking and image-based vehicle re-identification, with synchronized traffic video, cross-camera identities, and camera geometry intended to support the full MTMC pipeline rather than isolated image retrieval (Tang et al., 2019). CityFlowVS preserves the emphasis on city-scale, multi-camera urban surveillance, but shifts the task formulation from MTMC and crop-based ReID toward direct search over raw frames.

This change is methodological rather than merely cosmetic. In crop-based vehicle ReID, detection is assumed solved, and the model only compares already localized vehicle patches. CityFlowVS instead evaluates a setting closer to operational surveillance, where a system must first detect vehicles in gallery frames and only then determine whether a detected box matches the query identity. This makes the benchmark adjacent to person search rather than standard vehicle ReID. A plausible implication is that CityFlowVS should be understood not as a replacement for MTMC or ReID benchmarks, but as a benchmark for a different stage of the surveillance stack: instance-level retrieval under raw-frame uncertainty.

2. Dataset source, construction, and composition

CityFlowVS is built from CityFlowV2, described in the benchmark paper as a city-scale multi-target multi-camera vehicle tracking dataset containing 3.58 hours (215.03 minutes) of video from 46 cameras across 16 intersections in a mid-sized U.S. city. The CityFlowVS construction pipeline is explicit: all frames are extracted from each raw video, frames without vehicles are filtered out, one in every five frames is sampled, and the original vehicle bounding boxes and identity labels are inherited. The split is scenario-based, with S01, S02, S03, and S04 used for training and S05 used for testing. Within the test split, the query set is created by randomly selecting one image for each vehicle in each camera, cropping the vehicle with the ground-truth bounding box, and using that crop as a query image (Wang et al., 6 Aug 2025).

The resulting benchmark contains frame-level raw images, vehicle bounding boxes, and vehicle identity labels. The identity annotations are inherited from CityFlowV2’s multi-camera tracking labels, so identities may recur across multiple frames and multiple cameras. As in standard ReID protocols, training and test identities are disjoint.

Split	Identities	Frames	Boxes
Train	329	6,561	17,752
Test	337	11,550	32,877
Total	666	18,111	50,629

In addition, the test protocol contains 1,759 queries. These numbers make CityFlowVS a moderately sized but nontrivial benchmark for raw-frame search, especially because all performance must be achieved under joint localization and identification rather than over curated vehicle crops.

3. Task formulation and evaluation protocol

The training set is formalized as

$$\mathcal{D} = \{(\mathbf{x}_t,\mathbf{y}_t) \}_{t=1}^{T},$$

where $\mathbf{x}_t$ is the $t$-th raw frame and $\mathbf{y}_t$ is its annotation. Each annotation is

$\mathbf{y}_t=\{(b_t^j,c_t^j)\}_{j=1}^{N^{t}},$

where $N^t$ is the number of vehicles in frame $\mathbf{x}_t$, $b_t^j$ is the ground-truth bounding box, and $c_t^j\in \{1,2,\cdots, C\}$ is the identity label of the $j$-th vehicle. The learned model must therefore solve detection and identity representation learning in a single framework.

At test time, the query is a cropped vehicle image $\mathbf{x}_t$0, while the gallery is a set of raw frames

$\mathbf{x}_t$1

A retrieved gallery detection is counted as correct only if it satisfies two conditions simultaneously: it must have IoU $\mathbf{x}_t$2 with a ground-truth box, and that matched ground-truth box must have the same identity as the query. This criterion is central to CityFlowVS, because it couples retrieval quality to localization quality. A system cannot achieve strong search performance by learning identity features alone if its detector is weak or poorly aligned (Wang et al., 6 Aug 2025).

The benchmark reports mAP and CMC Top-1. These are familiar ReID-style metrics, but under CityFlowVS they quantify end-to-end search rather than crop-based retrieval. This distinction is crucial: errors now arise from false negatives, false positives, box misalignment, and identity confusion simultaneously.

4. Reference framework: CLIPVehicle

The benchmark was introduced alongside CLIPVehicle, which the paper presents as a unified end-to-end framework for vehicle search. Its backbone extracts a shared feature map

$\mathbf{x}_t$3

then an RPN generates proposals, RoI Align extracts proposal features, and a detection head with box predictor produces preliminary detections. A second RoI-based pathway then produces refined boxes and identity embeddings through an ID head, a regression head, and Norm-Aware Embedding (NAE). Detection is supervised by $\mathbf{x}_t$4, while identity learning uses Online Instance Matching (OIM) loss as $\mathbf{x}_t$5 (Wang et al., 6 Aug 2025).

Two additions define CLIPVehicle’s distinctive training strategy.

The first is dual-granularity semantic-region alignment. At the object granularity, CLIP-based text supervision aligns proposal regions with the prompts “A photo of a vehicle” and “Not a photo of a vehicle,” producing an object-level semantic loss. At the identity granularity, the method uses CoOp-style learnable prompt tokens and an attribute-aware prompt template,

$\mathbf{x}_t$6

to align region features with identity-discriminative text embeddings. The combined semantic objective is

$\mathbf{x}_t$7

The second is multi-level vehicle identification learning, which adds supervision at image, box, and feature levels. Image-level multi-ID classification uses the fact that one training frame may contain multiple identities; box-level single-ID classification learns directly from ground-truth boxes to reduce the effect of inaccurate predicted boxes; feature-level consistency distillation matches region features to features extracted by a separately pretrained crop-based vehicle ReID model. These terms are combined as

$\mathbf{x}_t$8

The full training objective is

$\mathbf{x}_t$9

Architecturally, the implementation uses ResNet50 as the main backbone, with backbone features from conv1 to conv4, separate conv5 blocks for the detection and ID heads, a frozen CLIP text encoder, $t$0 learnable prompt tokens, RoI pooled feature maps of size $t$1 and $t$2, batch size 5, image size $t$3, and SGD optimization. The framework is explicitly ReID-by-detection rather than a two-stage offline pipeline.

5. Empirical difficulty and benchmark results

CityFlowVS is difficult even for strong baselines. The paper compares crop-based vehicle ReID models evaluated on detected gallery boxes, person-search frameworks, and CLIPVehicle itself. The main reported results on CityFlowVS are as follows (Wang et al., 6 Aug 2025).

Method	mAP	Top-1
CLIP-ReID	9.9	52.6
MSINet	6.1	26.8
MBR	10.6	75.9
SeqNet	11.7	79.8
COAT	13.3	82.5
OIMNet++	12.8	83.0
CLIPVehicle	14.1	83.8

The strongest number reported in the paper is obtained by combining the CLIPVehicle training strategy with a stronger COAT backbone, yielding 15.2 mAP and 85.8 Top-1. The result pattern is more informative than the absolute ranking alone. Pure vehicle ReID methods degrade substantially in this end-to-end setting, especially in mAP, because they inherit detection errors rather than operating on ground-truth crops. By contrast, person-search frameworks transfer more effectively, which suggests that the CityFlowVS task is structurally closer to person search than to conventional vehicle ReID.

The paper’s ablations show that both major additions contribute. Starting from a SeqNet-style baseline at 11.7 mAP / 79.8 Top-1, object-granularity semantic alignment yields 12.2 / 81.4, ID-granularity semantic alignment yields 12.7 / 81.5, and using both yields 13.0 / 82.3. For multi-level identification learning, image-level supervision gives 11.9 / 80.6, box-level 12.4 / 82.2, feature-level 12.4 / 81.4, and using all three 12.8 / 82.5. The full model reaches 14.1 / 83.8. The paper also reports that the attribute-aware ID prompt outperforms a simpler prompt, improving from 12.1 mAP / 80.6 Top-1 to 12.7 mAP / 81.5 Top-1.

A central conclusion follows directly from these numbers: CityFlowVS is not saturated. Even the best reported mAP remains low, indicating that joint detection and fine-grained instance discrimination in urban traffic video remains an open problem.

6. Significance, scope, and open issues

CityFlowVS matters because it formalizes a task that earlier vehicle benchmarks largely avoided: instance-level vehicle retrieval from raw surveillance frames without pre-extracted gallery crops. Earlier CityFlow work established city-scale vehicle tracking and image-based ReID as benchmarkable problems in urban camera networks (Tang et al., 2019), while CityFlowVS redefines the retrieval problem around raw-frame galleries and joint detector-identification learning (Wang et al., 6 Aug 2025). In that sense, CityFlowVS is less a variant of crop-based ReID than a benchmark for an operational surveillance regime.

Its scope is also sharply defined. Queries remain cropped vehicle images rather than raw video segments or natural-language descriptions, so the task is not open-vocabulary search or full video-level retrieval. The benchmark provides train and test splits but does not report a separate validation split in the statistics table. It also does not report explicit manual color or type annotations for CityFlowVS itself, even though color and type cues are exploited through prompt design in CLIPVehicle. These are not defects so much as delimitations of the benchmark’s current formulation.

The presence of the synthetic companions SynVS-Day and SynVS-All further clarifies CityFlowVS’s role. Those datasets provide larger scale and controlled variation over day, rain, dawn, and night, whereas CityFlowVS remains the primary real-world benchmark. This suggests that domain robustness, especially under adverse conditions, is partly studied through synthetic complement rather than by condition-specific real-world splits. A plausible implication is that future work on CityFlowVS-like search systems will need to address three unresolved tensions simultaneously: robust detection under surveillance noise, identity discrimination among visually similar vehicles, and transfer across scene, weather, and illumination shifts.

In the benchmark literature, CityFlowVS can therefore be read as a transition point. It inherits the city-scale, multi-camera surveillance perspective of the CityFlow family, but reframes the unit of evaluation around end-to-end vehicle search. Its enduring significance lies less in the absolute performance numbers than in the task definition itself: a retrieved vehicle is only correct when the system both finds it and identifies it.