---
title: 'CityFlowVS: End-to-End Vehicle Search'
url: https://www.emergentmind.com/topics/cityflowvs
type: topic
---

# CityFlowVS: End-to-End Vehicle Search

CityFlowVS is a real-world benchmark for **end-to-end vision-based vehicle search**, defined as the joint problem of detecting vehicles in raw surveillance frames and retrieving instances that match a query vehicle image. It was introduced together with the CLIPVehicle framework to address a gap between conventional vehicle detection datasets, which localize vehicles, and crop-based vehicle re-identification datasets, which assume that gallery vehicles have already been perfectly detected and cropped. In CityFlowVS, the gallery consists of raw frames rather than pre-extracted vehicle patches, so localization and identity discrimination are evaluated simultaneously [2508.04120].

## 1. Position within the CityFlow benchmark lineage

CityFlowVS belongs to a broader research lineage shaped by the CityFlow family of urban traffic benchmarks. The earlier CityFlow benchmark was introduced for **multi-target multi-camera vehicle tracking** and **image-based vehicle re-identification**, with synchronized traffic video, cross-camera identities, and camera geometry intended to support the full MTMC pipeline rather than isolated image retrieval [1903.09254]. CityFlowVS preserves the emphasis on city-scale, multi-camera urban surveillance, but shifts the task formulation from MTMC and crop-based ReID toward direct search over raw frames.

This change is methodological rather than merely cosmetic. In crop-based vehicle ReID, detection is assumed solved, and the model only compares already localized vehicle patches. CityFlowVS instead evaluates a setting closer to operational surveillance, where a system must first detect vehicles in gallery frames and only then determine whether a detected box matches the query identity. This makes the benchmark adjacent to person search rather than standard vehicle ReID. A plausible implication is that CityFlowVS should be understood not as a replacement for MTMC or ReID benchmarks, but as a benchmark for a different stage of the surveillance stack: **instance-level retrieval under raw-frame uncertainty**.

## 2. Dataset source, construction, and composition

CityFlowVS is built from **CityFlowV2**, described in the benchmark paper as a city-scale multi-target multi-camera vehicle tracking dataset containing **3.58 hours (215.03 minutes)** of video from **46 cameras** across **16 intersections** in a mid-sized U.S. city. The CityFlowVS construction pipeline is explicit: all frames are extracted from each raw video, frames without vehicles are filtered out, one in every five frames is sampled, and the original vehicle bounding boxes and identity labels are inherited. The split is scenario-based, with **S01, S02, S03, and S04** used for training and **S05** used for testing. Within the test split, the query set is created by randomly selecting one image for each vehicle in each camera, cropping the vehicle with the ground-truth bounding box, and using that crop as a query image [2508.04120].

The resulting benchmark contains frame-level raw images, vehicle bounding boxes, and vehicle identity labels. The identity annotations are inherited from CityFlowV2’s multi-camera tracking labels, so identities may recur across multiple frames and multiple cameras. As in standard ReID protocols, training and test identities are disjoint.

| Split | Identities | Frames | Boxes |
|---|---:|---:|---:|
| Train | 329 | 6,561 | 17,752 |
| Test | 337 | 11,550 | 32,877 |
| Total | 666 | 18,111 | 50,629 |

In addition, the test protocol contains **1,759 queries**. These numbers make CityFlowVS a moderately sized but nontrivial benchmark for raw-frame search, especially because all performance must be achieved under joint localization and identification rather than over curated vehicle crops.

## 3. Task formulation and evaluation protocol

The training set is formalized as
\[
\mathcal{D} = \{(\mathbf{x}_t,\mathbf{y}_t) \}_{t=1}^{T},
\]
where \(\mathbf{x}_t\) is the \(t\)-th raw frame and \(\mathbf{y}_t\) is its annotation. Each annotation is
\[
\mathbf{y}_t=\{(b_t^j,c_t^j)\}_{j=1}^{N^{t}},
\]
where \(N^t\) is the number of vehicles in frame \(\mathbf{x}_t\), \(b_t^j\) is the ground-truth bounding box, and \(c_t^j\in \{1,2,\cdots, C\}\) is the identity label of the \(j\)-th vehicle. The learned model must therefore solve detection and identity representation learning in a single framework.

At test time, the query is a cropped vehicle image \(\mathbf{q}\), while the gallery is a set of raw frames
\[
\mathcal{G}=\{\mathbf{x}_k\}_{k=1}^{K}.
\]
A retrieved gallery detection is counted as correct only if it satisfies two conditions simultaneously: it must have **IoU \(>0.5\)** with a ground-truth box, and that matched ground-truth box must have the **same identity** as the query. This criterion is central to CityFlowVS, because it couples retrieval quality to localization quality. A system cannot achieve strong search performance by learning identity features alone if its detector is weak or poorly aligned [2508.04120].

The benchmark reports **mAP** and **CMC Top-1**. These are familiar ReID-style metrics, but under CityFlowVS they quantify end-to-end search rather than crop-based retrieval. This distinction is crucial: errors now arise from false negatives, false positives, box misalignment, and identity confusion simultaneously.

## 4. Reference framework: CLIPVehicle

The benchmark was introduced alongside **CLIPVehicle**, which the paper presents as a unified end-to-end framework for vehicle search. Its backbone extracts a shared feature map
\[
\mathbf{F}_t=f_{\mathrm{b}(\mathbf{x}_t|\theta),
\]
then an RPN generates proposals, RoI Align extracts proposal features, and a detection head with box predictor produces preliminary detections. A second RoI-based pathway then produces refined boxes and identity embeddings through an ID head, a regression head, and **Norm-Aware Embedding (NAE)**. Detection is supervised by \(\mathcal{L}_{\mathrm{det}}\), while identity learning uses **Online Instance Matching (OIM)** loss as \(\mathcal{L}_{\mathrm{reid}}\) [2508.04120].

Two additions define CLIPVehicle’s distinctive training strategy.

The first is **dual-granularity semantic-region alignment**. At the object granularity, CLIP-based text supervision aligns proposal regions with the prompts “A photo of a vehicle” and “Not a photo of a vehicle,” producing an object-level semantic loss. At the identity granularity, the method uses CoOp-style learnable prompt tokens and an attribute-aware prompt template,
\[
\texttt{`A photo of a }[X]_1[X]_2\cdots [X]_M \texttt{ vehicle with }[X]\texttt{ color and }[X]\texttt{ type'},
\]
to align region features with identity-discriminative text embeddings. The combined semantic objective is
\[
\mathcal{L}_\mathrm{sra} = \mathcal{L}_\mathrm{sra}^\mathrm{obj}+\mathcal{L}_\mathrm{sra}^\mathrm{id}.
\]

The second is **multi-level vehicle identification learning**, which adds supervision at image, box, and feature levels. Image-level multi-ID classification uses the fact that one training frame may contain multiple identities; box-level single-ID classification learns directly from ground-truth boxes to reduce the effect of inaccurate predicted boxes; feature-level consistency distillation matches region features to features extracted by a separately pretrained crop-based vehicle ReID model. These terms are combined as
\[
\mathcal{L}_\mathrm{mil} = \mathcal{L}_\mathrm{mil}^\mathrm{img} + \mathcal{L}_\mathrm{mil}^\mathrm{box} + \mathcal{L}_\mathrm{mil}^\mathrm{fea}.
\]

The full training objective is
\[
\mathcal{L}=\mathcal{L}_\mathrm{det}+\mathcal{L}_\mathrm{reid}+\mathcal{L}_\mathrm{sra}+\mathcal{L}_\mathrm{mil}.
\]

Architecturally, the implementation uses **ResNet50** as the main backbone, with backbone features from **conv1 to conv4**, separate **conv5** blocks for the detection and ID heads, a **frozen CLIP text encoder**, **\(M=4\)** learnable prompt tokens, RoI pooled feature maps of size \(h=w=14\) and \(d=1024\), batch size **5**, image size **\(900 \times 1500\)**, and **SGD** optimization. The framework is explicitly **ReID-by-detection** rather than a two-stage offline pipeline.

## 5. Empirical difficulty and benchmark results

CityFlowVS is difficult even for strong baselines. The paper compares crop-based vehicle ReID models evaluated on detected gallery boxes, person-search frameworks, and CLIPVehicle itself. The main reported results on CityFlowVS are as follows [2508.04120].

| Method | mAP | Top-1 |
|---|---:|---:|
| CLIP-ReID | 9.9 | 52.6 |
| MSINet | 6.1 | 26.8 |
| MBR | 10.6 | 75.9 |
| SeqNet | 11.7 | 79.8 |
| COAT | 13.3 | 82.5 |
| OIMNet++ | 12.8 | 83.0 |
| CLIPVehicle | 14.1 | 83.8 |

The strongest number reported in the paper is obtained by combining the CLIPVehicle training strategy with a stronger COAT backbone, yielding **15.2 mAP** and **85.8 Top-1**. The result pattern is more informative than the absolute ranking alone. Pure vehicle ReID methods degrade substantially in this end-to-end setting, especially in mAP, because they inherit detection errors rather than operating on ground-truth crops. By contrast, person-search frameworks transfer more effectively, which suggests that the CityFlowVS task is structurally closer to person search than to conventional vehicle ReID.

The paper’s ablations show that both major additions contribute. Starting from a SeqNet-style baseline at **11.7 mAP / 79.8 Top-1**, object-granularity semantic alignment yields **12.2 / 81.4**, ID-granularity semantic alignment yields **12.7 / 81.5**, and using both yields **13.0 / 82.3**. For multi-level identification learning, image-level supervision gives **11.9 / 80.6**, box-level **12.4 / 82.2**, feature-level **12.4 / 81.4**, and using all three **12.8 / 82.5**. The full model reaches **14.1 / 83.8**. The paper also reports that the attribute-aware ID prompt outperforms a simpler prompt, improving from **12.1 mAP / 80.6 Top-1** to **12.7 mAP / 81.5 Top-1**.

A central conclusion follows directly from these numbers: CityFlowVS is not saturated. Even the best reported mAP remains low, indicating that joint detection and fine-grained instance discrimination in urban traffic video remains an open problem.

## 6. Significance, scope, and open issues

CityFlowVS matters because it formalizes a task that earlier vehicle benchmarks largely avoided: **instance-level vehicle retrieval from raw surveillance frames without pre-extracted gallery crops**. Earlier CityFlow work established city-scale vehicle tracking and image-based ReID as benchmarkable problems in urban camera networks [1903.09254], while CityFlowVS redefines the retrieval problem around raw-frame galleries and joint detector-identification learning [2508.04120]. In that sense, CityFlowVS is less a variant of crop-based ReID than a benchmark for an operational surveillance regime.

Its scope is also sharply defined. Queries remain cropped vehicle images rather than raw video segments or natural-language descriptions, so the task is not open-vocabulary search or full video-level retrieval. The benchmark provides train and test splits but does **not** report a separate validation split in the statistics table. It also does **not** report explicit manual color or type annotations for CityFlowVS itself, even though color and type cues are exploited through prompt design in CLIPVehicle. These are not defects so much as delimitations of the benchmark’s current formulation.

The presence of the synthetic companions **SynVS-Day** and **SynVS-All** further clarifies CityFlowVS’s role. Those datasets provide larger scale and controlled variation over day, rain, dawn, and night, whereas CityFlowVS remains the primary **real-world** benchmark. This suggests that domain robustness, especially under adverse conditions, is partly studied through synthetic complement rather than by condition-specific real-world splits. A plausible implication is that future work on CityFlowVS-like search systems will need to address three unresolved tensions simultaneously: robust detection under surveillance noise, identity discrimination among visually similar vehicles, and transfer across scene, weather, and illumination shifts.

In the benchmark literature, CityFlowVS can therefore be read as a transition point. It inherits the city-scale, multi-camera surveillance perspective of the CityFlow family, but reframes the unit of evaluation around **end-to-end vehicle search**. Its enduring significance lies less in the absolute performance numbers than in the task definition itself: a retrieved vehicle is only correct when the system both **finds** it and **identifies** it.

Source: https://www.emergentmind.com/topics/cityflowvs