---
title: 'CityFlow: Benchmark & Traffic Simulator'
url: https://www.emergentmind.com/topics/cityflow
type: topic
---

# CityFlow: Benchmark & Traffic Simulator

CityFlow denotes two distinct but closely related research artifacts in intelligent transportation and computer vision: a **city-scale benchmark** for **multi-target multi-camera (MTMC) vehicle tracking** and **vehicle re-identification (ReID)**, and a **microscopic traffic simulator** designed as a **multi-agent reinforcement learning (MARL) environment** for large-scale traffic signal control. The former was introduced as a benchmark of synchronized urban traffic videos with calibration and identity annotations [1903.09254]; the latter was introduced as a high-throughput simulator for city-scale traffic scenarios with an RL-oriented Python interface [1905.05217]. Subsequent work expanded both lines into a broader ecosystem spanning language-conditioned retrieval, online MTMC tracking, cross-simulator RL platforms, and simulator extensions with embedded learned behavior models.

## 1. Dual usage and historical scope

In the literature, the name “CityFlow” refers to two primary systems.

| Sense of “CityFlow” | Core purpose | Canonical source |
|---|---|---|
| CityFlow benchmark | MTMC vehicle tracking, MTSC, detection, and ReID | [1903.09254] |
| CityFlow simulator | Microscopic traffic simulation and MARL for traffic signal control | [1905.05217] |

The benchmark version of CityFlow was introduced to address the gap between earlier vehicle ReID datasets and realistic urban deployment. It provides synchronized HD traffic videos from a distributed camera network, camera geometry, calibration, and benchmark tasks covering detection, MTSC, MTMC, and image-based vehicle ReID [1903.09254]. The simulator version was introduced from a different motivation: reinforcement learning for traffic signal control requires large numbers of simulated interactions, and existing open-source simulators were characterized as insufficiently scalable for large road networks and large traffic flow [1905.05217].

This naming collision has practical consequences. In computer vision, “CityFlow” usually denotes the MTMC benchmark and its extensions; in transportation systems and MARL, it usually denotes the simulator backend. A recurring source of confusion is to treat the two as interchangeable. They are instead complementary: one is a calibrated video benchmark for vehicle understanding, and the other is a microscopic traffic engine for control and optimization.

## 2. CityFlow as a city-scale benchmark for vehicle understanding

The benchmark paper describes CityFlow as a **city-scale traffic camera dataset** for **vehicle-based MTMC tracking**. It consists of **3.25 hours** of synchronized high-definition traffic video from **40 cameras** distributed over **10 intersections**, with the **longest distance between two simultaneous cameras** equal to **2.5 km** [1903.09254]. The dataset contains **229,680 annotated bounding boxes** corresponding to **666 vehicle identities**, and every annotated identity appears in **at least two cameras**. The benchmark is partitioned into **five scenarios**, with **Scenarios 1, 3, and 4 for training** and **Scenarios 2 and 5 for testing** [1903.09254].

CityFlow was designed to support four coupled tasks. For **object detection**, it provides frame-level vehicle boxes. For **multi-target single-camera tracking (MTSC)**, it evaluates identity maintenance within each video stream. For **MTMC tracking**, it requires linking trajectories across cameras into global identities. For **image-based vehicle ReID**, it exposes a dedicated subset, **CityFlow-ReID**, containing **56,277 bounding boxes**; the **training set** has **36,935 images from 333 identities**, and the **test set** has **18,290 images from another 333 identities**, with **1,052 query images** [1903.09254].

A technically important feature is its geometric annotation. CityFlow provides **camera homography matrices between the 2D image plane and the ground plane defined by GPS coordinates**, under a **flat-earth approximation**. These homographies were estimated from **5 to 14 landmark points**, and the reported **average reprojection error** is **11.52 pixels** [1903.09254]. This makes the benchmark unusually suitable for methods that combine appearance with spatio-temporal or geometric reasoning.

The benchmark also encodes several difficulties that shaped later methods. The videos include **city streets, residential areas, and highways**, multiple viewing angles, overlapping and non-overlapping camera pairs, **fish-eye lenses** in some cameras, **radial distortion**, and **motion blur** from vehicle speed [1903.09254]. Moreover, **license plates are masked and cannot be used** in the standard benchmark setting [2008.09785]. This forces algorithms toward robust visual, temporal, and topological cues rather than direct plate recognition.

Evaluation follows standard tracking and retrieval practice. For MTSC and MTMC, CityFlow uses **MOTA**, **MOTP**, **IDP**, **IDR**, and **IDF1**; for ReID it uses **mAP** and **rank-\(K\)** accuracy, with the evaluation server using **rank-100 mAP** [1903.09254]. For MTMC, the identity-preservation metric is
\[
\mathrm{IDF1}=\frac{2\mathrm{IDTP}}{2\mathrm{IDTP}+\mathrm{IDFP}+\mathrm{IDFN}}.
\]
The original benchmark study reported that MTMC remained difficult even when strong MTSC and ReID components were combined: the best reported MTMC baseline in that paper, **FVS + TC + BA ReID**, achieved **46.3\% IDF1** [1903.09254].

## 3. Methodological development on the tracking benchmark

Later work on CityFlow concentrated on the standard MTMC decomposition into **single-camera tracking (SCT)**, **cross-camera association**, and **vehicle ReID**, but increasingly emphasized that CityFlow is a **traffic network**, not merely a collection of disjoint cameras. A representative example is **“Traffic-Aware Multi-Camera Tracking of Vehicles Based on ReID and Camera Link Model”** [2008.09785], which proposed a four-stage pipeline: **traffic-aware single camera tracking (TSCT)**, a **trajectory-based camera link model (CLM)**, **cross-camera vehicle ReID with temporal attention**, and **hierarchical clustering**. On the CityFlow MTMCT benchmark it reported **IDF1 = 74.93\%**, with ablations showing that the **camera link model** contributed the largest gain and that **TSCT** substantially improved fragmented within-camera trajectories caused by queueing and long occlusions [2008.09785].

The importance of matching scope was made explicit by **“Adaptive Affinity for Associations in Multi-Target Multi-Camera Tracking”** [2112.07664]. That work argued that a global ReID distance is mismatched to the local temporal windows used in MTMCT association. On CityFlow, its **adaptive affinity** raised performance from **56.6 IDF1** with raw ReID distance to **63.0 IDF1**, while a globally trained Siamese metric without tracking-scope sampling reached only **57.1 IDF1** [2112.07664]. The central claim was that CityFlow associations are local in time and camera topology even when the overall dataset is city-scale.

More recent work pursued stronger spatio-temporal priors with less manual annotation. **“City-Scale Multi-Camera Vehicle Tracking System with Improved Self-Supervised Camera Link Model”** [2405.11345] targeted **CityFlow V2** in a **6 non-overlapping camera** setting and replaced manually specified spatial-temporal constraints with a self-supervised camera-link estimation process based on **feature similarity**, **matched-pair counts**, and **time variance**. It reported **IDF1 = 61.07\%**, explicitly claiming state of the art **among automatic camera-link-based methods** in that benchmark configuration [2405.11345].

Online MTMC tracking also became a major theme. **“Spatial-Temporal Multi-Cuts for Online Multiple-Camera Vehicle Tracking”** [2410.02638] replaced the common two-stage temporal-then-spatial or spatial-then-temporal pipeline with a single multicut graph over current detections and existing track summaries. On the **CityFlow S02 validation scene**, it reported **IDF1 = 79.58**, **IDP = 81.10**, and **IDR = 78.11**, and described this as the strongest online result in its comparison [2410.02638]. That paper also highlighted a benchmark caveat: in practice, CityFlow’s provided homographies are noisy and the cameras are **not synchronous**, so appearance becomes more reliable than geometry in some settings [2410.02638]. This suggests that CityFlow scores are strongly protocol-dependent and should not be compared across papers without attention to overlap, synchronization, and evaluation split.

CityFlow also became a major testbed for ReID-specific advances. For example, **“Self-supervised Geometric Features Discovery via Interpretable Attention for Vehicle Re-Identification and Beyond”** reported **37.14 imAP**, **60.08 Top-1**, and **67.21 Top-5** on CityFlow-ReID without extra supervision beyond official ID labels [2010.09221]. **“A Strong Baseline for Vehicle Re-Identification”** targeted **CityFlowV2-ReID** and reported **61.34\% mAP on the private CityFlow test set**, with the paper attributing gains to synthetic-to-real domain-gap reduction, multi-head attention, and adaptive loss weighting [2104.10850].

## 4. Language, retrieval, and other multimodal extensions

A major extension of the benchmark line is **CityFlow-NL**, introduced as **“CityFlow-NL: Tracking and Retrieval of Vehicles at City Scale by Natural Language Descriptions”** [2101.04741]. CityFlow-NL preserves the CityFlow visual and tracking backbone but adds **free-form natural language descriptions of vehicle targets**. The benchmark contains **666 target vehicles**, **3,028 single-view tracks**, **40 calibrated cameras**, and **5,289 unique natural language descriptions**, and was described as the first **multi-target multi-camera tracking with natural language descriptions** dataset and the largest NL-annotated tracking benchmark **in terms of number of NL descriptions** among the tracking-oriented datasets considered [2101.04741].

CityFlow-NL was positioned at the intersection of **MTMC tracking**, **language-based retrieval**, and **temporal localization**, with two foundational tasks emphasized in the paper: **Vehicle Retrieval by NL** and **Vehicle Tracking by NL** [2101.04741]. The practical use case is explicit: a target vehicle may be specified by a witness-style description such as a color, type, and maneuver sequence rather than by an image crop.

Early CityFlow-NL methods often reinterpreted retrieval as localization. **“SBNet: Segmentation-based Network for Natural Language-based Vehicle Search”** [2104.11589] treated language-based vehicle retrieval as a segmentation problem over scene images rather than only a global embedding-matching problem. Using CityFlow-NL, it reported **MRR = 0.1195**, compared with **0.0269** for the benchmark Siamese baseline, and achieved **10th place** in **AI City Challenge 2021 Track 5** [2104.11589]. Its design used weak segmentation supervision derived from bounding boxes, a substitution module for cross-modal alignment, and a future prediction module intended to capture motion expressions such as turning or going straight.

CityFlow also supported multimodal work beyond retrieval. **“Future Urban Scenes Generation Through Vehicles Synthesis”** used CityFlow as a real-world urban benchmark for vehicle-centric future scene generation, treating the dataset as a source of traffic-camera video, tracks, homographies, and realistic surveillance viewpoints [2007.00323]. That work reported stronger long-horizon visual realism than end-to-end frame-prediction baselines on vehicle crops in CityFlow, illustrating that the benchmark can function as more than a tracking or ReID dataset.

## 5. CityFlow as a microscopic traffic simulator and MARL environment

The simulator version of CityFlow was introduced as an **open-source-oriented microscopic traffic simulator** for **large-scale city traffic scenarios**, especially **traffic signal control** [1905.05217]. Its core premise is that RL for signal control requires vast numbers of samples, while real-world collection is impractical; the paper gave the concrete example that treating **each minute as a data sample** yields only **1,440 samples per day** from a real city [1905.05217]. CityFlow was therefore designed for high-throughput closed-loop interaction.

The simulator models vehicles microscopically with explicit per-vehicle state and behavior. Its road-network representation includes **Road**, **Lane**, **Intersection**, **Roadlink**, **Lanelink**, and **Cross** objects; each lane holds a **Linked List of vehicles**, is partitioned into **segments**, and intersections precompute **cross points between lanelinks** for conflict handling [1905.05217]. The intersection conflict algorithm is reported with time complexity
\[
\mathcal{O}(N_{\text{crosspoints}}).
\]
The car-following model is described as a modification of the model proposed by **Stephen Krauß**, and CityFlow uses a **ballistic position update rule** rather than an Euler update [1905.05217]. Lane changing uses **shadow vehicles** to maintain consistency between source and destination lanes during the maneuver.

A central contribution was performance. The paper stated that CityFlow is **more than twenty times faster than SUMO**, and on a **\(30 \times 30\)** road network with **tens of thousands of vehicles** and **8 threads** it reported **72 simulation steps per second**, corresponding to about **25× speedup** over SUMO in that setting [1905.05217]. The simulator core is implemented in **C++**, while the RL interface is exposed through **pybind11**, with stepwise control and query functions such as `get_lane_vehicle_count`, `get_lane_waiting_vehicle_count`, `get_lane_vehicles`, `get_vehicle_speed`, and `set_tl_phase` [1905.05217].

CityFlow does not prescribe a single MDP. Rather, it exposes sufficient lane-level and vehicle-level state for custom RL formulations. This flexibility made it the backend for a large body of traffic signal control work. In **MetaVIM**, CityFlow was used for decentralized traffic signal control on **Hangzhou**, **Jinan**, **New York**, and **Shenzhen**, with local observations defined as incoming-lane vehicle counts plus the current phase, and reward defined as negative queue length [2101.00746]. In **SocialLight**, CityFlow served as the benchmark for a distributed MARL method with locally centralized counterfactual critics; on the CityFlow **New York** benchmark, SocialLight reported **760.94** and **1114.47** average travel time on two traffic sets, corresponding to **22.23\%** and **24.53\%** gains over the best prior method in the paper’s comparison [2305.16145]. More recently, **GPLight+** evaluated a genetic-programming controller on CityFlow road networks in **Jinan**, **Hangzhou**, and **Manhattan**, claiming better **average travel time** than the earlier GPLight representation on **5 of 6 datasets** [2508.16090].

## 6. Ecosystem, interoperability, and continuing debates

CityFlow’s importance now lies not only in the original benchmark and simulator, but in the surrounding tooling. **LibSignal** treats CityFlow and SUMO as its two core backends, defines simulator-agnostic **road network** and **traffic flow** “atomic files,” and exposes a common **World** abstraction with standardized metrics such as **average travel time**, **average queue length**, **approximated delay**, and **throughput** [2211.10649]. Its experiments showed that, after calibration, CityFlow and SUMO can produce more comparable traffic signal control results on several datasets, while also showing that some RL methods remain simulator-sensitive [2211.10649].

**PyTSC** provides a different unification layer. It supports **SUMO and CityFlow** through simulator-specific `ConfigParser`, `Retriever`, `Simulator`, and `TrafficSignal` classes beneath a common `TrafficSignalNetwork` environment [2410.18202]. In the paper’s benchmark configuration, each signal observes traffic within a **50-meter range**, chooses one green phase, and, if it changes phase, the environment enforces the required yellow transition; the **action-selection interval and yellow phases are fixed to 5 simulation seconds** [2410.18202]. PyTSC also highlights a substantive modeling caveat: in its experiments, CityFlow is described as computationally efficient but as having **simpler vehicle dynamics and no dynamic routing** compared with SUMO [2410.18202]. A plausible implication is that algorithm rankings obtained on CityFlow may transfer imperfectly to richer simulators or real deployments.

CityFlow itself has also been extended. **CityFlowER** is explicitly presented as an extension built on top of CityFlow, embedding pre-trained ML behavior models directly inside the simulator through **LibTorch** rather than relying on external API loops [2402.06127]. It formalizes per-vehicle action as
\[
a_t := (v_t, l_t),
\]
where \(v_t \in \mathbb{R}\) is speed output and \(l_t \in \{\text{left}, \text{stay}, \text{right}\}\) is lane-change choice [2402.06127]. This retains CityFlow’s efficient road-network structure while allowing mixtures of rule-based and learned vehicle behaviors.

A separate line of work now positions new simulators directly against CityFlow. **“A GPU-accelerated Large-scale Simulator for Transportation System Optimization Benchmarking”** treats CityFlow as the best baseline among prior microscopic simulators in its largest performance experiment, but reports **42.81 s** versus **3806.7 s** for **3600 simulation steps** on the largest dataset, corresponding to roughly **84.09 Hz** and an **88.92 times** acceleration factor [2406.10661]. That paper also argues that CityFlow’s native optimization support is concentrated on **traffic signal phase control** and **vehicle route setting**, whereas broader scenarios such as dynamic lane assignment, tidal lane control, congestion pricing, and road planning require a wider API surface [2406.10661].

Survey literature has largely stabilized CityFlow’s position. One recent ITS survey describes CityFlow as **“a high-performance traffic simulator tailored for large-scale signal control environments”** and as particularly suited to **graph-based MARL research on coordinated intersection management** [2508.20315]. The most durable conclusion is therefore twofold: CityFlow is simultaneously a foundational benchmark for city-scale vehicle understanding and a foundational simulator for city-scale signal-control research, but results within either line remain highly sensitive to protocol, topology, and task definition.

Source: https://www.emergentmind.com/topics/cityflow