CityFlow: Benchmark & Traffic Simulator
- CityFlow is a dual-purpose platform featuring a city-scale vehicle tracking benchmark with synchronized HD videos, calibrated camera geometries, and tasks including detection, MTSC, MTMC tracking, and ReID.
- It provides rigorous evaluation using metrics like IDF1, MOTP, and mAP, addressing challenges such as occlusions, motion blur, and calibration imprecision in diverse urban scenarios.
- Additionally, its high-throughput microscopic traffic simulator supports MARL experiments for traffic signal control, offering efficient road network modeling and flexible vehicle dynamics.
CityFlow denotes two distinct but closely related research artifacts in intelligent transportation and computer vision: a city-scale benchmark for multi-target multi-camera (MTMC) vehicle tracking and vehicle re-identification (ReID), and a microscopic traffic simulator designed as a multi-agent reinforcement learning (MARL) environment for large-scale traffic signal control. The former was introduced as a benchmark of synchronized urban traffic videos with calibration and identity annotations (Tang et al., 2019); the latter was introduced as a high-throughput simulator for city-scale traffic scenarios with an RL-oriented Python interface (Zhang et al., 2019). Subsequent work expanded both lines into a broader ecosystem spanning language-conditioned retrieval, online MTMC tracking, cross-simulator RL platforms, and simulator extensions with embedded learned behavior models.
1. Dual usage and historical scope
In the literature, the name “CityFlow” refers to two primary systems.
| Sense of “CityFlow” | Core purpose | Canonical source |
|---|---|---|
| CityFlow benchmark | MTMC vehicle tracking, MTSC, detection, and ReID | (Tang et al., 2019) |
| CityFlow simulator | Microscopic traffic simulation and MARL for traffic signal control | (Zhang et al., 2019) |
The benchmark version of CityFlow was introduced to address the gap between earlier vehicle ReID datasets and realistic urban deployment. It provides synchronized HD traffic videos from a distributed camera network, camera geometry, calibration, and benchmark tasks covering detection, MTSC, MTMC, and image-based vehicle ReID (Tang et al., 2019). The simulator version was introduced from a different motivation: reinforcement learning for traffic signal control requires large numbers of simulated interactions, and existing open-source simulators were characterized as insufficiently scalable for large road networks and large traffic flow (Zhang et al., 2019).
This naming collision has practical consequences. In computer vision, “CityFlow” usually denotes the MTMC benchmark and its extensions; in transportation systems and MARL, it usually denotes the simulator backend. A recurring source of confusion is to treat the two as interchangeable. They are instead complementary: one is a calibrated video benchmark for vehicle understanding, and the other is a microscopic traffic engine for control and optimization.
2. CityFlow as a city-scale benchmark for vehicle understanding
The benchmark paper describes CityFlow as a city-scale traffic camera dataset for vehicle-based MTMC tracking. It consists of 3.25 hours of synchronized high-definition traffic video from 40 cameras distributed over 10 intersections, with the longest distance between two simultaneous cameras equal to 2.5 km (Tang et al., 2019). The dataset contains 229,680 annotated bounding boxes corresponding to 666 vehicle identities, and every annotated identity appears in at least two cameras. The benchmark is partitioned into five scenarios, with Scenarios 1, 3, and 4 for training and Scenarios 2 and 5 for testing (Tang et al., 2019).
CityFlow was designed to support four coupled tasks. For object detection, it provides frame-level vehicle boxes. For multi-target single-camera tracking (MTSC), it evaluates identity maintenance within each video stream. For MTMC tracking, it requires linking trajectories across cameras into global identities. For image-based vehicle ReID, it exposes a dedicated subset, CityFlow-ReID, containing 56,277 bounding boxes; the training set has 36,935 images from 333 identities, and the test set has 18,290 images from another 333 identities, with 1,052 query images (Tang et al., 2019).
A technically important feature is its geometric annotation. CityFlow provides camera homography matrices between the 2D image plane and the ground plane defined by GPS coordinates, under a flat-earth approximation. These homographies were estimated from 5 to 14 landmark points, and the reported average reprojection error is 11.52 pixels (Tang et al., 2019). This makes the benchmark unusually suitable for methods that combine appearance with spatio-temporal or geometric reasoning.
The benchmark also encodes several difficulties that shaped later methods. The videos include city streets, residential areas, and highways, multiple viewing angles, overlapping and non-overlapping camera pairs, fish-eye lenses in some cameras, radial distortion, and motion blur from vehicle speed (Tang et al., 2019). Moreover, license plates are masked and cannot be used in the standard benchmark setting (Hsu et al., 2020). This forces algorithms toward robust visual, temporal, and topological cues rather than direct plate recognition.
Evaluation follows standard tracking and retrieval practice. For MTSC and MTMC, CityFlow uses MOTA, MOTP, IDP, IDR, and IDF1; for ReID it uses mAP and rank- accuracy, with the evaluation server using rank-100 mAP (Tang et al., 2019). For MTMC, the identity-preservation metric is
The original benchmark study reported that MTMC remained difficult even when strong MTSC and ReID components were combined: the best reported MTMC baseline in that paper, FVS + TC + BA ReID, achieved 46.3\% IDF1 (Tang et al., 2019).
3. Methodological development on the tracking benchmark
Later work on CityFlow concentrated on the standard MTMC decomposition into single-camera tracking (SCT), cross-camera association, and vehicle ReID, but increasingly emphasized that CityFlow is a traffic network, not merely a collection of disjoint cameras. A representative example is “Traffic-Aware Multi-Camera Tracking of Vehicles Based on ReID and Camera Link Model” (Hsu et al., 2020), which proposed a four-stage pipeline: traffic-aware single camera tracking (TSCT), a trajectory-based camera link model (CLM), cross-camera vehicle ReID with temporal attention, and hierarchical clustering. On the CityFlow MTMCT benchmark it reported IDF1 = 74.93\%, with ablations showing that the camera link model contributed the largest gain and that TSCT substantially improved fragmented within-camera trajectories caused by queueing and long occlusions (Hsu et al., 2020).
The importance of matching scope was made explicit by “Adaptive Affinity for Associations in Multi-Target Multi-Camera Tracking” (Hou et al., 2021). That work argued that a global ReID distance is mismatched to the local temporal windows used in MTMCT association. On CityFlow, its adaptive affinity raised performance from 56.6 IDF1 with raw ReID distance to 63.0 IDF1, while a globally trained Siamese metric without tracking-scope sampling reached only 57.1 IDF1 (Hou et al., 2021). The central claim was that CityFlow associations are local in time and camera topology even when the overall dataset is city-scale.
More recent work pursued stronger spatio-temporal priors with less manual annotation. “City-Scale Multi-Camera Vehicle Tracking System with Improved Self-Supervised Camera Link Model” (Lin et al., 2024) targeted CityFlow V2 in a 6 non-overlapping camera setting and replaced manually specified spatial-temporal constraints with a self-supervised camera-link estimation process based on feature similarity, matched-pair counts, and time variance. It reported IDF1 = 61.07\%, explicitly claiming state of the art among automatic camera-link-based methods in that benchmark configuration (Lin et al., 2024).
Online MTMC tracking also became a major theme. “Spatial-Temporal Multi-Cuts for Online Multiple-Camera Vehicle Tracking” (Herzog et al., 2024) replaced the common two-stage temporal-then-spatial or spatial-then-temporal pipeline with a single multicut graph over current detections and existing track summaries. On the CityFlow S02 validation scene, it reported IDF1 = 79.58, IDP = 81.10, and IDR = 78.11, and described this as the strongest online result in its comparison (Herzog et al., 2024). That paper also highlighted a benchmark caveat: in practice, CityFlow’s provided homographies are noisy and the cameras are not synchronous, so appearance becomes more reliable than geometry in some settings (Herzog et al., 2024). This suggests that CityFlow scores are strongly protocol-dependent and should not be compared across papers without attention to overlap, synchronization, and evaluation split.
CityFlow also became a major testbed for ReID-specific advances. For example, “Self-supervised Geometric Features Discovery via Interpretable Attention for Vehicle Re-Identification and Beyond” reported 37.14 imAP, 60.08 Top-1, and 67.21 Top-5 on CityFlow-ReID without extra supervision beyond official ID labels (Li et al., 2020). “A Strong Baseline for Vehicle Re-Identification” targeted CityFlowV2-ReID and reported 61.34\% mAP on the private CityFlow test set, with the paper attributing gains to synthetic-to-real domain-gap reduction, multi-head attention, and adaptive loss weighting (Huynh et al., 2021).
4. Language, retrieval, and other multimodal extensions
A major extension of the benchmark line is CityFlow-NL, introduced as “CityFlow-NL: Tracking and Retrieval of Vehicles at City Scale by Natural Language Descriptions” (Feng et al., 2021). CityFlow-NL preserves the CityFlow visual and tracking backbone but adds free-form natural language descriptions of vehicle targets. The benchmark contains 666 target vehicles, 3,028 single-view tracks, 40 calibrated cameras, and 5,289 unique natural language descriptions, and was described as the first multi-target multi-camera tracking with natural language descriptions dataset and the largest NL-annotated tracking benchmark in terms of number of NL descriptions among the tracking-oriented datasets considered (Feng et al., 2021).
CityFlow-NL was positioned at the intersection of MTMC tracking, language-based retrieval, and temporal localization, with two foundational tasks emphasized in the paper: Vehicle Retrieval by NL and Vehicle Tracking by NL (Feng et al., 2021). The practical use case is explicit: a target vehicle may be specified by a witness-style description such as a color, type, and maneuver sequence rather than by an image crop.
Early CityFlow-NL methods often reinterpreted retrieval as localization. “SBNet: Segmentation-based Network for Natural Language-based Vehicle Search” (Lee et al., 2021) treated language-based vehicle retrieval as a segmentation problem over scene images rather than only a global embedding-matching problem. Using CityFlow-NL, it reported MRR = 0.1195, compared with 0.0269 for the benchmark Siamese baseline, and achieved 10th place in AI City Challenge 2021 Track 5 (Lee et al., 2021). Its design used weak segmentation supervision derived from bounding boxes, a substitution module for cross-modal alignment, and a future prediction module intended to capture motion expressions such as turning or going straight.
CityFlow also supported multimodal work beyond retrieval. “Future Urban Scenes Generation Through Vehicles Synthesis” used CityFlow as a real-world urban benchmark for vehicle-centric future scene generation, treating the dataset as a source of traffic-camera video, tracks, homographies, and realistic surveillance viewpoints (Simoni et al., 2020). That work reported stronger long-horizon visual realism than end-to-end frame-prediction baselines on vehicle crops in CityFlow, illustrating that the benchmark can function as more than a tracking or ReID dataset.
5. CityFlow as a microscopic traffic simulator and MARL environment
The simulator version of CityFlow was introduced as an open-source-oriented microscopic traffic simulator for large-scale city traffic scenarios, especially traffic signal control (Zhang et al., 2019). Its core premise is that RL for signal control requires vast numbers of samples, while real-world collection is impractical; the paper gave the concrete example that treating each minute as a data sample yields only 1,440 samples per day from a real city (Zhang et al., 2019). CityFlow was therefore designed for high-throughput closed-loop interaction.
The simulator models vehicles microscopically with explicit per-vehicle state and behavior. Its road-network representation includes Road, Lane, Intersection, Roadlink, Lanelink, and Cross objects; each lane holds a Linked List of vehicles, is partitioned into segments, and intersections precompute cross points between lanelinks for conflict handling (Zhang et al., 2019). The intersection conflict algorithm is reported with time complexity
The car-following model is described as a modification of the model proposed by Stephen Krauß, and CityFlow uses a ballistic position update rule rather than an Euler update (Zhang et al., 2019). Lane changing uses shadow vehicles to maintain consistency between source and destination lanes during the maneuver.
A central contribution was performance. The paper stated that CityFlow is more than twenty times faster than SUMO, and on a road network with tens of thousands of vehicles and 8 threads it reported 72 simulation steps per second, corresponding to about 25× speedup over SUMO in that setting (Zhang et al., 2019). The simulator core is implemented in C++, while the RL interface is exposed through pybind11, with stepwise control and query functions such as get_lane_vehicle_count, get_lane_waiting_vehicle_count, get_lane_vehicles, get_vehicle_speed, and set_tl_phase (Zhang et al., 2019).
CityFlow does not prescribe a single MDP. Rather, it exposes sufficient lane-level and vehicle-level state for custom RL formulations. This flexibility made it the backend for a large body of traffic signal control work. In MetaVIM, CityFlow was used for decentralized traffic signal control on Hangzhou, Jinan, New York, and Shenzhen, with local observations defined as incoming-lane vehicle counts plus the current phase, and reward defined as negative queue length (Zhu et al., 2021). In SocialLight, CityFlow served as the benchmark for a distributed MARL method with locally centralized counterfactual critics; on the CityFlow New York benchmark, SocialLight reported 760.94 and 1114.47 average travel time on two traffic sets, corresponding to 22.23\% and 24.53\% gains over the best prior method in the paper’s comparison (Goel et al., 2023). More recently, GPLight+ evaluated a genetic-programming controller on CityFlow road networks in Jinan, Hangzhou, and Manhattan, claiming better average travel time than the earlier GPLight representation on 5 of 6 datasets (Liao et al., 22 Aug 2025).
6. Ecosystem, interoperability, and continuing debates
CityFlow’s importance now lies not only in the original benchmark and simulator, but in the surrounding tooling. LibSignal treats CityFlow and SUMO as its two core backends, defines simulator-agnostic road network and traffic flow “atomic files,” and exposes a common World abstraction with standardized metrics such as average travel time, average queue length, approximated delay, and throughput (Mei et al., 2022). Its experiments showed that, after calibration, CityFlow and SUMO can produce more comparable traffic signal control results on several datasets, while also showing that some RL methods remain simulator-sensitive (Mei et al., 2022).
PyTSC provides a different unification layer. It supports SUMO and CityFlow through simulator-specific ConfigParser, Retriever, Simulator, and TrafficSignal classes beneath a common TrafficSignalNetwork environment (Bokade et al., 2024). In the paper’s benchmark configuration, each signal observes traffic within a 50-meter range, chooses one green phase, and, if it changes phase, the environment enforces the required yellow transition; the action-selection interval and yellow phases are fixed to 5 simulation seconds (Bokade et al., 2024). PyTSC also highlights a substantive modeling caveat: in its experiments, CityFlow is described as computationally efficient but as having simpler vehicle dynamics and no dynamic routing compared with SUMO (Bokade et al., 2024). A plausible implication is that algorithm rankings obtained on CityFlow may transfer imperfectly to richer simulators or real deployments.
CityFlow itself has also been extended. CityFlowER is explicitly presented as an extension built on top of CityFlow, embedding pre-trained ML behavior models directly inside the simulator through LibTorch rather than relying on external API loops (Da et al., 2024). It formalizes per-vehicle action as
where is speed output and is lane-change choice (Da et al., 2024). This retains CityFlow’s efficient road-network structure while allowing mixtures of rule-based and learned vehicle behaviors.
A separate line of work now positions new simulators directly against CityFlow. “A GPU-accelerated Large-scale Simulator for Transportation System Optimization Benchmarking” treats CityFlow as the best baseline among prior microscopic simulators in its largest performance experiment, but reports 42.81 s versus 3806.7 s for 3600 simulation steps on the largest dataset, corresponding to roughly 84.09 Hz and an 88.92 times acceleration factor (Zhang et al., 2024). That paper also argues that CityFlow’s native optimization support is concentrated on traffic signal phase control and vehicle route setting, whereas broader scenarios such as dynamic lane assignment, tidal lane control, congestion pricing, and road planning require a wider API surface (Zhang et al., 2024).
Survey literature has largely stabilized CityFlow’s position. One recent ITS survey describes CityFlow as “a high-performance traffic simulator tailored for large-scale signal control environments” and as particularly suited to graph-based MARL research on coordinated intersection management (Donatus et al., 27 Aug 2025). The most durable conclusion is therefore twofold: CityFlow is simultaneously a foundational benchmark for city-scale vehicle understanding and a foundational simulator for city-scale signal-control research, but results within either line remain highly sensitive to protocol, topology, and task definition.
