CityTrack: Urban MCMT Framework

• CityTrack is a city-scale multi-camera multi-target tracking framework designed for urban vehicle tracking, ensuring robust identity assignment across non-overlapping cameras.
• The system features a four-stage pipeline—detection, ReID feature extraction, single-camera tracking with Location-Aware SCMT, and inter-camera matching with Box-Grained Matching—to tackle urban challenges such as occlusions and similar appearances.
• Achieving an IDF1 score of 85.45 on the CityFlowV2 benchmark, CityTrack demonstrates enhanced tracking accuracy by integrating scene-aware trajectory repair and fine-grained appearance analysis.

CityTrack is a term used in recent tracking literature in two distinct but related senses. The primary usage denotes a city-scale multi-camera multi-target tracking framework for vehicles in urban traffic, introduced for the CityFlowV2 benchmark and organized around four stages—vehicle detection, vehicle ReID feature extraction, single-camera multi-target tracking, and inter-camera association—with two named contributions: a Location-Aware SCMT tracker and a Box-Grained Matching ICA module. In the 2023 paper, the method is reported as ranking 1st in the 2022 AI City Challenge, with the main results table giving 85.45 IDF1, while the abstract reports 84.91\% (Lu et al., 2023). A later paper reused the name CityTrack for a large real-world multi-view crowd-tracking dataset built from CityStreet, so the term now carries a substantive naming ambiguity across vehicle MCMT and ground-plane pedestrian tracking (Zhang et al., 21 Apr 2026).

1. Scope, task formulation, and research context

In its original vehicle-tracking sense, CityTrack addresses Multi-Camera Multi-Target Tracking (MCMT) in urban traffic. The objective is to assign consistent identities to vehicles as they move through a network of non-overlapping cameras, so that a vehicle observed in one camera can be recognized again in downstream cameras and its full cross-camera trajectory can be recovered. The problem setting is explicitly city-scale, with multiple cameras covering roads and intersections under severe occlusion, illumination change, viewpoint change, stop-and-go motion, small distant objects, and high inter-vehicle visual similarity (Lu et al., 2023).

This formulation places CityTrack within a broader CityFlow-centered line of work. "TrackNet" implements a modular pipeline of Faster R-CNN, DeepSORT, and triplet-metric cross-camera association, and reports an average IDF1 = 0.4733 under its leave-one-sequence-out protocol on CityFlowV2 (Serrano et al., 2022). "Traffic-Aware Multi-Camera Tracking of Vehicles Based on ReID and Camera Link Model" instead emphasizes traffic-aware single-camera repair, a trajectory-based camera link model, temporal attention, and hierarchical clustering, and reports IDF1 = 74.93\% on CityFlow (Hsu et al., 2020). This suggests that CityTrack belongs to a research strand in which detector quality, within-camera trajectory integrity, and structured inter-camera priors are all limiting factors, rather than cross-camera ReID alone.

A common misconception is to treat CityTrack as a dataset name only. That is inaccurate for the 2023 vehicle-tracking paper, where CityTrack is a systematic MCMT framework rather than a benchmark. The later crowd-tracking dataset usage is separate and arose afterward.

2. Pipeline architecture and representational design

The 2023 CityTrack framework is organized into four modules: Detection, ReID Feature Extraction, Single-Camera Multi-Target Tracking (SCMT), and Inter-Camera Association (ICA). Given multi-camera videos, the detector predicts vehicle boxes and categories per frame, the ReID module extracts a per-box appearance embedding, SCMT links detections over time within each camera, and ICA merges per-camera trajectories across connected cameras into global identities (Lu et al., 2023).

The paper formalizes a trajectory as

$\tau_i = [B_1, B_2, \dots, B_{l_i}],$

where $l_i$ is the trajectory length and each bounding box is

$B = [x, y, w, h, f],$

with box coordinates $(x,y,w,h)$ and ReID feature $f$. For two successive cameras, the ICA stage receives exiting trajectories

$T_{out} = [\tau_1, \tau_2, \dots, \tau_n]$

and entering trajectories

$$T_{in} = [\bar{\tau}_1, \bar{\tau}_2, \dots, \bar{\tau}_m].$$

The detector is Cascade R-CNN with a Swin-Transformer-Base backbone. The ReID extractor is an ensemble of five models—ResNet-50, ResNeXt101, Res2Net, ConvNext, and HRNet—trained with cross-entropy loss and triplet loss, with their features concatenated into the final appearance representation. The ReID training set combines real CityFlowV2 data and synthetic VehicleX-based data, totaling 2028 vehicles (666 real, 1362 synthetic) and 229,345 images (27,195 real, 192,150 synthetic) (Lu et al., 2023).

The SCMT baseline combines ideas from ByteTrack and StrongSORT. Detections with confidence $> 0.6$ are matched first; detections with confidence in $[0.1, 0.6]$ are matched afterward; lost tracklets are kept for 30 frames; and trajectory re-linking uses a maximum cosine-distance threshold of 0.4. The paper also states that if the cosine distance between a detection feature and a tracklet feature is less than 0.45, the matching is rejected. Because the paper records that condition exactly in this form, it is best treated as the implementation rule as documented, even though the wording is unusual (Lu et al., 2023).

3. Location-Aware SCMT

CityTrack’s first major contribution is Location-Aware SCMT, an augmented single-camera tracker designed to improve local trajectory quality before cross-camera association. Its motivation is that fragmented per-camera trajectories make ICA combinatorially harder and materially increase identity ambiguity at city scale (Lu et al., 2023).

The method introduces three add-on strategies. Stationary Sensitive Association (SSA) targets vehicles that stop and restart near intersections. When a tracklet and nearby tracklets all remain stationary, the method keeps the detection with the highest score as the vehicle’s location, and it uses a smoothed Mahalanobis distance to avoid abrupt motion-distance changes during nonlinear motion phases. Trajectory Re-Link (TRL) is an offline repair step that filters trajectories ending or starting in the middle of the scene and greedily merges likely fragments using cosine distance of ReID features with threshold 0.4. Bidirectional Tracking (BT) runs the tracker once forward and once backward, then merges tracklets from both directions to recover early or late low-confidence segments of a trajectory.

The three mechanisms are explicitly location-dependent in intent. SSA addresses stationary areas near traffic lights, TRL targets middle-scene occlusion breaks, and BT targets incomplete trajectories at scene entrances and exits. Their combined effect is measured on a dedicated SCMT benchmark constructed because CityFlowV2 annotations emphasize MCMT rather than full single-camera trajectories (Lu et al., 2023).

The ablation results quantify the contribution of each component. The baseline achieves IDF1 86.87, HOTA 75.94, MOTA 81.88, and IDSW 83. Adding SSA yields IDF1 89.09 and reduces IDSW to 56. Adding TRL yields IDF1 87.42 and IDSW 68. Adding BT yields MOTA 82.69 with IDSW 82. The full combination, baseline + SSA + TRL + BT, reaches IDF1 89.48, HOTA 76.75, MOTA 83.81, and IDSW 33. The paper interprets these results as showing that SSA produces the largest IDF1 gain, TRL sharply reduces identity switches, and BT mainly improves MOTA through better recall and trajectory completion (Lu et al., 2023).

These findings matter because CityTrack does not treat single-camera MOT as a solved precursor. Instead, it treats SCMT fragmentation as a first-order source of downstream ICA failure.

4. Inter-camera association and Box-Grained Matching

The second major contribution is the ICA module based on Box-Grained Matching (BGM). The baseline ICA first constrains the candidate pool using manually drawn exit and entry zones and camera-pair-specific time thresholds derived from road structure, then computes a trajectory-level distance matrix and solves assignment with the Hungarian algorithm (Lu et al., 2023).

For an exiting tracklet $\tau_i$ and entering tracklet $l_i$0, the baseline tracklet-wise distance is defined as

$l_i$1

that is, the mean of the smallest $l_i$2 cosine distances between box features from the two trajectories. The baseline also applies a travel-time penalty outside the valid interval $l_i$3, multiplying the distance by $l_i$4 when the transition is too early or too late (Lu et al., 2023).

CityTrack argues that such tracklet-wise aggregation is too coarse for urban vehicle MCMT because a trajectory may contain noisy, occluded, blurred, or poor-view boxes, and many vehicles are nearly indistinguishable at the whole-track level. BGM therefore computes distances between every box in the exiting zone and every box in the entering zone, preserving fine-grained appearance evidence rather than collapsing it prematurely to a single trajectory descriptor (Lu et al., 2023).

The raw box-level matrix is then refined in three stages. First, re-ranking reconstructs the original matrix into a refined matrix $l_i$5. Second, temporal prior refinement increases distances for candidate pairs whose interval lies outside the valid travel-time window. Third, occlusion-rate refinement penalizes heavily occluded boxes via

$l_i$6

with $l_i$7 and $l_i$8 (Lu et al., 2023).

Association is then performed not by Hungarian matching on the box matrix, but by k-reciprocal nearest neighbors. For a probe box $l_i$9, the paper defines its top-$B = [x, y, w, h, f],$0 neighbors as

$B = [x, y, w, h, f],$1

and its reciprocal set as

$B = [x, y, w, h, f],$2

A box votes for the entering trajectory that contributes the largest number of reciprocal neighbors, and an exiting trajectory inherits the global identity of the entering trajectory receiving the most box votes. The best-performing setting is $B = [x, y, w, h, f],$3 (Lu et al., 2023).

The ablations show that the design is not merely a matter of changing the distance granularity. Tracklet-wise + Hungarian gives 82.39 IDF1, tracklet-wise + k-reciprocal gives 84.16, box-grained + Hungarian drops to 80.16, and box-grained + k-reciprocal reaches 85.45. This is important: box-level evidence alone is insufficient without the reciprocal association rule, and k-reciprocal matching improves both trajectory-wise and box-grained variants. A second ablation shows incremental gains from each distance-optimization stage: without optimization the method gives 84.49 IDF1; adding re-ranking gives 85.23; adding travel-time-window refinement gives 85.39; adding occlusion-rate refinement gives 85.29; and combining all three yields 85.45 (Lu et al., 2023).

5. Evaluation, benchmark performance, and difficulty profile

CityTrack is evaluated primarily on CityFlowV2, which the paper describes as containing 46 cameras, 16 intersections, and splits of 3 scenarios for training, 2 for validation, and 1 for testing. The benchmark annotations cover only vehicles tracked across multiple cameras. For SCMT evaluation, the authors additionally manually annotate complete single-camera trajectories for two cameras in order to isolate the effect of the SCMT component (Lu et al., 2023).

For MCMT, the primary metric is IDF1, with IDP and IDR also reported. On the CityFlowV2 leaderboard table reproduced in the paper, the top five entries are: CityTrack 85.45, BOE 84.37, TAG 83.71, FraunhoferIOSB 83.48, and appolo 82.51. The abstract gives 84.91\%, whereas the main results table and leaderboard give 85.45; the latter is the more detailed figure reported in the experimental section (Lu et al., 2023).

The paper also compares Location-Aware SCMT against prior trackers under a common detector/ReID setting. DeepSORT achieves IDF1 82.37, HOTA 69.52, MOTA 76.52, IDSW 153; OCSORT gives 85.19, 73.47, 77.09, 70; ByteTrack gives 86.87, 72.78, 80.51, 68; StrongSORT gives 84.96, 73.93, 78.82, 108; and Location-Aware SCMT gives 89.48, 76.75, 83.81, 33 (Lu et al., 2023).

The broader significance of these numbers becomes clearer when placed next to adjacent large-scale benchmarks. "So you think you can track?" introduces the I24-Video multi-camera highway benchmark with 234 cameras, 234 hours of video, a 4.2 mile corridor, and manually corrected long-range trajectory ground truth; there, benchmarked tracking-by-detection systems obtain a best HOTA of 9.5\%, best recall 75.9\% at IoU 0.1, and 47.9 average IDs per ground-truth object, indicating severe long-horizon identity fragmentation even under georegistration and roadway constraints (Gloudemans et al., 2023). This adjacent evidence suggests that CityTrack’s urban-camera difficulty is part of a broader network-wide association problem: local recovery of positions is often easier than persistent identity maintenance across long spatial and temporal extents.

6. Later reuse of the name and resulting ambiguity

A later source of confusion is that CityTrack was reused in 2026 as the name of a large real-world multi-view crowd-tracking dataset rather than a vehicle MCMT framework. In that paper, CityTrack is built from CityStreet, resampled and re-annotated for identity-consistent multi-view pedestrian tracking on the ground plane. The reported dataset statistics are 3 cameras, 2704 × 1520 resolution, 2588 frames, 4 fps, a scene size of $B = [x, y, w, h, f],$4 meters, 950 people, and average trajectory length 228 frames. The first 1948 frames are used for training and the remaining 640 for testing; the ground plane is represented at $B = [x, y, w, h, f],$5 with 1 pixel = 0.1 m (Zhang et al., 21 Apr 2026).

The evaluation protocol for this later CityTrack is also different. All tracking metrics are computed on the ground plane, and the positive-association threshold is $B = [x, y, w, h, f],$6 for both MVCrowdTrack and CityTrack. The reported metrics are MOTA, MOTP, IDF1, MT, and ML. On this benchmark, MVTrackTrans reports MOTA 55.39, MOTP 22.71, IDF1 34.41, MT 25.07, and ML 12.69 (Zhang et al., 21 Apr 2026).

Accordingly, the name CityTrack now denotes two materially different objects in the literature. In the 2023 vehicle paper, it is a CityFlowV2-oriented MCMT method whose principal innovations are Location-Aware SCMT and Box-Grained Matching. In the 2026 crowd paper, it is a ground-plane pedestrian benchmark derived from CityStreet. The overlap is lexical rather than methodological. A precise reading therefore requires attention to domain, task, and citation context.

In the stricter historical sense established by the titled 2023 paper, CityTrack is best understood as a vehicle-oriented city-scale MCMT framework that improves cross-camera tracking by treating single-camera fragmentation and inter-camera matching granularity as coupled problems. Its enduring technical significance lies in that coupling: better urban MCMT required not just stronger ReID, but scene-aware repair of local tracklets and box-level reasoning during cross-camera association (Lu et al., 2023).