SynMVCrowd Benchmark for Multi-view Crowd Analysis

• SynMVCrowd is a synthetic large-scale benchmark featuring calibrated multi-view geometry and dense 2D/3D crowd annotations for precise counting and localization.
• It comprises 500,000 high-resolution images from 50 urban scenes with 50 cameras per scene, facilitating robust evaluation across diverse settings.
• The benchmark supports cross-view and cross-scene analysis with an innovative OT-based loss, enhancing synthetic-to-real transfer and performance in high-density environments.

SynMVCrowd is a synthetic, large-scale benchmark for multi-view crowd counting and localization, proposed to make evaluation and comparison practical under large scenes, many camera views, many frames, and much larger crowd numbers than earlier datasets (Zhang et al., 25 Mar 2026). It is built from GTA-V using an extended version of the GCC framework and is organized as a simulated world with calibrated camera geometry, cross-view correspondences, dense 2D and 3D annotations, standardized train/validation/test splits, and baseline methods for both counting and localization. The benchmark also serves as a large and challenging single-image crowd counting and localization dataset, and it is explicitly positioned as a source domain for synthetic-to-real transfer.

1. Motivation and problem setting

Most existing multi-view crowd datasets are small in at least one crucial dimension: they often contain only one scene, use only 3–7 cameras, include low-to-moderate crowd sizes, and provide only a few hundred frames per scene (Zhang et al., 25 Mar 2026). The benchmark paper identifies two consequences. First, modern deep multi-view methods can overfit these tiny benchmarks and fail to generalize to new scenes. Second, such datasets do not reflect deployments such as transport hubs, festivals, shopping streets, or beaches, where large areas and hundreds to thousands of people are common.

SynMVCrowd is intended to address this gap by supporting high-density crowds, large scenes, many cameras per scene, and cross-scene evaluation. Its stated goals are to provide a realistic large-scale testbed for multi-view crowd counting and multi-view crowd localization, to serve as a very large single-image benchmark, to enable meaningful cross-scene and cross-view evaluation, and to support synthetic pretraining and domain transfer to real-world datasets (Zhang et al., 25 Mar 2026).

A common misconception is that SynMVCrowd is merely a collection of rendered images. The benchmark is instead defined around calibrated multi-view geometry, explicit Region of Interest design, 3D person coordinates in a unified scene coordinate system, cross-view person IDs, ground-plane occupancy supervision, and task-specific baseline models. This makes it a benchmark for geometry-aware crowd analysis rather than a conventional 2D image corpus.

2. Dataset composition and scene design

At the dataset level, SynMVCrowd contains 50 different urban scenes, 50 calibrated cameras per scene, and 200 multi-view frames per scene, yielding 500,000 images at 1920 × 1080 resolution (Zhang et al., 25 Mar 2026). Typical scene size is around 100 m × 120 m. The benchmark covers 15 scene categories, including parks, curbside streets, walking streets, beaches, shopping centers, and churches.

Component	Value
Scenes	50 different urban scenes
Scene categories	15 scene categories
Cameras per scene	50 calibrated cameras
Frames per scene	200 multi-view frames
Total images	500,000 images
Resolution	1920 × 1080
Typical scene size	around 100 m × 120 m
Per-image crowd count	roughly 200–1,000 people, mean ~530
Total human instances	about 5.3 million

The benchmark is explicitly high-density. Per-image crowd count is roughly 200–1,000 people, with mean approximately 530, and the full dataset contains about 5.3 million human instances (Zhang et al., 25 Mar 2026). The count distribution is skewed toward medium and high densities, which is why the paper characterizes the dataset as significantly more challenging than earlier multi-view benchmarks.

Environmental diversity is also a core design dimension. Weather is sampled from seven GTA-V weathers—Clear, Extra Sunny, Overcast, Clouds, Rain, Foggy, and Thunder—with final proportions of 56.3%, 21.8%, 8.3%, 4.5%, 4.5%, 3.8%, and 0.8%, respectively (Zhang et al., 25 Mar 2026). Time-of-day spans the full 0–24h range and is grouped into Morning, Noon, Afternoon, Sunset, and Evening, with Evening further split into Night, Nightfall, Midnight, and Early Morning. The final distribution is reported as roughly uniform across the coarse time groups. The result is substantial variation in illumination, shadows, visibility, and low-light imagery.

The benchmark uses 265 unique GTA-V character models with diversity in gender, skin tone, body shape, height, clothing, and hair (Zhang et al., 25 Mar 2026). Each person is assigned a unique ID and a static but subtle action such as smoking, drinking, or listening to music. These upper-body motions are designed to preserve static spatial annotations while reducing a frozen appearance.

3. Camera geometry, synthesis pipeline, and annotations

Camera design is central to SynMVCrowd. Four initial cameras are placed roughly at north, east, south, and west around the Region of Interest, after which a circular camera ring is constructed so that adjacent views differ by approximately 10° in yaw and each camera looks roughly toward the scene center (Zhang et al., 25 Mar 2026). The field of view is 40°. This arrangement provides both large baselines and overlapping fields of view, which are useful for projection, fusion, and localization on the ground plane.

All cameras are calibrated with known intrinsics and extrinsics. Projection is expressed as

$$s \begin{pmatrix} u\ v\ 1 \end{pmatrix} = P \begin{pmatrix} x\ y\ z\ 1 \end{pmatrix} = A [R|t] \begin{pmatrix} x\ y\ z\ 1 \end{pmatrix},$$

where $A$ is the intrinsic matrix, $[R|t]$ is the extrinsic matrix, and $P = A[R|t]$ is the full projection matrix (Zhang et al., 25 Mar 2026). For ground-plane projections, person height is fixed to $z = 1750\text{ mm}$, following prior work.

Because GTA-V limits the number of people per scene, the synthesis procedure partitions the Region of Interest into multiple sub-areas until each area can host at most 256 people (Zhang et al., 25 Mar 2026). For each area, the pipeline places a subset of people, captures images from all 50 cameras, records each person’s 3D and per-camera 2D coordinates together with the unique ID, removes those people, and repeats the process for the next area. Time is controlled so that background conditions remain consistent. The area-specific crowd images are then composited onto a background image per camera and frame index. This procedure yields final frames with all people present while preserving consistent global 3D coordinates and cross-view correspondences.

Annotations include 3D world coordinates for each person, person IDs for tracking and cross-view association, precise image-plane coordinates for each person, accurate semantic segmentation of crowd pixels, ground-plane locations, full camera intrinsics and extrinsics, scene type, scene size, Region of Interest polygon, weather, time-of-day, and crowd counts per frame (Zhang et al., 25 Mar 2026). Derived supervision includes 2D density maps and ground-plane occupancy maps in which each grid cell corresponds to 0.2 m on the ground.

The split protocol is scene-disjoint rather than frame-disjoint. Training uses 30 scenes and 14 scene types; validation uses 10 scenes; test uses 10 scenes (Zhang et al., 25 Mar 2026). In image counts, this corresponds to 300,000 images for training and 100,000 images each for validation and test. The split design is intended to enforce cross-scene generalization rather than within-scene memorization.

4. Supported tasks, baselines, and evaluation protocol

SynMVCrowd directly supports three benchmarked tasks: multi-view crowd localization, multi-view crowd counting, and single-image crowd counting and localization (Zhang et al., 25 Mar 2026). In the multi-view setting, the input is a subset of cameras for a scene at one time step, commonly five views, although experiments also use three, seven, and nine. For localization, the output is a ground-plane occupancy map or a set of person locations in world or BEV coordinates. For counting, the output is a scalar total number of people or a density map integrated over the ground plane.

The proposed baseline architecture has four stages. First, it performs single-view feature extraction, with ResNet-18 identified as the best backbone in the ablation. Second, it applies spatial feature selection by estimating a spatial attention map for each view and normalizing attention across views with a softmax over views. Third, it projects features onto the ground plane using a Spatial Transformer Network and known camera geometry, then fuses them with view-wise max pooling. Fourth, it decodes the fused BEV feature map into either a localization occupancy map or a counting density map (Zhang et al., 25 Mar 2026).

Two training regimes are emphasized. The first is a standard MSE loss on density or occupancy maps,

$$L_{\text{MSE}} = \frac{1}{N}\sum_{i=1}^{N} \left\| \hat{D}_i - D_i \right\|_2^2,$$

where $\hat{D}_i$ and $D_i$ are predicted and ground-truth maps (Zhang et al., 25 Mar 2026). The second, and the key methodological contribution, is a point-supervision Optimal Transport loss on the ground plane:

$$\mathcal{L}^{\mathcal{T}_C} = \min_{\mathbf{P} \in \mathbb{R}_+^{n \times m}} \langle C, P \rangle - \varepsilon H(P) + \tau\|P \mathbf{1}_{m} - a\|_2^2 + \tau\|P^{T} \mathbf{1}_{n} - b\|_1.$$

The paper presents this OT formulation as a joint mechanism for precise BEV localization and accurate counting, especially in high-density scenes.

For multi-view localization, the benchmark uses MODA, MODP, Precision, Recall, and F1. In particular,

$$\text{MODA} = 1 - \frac{\text{FP} + \text{FN}}{\text{TP} + \text{FN}},$$

and

$A$0

For counting, the benchmark uses MAE, the quantity labeled MSE in the paper but written with an RMSE formula, and NAE:

$A$1

Single-image localization uses 2D point matching with a 3-pixel threshold (Zhang et al., 25 Mar 2026).

5. Empirical results and synthetic-to-real transfer

Under cross-scene evaluation on SynMVCrowd with five randomly selected views per frame, the OT-based baseline achieves the strongest localization results reported in the paper: MODA 49.6, MODP 70.2, Precision 88.6, Recall 57.0, and F1 69.4 (Zhang et al., 25 Mar 2026). The paper notes that all existing multi-view state-of-the-art methods show significant drops relative to their performance on WildTrack or MultiviewX, which is consistent with the larger scenes, higher density, and scene-disjoint evaluation of SynMVCrowd.

For multi-view counting on SynMVCrowd, the OT-based baseline also gives the best MAE among the compared methods, with MAE 50.2, compared with 56.3 for CountFormer, 67.7 for CVCS, 70.6 for MVMSR, 74.6 for 3DCounting, and 68.8 for the same baseline trained with MSE (Zhang et al., 25 Mar 2026). The benchmark therefore serves not only as a harder dataset but also as an environment in which the choice of supervision, especially OT, materially changes performance.

Ablation studies are part of the benchmark definition. ResNet-18 outperforms VGG19 and the Swin-based alternative within the same pipeline. Removing the spatial feature selection module degrades both counting and localization. Increasing the number of input views improves performance for the OT baseline. Cameras with more downward-facing pitch angles perform better than more frontal views because frontal views suffer more from occlusion and scale variation (Zhang et al., 25 Mar 2026).

The paper also studies domain transfer. For multi-view localization on WildTrack and MultiviewX, direct testing from SynMVCrowd is described as modest, but fine-tuning improves results rapidly, and with 100% labels plus domain adaptation the baseline reaches MODA 88.7 and F1 94.6 on WildTrack, and MODA 87.6 and F1 93.9 on MultiviewX (Zhang et al., 25 Mar 2026). For multi-view counting on CityStreet and PETS2009, direct transfer produces high error, whereas 10–30% annotations plus domain adaptation reduce MAE substantially, and 100% labels plus domain adaptation make the baseline comparable to MVMSR and 3DCounting, although still somewhat behind CountFormer.

Single-image transfer is treated as an additional use of the benchmark rather than a separate dataset. On ShanghaiTech B, direct testing from SynMVCrowd yields MAE 18.1, which is better than direct testing from GCC at 22.8 and better than SE CycleGAN from GCC at 19.9, while unsupervised domain-adversarial training from SynMVCrowd reaches MAE 11.2 (Zhang et al., 25 Mar 2026). The paper interprets these results as evidence that SynMVCrowd can function as a strong synthetic source domain for both multi-view and single-image crowd analysis, while also making clear that the synthetic-to-real gap remains non-trivial.

6. Position in the literature, limitations, and related synthetic benchmarks

Within the synthetic crowd benchmark literature, SynMVCrowd occupies the multi-view, large-scene, high-density end of the design space (Zhang et al., 25 Mar 2026). UniCrowd, by contrast, is a human crowd simulator plus a validation pipeline designed for crowd analysis in computer vision, with a behavioral module based on the Social Forces Model and a visual rendering module connected through TCP; it is oriented toward detection, segmentation, crowd counting, human pose estimation, trajectory analysis and prediction, and anomaly detection rather than a fixed large-scale multi-view benchmark (Bisagno et al., 2023). ANTHROPOS-V introduces the distinct task of Crowd Volume Estimation, defines CVE as estimating the undergarment total body volume occupied by all humans in a scene using only RGB images, and provides a synthetic photorealistic video benchmark with per-person volume, per-part volumes for nine body regions, SMPL meshes, and keypoints, but it does not define a synchronized multi-camera protocol (Collorone et al., 3 Jan 2025).

This comparison helps situate SynMVCrowd precisely. UniCrowd emphasizes dual fidelity—visual and behavioral—and a generator architecture. ANTHROPOS-V emphasizes anthropometric realism and volume-aware supervision. SynMVCrowd emphasizes calibrated multi-view geometry, cross-view correspondence, large-scene density, and cross-scene evaluation. A plausible implication is that these lines of work are complementary: SynMVCrowd supplies large-scale multi-view geometry and evaluation protocols, UniCrowd contributes behavior-centric simulation and broader task coverage, and ANTHROPOS-V contributes physiologically meaningful labels such as volume.

The paper also states several limitations of SynMVCrowd itself (Zhang et al., 25 Mar 2026). Despite GTA-V’s fidelity, the benchmark does not perfectly capture the full diversity of real-world lighting, weather, motion blur, camera noise, and behavior. People are static in each frame, apart from local idle motion, so temporal dynamics, trajectories, and appearance changes are limited. Cameras are well calibrated, synchronized, and arranged in a controlled ring, which is not always true in real deployments. Extreme events such as panic or stampedes are not modeled. These limitations constrain direct deployment claims and explain why domain adaptation and evaluation on real datasets remain necessary.

In the broader literature, SynMVCrowd is intended to function as a stress test for algorithms’ ability to handle realistic, large-scale, high-density settings, and as a foundation for robust pretraining and domain adaptation (Zhang et al., 25 Mar 2026). Its combination of 50 scenes, 50 cameras per scene, scene-disjoint splits, 3D world coordinates, per-view projections, occupancy supervision, environmental variation, and strong OT-based baselines makes it a reference point for multi-view crowd counting and localization in synthetic environments.