FusionCounting: Fusion for Crowd Counting

• FusionCounting is a research domain that employs fusion techniques to integrate multiscale, multimodal, and multi-view data for accurate crowd counting.
• It utilizes strategies such as encoder-layer fusion, branch aggregation, and cross-modal integration to tackle challenges like scale variation, occlusion, and modality mismatch.
• The approach improves efficiency and accuracy in density estimation, with applications ranging from surveillance to smart city analytics.

FusionCounting denotes a strand of crowd-counting research in which fusion is the primary mechanism for resolving scale variation, density variation, occlusion, modality mismatch, and view inconsistency. In this literature, fusion may occur across encoder layers, across multiscale branches, across RGB and thermal streams, across visual and wireless sensors, across calibrated camera views, or across tasks such as visible–infrared image fusion and crowd counting. The resulting predictors may output a density map, a set of point coordinates, or a scalar count, but the unifying theme is that complementary information is aggregated before final count estimation (Ma et al., 2022, Ma et al., 2024, Zhang et al., 2020, Li et al., 28 Aug 2025).

1. Scope and formulations

In the standard density-estimation setting, an input image is mapped to a density map whose integral, or sum over pixels, equals the number of people. Fusion enters this formulation because perspective distortion causes large within-image scale variation: small distant heads and large nearby bodies coexist, and a network that relies on a single high-level feature map with a narrow or fixed range of receptive fields can struggle to represent both. This is why many encoder–decoder crowd counters either append multiscale modules to the encoder output or fuse intermediate encoder features directly (Ma et al., 2022).

A distinct formulation replaces density regression with direct point prediction. FGENet, for example, predicts a set of coordinate points and confidence scores, then establishes correspondences with ground-truth points by the Hungarian algorithm. Its prediction set is written as

$P=\{\mathbf{p}_i\}_{i=1}^{N}, \qquad \hat{P}=\{\hat{\mathbf{p}_j\}_{j=1}^{M}, \qquad N \le M,$

and training combines regression, classification, and counting losses in the Three-Task Combination objective

$$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$

This point-based line preserves person-level localization more explicitly than density maps, while still relying on multiscale feature fusion in the neck (Ma et al., 2024).

Outside computer vision, the expression has unrelated meanings. In one usage it refers to bounds on total count under overlapping sensor regions, formalized through a linear program and the fusion polytope; in another it refers to combinatorial formulas for type-$A$ fusion coefficients via cylindric tableaux. These usages are mathematically distinct from crowd counting, even though all of them are concerned with counting under some notion of fusion (Moran et al., 2014, Morse et al., 2012).

2. Multiscale feature fusion in density-estimation networks

A central line of work treats multiscale fusion as the core architectural primitive. FusionCount is exemplary: it uses the first 17 layers of VGG-16, preserves 15 feature maps from the third layer onward, groups them by spatial resolution, and replaces a post-encoder multiscale block with adaptive fusion of the “large majority” of encoded features. For an input of size $224\times224\times3$, the preserved encoder features span receptive fields from 6 to 192 pixels. Within each resolution group, contrast features

$$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$

generate spatially adaptive weights

$$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$

and the fused feature is

$$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$

The decoder then merges the four fused scales coarsest-to-finest, with a channel reduction block combining a dilated convolution branch and a point-wise bottleneck branch (Ma et al., 2022).

Other density-estimation systems impose more structured fusion paths. MBTTBF-SCFB uses VGG16 plus conv6, then performs bidirectional multi-level fusion: a bottom-top path injects localization detail into deeper features, a top-bottom path injects context into shallower features, and Scale Complementary Feature Extraction Blocks exchange adjacent-scale residuals via

$$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$

The final bottom-top and top-bottom outputs are then fused by self-attention weighting (Sindagi et al., 2019).

SFANet organizes fusion as two parallel top-down pyramids over VGG16-bn features $$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$: a Density Map Path and an Attention Map Path. After coarse-to-fine fusion in both paths, the attention branch produces

$M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$

which gates density features through

$$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$0

Here fusion is simultaneously multiscale and cross-path, since the attention path modulates the density path spatially (Zhu et al., 2019).

A simpler school of design argues that effective fusion need not imply architectural inflation. FFNet keeps only a backbone and a three-branch multiscale fusion structure; each branch uses a Focus Transition Module, and the three refined features are aligned by transposed convolution and fused by channel concatenation. RepSFNet similarly pursues a lightweight route: a RepLK-ViT backbone with large reparameterized kernels is followed by a Feature Fusion module combining ASPP and CAN, then a Concatenate Fusion module for density prediction. This suggests two distinct interpretations of FusionCounting: one as increasingly structured hierarchical fusion, and another as deliberately simplified single-fusion pipelines (Chen et al., 2024, Achmadiah et al., 28 Jan 2026).

3. Supervision, optimization, and efficiency

Fusion-based crowd counting is not defined by architecture alone; supervision design is often equally decisive. FusionCount uses the DM-Count loss for training, Adam with learning rate $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$1, batch size 2, and patch-based training with two $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$2 crops per image. Its main comparison reports Multi-Adds on a $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$3 RGB image: FusionCount uses 815.00 G, versus 853.70 G for BL and DM-Count, 856.99 G for CSRNet, and 908.05 G for CAN. On ShanghaiTech Part A it achieves $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$4 MAE/RMSE, while on Part B it achieves $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$5, the best MAE and tied-best RMSE in that table. Its ablations also show that replacing contrast-based weights with weights derived directly from encoded features degrades ShanghaiTech Part B from $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$6 to $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$7 (Ma et al., 2022).

MBTTBF-SCFB couples architectural fusion with scale-aware supervision. It generates four scale-aware density maps $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$8 from head-size estimation using SLIC superpixels, watershed segmentation, and an MRF framework, then applies Euclidean loss both at the final density map and inside SCFB branches. The full method improves ShanghaiTech Part A from a baseline $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$9 to $A$0, and UCF-QNRF from $A$1 to $A$2, illustrating that fusion hierarchy and supervision design are co-dependent rather than separable (Sindagi et al., 2019).

SPF-Net makes this coupling explicitly probabilistic. It models noisy annotations as

$A$3

defines scale-specific random density maps, approximates each scale’s joint density distribution by a Gaussian with covariance $A$4, and uses a low-rank approximation to obtain a tractable negative log-likelihood. The final loss is

$A$5

With three scales and fusion enabled, SFP-Net reports $A$6 on UCF-QNRF, $A$7 on ShanghaiTech Part A, and $A$8 on Part B (Hsieh et al., 2022).

Efficiency remains a recurring point of contention. FFNet reports 29.02M parameters and 23.67G FLOPs while achieving $A$9 on ShanghaiTech Part A and $224\times224\times3$0 on Part B. RepSFNet reports MACs 62.59, Params 26.06, and latency $224\times224\times3$1 ms at $224\times224\times3$2, $224\times224\times3$3 ms at $224\times224\times3$4, and $224\times224\times3$5 ms at $224\times224\times3$6, with $224\times224\times3$7 on ShanghaiTech Part A and $224\times224\times3$8 on NWPU. At the same time, RepSFNet’s own loss presentation is internally inconsistent: the abstract states Mean Squared Error plus Optimal Transport, whereas the detailed loss subsection writes a combination of MAE and OT (Chen et al., 2024, Achmadiah et al., 28 Jan 2026).

4. Point-based and multimodal extensions

Fusion in crowd counting is no longer confined to density-map back-ends. FGENet replaces density regression with direct point localization, yet retains a fusion-centric neck. Its Fine-Grained Feature Pyramid fuses $224\times224\times3$9, $$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$0, and $$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$1 features using EVC, CSPLayer, and CBAM before regression and classification heads predict coordinates and confidences. Matching is defined by a cost

$$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$2

and the robust TTC loss combines HSL$$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$3, weighted cross-entropy, and Highly Robust Count loss. On ShanghaiTech Part A it reports $$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$4, on Part B $$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$5, and on UCF-QNRF $$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$6. Its FGFP ablation is particularly revealing: removing FGFP degrades ShanghaiTech Part A from $$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$7 to $$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$8, so fusion remains central even when the prediction target is a point set rather than a density field (Ma et al., 2024).

RGB-T crowd counting extends fusion to cross-modal settings. TAFNet uses a three-stream design with one main RGB-T stream and two auxiliary modality-specific streams; after each VGG16 stage, an Information Improvement Module fuses pyramid-pooled contextual RGB, thermal, and combination features, with thermal context further refined by channel and spatial attention. On RGBT-CC, TAFNet reports GAME(0) $$\boldsymbol{c}_{j} = \boldsymbol{x}_{i+j} - \boldsymbol{x}_i^*$$9, GAME(1) $$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$0, GAME(2) $$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$1, GAME(3) $$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$2, and RMSE $$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$3, improving over CMCRL by $$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$4 on GAME(0) and $$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$5 on RMSE. A transformer-based RGB-T alternative introduces a learnable count token, a Multi-Scale Token Transformer for count-guided multimodal fusion, and a multi-scale deformable transformer decoder for modal-guided count enhancement, reaching $$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$6 on GAME(0–3) and $$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$7 RMSE on RGBT-CC (Tang et al., 2022, Liu et al., 2023).

A further extension replaces thermal imagery with wireless sensing. TransFusion fuses image patches and WiFi CSI amplitude sequences through bidirectional cross-modal Transformer attention, multi-scale CNN refinement, and temporal self-attention, then regresses a scalar count with an $$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$8 loss. On its self-collected indoor dataset it reports MAE $$\boldsymbol{\omega}_{j} = \sigma (\boldsymbol{c}_{j}),$$9, MSE $$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$0, MAPE $$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$1, and $$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$2. The paper also contains two reporting inconsistencies: its text claims MSE improvement over LF-LSTM although the table gives $$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$3, and the linear-attention ablation row duplicates the full-model metrics while the deltas indicate degradation (Cui et al., 28 Apr 2025).

5. Geometric, 3D, and semi-supervised multi-view fusion

When multiple synchronized cameras observe a large scene, fusion becomes explicitly geometric. “Wide-Area Crowd Counting: Multi-View Fusion Networks for Counting in Large Scenes” formulates the target as a scene-level density map on the ground plane and compares three variants: late fusion of projected per-view density maps, naive early fusion of projected feature maps, and multi-view multi-scale early fusion with a rotation selection module. The core principle is that camera-view representations should be projected into a common world-space frame before learning the fused scene density (Zhang et al., 2020).

A stronger geometric formulation lifts feature fusion from 2D ground-plane maps to 3D voxels. “3D Crowd Counting via Geometric Attention-guided Multi-View Fusion” predicts a 3D scene-level density map $$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$4 and supervises it with

$$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$5

Each camera-view feature map is projected to multiple height planes,

$$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$6

then multi-view 3D volumes are concatenated and decoded by a 3D CNN. The model also adds a projection consistency term

$$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$7

Despite the title, the paper does not actually contain a learned height estimation network or a geometric attention module that selects one voxel per ray; it uses deterministic multi-height projection and 3D fusion. It reports MAE $$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$8 on PETS2009 and $$\boldsymbol{f} = \text{Conv} \left(\left[ \boldsymbol{x}_i \, \middle| \, \sum_{j=1}^k \boldsymbol{\omega}_j \odot \boldsymbol{x}_{i + j} \right] \right).$$9 on CityStreet, but $$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$0 on DukeMTMC, where MVMS remains better at $$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$1 (Zhang et al., 2020).

Semi-supervised work has recently turned view multiplicity itself into a supervision signal. “Semi-Supervised Multi-View Crowd Counting by Ranking Multi-View Fusion Models” trains a family of fusion models indexed by the number of input views. The vanilla MVPR method ranks density predictions so that fewer-view outputs should not exceed more-view outputs,

$$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$2

whereas MVUR ranks estimated uncertainty maps on the common visible area,

$$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$3

At $$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$4 labels, MVUR reports MAE $$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$5 on CVCS, $$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$6 on CityStreet, and $$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$7 on PETS2009. This suggests that in multi-view FusionCounting, uncertainty monotonicity is often a more stable prior than direct prediction monotonicity (Zhang et al., 18 Dec 2025).

6. FusionCounting as count-guided visible–infrared image fusion

The 2025 framework titled “FusionCounting” shifts the meaning of the term from “fusion for counting” to “counting for fusion.” It jointly optimizes visible–infrared image fusion and RGB-T crowd counting with a shared two-stream encoder, a fusion decoder, a counting decoder, dynamic task weighting, and PGD-based adversarial training. The visible and infrared encoders $$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$8 and $$\hat{F}_i = F_i + F_j^r, \qquad \hat{F}_j = F_j + F_i^r.$$9 interact at every layer through

$$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$0

and the fusion decoder reconstructs hierarchical fused features by

$$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$1

The counting decoder keeps the original regression head of the chosen RGB-T crowd-counting backbone and predicts a density map from deep shared features (Li et al., 28 Aug 2025).

A defining feature of this framework is that crowd counting guides visible–infrared fusion indirectly through the shared encoder rather than by injecting density maps into the fusion decoder. The overall objective is

$$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$2

where the fusion loss combines SSIM, texture, intensity, and MSE terms, and the counting branch uses Bayesian loss. To balance tasks, the dynamic weighting strategy defines

$$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$3

with

$$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$4

and the paper uses $$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$5. Robustness is improved by PGD adversarial training with $$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$6, $$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$7, and 7 iterations.

Empirically, the framework is instantiated on IADM, CSCA, MC$$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$8Net, and DEFNet. On the aligned RGBT-CC subset, MC$$\{\text{conv2-2},\text{conv3-3},\text{conv4-3},\text{conv5-3}\}$$9Net + FusionCounting achieves the best counting results with GAME(0) $M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$0, GAME(1) $M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$1, GAME(2) $M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$2, GAME(3) $M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$3, and RMSE $M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$4, improving markedly over the original MC$M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$5Net’s GAME(0) $M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$6 and RMSE $M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$7. For fusion quality, IADM + FusionCounting reports SSIM $M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$8 and PSNR $M_{Att} = Sigmoid(W \textcircled{c} f_{att} + b),$9. The framework also shows that a parallel shared-encoder design is far stronger than a serial cascade: for example, IADM in series yields RMSE $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$00, versus $$L=\lambda_1 L_{cls}+\lambda_2 L_{reg}+\lambda_3 L_{cou}.$$01 in the proposed parallel version. The main limitation identified by the authors is RGB–thermal misalignment, which they leave as future work.

Taken together, the literature presents FusionCounting not as a single architecture but as a research program. In one direction, fusion is used to improve counting by combining scales, modalities, or views; in another, counting is used to improve fusion by imposing semantically meaningful supervision. The field therefore spans adaptive VGG feature aggregation, bidirectional residual-scale fusion, probabilistic multiscale supervision, point-set prediction, RGB-T and CSI multimodality, 3D voxel fusion, semi-supervised ranking across view subsets, and multi-task visible–infrared fusion guided by density estimation. The common invariant is that counting quality improves when the fused representation encodes complementary information at the right geometric, semantic, and supervisory level.