Dual-Stream Adaptive Counting Decoder

• The Dual-Stream Adaptive Counting Decoder is a network architecture that fuses a Transformer stream for global context with a CNN stream for local detail.
• It employs Transformer-to-CNN enhancement adapters and a learned gating net to adaptively fuse outputs for accurate density map predictions.
• Ablation studies reveal that combining both streams significantly reduces validation and test MAE compared to utilizing individual streams.

Searching arXiv for the specified paper and closely related dual-stream counting work to support the article with fresh citations. The Dual-Stream Adaptive Counting Decoder is a decoder architecture for object or crowd counting that combines two parallel processing streams with complementary inductive biases and then adapts their interaction during density estimation or count prediction. In the most direct formulation, introduced as the DAC-decoder in QUANet, the two streams are a Transformer stream for globally aware modeling and a CNN stream for locally detailed modeling; these are coupled by Transformer-to-CNN enhancement adapters (T2C-adapters) and a learned gating net that fuses stream outputs into a final density map (Shi et al., 9 Jul 2025). More broadly, related counting systems use dual-stream designs in different senses: graph-based density and representation branches in DSGC-Net (Wu et al., 2 Sep 2025), density and attention decoder paths in M-SFANet (Thanasutives et al., 2020), and dorsal–ventral what/where pathways in zero-shot counting models inspired by parietal and ventral cortex (Thompson et al., 2024). Across these variants, the central theme is adaptive fusion of complementary signals for improved counting under density variation, viewpoint change, clutter, or open-vocabulary prompting.

1. Conceptual definition and scope

In QUANet, the Dual-Stream Adaptive Counting Decoder is the density regression head placed after category-conditioned vision–language features have been formed. The model uses a vision–language backbone with a DINOv2 ViT-B/14 vision encoder and a BERT-base text encoder, then applies four cross-attention blocks to inject category-only text information into visual embeddings and obtain a category-prompted visual feature $F^V$. The DAC-decoder takes $F^V$ as input and is responsible for decoding it into density maps while explicitly leveraging both global and local modeling, supporting cross-stream knowledge transfer, adaptive fusion, and patch-level prediction consistency (Shi et al., 9 Jul 2025).

The designation “adaptive” has a specific meaning in this context. In QUANet, adaptivity operates at two levels. First, feature-level adaptation is performed by T2C-adapters, which inject Transformer-derived context into the CNN stream. Second, prediction-level adaptation is performed by a gating network that computes data-dependent scalar weights $w_1$ and $w_2$ and forms the fused density map

$D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$

This is explicitly not a simple ensemble; it is a coupled decoder trained jointly with stream-specific density supervision and a cross-stream quantity ranking objective (Shi et al., 9 Jul 2025).

A broader reading of the term encompasses other dual-stream counting decoders that separate complementary cues and refine them before final counting. DSGC-Net, for example, uses a Density Approximation branch and a Representation Approximation branch, each defined by a different semantic graph, and fuses their outputs with a shared feature map before the final prediction head (Wu et al., 2 Sep 2025). M-SFANet uses dual decoder paths for density map and attention map generation, with the attention path suppressing background responses in the density path (Thanasutives et al., 2020). These systems do not share QUANet’s exact mechanics, but they instantiate the same architectural principle: dual streams are introduced because a single homogeneous decoder is insufficiently expressive for the full range of counting-relevant structure.

2. QUANet’s DAC-decoder architecture

The DAC-decoder in QUANet is organized as two parallel streams derived from the same category-prompted feature $F^V$. The Transformer stream processes $F^V$ with Transformer blocks to obtain globally aware features, while the CNN stream processes $F^V$ with convolutional operations to obtain locally enriched features (Shi et al., 9 Jul 2025). The paper states that the decoder “devises a Transformer stream in parallel to the CNN stream, while several layer-wise T2C-adapters are introduced to effectively transfer the knowledge from the Transformer stream into the CNN stream on the patch level” (Shi et al., 9 Jul 2025).

The Transformer stream applies a sequence of Transformer blocks to the input feature: $F^{V}_{Trans} = \text{TransformerBlocks}(F^V).$ After the last Transformer layer, the resulting feature map at patch resolution is projected to a density map through a $1 \times 1$ convolution: $F^V$0 The CNN stream applies convolutional layers: $F^V$1 and similarly produces a density map

$F^V$2

The final prediction is obtained by the learned weighted fusion already noted above (Shi et al., 9 Jul 2025).

The role separation between the streams is empirically motivated. The ablation analysis reports that a decoder without the Transformer stream yields VAL MAE 16.84, TEST MAE 16.92, while a decoder without the CNN stream yields VAL MAE 16.05, TEST MAE 16.30. The full DAC-decoder in QUANet reaches VAL MAE 14.22, TEST MAE 13.24, indicating that neither stream subsumes the other (Shi et al., 9 Jul 2025). The accompanying GAME analysis further associates the Transformer stream with better coarse global counts and the CNN stream with better fine local counts, while the full decoder achieves the lowest error across all GAME levels, supporting a global–local complementarity interpretation (Shi et al., 9 Jul 2025).

3. Transformer-to-CNN enhancement adapters

The most distinctive internal mechanism of QUANet’s DAC-decoder is the Transformer-to-CNN enhancement adapter, or T2C-adapter. Its function is strictly directional: it transfers knowledge from the Transformer stream to the CNN stream on the patch level, providing contextual information for the overall object count (Shi et al., 9 Jul 2025). The reverse direction, C2T, and bidirectional exchange, BiD, are reported to degrade performance and to “break the contextual cues originally stored in the Transformer stream,” according to the ablation summary (Shi et al., 9 Jul 2025).

Each T2C-adapter consists of two components:

1. A Cross-Attention block (CA), which facilitates interaction in the spatial dimension. The query is the CNN local feature $F^V$3, and the key and value are the Transformer global feature $F^V$4:

$F^V$5

The block contains a cross-attention layer, layer normalization, and an MLP (Shi et al., 9 Jul 2025).

2. A Channel-Excitation block (CE), inspired by Squeeze-and-Excitation, in which a channel descriptor is computed from Transformer features by Global Average Pooling and an MLP, then applied to recalibrate CNN channels:

$F^V$6

Its stated purpose is to “recalibrate the importance of local features along the channel dimension” (Shi et al., 9 Jul 2025).

The adapter output is the sum of these two contributions,

$F^V$7

which is then added residually to the CNN feature: $F^V$8 This yields a CNN representation enriched by both spatially informed and channel-wise cues from the Transformer stream (Shi et al., 9 Jul 2025).

Ablation results quantify the contribution of the adapters. Removing T2C-adapters entirely gives VAL MAE 15.12, TEST MAE 14.73. Removing only CE gives VAL MAE 14.91, TEST MAE 14.39, and removing only CA gives VAL MAE 14.83, TEST MAE 14.26, compared with 14.22 / 13.24 for the full DAC-decoder (Shi et al., 9 Jul 2025). This indicates that both CA and CE matter, and that partial T2C remains better than no T2C.

4. Prediction, ranking, and training objective

QUANet supervises the DAC-decoder with a density regression loss applied to the fused prediction and to each stream-specific prediction: $F^V$9 This enforces both branch competence and competence of the fused estimator (Shi et al., 9 Jul 2025).

The second decoder-specific objective is the cross-stream quantity ranking loss $w_1$0, introduced to enforce patch-level ordering consistency within and across streams. The density map is partitioned into $w_1$1 non-overlapping patches, sorted by ground-truth patch count in descending order, and the predicted counts per ranked patch are collected as $w_1$2 and $w_1$3. The loss is

$w_1$4

where $w_1$5 is ReLU and the paper uses $w_1$6 (Shi et al., 9 Jul 2025). Two terms impose within-stream monotonicity, and two impose cross-stream ranking coherence.

The reported ablation is strong. Removing $w_1$7 produces a significant performance drop, specifically MAE +1.70 on VAL and +1.81 on TEST; the explicit figures are VAL MAE 15.92, TEST MAE 15.05 compared with 14.22 / 13.24 for the full model (Shi et al., 9 Jul 2025). Using only within-stream or only cross-stream ranking constraints performs worse than the complete formulation.

The full QUANet objective couples counting, vision–text quantity alignment, and cross-stream ranking: $w_1$8 with $w_1$9 (Shi et al., 9 Jul 2025). The vision–text quantity alignment loss $w_2$0 acts at the encoder level using quantity-oriented text prompts, but it conditions the decoder indirectly by shaping $w_2$1 and thus $w_2$2. The DAC-decoder therefore does not consume text directly; it consumes a quantity-sensitive visual representation learned by the upstream prompt-aligned encoder (Shi et al., 9 Jul 2025).

Implementation details reported as relevant to the DAC-decoder include image size $w_2$3, a DINOv2 ViT-B/14 vision encoder with the last six layers fine-tuned, a frozen BERT-base top-9-block text encoder, four cross-attention blocks to form $w_2$4, $w_2$5 patches for $w_2$6, $w_2$7 counterfactual prompts, optimizer AdamW with learning rate $w_2$8 and decay factor $w_2$9, training for 200 epochs with batch size 32 on an NVIDIA A40 GPU (Shi et al., 9 Jul 2025).

5. Empirical behavior and decoder-level evidence

The decoder-specific evidence in QUANet is unusually explicit. Four ablation themes are especially relevant: stream complementarity, T2C-adapters, adaptive fusion, and portability across counting frameworks.

The following table summarizes the principal DAC-decoder ablations reported for QUANet (Shi et al., 9 Jul 2025).

Configuration	VAL MAE	TEST MAE
DAC-decoder w/o Transformer	16.84	16.92
DAC-decoder w/o CNN	16.05	16.30
DAC-decoder w/o T2C	15.12	14.73
DAC-decoder w/o CE	14.91	14.39
DAC-decoder w/o CA	14.83	14.26
avg-$D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$0 fusion instead of gating	15.31	13.86
w/o $D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$1	15.92	15.05
Full DAC-decoder (QUANet)	14.22	13.24

The learned gating net is a genuine performance contributor rather than a cosmetic addition. Replacing it with simple average fusion, denoted avg-$D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$2, degrades the result from 14.22 / 13.24 to 15.31 / 13.86 on VAL/TEST MAE (Shi et al., 9 Jul 2025). This supports the claim that the decoder should not statically commit to one stream or an invariant mixture.

The DAC-decoder is also evaluated as a drop-in counting head. Replacing the decoders of CLIP-Count and VLCounter with DAC-decoder improves both methods: CLIP-Count improves from VAL MAE 18.79 → 17.17 and TEST MAE 17.78 → 16.20; VLCounter improves from VAL MAE 18.06 → 16.51 and TEST MAE 17.05 → 15.56 (Shi et al., 9 Jul 2025). This suggests that the architectural contribution is not confined to QUANet’s particular prompt design, although the paper’s direct evidence remains within the counting settings tested there.

On benchmark scope, QUANet reports extensive experiments on FSC-147, CARPK, PUCPR+, and ShanghaiTech, framed as evidence of strong generalizability for zero-shot class-agnostic counting (Shi et al., 9 Jul 2025). A plausible implication is that the decoder’s benefit becomes especially salient when category semantics are weakly supervised or shift across datasets, because global context and local density cues must be reconciled without class-specific retraining.

6. Related dual-stream counting decoders

The term “Dual-Stream Adaptive Counting Decoder” is most precisely instantiated by QUANet’s DAC-decoder, but related literature shows that dual-stream decoder design has multiple technical realizations.

DSGC-Net introduces a Dual-Stream Graph Convolutional Correlation Mining module for crowd counting via feature correlation mining (Wu et al., 2 Sep 2025). Its two streams are the Density Approximation (DA) branch and the Representation Approximation (RA) branch. Both operate on a fused feature map $D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$3, but construct different semantic graphs: DA uses a density distribution map $D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$4 and a density-driven semantic graph based on

$D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$5

while RA uses cosine similarity of flattened features,

$D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$6

Each graph is processed by a GCN and reshaped into refined feature maps $D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$7 and $D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$8, which are fused with $D^{*} = w_1 D^{*}_{CNN} + w_2 D^{*}_{Trans}.$9 and sent to a prediction head (Wu et al., 2 Sep 2025). Although this is not a Transformer/CNN decoder, it is dual-stream and adaptive in the sense that the graph neighborhoods are data-dependent.

The DSGC-Net ablation hierarchy also mirrors the logic seen in QUANet. On ShanghaiTech Part A/B, the Baseline has MAE 53.6 / 7.4 and MSE 85.1 / 11.6; adding the Density Prediction Module gives 51.3 / 6.6 and 79.9 / 10.7; adding DP + DA gives 49.8 / 6.5 and 79.5 / 10.2; RA only gives 52.0 / 6.9 and 83.7 / 11.1; and the full DP + DA + RA model yields 48.9 / 5.9 and 77.8 / 9.3 (Wu et al., 2 Sep 2025). This again shows that complementary streams become more effective when fused than when used separately.

M-SFANet and M-SegNet provide an earlier dual-path decoder lineage in density-map-based crowd counting (Thanasutives et al., 2020). M-SFANet uses a density map path and an attention map path, each with the same multi-scale fusion strategy. The decoder combines ASPP output, CAN output, and skip-connected encoder features, then applies element-wise multiplication of the attention map and density feature maps to produce a refined density map: $F^V$0 The attention path is thus an adaptive gating mechanism aimed at suppressing background clutter before density regression (Thanasutives et al., 2020). The model is not “adaptive” in the same sense as QUANet’s T2C-adapters or learned scalar gating of two density maps, but it is adaptive through location-dependent contextual scaling in CAN and through learned attention masking.

Finally, the dual-stream neural network for zero-shot counting in (Thompson et al., 2024) uses a dorsal–ventral split rather than a dual decoder on a shared spatial tensor. The ventral stream encodes glimpse contents, the dorsal stream encodes fixation coordinates, and a recurrent parietal module integrates both streams over time, followed by a map layer and a numerosity readout: $F^V$1 Here the decoder is adaptive through recurrent integration over what/where evidence rather than through parallel density heads (Thompson et al., 2024). The relation is conceptual rather than architectural, but it underscores that counting systems frequently require distinct channels for identity and spatial structure.

7. Interpretation, misconceptions, and design implications

A common misconception is that a dual-stream counting decoder is simply a late-fusion ensemble of two predictors. The QUANet evidence directly contradicts this interpretation. The DAC-decoder is coupled through layer-wise T2C-adapters, constrained by a cross-stream quantity ranking loss, and fused by a learned gating net rather than fixed averaging (Shi et al., 9 Jul 2025). The measured gap between learned gating and average fusion, and between full T2C and no T2C, shows that coupling and adaptivity are central, not incidental.

A second misconception is that the “adaptive” designation refers only to per-image weighting at the final prediction layer. In QUANet, adaptivity is distributed: feature-level adaptation occurs repeatedly via T2C-adapters; prediction-level adaptation occurs through gating; and rank-level adaptation occurs through the patch ordering constraints in $F^V$2 (Shi et al., 9 Jul 2025). In DSGC-Net, adaptivity is instead graph-structural, since the neighborhood system is built per image using density similarity or representation similarity (Wu et al., 2 Sep 2025). In M-SFANet, adaptive behavior is tied to context-aware scale encoding and attention masking (Thanasutives et al., 2020). The term therefore does not denote a single mechanism.

The design rationale shared across these systems is that counting is neither purely global nor purely local. QUANet states that Transformers excel at global context but “have not been successfully adopted in object density estimation,” whereas CNNs excel at local density modeling but may lack long-range semantic context (Shi et al., 9 Jul 2025). DSGC-Net frames the issue differently, emphasizing density distribution differences between regions and representation inconsistency caused by viewpoint and posture changes (Wu et al., 2 Sep 2025). M-SFANet emphasizes dense versus sparse scenes and the need for both adaptive and static multi-scale context (Thanasutives et al., 2020). These explanations differ in formulation but converge on a single architectural lesson: effective counting decoders often require heterogeneous operators with distinct spatial biases.

This suggests several stable design principles for the topic as a whole. A plausible synthesis is that a dual-stream adaptive counting decoder is most useful when at least one of the following holds: the input representation is semantically rich but spatially coarse, the scene contains simultaneous long-range context and fine local structure, counting is performed across categories or prompts, or the data exhibit strong density and appearance heterogeneity. Within the evidence provided, QUANet’s DAC-decoder is the clearest modern exemplar of this pattern for text-promptable zero-shot class-agnostic counting (Shi et al., 9 Jul 2025), while DSGC-Net, M-SFANet, and the dorsal–ventral zero-shot model show how the same dual-stream principle has been instantiated in graph-based crowd counting, attention-guided density regression, and neurophysiologically inspired sequential counting, respectively (Wu et al., 2 Sep 2025, Thanasutives et al., 2020, Thompson et al., 2024).