QUANet: Zero-Shot Object Counting Model

• QUANet is a text-promptable object counting model that counts objects in images via detailed numerical prompts and density map predictions.
• It employs a vision-text alignment loss with both factual and counterfactual prompts to enforce a structured, ordinal similarity for accurate counting.
• The dual-stream adaptive counting decoder fuses Transformer global context with CNN local details, enhancing performance across varied datasets.

Searching arXiv for QUANet and similarly named methods to ground the article in the relevant papers. QUANet is a text-promptable object counting model introduced for zero-shot class-agnostic counting. It addresses the setting in which an image $I \in \mathbb{R}^{H \times W \times 3}$ and a text prompt specifying a category $C$ are given, and the model predicts a density map $D^* \in \mathbb{R}^{H \times W}$ whose integral equals the number of objects of class $C$ in the image. The method combines quantity-oriented text prompts, a vision-text quantity alignment loss, and a dual-stream adaptive counting decoder with Transformer-to-CNN enhancement adapters in order to improve quantity awareness beyond category-only prompting (Shi et al., 9 Jul 2025).

1. Problem setting and motivation

QUANet is situated in the line of work on text-promptable object counting. In the formulation adopted by the paper, the task is to count objects of a prompted category without requiring exemplar patches. Traditional class-agnostic counters use visual exemplars to indicate the target category, but this requires manual annotation and limits flexibility. Recent zero-shot methods instead leverage large pre-trained VLMs such as CLIP and simple category-only prompts, for example “a photo of dogs,” together with a standard vision-text contrastive loss. According to the paper, these methods can achieve class-agnostic counting, but they do not teach the model to discriminate absolute object counts; they only distinguish presence from absence (Shi et al., 9 Jul 2025).

A central claim of QUANet is that category information alone is insufficient for counting supervision. The paper also contrasts its approach with a recent work, “Teaching CLIP to Count to Ten,” which inserts fixed numbers into prompts but provides only a coarse binary supervisory signal, described as matched versus mismatched up to ten, while neglecting fine-grained numerical differences beyond 10. This motivates the introduction of explicit numerical variation in the textual input and losses that enforce ordering over numerical alternatives.

This framing suggests that QUANet is not merely a decoder modification; it is a reformulation of the supervision signal used in text-promptable counting. The architecture treats counting as a joint vision-language alignment problem in which the numeric token is semantically consequential rather than incidental.

2. Quantity-oriented prompting and vision-text alignment

QUANet injects detailed numerical variation into textual supervision through a prompt template of the form “a photo of [num] [class].” The model uses two prompt types during training. A factual prompt $P^f$ replaces $[num]$ with the ground-truth count $a^p$, and counterfactual prompts $\{P^n_i\}_{i=1\ldots N}$ replace it with $a^p \pm k\Delta$, where $k=1\ldots N/2$ and $C$0 is adaptively chosen based on $C$1. The example given is $C$2 for “kiwis,” with factual prompt “a photo of 14 kiwis” and counterfactuals $C$3 (Shi et al., 9 Jul 2025).

During training, each factual prompt forms one positive vision-text pair and each counterfactual prompt forms a negative pair. These prompts are encoded to produce quantity-aware text embeddings $C$4. A separate category embedding $C$5 is formed by dropping the $C$6-token embedding from $C$7. This separation allows the model to use count-aware text for alignment while retaining category-prompted features for downstream density estimation.

The vision-text quantity alignment loss imposes an explicit order on similarity scores. Let

$C$8

The loss is

$C$9

The first term enforces $D^* \in \mathbb{R}^{H \times W}$0 for all negatives, while the second term enforces an ordering among negatives so that prompts with counts closer to $D^* \in \mathbb{R}^{H \times W}$1 have higher similarity than prompts farther away. This is more stringent than ordinary contrastive supervision because it encodes relative numerical proximity, not only positive-negative separation.

A plausible implication is that the method converts numerical counting supervision into an ordinal structure over text-image similarity scores, thereby exploiting the ranking capacity of contrastive VLM features rather than relying only on regression in the density head.

3. Dual-stream Adaptive Counting decoder

QUANet’s density estimator is the Dual-stream Adaptive Counting, or DAC, decoder. It consists of a Transformer stream, a CNN stream, and Transformer-to-CNN enhancement adapters. The decoder operates on the category-prompted feature $D^* \in \mathbb{R}^{H \times W}$2, which is obtained by cross-attention between image features and the category embedding $D^* \in \mathbb{R}^{H \times W}$3.

The Transformer stream is described as a small stack of Vision Transformer decoder blocks that process $D^* \in \mathbb{R}^{H \times W}$4 to capture global context, producing an intermediate feature $D^* \in \mathbb{R}^{H \times W}$5 that is projected by a $D^* \in \mathbb{R}^{H \times W}$6 convolution to a density map $D^* \in \mathbb{R}^{H \times W}$7. In parallel, a lightweight convolutional decoder processes $D^* \in \mathbb{R}^{H \times W}$8 to preserve spatial locality and fine details, producing $D^* \in \mathbb{R}^{H \times W}$9 and then $C$0 via another $C$1 convolution (Shi et al., 9 Jul 2025).

The T2C adapter transfers global signals from the Transformer branch into the CNN branch through two sub-blocks. The first is Cross-Attention:

$C$2

The second is Channel-Excitation. A channel weight vector is obtained by

$C$3

followed by

$C$4

The adapter output is

$C$5

and the CNN feature is updated as

$C$6

The paper emphasizes that this is a unidirectional information flow, intended to avoid corrupting the Transformer’s contextual cues.

Final prediction uses a gating net that fuses the two density maps pixel-wise with learned weights:

$C$7

The accompanying GAME analysis reports that the Transformer excels at global but lags local, while the CNN shows the opposite pattern; the fused decoder outperforms both. This supports the architectural premise that global context and local detail should be modeled in parallel rather than forced into a single branch.

4. Cross-stream quantity ranking and optimization

In addition to the alignment term, QUANet introduces a cross-stream quantity ranking loss to regularize the two decoder branches. Each predicted density map is partitioned into $C$8 non-overlapping patches. Let

$C$9

denote the predicted counts in descending-GT-order. For a patch interval $P^f$0 with default $P^f$1, the loss is

$P^f$2

The first two terms couple the two streams, while the last two terms enforce monotonic ranking within each stream (Shi et al., 9 Jul 2025).

The forward-pass summary in the paper combines prompt generation, embedding extraction, vision-text alignment, category-prompted feature construction, DAC decoding, and a final objective

$P^f$3

where

$P^f$4

This design places quantity structure in two locations simultaneously: at the vision-text interface through $P^f$5 and inside the density decoder through $P^f$6. This suggests that QUANet treats counting as both semantic discrimination over numbers and structured prediction over spatial mass.

5. Training protocol and empirical results

The reported training protocol uses FSC-147 for training, with 6,135 images from 147 categories, and zero-shot evaluation without fine-tuning on FSC-147 val/test, CARPK, PUCPR+, and ShanghaiTech (SHA/SHB). The vision encoder is DINOv2 ViT-B/14 with the last 6 layers fine-tuned, while the text encoder is BERT-base with the top 9 blocks frozen. The counterfactual interval $P^f$7 is chosen per ground-truth bin as $P^f$8. The paper further specifies $P^f$9 negative prompts, image size $[num]$0, patches $[num]$1, loss-balance $[num]$2, and optimization with AdamW, learning rate $[num]$3, weight decay $[num]$4, batch size $[num]$5, and $[num]$6 epochs on A40 GPU (Shi et al., 9 Jul 2025).

On the FSC-147 test set among text-promptable methods, QUANet reports $[num]$7 and $[num]$8. The prior best CountDiff is listed at $[num]$9 and $a^p$0, while a baseline using the same encoders and a CNN decoder reports $a^p$1 and $a^p$2. For cross-dataset zero-shot generalization, the paper reports CARPK $a^p$3, PUCPR+ $a^p$4, ShanghaiTech-SHA $a^p$5, and SHB $a^p$6.

The ablation study attributes performance gains to all major components. On FSC-147 test MAE, removing quantity-oriented text prompts and $a^p$7 gives $a^p$8; removing the factual prompt gives $a^p$9; removing T2C adapters gives $\{P^n_i\}_{i=1\ldots N}$0; removing $\{P^n_i\}_{i=1\ldots N}$1 gives $\{P^n_i\}_{i=1\ldots N}$2; using within-stream-only ranking gives $\{P^n_i\}_{i=1\ldots N}$3; and using the alignment-only term without ranking among negatives gives $\{P^n_i\}_{i=1\ldots N}$4. These results indicate that both the prompt design and the dual-stream regularization contribute measurably.

Qualitatively, the model is reported to produce high-quality density maps under heavy clutter. The listed failure modes are severe occlusion or unusual object grouping, with eyeglasses counted as separate lenses given as an example.

6. Generalizability, limitations, and nomenclature

The paper states that the DAC decoder can replace existing CNN decoders, including those in CLIP-Count and VLCounter, to yield consistent improvements. It also states that QUANet demonstrates strong zero-shot cross-dataset performance for category-agnostic counting, and code is available at https://github.com/viscom-tongji/QUANet (Shi et al., 9 Jul 2025).

The limitations identified in the paper are specific. Severe occlusion or stacking remains challenging, as do atypical part-based counts such as eyeglasses. Counting extremely large crowds, described as $\{P^n_i\}_{i=1\ldots N}$5, may require finer numerical granularity or hierarchical prompts. The future directions proposed in the paper include explicit ordinal reasoning modules, object prototypes or bounding-box proposals for extreme overlap, end-to-end adaptation of $\{P^n_i\}_{i=1\ldots N}$6-intervals or prompt templates, fusion with object-detection priors for part-versus-whole disambiguation, and extension of quantity-awareness losses to video counting.

The name QUANet should also be distinguished from visually similar model names in other areas. “QuantNet” denotes a meta-based quantizer for network binarization that maps full-precision weights to binarized weights within a fully differentiable framework and avoids the Straight-Through Estimator (Liu et al., 2020). “QuaNet” denotes a recurrent neural network for sentiment quantification that learns a quantification embedding from ranked classifier confidences and auxiliary quantification statistics (Esuli et al., 2018). These methods address different problem domains—object counting, neural network quantization, and class-prevalence estimation—even though their names are orthographically similar.

In that sense, QUANet occupies a specific position within vision-language counting: it is a text-promptable counting architecture that operationalizes quantity awareness through prompt construction, similarity ordering, and cross-stream density ranking, rather than through exemplar matching or category-only prompting.