---
title: 'Count Anything: A Unified Counting Framework'
url: https://www.emergentmind.com/topics/count-anything
type: topic
---

# Count Anything: A Unified Counting Framework

In the literature surveyed here, “Count Anything” denotes a family of counting problems in which the counted set is not fixed by a single pre-specified class. Instead, the target is specified by exemplars, text, literal assumptions, stream definitions, or combinatorial constraints, and the system returns either a scalar count or a set whose cardinality is the count. The phrase is most visible in class-agnostic and open-world visual counting, but closely related formulations also appear in zero-shot numerosity, semantic-invariant list counting, answer set counting, streaming frequency moments, machine-learning counting engines, and generalized combinatorial counting functions [1811.00472][2605.30846][2311.07233][0802.2305][1301.4550].

## 1. Formal problem formulations

A common thread across these works is that counting is defined by an explicit specification of the target set. In few-shot object counting, the support set is given as $S=\{(I_s^j,b^j)\}_{j=1}^k$, where each $b^j$ is a bounding box around one instance of the target category in support image $I_s^j$; for a query image $I_q$ containing an unknown number $y$ of instances, the goal is to estimate $\hat y$ [2304.10817]. In multi-grained visual counting, the input is an image $I$, visual exemplars $E$, text $T$, and an intended granularity $g$, and the counting function is written as $C(I,E,T,g)\in\mathbb{N}$ [2605.10887]. In text-guided cross-domain counting, the output is an instance-grounded point set $\hat{\mathcal{P}}_T=\{(\hat p_n,\hat s_n)\}_{n=1}^{\hat N}$, with final count $\hat c_T=|\hat{\mathcal{P}}_T|$ [2605.30846]. Outside vision, ASP navigation asks for $|AS(\Pi^A)|$ under assumptions $A$ on literals [2311.07233], stream counting estimates the $p$th frequency moment $F_p(t)=\sum_{i=1}^{D} A_t[i]^p$ [0802.2305], and database-style counting engines answer queries of the form $c(S=s)=\sum_{t=1}^{N}\mathbf{1}\{X_S^{(t)}=s\}$ [1804.04640].

| Setting | Target specification | Count output |
|---|---|---|
| Few-shot object counting | Support set $S=\{(I_s^j,b^j)\}_{j=1}^k$ and query image $I_q$ | Estimate $\hat y$ |
| Multi-grained/open-world visual counting | Image $I$, exemplars $E$, text $T$, granularity $g$ | $C(I,E,T,g)$ or $\hat c_T=|\hat{\mathcal{P}}_T|$ |
| ASP counting under assumptions | Ground ASP program $\Pi$ and assumptions $A$ | $|AS(\Pi^A)|$ |
| Stream moment counting | Turnstile updates to $A_t$ | $F_p(t)=\sum_i A_t[i]^p$ |
| ML contingency counting | Dataset $D$ and equality predicate $S=s$ | $c(S=s)$ |

The output representation differs substantially by domain. Some systems infer counts by summing a density map, some by thresholding localized proposals, some by counting masks or points, and some by exact symbolic enumeration. This suggests that “count anything” is not a single algorithmic template, but a broader design objective: decouple the specification of the target from a fixed, domain-specific detector or class label.

## 2. Exemplar-based class-agnostic visual counting

A foundational visual formulation is class-agnostic counting as matching. “Class-Agnostic Counting” introduced the Generic Matching Network (GMN), which reformulated counting as exemplar matching via image self-similarity rather than class-specific detection [1811.00472]. An exemplar patch $z$ and image $x$ are embedded as $v=\phi(z;\theta_1)$ and $f=\psi(x;\theta_2)$, and a learned relation module predicts a similarity map,
$$
\mathrm{Sim}=\gamma([\mathrm{broadcast}(v):f];\theta_3).
$$
When this similarity map is trained as a density map, the count is obtained by summation; when objects are well separated, local maxima can be counted as detections. GMN also introduced residual adapters for few-shot customization, training only 178K parameters out of 6.0M total.

“Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting” argued that a standard exemplar pipeline depends critically on both representation and similarity metric, and that naive inner-product matching produces noisy similarity maps [2203.08354]. BMNet defined a learnable bilinear similarity
$$
S_{ij}(x,z)=(Px_{ij}+b_x)^\top(Qz+b_z),
$$
while BMNet+ added self-similarity representation enhancement, dynamic exemplar-conditioned channel attention, and a similarity-aware loss. These models remained density-based, with the final count computed as
$$
\hat C=\sum_{u,v} D^{pr}(u,v).
$$
On FSC147, BMNet+ reported Test MAE 14.62 and MSE 91.83, improving over FamNet and BMNet under the reported protocol [2203.08354].

“SQLNet: Scale-Modulated Query and Localization Network for Few-Shot Class-Agnostic Counting” replaced density regression with localization-based counting and made scale a first-class supervisory signal [2311.10011]. SQLNet comprises the Hierarchical Exemplars Collaborative Enhancement (HECE) module, the Exemplars-Unified Query Correlation (EUQC) module, and the Scale-aware Multi-head Localization (SAML) module. It predicts per-instance confidence, center, and size, and counts by thresholding confidences:
$$
C=|\{i\mid p_i\ge \tau\}|.
$$
Its scale-aware localization loss combines weighted BCE for confidence, $\mathcal{L}_{loc}$ for center localization, and $\mathcal{L}_{size}$ for exemplar-driven size regression. On FSC-147, SQLNet reported Val MAE 12.40 and RMSE 42.30, and Test MAE 12.49 and RMSE 80.85, while also producing approximate bounding boxes [2311.10011].

These exemplar-based systems differ primarily in the representation of similarity. GMN uses learned matching over dense features, BMNet/BMNet+ jointly learn feature space and metric, and SQLNet explicitly localizes points and sizes. A plausible implication is that the central difficulty in class-agnostic counting is not only target specification but also the geometry of the matching space: whether the model produces a smooth density, a sparse localization set, or a similarity field strongly affects failure modes and downstream utility.

## 3. Reference-less and text-guided open-world counting

“Learning to Count Anything: Reference-less Class-agnostic Counting with Weak Supervision” removed the test-time reference image entirely and recast counting as repetition recognition [2205.10203]. The method uses a ViT-Small backbone initialized with DINO, flattens globally contextual token features $f=\mathrm{vec}(G)$, and predicts the count with a linear regressor,
$$
\hat c=w^\top f+b.
$$
Training uses only image-level count labels with the Absolute Percentage Error loss,
$$
L=\frac{|c-\hat c|}{c}.
$$
On FSC-147, the method reported Test MAE 17.12 and RMSE 104.53; on the cleaned FSC-133 benchmark it reported Test MAE 14.23 and RMSE 43.83 [2205.10203]. The paper’s central claim is that globally contextual ViT features are sufficient for weakly supervised reference-less enumeration.

“Can SAM Count Anything? An Empirical Study on SAM Counting” examined whether the frozen Segment Anything Model can perform few-shot counting without additional classifiers or detectors [2304.10817]. The pipeline uses SAM’s ViT-H dense features, segments support exemplars from bounding-box prompts, generates query masks from a $32\times 32$ grid of point prompts, converts each mask to a masked-average feature vector, and counts the masks whose cosine similarity to the support prototype exceeds a threshold:
$$
\hat y=\sum_i \mathbf{1}[s^i\ge \tau].
$$
No fine-tuning, NMS, clustering, or IoU-based de-duplication is used. On COCO val2017, SAM reported average MAE 3.87 and RMSE 8.03; on FSC-147 (3-shot), it reported Test MAE 27.97 and RMSE 131.24, with the gap widening on small and crowded scenes [2304.10817].

A more explicit open-world formulation appears in “Count Anything at Any Granularity” [2605.10887]. This work argues that open-world counting is brittle because counting granularity is usually left implicit. It formalizes multi-grained counting with five explicit levels: identity-level, attribute-level, category-level, instance-level, and concept-level. The proposed KubriCount dataset contains 110,507 images, about 7.3M instances, 157 categories, and five granularity levels, while HieraCount combines text and visual exemplars as complementary target specifications. HieraCount reported overall MAE 4.67 and RMSE 11.07 on KubriCount, with L4 remaining the hardest level and reporting MAE 8.37 and RMSE 17.14 [2605.10887].

A closely related but distinct text-guided formulation is given in “Count Anything” [2605.30846]. This work defines text-guided object counting over CLOC, a Cross-domain Large-scale Object Counting dataset with about 220K images, 619 categories, and 15.356M instances across General Scene, Remote Sensing, Histopathology, Cellular Microscopy, Agriculture, and Microbiology. The model outputs discrete instance points rather than density maps, using a Region-level Sparse Counter, a Pixel-level Dense Counter, and Complementary Count Fusion. On the full CLOC test set it reported 9.34 MAE, 33.34 RMSE, and 0.75 NAE [2605.30846].

Taken together, these works show a progression from exemplar-conditioned matching, to weakly supervised repetition recognition, to text-guided and multi-grained prompt following. They also show that removing or broadening the reference mechanism does not remove the need for precise target specification; rather, the specification shifts from a cropped exemplar to a combination of global context, text, and explicit granularity labels.

## 4. Zero-shot numerosity and semantic invariance

A distinct line of work studies whether counting itself generalizes independently of object identity. “Zero-shot counting with a dual-stream neural network model” defines zero-shot counting as correctly reporting how many target items are present when those items were never seen during training [2405.09953]. The model uses 12 saliency-based, log-polar glimpses, a ventral pathway for glimpse contents, a dorsal pathway for fixation coordinates, a recurrent PPC-like module, a 36-unit sigmoid map layer, and a 5-way numerosity readout for counts 1–5. In simple counting, the dual-stream RNN reported validation accuracy 99.0% ± 0.1, OOD shape 98.1% ± 0.2, OOD luminance 98.4% ± 0.2, and OOD both 97.1% ± 0.3; in the ignore-distracters setting it reported OOD both 95.5% ± 0.6 [2405.09953]. The recurrent units developed bell-shaped numerosity tuning better fit by Gaussians in $\log(n)$ than in $n$, consistent with a lognormal code:
$$
r_k(n)=A_k \exp\!\Big(-\frac{(\ln n-\mu_k)^2}{2\sigma_k^2}\Big)+b_k.
$$

At a different level of abstraction, “Semantic Content Determines Algorithmic Performance” argues that counting should be invariant to what is being counted [2601.21618]. It defines a semantic invariance desideratum for a counting operator $f$ over delimited lists:
$$
f(\mathrm{map}(\sigma,L))=f(L),
$$
and measures semantic dependence by
$$
\Delta_{sem}(m)=\max_{e\in E}\mathrm{Acc}(m,e)-\min_{e\in E}\mathrm{Acc}(m,e).
$$
On the WhatCounts benchmark, frontier LLMs exhibited over 40 percentage points of accuracy variation depending solely on semantic category, even when structure and length were held constant [2601.21618]. Explicit separator prompts, XML wrapping, and token-count controls did not eliminate the semantic gap, and agentic tool use increased accuracy while widening the gap.

These two strands use different experimental objects—small visual arrays in one case, delimited text lists in the other—but they converge on a common theoretical issue. Counting generalizes when the system separates content from structure, as in the dorsal/ventral factorization, and fails when semantic content contaminates aggregation, as in WhatCounts. This suggests that “count anything” depends not only on supervision scale but on whether the architecture explicitly isolates the operation of aggregation from the semantics of the operands.

## 5. Exact counting in answer set programming

In symbolic reasoning, “count anything” takes the form of counting solutions of declarative programs under changing assumptions. “IASCAR: Incremental Answer Set Counting by Anytime Refinement” addresses repeated counting of stable models of a ground ASP program under literal assumptions [2311.07233]. The method compiles Clark’s completion of the program into an sd-DNNF, counts supported models under assumptions by conditioning, and then corrects the supported-model overcount through inclusion–exclusion over unsupported cycle constraints:
$$
a_d^L=\sum_{i=0}^{d}(-1)^i\sum_{\Gamma\in\Lambda_i(\Pi)}|SMod(\Pi^{L\cup B(\Gamma)})|.
$$
At full depth $n=|Cycles(\Pi)|$, $a_n^L$ equals the exact answer set count $|AS(\Pi^L)|$. The method is incremental and anytime: after the offline compilation phase, conditioning and recounting are very fast, often milliseconds, while depth-2 inclusion–exclusion often gives useful bounds [2311.07233].

“Exact ASP Counting with Compact Encodings” takes a different route to exactness [2312.11936]. sharpASP constructs Clark’s completion $F=\mathrm{Comp}(\Pi)$ together with a compact copy layer $G=\mathrm{Copy}(\Pi)$ only for loop atoms, and proves the equivalence
$$
M_\tau\in SM(\Pi)\iff \tau\models \mathrm{Comp}(\Pi)\ \text{and}\ G|\tau=\emptyset.
$$
This encoding lets standard #SAT machinery—component decomposition, caching, unit propagation, and knowledge compilation—be reused with minimal changes. On 1470 benchmarks, the hybrid sharpASP solver solved 1062 instances with PAR2 score 3082, compared with 895 instances and PAR2 score 4205 for the previous best hybrid under the same conditions [2312.11936].

The contrast between IASCAR and sharpASP is instructive. IASCAR compiles an overapproximation and corrects it online by inclusion–exclusion; sharpASP builds a compact exact encoding whose residual copy structure captures the stability condition. Both rely on the separation between a tractable compiled representation and a domain-specific correction or semantic test. In this literature, “count anything” therefore means count arbitrary solution sets of the same declarative theory under many queries, not detect arbitrary visual categories.

## 6. Streaming, query-engine, and combinatorial counting

In streaming algorithms, “Count Anything” corresponds to maintaining generic aggregate statistics of a stream rather than enumerating objects. “Compressed Counting” estimates the $p$th frequency moment $F_p(t)=\sum_i A_t[i]^p$ for $0<p\le 2$ in the Turnstile model under the condition that $A_t[i]\ge 0$ at evaluation time [0802.2305]. It uses maximally skewed stable random projections and provides estimators for $F_p$ from projected samples. Near $p=1$, the method achieves sample complexity $k=O(1/\epsilon)$ instead of $O(1/\epsilon^2)$:
$$
\Pr[|\hat F_p-F_p|\le \epsilon F_p]\ge 1-\delta,
$$
with $k\ge G(1/\epsilon^2)\log(2/\delta)$ and $G=O(\epsilon)$ as $p\to 1$. The same sketch supports Rényi and Tsallis entropy estimation, logarithmic norms, logarithmic distances, and method-of-moments statistics [0802.2305].

“Fast Counting in Machine Learning Applications” abstracts counting queries over discrete datasets into a streaming accumulator interface [1804.04640]. The basic query is
$$
c(S=s)=\sum_{t=1}^{N}\mathbf{1}\{X_S^{(t)}=s\},
$$
with applications to BN sufficient statistics and association rule mining. The paper proposes Bitmap and Radix strategies that stream counts to user-defined accumulators rather than materializing heavy-weight indexes. In empirical evaluation, Radix was about 20× faster than hash tables on average in random queries, 5–25× faster than hash tables in Bayesian network learning, and up to 100× faster than ADtrees on high-arity datasets [1804.04640].

“A Counting Function” provides a combinatorial unification of many counting problems through the number of $(n+k)$-insets of a set system with $n$ main blocks of sizes $Q=\{q_1,\dots,q_n\}$ and an additional block of size $m$ [1301.4550]. The counting function
$$
C(m,n,k;Q)
$$
admits the inclusion–exclusion formula
$$
C(m,n,k;Q)=\sum_{I\subseteq [n]}(-1)^{|I|}\binom{m+\sum_{i\in [n]\setminus I} q_i}{n+k}.
$$
The paper shows that this same function counts, among other objects, $(0,1)$-matrices with no zero rows or columns, Delannoy numbers, Catalan-related quantities, crystal ball sequences of the cubic lattice, spanning subgraphs of complete bipartite graphs, squares in a square, and the number of rook, bishop, and queen moves on a chessboard [1301.4550].

These works are methodologically distant from visual counting, but they share a structural aim: define a count operator that can be reused across many targets by changing only the specification of the set being aggregated. In streaming this specification is a moment or sketch; in tabular ML it is a predicate context; in combinatorics it is a block system and inset constraint.

## 7. Recurrent failure modes and open problems

Across modalities, the main obstacle to “count anything” is not the arithmetic of counting but the fidelity of target specification and instance separation. In frozen SAM counting, the two core issues are mask fragmentation or merging in crowded scenes and class-agnostic ambiguity, producing undercounting for tightly packed small objects and confusions among visually similar categories [2304.10817]. SQLNet explicitly notes that very heavy occlusions and extreme scale variations can still challenge size estimation, and that optional IoU-based NMS may be needed in extremely crowded scenes [2311.10011]. HieraCount reports that L4, the instance-level distinction within a category, remains the hardest granularity [2605.10887]. Count Anything on CLOC identifies fusion edge cases in extremely crowded or heavily occluded scenes and notes residual annotation noise and domain imbalance [2605.30846]. RCC reports that very dense images with more than 1000 instances are a failure mode because the 28×28 token grid under-resolves tiny objects [2205.10203].

A second recurrent problem is semantic grounding. Fine-grained prompt following fails systematically in multi-grained open-world counting, especially when positive and negative sets differ by attributes or near-neighbor instance types [2605.10887]. In language-only list counting, WhatCounts shows that aggregation itself can depend strongly on semantic category, even when item boundaries are unambiguous and explicit [2601.21618]. The failure is therefore not reducible to low-level segmentation alone; it can also arise from argument-dependent aggregation.

A third problem is computational complexity. IASCAR amortizes expensive compilation over repeated queries, but its online inclusion–exclusion stage remains worst-case exponential in the number of positive cycles [2311.07233]. sharpASP inherits the #P-completeness of exact ASP counting and can lose efficiency on hard-to-decompose instances [2312.11936]. Compressed Counting achieves strong guarantees only under the evaluation-time nonnegativity restriction and for $0<p\le 2$ [0802.2305]. Fast counting engines for ML workloads assume discrete features and equality predicates, not arbitrary relational or continuous queries [1804.04640].

Taken together, these results suggest that robust “count anything” systems require three aligned components. The target set must be specified explicitly, whether by exemplars, text, assumptions, or predicates; the output representation must preserve enough instance structure to support aggregation; and the gap between tractable approximation and target semantics must be corrected, whether by supervised localization, granularity-aware prompting, inclusion–exclusion, compact encodings, deterministic counting tools, or sketching guarantees. That synthesis is a plausible interpretation of the surveyed literature rather than a single stated theorem, but it captures the most consistent pattern across these otherwise disparate lines of work.

Source: https://www.emergentmind.com/topics/count-anything