CountCluster: Dual Clustering Methods

• CountCluster is a dual-purpose approach: one variant recovers unknown clusters via adaptive same-cluster queries, while the other partitions attention maps to match specified counts.
• The interactive clustering method employs D²-sampling, rejection sampling, and oracle queries to incrementally identify clusters without presetting the cluster number.
• The text-to-image variant refines early cross-attention maps using Gaussian smoothing and KL-loss minimization to guide object count accuracy in generated images.

CountCluster denotes two distinct methods in the literature, both centered on quantity determination through clustering but operating in different problem domains. In interactive clustering, CountCluster is the name of two algorithms for discovering and counting an unknown number of ground-truth clusters in a point set $X\subset\mathbb{R}^d$ by issuing same-cluster queries to an oracle (Li et al., 2021). In text-to-image generation, CountCluster is a training-free inference-time method that guides an object cross-attention map to be partitioned into $k$ spatially separated regions matching the object count specified in a prompt (Lee et al., 14 Aug 2025). The shared name reflects a common emphasis on recovering or enforcing discrete multiplicity, but the two methods differ fundamentally in assumptions, observables, optimization targets, and theoretical framing.

1. Scope and research context

The need to determine the number of clusters arises across several lines of work. In classical text clustering, the central issue is that many clustering algorithms require the number of clusters to be specified apriori; one response is to vary $k$ and monitor cluster quality measures such as entropy and purity until they stabilize (Grace et al., 2015). Another line uses a cluster ensemble to build a consensus similarity matrix and then identifies the true cluster count from a Perron cluster of eigenvalues near 1 in an associated random-walk matrix, with iterative refinement for noisy or high-dimensional data (Race et al., 2014). A further alternative targets heavily overlapping mixtures by introducing a quantum path-integral formulation in which interference suppresses spurious merged modes and reveals the number of clusters $K$ from peaks in $P_q(y)$ (Muthyala et al., 2020).

Within this broader landscape, the CountCluster algorithms of "Learning to Cluster via Same-Cluster Queries" remove two assumptions that are common in earlier work: they do not assume that the total number of clusters is known at the beginning and do not require that the true clusters are consistent with a predefined objective function such as the K-means (Li et al., 2021). The CountCluster method of "Training-Free Object Quantity Guidance with Cross-Attention Map Clustering for Text-to-Image Generation" addresses a different counting problem: diffusion models often fail to generate images that accurately reflect the number of objects specified in the input prompt, and the method exploits the observation that the number of object instances is largely determined in the early timesteps of the denoising process (Lee et al., 14 Aug 2025).

CountCluster usage	Domain	Core mechanism
CountCluster	Learning to cluster	Same-cluster queries, $D^2$-sampling, centroid recovery
CountCluster	Text-to-image generation	Cross-attention map clustering, KL loss, latent refinement

This suggests that "CountCluster" is not a single unified framework but a reused name for two technical programs: one estimates an unknown $K$ from oracle-mediated structure in $\mathbb{R}^d$, and the other imposes a desired $k$ on generative attention dynamics.

2. Oracle-based CountCluster: problem formulation

In the clustering setting, one is given a finite set $X\subset\mathbb{R}^d$ of $k$0 points and an unknown ground-truth partition $k$1, with $k$2 itself unknown (Li et al., 2021). For each true cluster $k$3, the centroid is

$k$4

The available supervision is an oracle for same-cluster queries,

$k$5

which returns "Yes" if $k$6 and $k$7 lie in the same $k$8, and "No" otherwise.

The theoretical stopping condition is defined through a reducibility notion. Let $k$9 be any subset of recovered cluster-indices and let $k$0 be their approximate centroids. The squared-$k$1 coverage cost is

$k$2

The true clustering is said to be $k$3-reducible with respect to $k$4 if for every $k$5,

$k$6

The stated intuition is that if the remaining clusters can already be covered by the recovered ones at tolerable extra cost, recovery may stop (Li et al., 2021).

This formulation is structurally different from methods that infer $k$7 from objective-function elbows or eigengaps. CountCluster does not begin with a fixed candidate $k$8, and it does not define success as optimizing a global partition objective over all points. Instead, it incrementally discovers clusters through adaptive sampling and oracle queries until no new cluster label is found.

3. Algorithmic structure of Basic CountCluster and Improved CountCluster

The first algorithm, Basic CountCluster, runs in rounds $k$9, maintaining the set $K$0 of recovered cluster-indices and an approximate centroid $K$1 for each $K$2, with

$K$3

Each round has four phases (Li et al., 2021).

In Phase 1, Discover, the algorithm draws

$K$4

points by $K$5-sampling with respect to $K$6, classifies each new sample via same-cluster queries, and terminates if no new cluster appears. Otherwise the set $K$7 contains the newly discovered labels. In Phase 2, Identify largest, it continues $K$8-sampling until for each $K$9 it has seen at least

$P_q(y)$0

samples in cluster $P_q(y)$1, then selects $P_q(y)$2 with the largest empirical sample count. In Phase 3, Pivot selection, it continues sampling until it sees

$P_q(y)$3

total points and chooses a pivot $P_q(y)$4 among samples labeled $P_q(y)$5 minimizing $P_q(y)$6. In Phase 4, Rejection sampling + centroid, when a sampled point $P_q(y)$7 is classified into $P_q(y)$8, it is accepted with probability

$P_q(y)$9

After collecting

$D^2$0

such points for cluster $D^2$1, the algorithm sets $D^2$2 to their empirical mean and adds $D^2$3 to $D^2$4 (Li et al., 2021).

Improved CountCluster preserves the same high-level loop but recovers a whole batch of new clusters $D^2$5 instead of a single $D^2$6 (Li et al., 2021). After collecting a moderate number of $D^2$7 samples, it computes empirical frequencies $D^2$8 for each discovered $D^2$9, partitions $K$0 into logarithmic bands by

$K$1

and declares a band heavy if the sum of frequencies in the band is at least $K$2. All clusters in heavy bands form $K$3; pivots $K$4 are selected for every $K$5, rejection sampling is performed simultaneously, and $K$6 is batch-added to $K$7. To cope with unknown true $K$8, the algorithm outer-loops over guesses $K$9, accepting rounds while $\mathbb{R}^d$0 and stopping when no new clusters appear and $\mathbb{R}^d$1.

Both algorithms return

$\mathbb{R}^d$2

They therefore treat cluster counting as a byproduct of certified cluster recovery rather than as a standalone model-selection score.

4. Theoretical guarantees and practical behavior of oracle-based CountCluster

The Basic algorithm has the following guarantee: with probability at least $\mathbb{R}^d$3, upon termination it returns a set $\mathbb{R}^d$4 satisfying $\mathbb{R}^d$5-reducibility, and for each $\mathbb{R}^d$6,

$\mathbb{R}^d$7

Its oracle-query complexity is

$\mathbb{R}^d$8

where $\mathbb{R}^d$9 is the number of recovered clusters and $k$0 the total number of discovered but not necessarily recovered clusters (Li et al., 2021).

The Improved algorithm has the same success probability and approximate-centroid guarantees, with query complexity

$k$1

The stated source of the $k$2 factor is the outer loop over guesses $k$3, and the paper notes that this factor can be removed at the cost of more careful bookkeeping (Li et al., 2021). Concentration is controlled phasewise by additive and multiplicative Chernoff bounds, with insufficient new-cluster mass bounded by

$k$4

and bad center approximation bounded by

$k$5

A union bound over rounds yields total failure at most $k$6.

The count estimate is exact under the appropriate structural condition. Both algorithms terminate precisely when no new cluster label is discovered in Phase 1, and under the $k$7-reducibility assumption the true number of clusters $k$8 satisfies $k$9. More generally, the algorithms recover all clusters whose coverage costs exceed the $X\subset\mathbb{R}^d$0-threshold; the remaining clusters are covered by the recovered ones. Thus $X\subset\mathbb{R}^d$1 is zero when the entire true set is $X\subset\mathbb{R}^d$2-irreducible (Li et al., 2021).

Experiments on synthetic data with million-point, power-law cluster sizes and varying collision parameter $X\subset\mathbb{R}^d$3, and on real data including Shuttle and KDD99, show that the basic and improved algorithms recover more clusters under the same query budget than uniform sampling; approximate-centroid error is below $X\subset\mathbb{R}^d$4 in all settings; the batch version runs in fewer rounds and less wall-clock time; and a nearest-approximate-center heuristic classifies new points correctly with just one extra query in approximately $X\subset\mathbb{R}^d$5–$X\subset\mathbb{R}^d$6 of cases (Li et al., 2021). A plausible implication is that CountCluster is designed for query efficiency under weak structural assumptions rather than for unsupervised operation without interaction.

5. CountCluster for text-to-image generation: attention-space formulation

In diffusion-based text-to-image generation, CountCluster is a training-free object quantity guidance method based on clustering object cross-attention maps at inference time (Lee et al., 14 Aug 2025). The latent at timestep $X\subset\mathbb{R}^d$7 is denoted $X\subset\mathbb{R}^d$8. The prompt contains an object token $X\subset\mathbb{R}^d$9 with desired count $k$00. At each timestep $k$01, the cross-attention layer for token $k$02 produces a two-dimensional attention map

$k$03

where $k$04 is the attention weight on spatial patch $k$05 for token $k$06.

The method extracts $k$07 from one or more early layers of the U-Net, with examples given as layers at $k$08, and applies Gaussian smoothing with kernel size $k$09 and $k$10, min-max normalization to $k$11, and thresholding at $k$12, for example $k$13, to discard low-attention patches (Lee et al., 14 Aug 2025). The set of early timesteps where clustering is applied is

$k$14

with the example $k$15.

The goal is to partition the set of candidate patch coordinates

$k$16

into exactly $k$17 clusters, so that each cluster corresponds to one object. Cluster centers $k$18 are selected by a greedy max-min procedure: patches are sorted by descending attention score, and a patch is accepted as a center if its Euclidean distance to all previously chosen centers is at least

$k$19

The assignments are then defined by nearest center,

$k$20

The stated role of $k$21 is to enforce roughly one object per $k$22 distance (Lee et al., 14 Aug 2025).

This formulation operationalizes the claim that the number of object instances in the generated image is largely determined in the early timesteps of the denoising process, and that the highly activated regions in the object cross-attention map at those timesteps should match the input object quantity while remaining clearly separated (Lee et al., 14 Aug 2025).

6. Target distribution, clustering loss, and inference-time latent refinement

For each cluster $k$23, CountCluster defines an ideal distribution in which attention forms a single blob decaying from center $k$24 to a boundary where the ideal attention equals $k$25 (Lee et al., 14 Aug 2025). With $k$26 and radius $k$27 given by the distance from $k$28 to the farthest assigned patch, or simply $k$29, the standard deviation is

$k$30

The target Gaussian is

$k$31

The normalized actual attention on patches assigned to cluster $k$32 is

$k$33

Similarity is measured by

$k$34

The clustering loss is the sum of these divergences normalized by $k$35: $k$36 The stated reason for the $k$37 normalization is that when $k$38 increases, each cluster’s area shrinks and its individual KL tends to drop, whereas dividing by $k$39 keeps the loss scale roughly independent of $k$40 and allows the overall penalty to rise if clustering fails (Lee et al., 14 Aug 2025).

Inference interleaves standard diffusion denoising with gradient steps on the latent $k$41. At each $k$42, the gradient

$k$43

is computed and the latent is updated as

$k$44

Example step sizes are $k$45 for SDXL and $k$46 for SD2.1, and refinement stops when $k$47 drops below thresholds such as $k$48 at $k$49 and $k$50 at $k$51 (Lee et al., 14 Aug 2025).

The full inference algorithm begins by tokenizing the prompt, running the text encoder, and initializing $k$52. For each timestep $k$53, the method computes the standard DDIM or ancestral update, and if $k$54, it extracts and normalizes $k$55, smooths and thresholds it at $k$56, selects centers, assigns patches, constructs Gaussian targets, forms $k$57, and performs the gradient step. The final latent $k$58 is then decoded to an image (Lee et al., 14 Aug 2025).

A common misconception would be to treat this CountCluster as a conventional clustering algorithm for unlabeled datasets. The paper’s formulation is narrower: it is an inference-time control method for quantity alignment in diffusion models, and its "clusters" are clusters of attention patches rather than clusters of input data points.

7. Empirical results, limitations, and relation to other count-estimation methods

For text-to-image generation, the reported models are Stable Diffusion 2.1 and SDXL. The prompt set is “A photo of [count] [object]” with $k$59 and $k$60 object categories, yielding $k$61 prompt variants and $k$62 images over $k$63 seeds (Lee et al., 14 Aug 2025). Clustering hyperparameters are $k$64, $k$65, and $k$66, with refinement thresholds $k$67. Accuracy, MAE, and RMSE are evaluated using predictions by CountGD and TIFA.

On SD2.1, the reported values are $k$68 accuracy, $k$69 MAE, and $k$70 RMSE for the baseline SD2.1; $k$71, $k$72, and $k$73 for Counting Guidance; and $k$74, $k$75, and $k$76 for CountCluster. On SDXL, the values are $k$77, $k$78, and $k$79 for SDXL; $k$80, $k$81, and $k$82 for Zafar et al.; $k$83, $k$84, and $k$85 for CountGen; and $k$86, $k$87, and $k$88 for CountCluster (Lee et al., 14 Aug 2025). The paper states that Ours(SDXL) beats CountGen by $k$89p in accuracy and by $k$90p over plain SDXL, and that TIFA VQA scores improve by $k$91p versus baselines. Ablation on SDXL reports $k$92 accuracy without the min-distance constraint and $k$93 accuracy without $k$94 scaling (Lee et al., 14 Aug 2025).

Inference cost is also reported: on SDXL, $k$95 s and $k$96 GB for SDXL, $k$97 s and $k$98 GB for CountGen, and $k$99 s and $k$00 GB for CountCluster, with CountGen using $k$01 GPU and the others using a single $k$02 (Lee et al., 14 Aug 2025). This suggests that the method targets a trade-off between count accuracy and inference overhead while remaining training-free and external-tool-free.

For the oracle-based clustering CountCluster, the main limitations arise from its assumptions and oracle dependence. Its guarantees are conditional on $k$03-reducibility, and its operation requires access to same-cluster queries (Li et al., 2021). For the diffusion CountCluster, the method presumes access to internal cross-attention maps at selected early timesteps and depends on hyperparameters such as $k$04, $k$05, timestep selection, and latent step sizes (Lee et al., 14 Aug 2025). In that sense, the two CountCluster methods occupy distinct methodological niches. One is an active learning algorithm with provable query complexity and stopping guarantees; the other is an inference-time control mechanism that shapes attention geometry to enforce an externally specified quantity.

Relative to broader count-estimation approaches, these methods illustrate two general paradigms. Classical cluster-number estimation via entropy and purity stabilization, consensus spectral gaps, or quantum interference derives $k$06 from structural signatures in data representations [(Grace et al., 2015); (Race et al., 2014); (Muthyala et al., 2020)]. CountCluster in the oracle setting instead discovers $k$07 through adaptive querying (Li et al., 2021). CountCluster in text-to-image generation does not estimate a latent unknown $k$08; it conditions generation on a known target count and uses clustering as a control primitive (Lee et al., 14 Aug 2025). The commonality is therefore conceptual rather than algorithmic: in both cases, clustering is used as the mechanism by which multiplicity becomes explicit.