Anchor-Vertex Pooling: Methods & Applications

• Anchor-vertex pooling is a design pattern that uses representative anchors to aggregate local evidence into coarser, task-relevant summaries.
• It is applied in areas such as graph pooling, video temporal grounding, scene text recognition, and anchor-free detection through varied mechanisms like supernodes and anchor tokens.
• Pooling strategies leverage explicit or learned anchor selection with optimization objectives (contrastive, focal, and recognition losses) to enhance accuracy and efficiency.

Anchor-Vertex Pooling denotes a family of pooling mechanisms in which representative anchors, anchor sets, or anchor tokens are used to aggregate information from finer units and to produce coarser, task-relevant summaries. Across the cited literature, this role is instantiated by supernodes in graph pooling, cluster-induced meta-vertices in simplicial coarsening, pooled temporal anchors in video temporal grounding, character anchors in scene text recognition, anchor tokens in text embedding, and pooled feature-map vertices in anchor-free detection (Grattarola et al., 2021, McGuire et al., 2023, An et al., 27 Oct 2025, Long et al., 2020, Pan et al., 31 Aug 2025, Samet et al., 2020). The term itself is not used uniformly: several works instead describe representative nodes, supernodes, anchor sets, or anchor tokens, but they share the same operational pattern of selecting or constructing anchors, reducing local evidence onto them, and using the resulting coarse representation for downstream reasoning.

1. Terminology and formal abstraction

A common formalization of pooling in graph-structured settings is the Select–Reduce–Connect framework, in which pooling is written as

$$S=\{S_1,\dots,S_K\}=\mathrm{Select}(G), \qquad X'=\{\mathrm{Reduce}(G,S_k)\}_{k=1,\dots,K}, \qquad E'=\{\mathrm{Connect}(G,S_k,S_l)\}_{k,l=1,\dots,K}.$$

In matrix form, selection is typically represented by an assignment matrix $S \in \mathbb{R}^{n \times k}$ whose entries encode hard or soft membership of original nodes in pooled supernodes (Grattarola et al., 2021). This formulation captures the most direct graph interpretation of anchor-vertex pooling: anchors are representative vertices or prototypes, and the pooled graph is built by assigning the original nodes to these anchor-centered supernodes.

Several cited works broaden this abstraction beyond ordinary graphs. NervePool treats learned vertex clusters $U_i$ as “anchor sets,” then extends them to the full simplicial complex through unions of vertex stars, producing a pooled complex by a nerve construction (McGuire et al., 2023). HieraMamba states that it does not use the word “vertex” explicitly, but frames, clips, or tokens can be viewed as vertices in a temporal graph, while anchors function as representative vertices summarizing fixed-size temporal neighborhoods (An et al., 27 Oct 2025). ATA pooling defines anchor tokens as positions whose final-layer self-attention behavior indicates that they aggregate information from many other tokens, and then weights the final embedding toward those tokens (Pan et al., 31 Aug 2025). CAPNet similarly uses character anchors as localized positions on a feature map, then pools along the Character Anchor Line to form a sequence (Long et al., 2020).

This suggests a unifying interpretation: anchor-vertex pooling is less a single algorithm than a design pattern in which local evidence is first attached to representative units and then propagated through a coarsened computational graph, sequence, or hierarchy.

2. Graph and simplicial constructions

In graph pooling, the canonical coarsening formulas are

$X' = S^\top X, \qquad A' = S^\top A S,$

where $X$ is the node-feature matrix, $A$ the adjacency, and $S$ the assignment matrix (Grattarola et al., 2021). When $S$ is sparse and anchor-centered, this implements a direct anchor-vertex reduction: each anchor corresponds either to a selected node or to a cluster index, and the pooled node features and adjacency are induced by aggregating the nodes assigned to that anchor.

NervePool generalizes this idea from graphs to simplicial complexes. Let $U_i$ be a vertex cluster. NervePool extends the vertex cover to the entire complex by

$\widetilde{U}_i = \bigcup_{v \in U_i} St(v),$

and defines the pooled complex as

$S \in \mathbb{R}^{n \times k}$0

Feature pooling and structural coarsening are then expressed dimension-wise as

$S \in \mathbb{R}^{n \times k}$1

Under a hard partition, the paper proves equivalence between the topological nerve construction and the matrix implementation, proves an identity-function property for a singleton-vertex cover, and proves permutation invariance with respect to vertex permutations (McGuire et al., 2023).

MIVSPool provides an explicitly anchor-centered graph construction. It selects surviving vertices by a Maximal Independent Vertex Set and assigns every remaining vertex to one of those survivors. If $S \in \mathbb{R}^{n \times k}$2 denotes the resulting one-hot assignment matrix, the pooled graph is again

$S \in \mathbb{R}^{n \times k}$3

while features are aggregated by a score-weighted average inside each anchor-centered cluster. Because every non-surviving vertex is assigned to exactly one survivor, the method does not discard node information; because the assignment is hard, it avoids the near-complete pooled graphs associated with dense soft-clustering methods (Stanovic et al., 2022).

A recurring contrast in the literature is therefore between explicit anchor selection and implicit anchor formation. Selection-based methods choose representative nodes directly; clustering-based methods let cluster indices act as anchors; simplicial methods use anchor sets and cover intersections; and MIS-based methods impose a structural coverage constraint on the anchor set itself.

3. Anchor formation across temporal, spatial, and token domains

In HieraMamba, Anchor-MambaPooling (AMP) forms anchors from temporal windows. Given stride $S \in \mathbb{R}^{n \times k}$4 and current-layer tokens $S \in \mathbb{R}^{n \times k}$5, anchors are initialized by mean pooling over local windows:

$S \in \mathbb{R}^{n \times k}$6

The anchor-interleaved sequence is then

$S \in \mathbb{R}^{n \times k}$7

This deterministic layout places each anchor immediately before the frames it summarizes, after which Hydra performs bidirectional selective scanning over the interleaved sequence, a local windowed Transformer adds fine-grained attention, and a learnable sigmoid gate fuses the branches. The outputs are refined current-resolution tokens $S \in \mathbb{R}^{n \times k}$8 and the next coarser anchor set $S \in \mathbb{R}^{n \times k}$9 (An et al., 27 Oct 2025).

In scene text recognition, CAPNet couples a Character Anchoring Module with an Anchor Pooling Module. CAM predicts a dense anchor heat map on shared feature maps; adjacent positives are grouped into connected components; the midpoint of each component becomes a character anchor; the anchors are sorted left-to-right; and a Character Anchor Line is formed by the resulting polyline. APM then samples a fixed number of markers along this line and bilinearly interpolates features from recognition maps, producing an ordered sequence suitable for BiLSTM encoding and attentional decoding (Long et al., 2020).

ATA pooling uses attention-derived anchor scores rather than geometric neighborhoods. Let $U_i$0 be the final-layer attention matrix of head $U_i$1, and let $U_i$2 be the final-layer token embeddings. The unnormalized anchor score is

$U_i$3

the normalized weight is

$U_i$4

and the pooled embedding is

$U_i$5

The method adds no learnable parameters; anchors are inferred directly from the model’s own final-layer attention patterns (Pan et al., 31 Aug 2025).

In anchor-free detection, PPDet treats feature-map locations inside a positive central region as the pooled set. For class probabilities $U_i$6 at locations $U_i$7, classification pooling is uniform sum-pooling:

$U_i$8

The object-level classification loss is applied only to the pooled score, while regression remains location-wise (Samet et al., 2020).

Across these domains, anchor formation alternates between explicit geometry, structural constraints, content-aware local averaging, and attention-derived centrality. The aggregation operator may therefore be a cluster assignment, a temporal window pool, a polyline sampler, an attention-weighted token sum, or a sum over positive detector vertices.

4. Optimization objectives and decoding strategies

In HieraMamba, anchor formation is supervised by two contrastive objectives applied at each layer. The anchor-conditioned contrastive loss uses an anchor as query, its own refined window tokens as positives, and distant anchors as negatives:

$U_i$9

The segment-pooled contrastive loss pools refined tokens from a ground-truth interval into a segment prototype $X' = S^\top X, \qquad A' = S^\top A S,$0 and contrasts in-interval against out-of-interval tokens. The combined contrastive objective is

$X' = S^\top X, \qquad A' = S^\top A S,$1

At inference, the multi-scale pyramid is fused with text by cross-attention, each scale outputs classification scores and normalized start/end offsets, proposals are ranked across scales, Soft-NMS merges overlaps, and top-$X' = S^\top X, \qquad A' = S^\top A S,$2 moments are returned (An et al., 27 Oct 2025).

CAPNet uses a joint localization and recognition objective. Pixel-level anchor localization is supervised by binary cross-entropy,

$X' = S^\top X, \qquad A' = S^\top A S,$3

recognition is supervised by sequence negative log-likelihood,

$X' = S^\top X, \qquad A' = S^\top A S,$4

and the joint loss is

$X' = S^\top X, \qquad A' = S^\top A S,$5

The anchor-pooled sequence is then decoded by a one-layer BiLSTM encoder and an attentional unidirectional LSTM decoder (Long et al., 2020).

ATA is integrated into a standard contrastive InfoNCE objective,

$X' = S^\top X, \qquad A' = S^\top A S,$6

with cosine similarity $X' = S^\top X, \qquad A' = S^\top A S,$7 and $X' = S^\top X, \qquad A' = S^\top A S,$8. Because ATA weights are functions of final-layer attention probabilities, gradients flow through the attention computation, but no additional loss term or regularizer is introduced for the anchor weights themselves (Pan et al., 31 Aug 2025).

PPDet uses pooled focal loss for positive objects,

$X' = S^\top X, \qquad A' = S^\top A S,$9

standard focal loss on negatives, and location-wise Smooth L1 regression:

$X$0

At inference, same-class detections with IoU above $X$1 mutually boost one another’s scores by a factor involving $X$2 before class-aware NMS (Samet et al., 2020).

A shared theme is that anchor-vertex pooling is rarely only a reduction operator. It is usually paired with objectives that enforce either compactness, discriminability, sequence alignment, or noise robustness, and inference often reuses the anchor structure through proposal decoding, attention-based decoding, or overlap-aware voting.

5. Expressiveness, invariance, and structural guarantees

The most explicit expressiveness analysis appears in the graph-pooling literature. For a pooling operator placed after $X$3 message-passing layers, sufficient conditions for preserving the discriminative power of the preceding message-passing network are: every node contributes to supernodes with a fixed total membership

$X$4

and reduction is of the form

$X$5

Under these conditions, together with an expressive preceding message-passing stack, pooled graphs remain WL-distinguishable up to row permutation of the pooled feature matrix (Bianchi et al., 2023). The same exposition states that anchor-vertex pooling fails these sufficient conditions when it is implemented as anchor selection plus node dropping, because non-selected nodes then have row-sum $X$6 rather than $X$7.

NervePool contributes topological and combinatorial guarantees. Under hard partitions, the topological nerve/cover construction and the matrix formulation produce the same pooled simplicial complex. There also exists a cover choice making NervePool the identity function, and NervePool is permutation invariant on the input simplicial complex and partition of vertices. At the same time, the nerve lemma is not invoked, because the extended cover elements need not be contractible; hence homotopy equivalence to the original complex is not guaranteed (McGuire et al., 2023).

MIVSPool gives a different guarantee profile. If the original graph is connected, the quotient graph induced by the anchor-centered partition remains connected. Cluster sizes satisfy

$X$8

and the number of nonzero edges in the pooled graph cannot exceed the number of original edges, so the method cannot become artificially dense in the way dense soft-clustering methods may (Stanovic et al., 2022).

The broader taxonomy of pooling methods also complicates any universal claim about anchor-based coarsening. The comparative study of graph pooling reports that no pooling family dominates across tasks: sparse, dense, trainable, and non-trainable poolers trade off node-attribute preservation, structural preservation, task-specific information, and scalability differently (Grattarola et al., 2021). A plausible implication is that the theoretical desirability of anchor-vertex pooling depends strongly on whether the application values locality, strict coverage, sparsity, or end-to-end differentiable assignments.

6. Empirical performance and limitations

HieraMamba reports new state-of-the-art results for video temporal grounding. On Ego4D-NLQ it achieves average recall $X$9, with $A$0, $A$1, $A$2, and $A$3. On MAD v2 it reaches average recall $A$4, surpassing SnAG by $A$5 percentage points, and on TACoS it reaches average recall $A$6. The paper also reports that on MAD-v2 it improves average recall by $A$7 percentage points over SnAG (Global) while using roughly $A$8 fewer FLOPs than a strong global-attention baseline. Ablations show that removing interleaving, bidirectional scan, local encoding, or gates degrades Ego4D performance, and mean pooling is the best anchor initialization in the base configuration (An et al., 27 Oct 2025).

CAPNet reports strong recognition results on irregular text benchmarks, including $A$9 on ICDAR 2015, $S$0 on SVT-Perspective, $S$1 on CUTE, and $S$2 on Total-Text. The reported interpretation is that CAM plus APM, together with sequence learning, is especially effective on curved and perspective text. The same source also notes failure modes: severe occlusion, large gaps between anchors, touching characters, dense scripts, and extreme perspective can corrupt anchor grouping or left-to-right ordering, and the system is designed for case-insensitive English letters and digits rather than vertical reading orders or dense ligatures (Long et al., 2020).

ATA pooling improves text embedding performance without introducing parameters. In the pooling ablation on public retrieval data only, mean pooling scores $S$3, last-token pooling $S$4, NV-Embed pooling $S$5, and ATA pooling $S$6. The method reports essentially identical wall-clock training time with and without ATA, $S$7 versus $S$8 hours for $S$9 steps on a single H100. The detailed exposition also notes practical sensitivity: special instruction tokens can emerge as strong anchors, and poorly calibrated last-layer attention under domain shift can make anchor scores noisy (Pan et al., 31 Aug 2025).

MIVSPool reports strong graph-classification results under a unified GNN backbone. Reported accuracies include $S$0 on D&D and $S$1 on PROTEINS for MIVSPool, with MIVSPool_comp reaching $S$2 on D&D and $S$3 on PROTEINS. On ENZYMES, MIVSPool reports $S$4, substantially above gPool, SAGPool, and MVPool-SL in the cited comparison. The method’s limitations are its non-differentiable MIVS selection, dependence of the pooling ratio on graph degree, the need for careful tie-breaking, and the possibility that 1-hop anchor assignment is too local on near-linear graphs (Stanovic et al., 2022).

PPDet demonstrates the anchor-vertex idea in detection. On COCO test-dev, PPDet with ResNeXt-101-FPN and multi-scale testing reports $S$5, $S$6, $S$7, and $S$8, with the cited source stating that this $S$9 is the best among all reported one-stage and two-stage detectors in its table. Ablation finds the positive-area shrink factor $U_i$0 best on COCO, and removing inference-time pooling from PPDet with ResNet-101-FPN drops AP by $U_i$1 on val2017. The same source notes limitations in crowded scenes, heavy occlusion, and the absence of regression pooling during training (Samet et al., 2020).

Taken together, these results show that anchor-vertex pooling is empirically strongest when the task demands both compression and localized evidence retention: long-video grounding, irregular text recognition, graph coarsening without densification, text embeddings that should emphasize information hubs, and anchor-free detection under noisy positive labels. The principal limitations recur across domains: fixed or imperfect anchor allocation, memory or combinatorial growth in higher-order structures, non-differentiable discrete selection, susceptibility to noisy upstream features, and task-specific failure when anchors do not align with the true semantic support.