Landmark Pooling (LMK): Techniques & Applications

• Landmark Pooling (LMK) is a family of pooling strategies that insert key landmark tokens or nodes to preserve both local and global information in dense representations.
• LMK methods improve on traditional [CLS] and mean pooling by distributing attention, reducing positional bias, and preventing information dilution in multimodal tasks.
• Empirical results show LMK-driven models enhance long-context retrieval, graph classification accuracy, and visual grounding efficiency with minimal computational overhead.

Landmark Pooling (LMK) is a family of pooling strategies that aggregate intermediate representations by identifying or inserting "landmarks"—special tokens, nodes, or features—at key positions within a sequence, graph, or spatial feature map, and constructing the final embedding or prediction via aggregation over these landmarks. LMK pooling mechanisms address the information collapse, bias, or dilution often observed in standard pooling operators such as [CLS] token selection or global mean pooling. Landmark Pooling has been applied across a variety of modalities, including Transformer-based text encoders (Doshi et al., 29 Jan 2026), graph neural networks via topological pooling (Chen et al., 2023), and visual feature aggregation for grounding tasks (Huang et al., 2021).

1. LMK Pooling for Dense Embeddings in Transformers

In Transformer-based sequence encoders, the canonical "collapse-and-pool" paradigm typically employs either a [CLS] token or mean pooling to produce a fixed-size vector representation:

• [CLS]-pooling: $$\mathcal{P}_{\mathrm{CLS}}(\mathbf{H}) = \mathbf{h}_{[\mathrm{CLS}]}$$
• Mean pooling: $$\mathcal{P}_{\mathrm{mean}}(\mathbf{H}) = \frac{1}{|\mathbf{H}|}\sum_i\mathbf{h}_i$$

Empirical and theoretical examinations reveal systematic weaknesses in both:

• [CLS]-pooling is subject to positional bias, with attention and representational capacity concentrated near the initial sequence positions, leading to under-representation of evidence from later tokens in long contexts.
• Mean pooling indiscriminately weighs all tokens, which may drown salient local signals within a global average, particularly problematic when local cues are sparse but decisive.

Landmark Pooling (LMK) introduces an alternative:

1. Partition the sequence into $K = \lceil\frac{N}{g}\rceil$ chunks, where $g$ is the granularity (chunk size).
2. Insert landmark tokens ($t_\mathrm{LMK}$, often implemented via the [SEP] token) after each chunk and [CLS] at the sequence head, yielding an augmented token sequence $\widetilde{T}$.
3. Contextual encode the sequence with a Transformer $\theta_{\mathrm{enc}}$ to yield embeddings $$\mathbf{H}^{\mathrm{enc}} \in \mathbb{R}^{S \times D}$$, $S = N+K+2$.
4. Pool only landmark embeddings: form the final embedding by mean-pooling over $\mathcal{I}_L \subset \{0, ..., S-1\}$, the indices of unmasked landmark tokens:

$$\mathbf{x}^{\mathrm{LMK}} = \frac{1}{|\mathcal{I}_L|}\sum_{i\in\mathcal{I}_L}\mathbf{H}^{\mathrm{enc}}_{i,:}$$

This architecture distributes pooling load across $K$ landmarks, mitigating the attenuation of distant information and preserving local structure. Landmark insertion increases sequence length marginally (e.g., $<3\%$ overhead for $g=128$ at 4K tokens), imposes no additional learnable parameters beyond standard token embeddings, and is compatible with off-the-shelf Transformern-based models with minimal tokenizer adjustments (Doshi et al., 29 Jan 2026).

2. LMK Pooling in Topological Graph Representation Learning

Landmark pooling for graphs manifests in methods such as Wit-TopoPool (Chen et al., 2023), which systematically integrates local and global topological information using persistent homology and the concept of landmark nodes.

The process involves:

• Local Landmark Selection: For each node $u$, a local neighborhood $Z_u^\phi$ is extracted (by feature similarity threshold $\phi$) in $\mathbf{H}^{(\ell)} \in \mathbb{R}^{N \times d}$. Computing the persistence diagram $D_u$ of the Vietoris–Rips complex on $Z_u^\phi$, a scalar topological score $y_u = \sum_{(b,d)\in D_u}(d-b)$ quantifies the "shape" encoded locally. The top $\lceil\tau N\rceil$ nodes by $y_u$ are selected as $L_\mathrm{local}$ for TopK pooling.
• Global Landmark Selection: Subset $L_\mathrm{global}\subset V$, $|L_\mathrm{global}| = \psi N$, is selected via random sampling, degree centrality, or betweenness centrality.
• Topological Pooling:
  • Local: Structures and features are pooled over $L_\mathrm{local}$.
  • Global: A weak witness complex $\mathcal{W}(L_\mathrm{global},V)$ is constructed using the shortest-path metric. Homology computations across a filtration parameter $\epsilon$ yield birth-death pairs in persistence diagrams, which are embedded as persistence images then processed via MLP to yield global descriptors.
• Integration: The topological pooling layer and global embedding are concatenated or pooled via attention, providing both local and global topological summaries.

Wit-TopoPool is computationally efficient due to the sparse use of landmarks and yields state-of-the-art performance in graph classification tasks across chemistry, biology, and social networks, highlighting the utility of landmark-based topological summarization (Chen et al., 2023).

3. LMK Pooling via Dynamic Max-Pooling in Visual Grounding

The LBYL-Net architecture introduces a variant of LMK pooling in the context of one-stage visual grounding, leveraging spatial context via dynamic max-pooling over regions that represent landmark-relative directions (Huang et al., 2021).

The key design is:

• Partition the $C \times H \times W$ spatial feature map into $k$ directionally-defined groups (e.g., quadrants relative to each position).
• For each group and each position $(i,j)$, compute:

$$H_\ell(i,j) = \max_{(p,q)\in G_\ell(i,j)} f_\ell(x_{p,q})$$

where $$f_\ell(x_{i,j}) = \mathrm{ReLU}(W_\ell \cdot x_{i,j})$$ and $G_\ell(i,j)$ is the region for landmark group $\ell$.

• Efficient dynamic programming recursively computes the channel-wise maxima over O($kHW$) time and memory. This mechanism yields a global receptive field in linear complexity—far more efficient than non-local attention.
• Aggregate the direction-aware features via

$$y_{i,j} = \sum_{\ell=1}^k W'_\ell \cdot H_\ell(i,j)$$

integrating context from all landmark directions.

This approach enables the network to mimic human spatial reasoning by encoding spatial relationships to "landmarks" and offers measurable speed and accuracy improvements on standard visual grounding benchmarks, outperforming prior two-stage and one-stage methods on ReferitGame and achieving competitive results on RefCOCO and RefCOCO+ (Huang et al., 2021).

4. Comparative Analysis with Conventional Pooling

Landmark pooling resolves key limitations present in conventional pooling operations:

• [CLS] pooling's position-centric bias leads to degraded long-context performance due to information attenuation, especially under absolute/rotary positional encodings.
• Mean-pooling's uniform weighting can obliterate the influence of locally salient signals, inhibiting the representation of sparse but significant evidence.

Landmark pooling achieves:

• Mitigation of positional bias by dispersing "attentive pooling" to all landmarks, not just a single token or node.
• Preservation of local and global information by selective inclusion (sequence, graph) or directional context (vision).
• Minimal computational and parameter overhead: token count increases sub-linearly with context length, leveraging existing embedding mechanisms; dynamic programming ensures tractable computation even for high-resolution feature maps.

Ablations confirm that simple alternatives—such as Mean@k (pooling every $k$th token) or sentence-based chunking—do not consistently match the effectiveness of explicit landmark tokens or nodes with learnable embeddings (Doshi et al., 29 Jan 2026, Chen et al., 2023).

5. Empirical Performance and Applications

Landmark pooling demonstrates consistent advantages in diverse tasks and modalities:

• Text retrieval and classification: LMK pooling matches or slightly exceeds [CLS]-based pooling on short-context benchmarks (e.g., MSMarco Dev NDCG@10: LMK 40.0 vs CLS 39.8; BEIR/MTEB-v2/MIRACL short-context: LMK 45.6 vs CLS 45.2), and yields dramatic improvements in long-context settings (MLDR/COIR/LongEmbed: LMK 47.1 vs CLS 37.2, +9.9) (Doshi et al., 29 Jan 2026). In multilingual scenarios, LMK maintains and expands these gains.
• Graph classification: Wit-TopoPool surpasses 18 baseline methods, achieving an average 5.1% higher accuracy on molecular graphs and leading all social graph benchmarks. Both local and global landmark modules are shown empirically essential (Chen et al., 2023).
• Visual grounding: LBYL-Net leveraging landmark feature convolution via dynamic max-pooling reaches or surpasses state-of-the-art results with only a minor computational increase (~3 ms on 256×256 grids). (Huang et al., 2021)

Potential applications include long-document retrieval (legal, scientific, code), chunk-level multi-vector retrieval (each landmark as anchor), and retrieval-augmented generation pipelines requiring robust document-level embeddings.

6. Design Considerations, Trade-offs, and Limitations

Selection and granularity of landmarks are critical hyperparameters. In text, chunk size $g$ trades off the number of landmarks versus overhead; variable chunking during training increases robustness and avoids test-time hyperparameter sensitivity. In graphs, landmark selection method (random, degree, betweenness) and neighborhood definition ($\phi$) impact information capture.

Notably:

• Further reduction in $g$ (more landmarks) produces diminishing returns beyond $K \approx 128$ (Doshi et al., 29 Jan 2026).
• Simple fixed schemes (sentence chunking) may suffer from domain variance; explicit landmark tokens generalize better.
• LMK's format remains a single-vector summary; ultra-fine granularity or per-chunk retrieval would require multi-vector or hybrid mechanisms.
• Optimal granularity and landmark selection remain open; learned or adaptive strategies are suggested as future directions (Doshi et al., 29 Jan 2026).

7. Extensions and Future Prospects

Extensions of LMK pooling beyond transformation-invariant tasks are plausible, such as hybrid landmark+latent attention designs, end-to-end trainable landmark insertions, or task-adaptive granularity schedulers.

In topological graph pooling, persistence-image embeddings, alternative filtration strategies, or parameterized landmark allocation suggest new research directions. Dynamically adaptive max-pooling architectures in vision may integrate spatial and semantic cues for multimodal reasoning.

A plausible implication is that landmark pooling strategies will serve as default or foundational components in high-context dense embedding architectures, especially where preserving multi-scale or global structure is paramount.

---

Key References:

• LMK > CLS: Landmark Pooling for Dense Embeddings (Doshi et al., 29 Jan 2026)
• Topological Pooling on Graphs (Chen et al., 2023)
• Look Before You Leap: Learning Landmark Features for One-Stage Visual Grounding (Huang et al., 2021)