ToMMeR: Lightweight Entity Mention Detection

• The paper introduces a lightweight probing architecture that recovers mention boundaries with 93% zero-shot recall across 13 NER benchmarks.
• It utilizes low-rank query-key matching and scalar value features to extract token-binding information from early transformer layers efficiently.
• Extensive experiments demonstrate that ToMMeR, with only 274K parameters, supports efficient mention extraction and achieves near-state-of-the-art span classification performance.

ToMMeR is a lightweight probing architecture for entity mention detection from LLM representations. Introduced in “ToMMeR -- Efficient Entity Mention Detection from LLMs,” it targets the boundary-detection component of information extraction: given a text, identify which contiguous spans correspond to entity mentions, independent of semantic type. The paper explicitly expands the name as “Token Matching for Mention Recognition,” argues that mention boundaries are already encoded in early transformer layers, and reports that this capability can be recovered with fewer than 300K additional parameters from frozen backbones, with 93% recall zero-shot across 13 NER benchmarks and near-SOTA performance once span classification heads are added (Morand et al., 22 Oct 2025).

1. Task definition and scope

ToMMeR addresses mention detection rather than full named entity recognition. In the paper’s formulation, mention detection is the problem of deciding whether a span is a valid mention, while full NER adds a later type-assignment stage. This distinction is central because many benchmark datasets annotate only a narrow ontology, so a detector that recovers a broader notion of mentionhood may appear low-precision against gold labels even when many predicted spans are legitimate mentions omitted by the benchmark (Morand et al., 22 Oct 2025).

Formally, for a tokenized sentence

$(t_1,\ldots,t_n)\in\mathcal V^n$

and hidden states at layer $\ell$,

$(h_1^\ell,\ldots,h_n^\ell)\in\mathbb R^d,$

the gold mention set is

$E=\{(s_k,e_k)\in[1,n]^{2\times N_E}\}.$

The task is binary span classification over contiguous spans. The work restricts candidate spans to length $\le 25$, and the appendix states that this covers 99.8% of mentions in Pile-NER. The restriction is therefore presented as a computational choice rather than a semantic one (Morand et al., 22 Oct 2025).

The paper positions mention detection as foundational for information extraction pipelines, including named entity recognition, entity linking, relation extraction, and coreference pipelines. A practical implication is that ToMMeR isolates the boundary-recovery problem that is often bundled together with typing. This suggests that its main contribution is not a new label ontology, but an efficient readout of “entityhood” from pretrained representations (Morand et al., 22 Oct 2025).

2. Architecture and span scoring

ToMMeR uses a frozen pretrained LM or LLM as a feature extractor and probes one selected layer at a time. The backbone parameters are not updated. Instead, the model reads hidden states $h_i^\ell$ from a chosen layer $\ell$, which the paper analyzes empirically rather than learning jointly across layers (Morand et al., 22 Oct 2025).

The probe combines a low-rank query-key matching mechanism with token-value features. To capture token-to-token binding, it learns

$(W_Q, W_K)\in\mathbb{R}^{r\times d}$

and computes a cosine-similarity-based matching score

$$m_{ij} = \cos \left ( W_Q h_i^\ell \ \middle| \ W_K h_j^\ell \right) \in \mathbb [0,1].$$

The main experiments use rank $r=64$. In parallel, each token is mapped through a scalar value probe,

$\ell$0

The paper motivates the value term by prior observations that autoregressive LMs often concentrate entity information in the final token of an entity (Morand et al., 22 Oct 2025).

For a candidate span $\ell$1, ToMMeR combines five features through a tiny logistic model: $\ell$2 These features are the direct start-end match, the strongest and weakest internal token-to-end match, the value of the last token, and the value of the next token. The architecture therefore scores span existence directly rather than predicting BIO tags or separate start/end labels. The paper describes this as a small probe recovering latent token-binding structure already present in the backbone (Morand et al., 22 Oct 2025).

The resulting detector remains extremely small. The main zero-shot model on Llama-3.2-1B layer 6 is reported at 274K parameters, and an architecture-comparison table reports 264,198 parameters for the main ToM variant. This compactness is a direct consequence of using only low-rank projections, a scalar value head, and a 5-feature combiner (Morand et al., 22 Oct 2025).

3. Training regime and inference procedure

The main supervision source is Pile-NER, described as a GPT-3.5-annotated dataset built from 45,889 samples from The Pile. The paper presents this choice as a way to learn a broad notion of mentionhood rather than the narrow annotation conventions of a single NER benchmark. Training uses only mention boundaries, not entity types (Morand et al., 22 Oct 2025).

Because negative spans vastly outnumber positive spans, ToMMeR is trained with Balanced Binary Cross-Entropy. The extracted formula is

$\ell$3

with

$\ell$4

The paper also uses a two-stage self-distillation procedure: first train on available annotations, then augment training with high-confidence predicted mentions that were unlabeled, then retrain or continue training on the expanded set. Reported hyperparameters include 8 epochs, AdamW, learning rate $\ell$5, batch size 16, rank 64, sliding window 25, and teacher threshold probability 0.90 (Morand et al., 22 Oct 2025).

At inference time, ToMMeR enumerates candidate spans up to length 25 and scores them. The default raw setting is threshold decoding, which allows nested and overlapping mentions. For flat NER benchmarks, the paper also uses a greedy non-overlap decoder: spans are sorted by score, and the highest-scored non-overlapping spans are retained. This is presented as a simple post-processing step rather than a structured decoder such as a CRF (Morand et al., 22 Oct 2025).

The reported training cost is low. For the rank-64 model probing layer 6 of Llama-3.2-1B, training takes about 4 hours on an NVIDIA H100 80GB, with peak GPU memory only 6GB. The paper therefore presents ToMMeR as suitable for efficient deployment on frozen backbones without prompting, schema fine-tuning, or full-sequence generative inference (Morand et al., 22 Oct 2025).

4. Empirical performance

The headline zero-shot result is 93% recall across 13 NER benchmarks. More precisely, for ToMMeR on layer 6 of Llama-3.2-1B with 274K parameters, aggregated threshold decoding yields recall 92.6, precision 23.2, and F1 37.1; averaged across datasets, the corresponding values are recall 88.2, precision 29.8, and F1 42.6. With greedy flat decoding, the aggregated numbers become recall 84.0, precision 31.2, and F1 45.5, while the averaged numbers are recall 75.3, precision 41.3, and F1 52.1 (Morand et al., 22 Oct 2025).

Dataset-level recall is very high on many benchmarks: 98.6 on MultiNERD, 94.8 on CoNLL 2003, 97.0 on CrossNER politics, 97.0 on CrossNER AI, 94.4 on CrossNER literature, 95.7 on CrossNER science, 95.5 on CrossNER music, 91.9 on ncbi, 97.8 on WikiNeural, and 95.7 on GENIA NER. The notable low-recall outlier is ACE 2005 at 42.0, which the paper later links to span-convention mismatch rather than total failure of localization (Morand et al., 22 Oct 2025).

The paper argues that standard benchmark precision is artificially depressed by incomplete annotation. To estimate broader-mention precision, it introduces LLM-judged datasets. On MultiNERD-gpt-4.1-mini, threshold decoding reaches precision 92.8, recall 71.1, and F1 80.5. On GENIA-gpt-4.1-mini, the corresponding values are precision 92.3, recall 68.4, and F1 78.6. Human validation agreement with the LLM judge is reported as 91.50% on MultiNERD, 78.50% on GENIA, and 87.17% aggregated over 1,800 judgments. The paper therefore argues that ToMMeR rarely produces spurious predictions despite high recall (Morand et al., 22 Oct 2025).

When extended with span classification heads, ToMMeR reaches near-SOTA typed NER performance. Reported micro-F1 values include 84.8, 86.8, and 85.0 on CoNLL 2003 for LLaMA-3.2-1B, LLaMA-3.2-3B, and LLaMA-3.1-8B respectively; 92.2, 93.3, and 92.4 on MultiNERD; and 80.4, 81.7, and 80.0 on OntoNotes. Encoder-based variants also perform strongly, including 87.3 on CoNLL 2003 and 85.4 on OntoNotes for RoBERTa-base. This supports the paper’s summary that ToMMeR-based span classification reaches 80–87% F1 on standard benchmarks (Morand et al., 22 Oct 2025).

5. Interpretation, significance, and limitations

A central interpretive claim is that structured entity representations emerge naturally in language modeling and are accessible from early layers. The paper reports that for Llama-3.2-1B, strong mention-detection performance appears almost from layer 0 onward, with example aggregated recalls of 0.9349 at layer 0, 0.9262 at layer 6, and 0.9474 at layer 15. Cross-model analysis further reports DICE scores above 75% across diverse autoregressive architectures, which the authors interpret as evidence that different models converge on similar mention boundaries (Morand et al., 22 Oct 2025).

This claim is strengthened by probe comparisons. In an appendix architecture ablation on Llama-3.2-1B, the main ToM variant obtains aggregated recall 92.6, aggregated precision 23.2, aggregated F1 37.1, LLM-judged precision 92.5, and 264,198 parameters. A reduced LTQK variant preserves recall at 94.4 but lowers LLM-judged precision to 78.8, while an LCAttn variant yields 94.3 recall but only 44.2 LLM-judged precision. The paper therefore treats the main ToMMeR design as the best precision-recall tradeoff among tiny probes (Morand et al., 22 Oct 2025).

The paper also makes several limitations explicit. There is no gold standard for broad, untyped mention detection; standard NER datasets are incomplete with respect to mentionhood. The method only handles contiguous spans, imposes a span-length cap of 25, depends on tokenization, and for decoder-only backbones operates on causal hidden states without explicit right-context access. The ACE 2005 failure case illustrates annotation mismatch sharply: the paper notes that ACE often includes determiners such as “the president,” whereas ToMMeR often predicts “president.” The authors interpret this as a span-convention discrepancy rather than a total mention-detection breakdown (Morand et al., 22 Oct 2025).

A broader implication is that ToMMeR functions both as an extraction component and as an interpretability result. The practical reading is that a high-recall mention proposal system can be recovered cheaply from frozen LLMs. The theoretical reading is that early transformer layers contain robust boundary information that does not need full-model finetuning to become usable (Morand et al., 22 Oct 2025).

6. Disambiguation and related acronym landscapes

Despite its name, ToMMeR in this literature is not a Theory-of-Mind method. It is an entity mention detector. This matters because a large neighboring literature uses “ToM” for Theory of Mind: “ToMBench” benchmarks social-cognitive abilities in LLMs (Chen et al., 2024), “PersuasiveToM” studies persuasive dialogues (Yu et al., 28 Feb 2025), “EgoToM” evaluates egocentric-video mental-state inference (Li et al., 28 Mar 2025), “ToMCAT” generates ToM-conditioned trajectories for cooperative agents (Sequeira et al., 25 Feb 2025), and “RecToM” introduces recursive perspective construction for nested-belief reasoning (Lei et al., 10 Jun 2026). These works are conceptually unrelated to ToMMeR’s mention-detection objective.

A similar disambiguation applies outside the Theory-of-Mind literature. The diffusion-efficiency paper “ToMA: Token Merge with Attention for Image Generation with Diffusion Models” explicitly states that it does not mention or compare against any method called “ToMMeR” (Lu et al., 13 Sep 2025). This suggests that acronym overlap can produce misleading retrieval results across otherwise unrelated subfields.

A plausible implication is that ToMMeR occupies a distinctive position: it is a compact probe for recovering entity boundaries from LLM representations, rather than a benchmark, a social-reasoning module, or a token-merging method. In practice, correct identification of the paper usually depends on its full title, “ToMMeR -- Efficient Entity Mention Detection from LLMs” (Morand et al., 22 Oct 2025).