ADoBo 2025: Anglicism Detection in Spanish

• ADoBo 2025 is a shared task that targets the extraction of unassimilated anglicisms from Spanish journalistic text using precise span-level evaluation.
• The benchmark BLAS, with 1,836 sentences and 2,076 annotated spans, challenges systems with orthotypographic variability and strict boundary precision.
• Multiple approaches including LLM prompting, rule-based gazetteers, and Transformer models were evaluated, highlighting persistent challenges in ambiguity resolution and multiword segmentation.

Searching arXiv for ADoBo 2025 and closely related anglicism-detection work. Tool call: arxiv_search({"query":"ADoBo 2025 anglicisms Spanish IberLEF", "max_results": 10, "sort_by": "relevance"}) ADoBo 2025 is the second edition of the ADoBo shared task at IberLEF devoted to the automatic detection of anglicisms in Spanish, specifically the identification of exact spans corresponding to unassimilated English lexical borrowings in Spanish journalistic text. The 2025 edition evaluates systems on BLAS, a hand-crafted benchmark designed to stress orthotypographic variability and boundary precision, and it compares LLMs, Transformer-based models, deep learning systems, and rule-based approaches under strict span-level scoring. Reported test-phase results range from F1 scores of 0.17 to 0.99, with the strongest systems approaching ceiling performance on the benchmark while also exposing persistent difficulties in ambiguity resolution and multiword span segmentation (Alvarez-Mellado et al., 29 Jul 2025).

1. Task definition and linguistic scope

ADoBo 2025 targets the automatic detection of English lexical borrowings in Spanish. Its official scope is narrower than generic foreign-word detection: it focuses on unassimilated anglicisms, defined as words borrowed from English and used in Spanish without orthographic adaptation. The task includes both single-word and multiword expressions, and evaluation is based on strict span identification rather than token-wise tagging.

The overview highlights examples such as running, smartwatch, influencer, holding, look, hype, prime time, and lawfare. It also emphasizes linguistically ambiguous cases that are difficult even for strong systems, including total red, total black, casual looks, fatal error, global director, pie, and natural time. These cases are important because the challenge is not limited to visually marked foreign forms; many candidate spans overlap with forms that can also function inside ordinary Spanish discourse.

The task definition is also bounded by explicit exclusions. Calques, hybrid or code-switched spans, and assimilated forms are not the focus of the 2025 overview. Code-switching is not targeted. Morphologically integrated forms are implicitly excluded by the unassimilated criterion, although the paper does not enumerate a full rule inventory for integrated forms. Multiword expressions, by contrast, are fully in scope and are evaluated as unified spans.

From a methodological standpoint, ADoBo 2025 operationalizes anglicism detection as a span extraction problem with strong lexical and contextual components. Dictionary lookup alone is insufficient, because borrowings may be multiword, context-dependent, and confusable with native lexemes. This framing places the task at the intersection of lexical borrowing research, NER-like span extraction, and robust sequence labeling.

2. Dataset design, annotation regime, and submissions

The test set for ADoBo 2025 is BLAS, described as a hand-crafted, linguistically motivated collection of Spanish journalistic-style sentences. BLAS contains 1,836 sentences, 37,344 tokens, and 2,076 spans labeled as anglicisms. Every sentence contains at least one anglicism span. The benchmark was deliberately constructed so that orthotypographic cues such as casing, quotation marks, punctuation, and sentence position vary across examples, allowing organizers to probe robustness rather than simple cue memorization.

No training set was provided in 2025. A development set was provided and was derived from prior work, filtered to include only anglicisms and exclude borrowings from other languages. Participants were explicitly encouraged to use any resources at their disposal, including lexica, prior corpora, and the 2021 ADoBo dataset. This design makes ADoBo 2025 closer to an open-resource shared task than to a closed-data benchmark.

The annotation and submission regime is span-based. Participants do not return token-level BIO labels in the official evaluation; instead, each submission line consists of semicolon-separated values of the form:

sentence;span1;span2;span3;etc.

BLAS includes multiword spans such as prime time, and the overview repeatedly stresses that boundary fidelity is central. The 2025 paper does not fully specify a negative-case ontology for categories such as brand names, hashtags, acronyms, or technical terms, but it does note through error analysis and participant heuristics that named entities are commonly mistaken for anglicisms and should not be counted as such.

3. Evaluation protocol and scoring assumptions

ADoBo 2025 uses strict exact-match span-level evaluation. Partial overlaps do not count as correct. The official metrics are precision, recall, and F1, computed from aggregate TP, FP, and FN counts over the corpus:

$P = \frac{TP}{TP + FP}$

$R = \frac{TP}{TP + FN}$

$F1 = \frac{2PR}{P + R}$

The overview indicates that the scoring is effectively micro-averaged over all spans; no macro-averaging is reported. This choice favors systems that maintain both high recall and precise boundary control over the full corpus.

The scoring script includes three accommodations intended to avoid superficial penalties, especially for LLM outputs. First, matching is case-insensitive, so SMARTWATCH matches smartwatch. Second, trailing quotation marks are ignored, so a span such as “smartwatch” can match smartwatch. Third, if an identical span occurs multiple times in a sentence, outputting it once suffices to count as a match. These rules make the evaluation more tolerant of formatting variation while remaining strict about lexical content and span identity.

The competition was hosted on Codabench at the IberLEF platform. No threshold optimization or decision calibration beyond exact span matching is part of the official metric definition. The evaluation protocol therefore places the main burden on extraction quality, prompt design, post-processing, and lexical coverage rather than on confidence tuning.

4. Baselines and participating system families

Six baselines were evaluated on BLAS. These span supervised sequence models, multilingual and Spanish-specific Transformer encoders, hybrid BiLSTM-CRF systems, and an LLM few-shot baseline. Their reported results show that BLAS is considerably harder than conventional in-domain borrowing detection settings, especially in recall.

Baseline	System type	Precision / Recall / F1
CRF	Supervised	62.21 / 6.50 / 11.77
BETO	Supervised Transformer	76.05 / 23.55 / 35.96
mBERT	Supervised Transformer	84.01 / 23.80 / 37.09
BiLSTM-CRF A	Hybrid neural	82.74 / 23.55 / 36.66
BiLSTM-CRF B	Hybrid neural	85.15 / 23.75 / 37.14
8B-Llama3 few-shot	LLM baseline	90.96 / 36.37 / 51.96

The strongest baseline is the 8B-Llama3 few-shot system, with F1 = 51.96, substantially above the supervised baselines. Even so, its recall is only 36.37, leaving considerable room for improvement. This baseline profile is central to the interpretation of ADoBo 2025: the benchmark is not trivially solved by standard token classifiers trained on earlier anglicism corpora.

The participating systems covered four broad methodological families. Qilex relied on commercial OpenAI LLMs and explored prompt variants with extended guidelines, reminders, self-refinement, chain-of-thought, and in-context learning. Shentzu used a rule-based pipeline centered on a semi-automatically compiled gazetteer of 37,000 borrowing candidates extracted from a 6.6B-token Spanish news corpus, with optional NER pre-processing and contextual rules. Mheredia / HiTZ combined instruction-tuned Llama 3.3 70B prompting with Transformer sequence-labeling models such as ModernBERT, BETO, IXABERT, XLM-R large, and mDeBERTa v3. Trockti / HULAT-UC3M approached the task as BIO token classification with BERT and XLM-R and then added post-processing via the RAE dictionary and spaCy. Hammond used logistic regression and FFNN models with handcrafted binary features plus heuristics for multiword units.

These submissions collectively show that ADoBo 2025 was not dominated by a single modeling paradigm. LLM prompting, lexicon-heavy rule systems, and Transformer pipelines all reached competitive performance, albeit with different failure modes.

5. Reported results and comparative findings

The leaderboard results show a sharp separation between the strongest systems and the baselines, with the top four leaderboard teams all exceeding F1 = 91.84 (Alvarez-Mellado et al., 29 Jul 2025).

Team	Best approach	Precision / Recall / F1
Qilex	OpenAI o3 with explicit guidelines and reminders	98.84 / 98.74 / 98.79
Shentzu	Lexicon + rules with contextual features	96.68 / 95.47 / 96.07
Mheredia	Instruction-based 5-shot Llama 3.3	92.60 / 94.07 / 93.33
Trockti	Transformer token classifier + post-processing	96.20 / 87.86 / 91.84

The paper also reports TP/FP/FN counts for these systems: Qilex 2050 / 24 / 26, Shentzu 1982 / 68 / 94, Mheredia 1953 / 156 / 123, and Trockti 1824 / 72 / 252. Across all test-phase submissions, results ranged from F1 = 0.17 to 0.99. Of the 27 test submissions, 25 surpassed the best baseline. Participation statistics were: 14 teams registered, 38 submissions from 6 teams overall (11 dev, 27 test), 5 teams submitted in the test phase, and 4 teams published leaderboard entries.

The most striking empirical result is the performance of prompt-engineered LLMs. Qilex found that OpenAI o3 with explicit guidelines and reminders achieved F1 = 98.79, whereas the same o3 model without guidelines dropped to F1 ≈ 45. Applying the same guideline-enriched prompt to 4.1 nano yielded F1 ≈ 24. These observations are reported as ablation findings rather than as a general theory of LLM behavior, but they establish that prompt formulation was decisive on BLAS.

Rule-based systems also remained highly competitive. Shentzu’s gazetteer-driven approach reached F1 = 96.07, indicating that large lexical coverage plus contextual rules can rival contemporary LLM prompting in this benchmark setting. Transformer pipelines remained viable, especially when combined with lexicon-based or NER-based post-processing to reduce false positives.

The development-set snapshot is broadly consistent with the final ranking: trockti 0.86/0.83, qilex 0.82, shentzu 0.82, and other systems between 0.23 and 0.74. The overview does not report confidence intervals or statistical significance tests, so the results are primarily descriptive.

6. Error typology, benchmark interpretation, and future directions

The organizers discuss errors using an extended MUC-style typology. The most frequent category is Missing, in which a gold span is not predicted. Additional categories include Overlap missing, where a partial match omits at least one gold token; Split, where a multiword gold span is returned as two adjacent shorter spans; and Fused, where two adjacent gold spans are merged into one longer prediction. The paper also mentions Spurious, Overlap spurious, and Missegmented errors. Type errors are defined in the typology, but label variation is not central in this edition because the task is fundamentally binary: anglicism versus non-anglicism.

Qualitative analysis identifies two dominant challenges. The first is lexical ambiguity, especially for spans that resemble native Spanish forms or can be plausibly interpreted in more than one way. Examples cited in the paper include fatal error, global director, Pie in Pie de limón, total red, total black, and natural time. The second is boundary instability in multiword expressions. The overview gives examples such as red predicted for total red, time predicted for NATURAL TIME, “Casual Looks” predicted as casual, looks, “Ugly Shoes” predicted as ugly, shoes, and “Marketing Online” predicted as marketing online.

A central interpretive point of the benchmark is that BLAS was engineered to probe recall and contextual variability of true positives. It is not designed to be rich in negative cases prone to false positives, such as foreign named entities, literal quotations of titles, or odd-looking native words. As a consequence, simple heuristics such as lowercasing, quote normalization, or expanding quoted regions can improve scores on BLAS but may not generalize to precision-demanding real-world settings. The near-ceiling top-line F1 values should therefore not be read as evidence that automatic anglicism detection in Spanish is a solved problem.

The future directions proposed in the overview follow directly from this diagnosis. Recommended dataset work includes constructing evaluation sets richer in precision stressors, especially foreign named entities, literal quotations, brand names, and odd-looking native words, while preserving strict span-level evaluation and orthotypographic variation. Recommended modeling directions include hybrid systems that combine LLM reasoning, lexicon grounding, robust NER, and explicit boundary modeling for multiword spans, as well as methods tailored to ambiguity resolution for borrowings that overlap with Spanish lexemes. The paper also identifies domain transfer beyond journalistic prose, especially to social media and other settings in which orthotypographic cues are less reliable, as an important next step.

In this sense, ADoBo 2025 serves both as a competitive shared task and as a methodological probe. It demonstrates that carefully prompted LLMs can achieve near-perfect span-level identification on a recall-oriented benchmark, that large gazetteer-based rule systems remain formidable, and that multiword boundary control and ambiguity resolution remain the main unresolved problems.