CLEARS Shared Task Overview

• CLEARS Shared Task is a benchmark evaluating the plausibility of candidate clarifications in instructional texts, primarily demonstrated in SemEval-2022 and IberLEF-2025.
• It employs a cloze-style format combined with rigorous data construction from wikiHow guides and uses advanced Transformer architectures to assess implicit meaning.
• The task highlights methodological challenges in ambiguity resolution and model evaluation, promoting transparency and nuanced interpretation of multiple plausible answers.

“CLEARS Shared Task” refers to more than one shared-task usage in the recent literature. In the official overview of SemEval-2022 Task 7, it denotes a benchmark on “Identifying Plausible Clarifications of Implicit and Underspecified Phrases in Instructional Texts,” centered on plausibility judgments for candidate clarifications in instructional discourse (Roth et al., 2023). In later IberLEF work, “CLEARS-2025” denotes a shared task on Spanish text adaptation for accessibility, with Plain Language and Easy-to-Read subtasks (Ayesh et al., 5 Aug 2025). The term is therefore not a single stable benchmark family in the available sources. This is reinforced by the NFDI4DS survey of scholarly-document shared tasks, which explicitly lists twelve tasks and does not include any task named CLEARS (Ahmad et al., 26 Sep 2025).

1. Scope and nomenclature

The best-documented use of the term is the SemEval-2022 shared task officially overviewed as “Identifying Plausible Clarifications of Implicit and Underspecified Phrases in Instructional Texts,” which the source explicitly presents as the CLEARS shared task (Roth et al., 2023). Its subject matter is not simplification, paraphrasing, or generation in the usual sense; instead, it evaluates whether a proposed clarification is plausible in the local and discourse context of an instructional sentence.

A second use appears in IberLEF 2025. There, CLEARS-2025 is described in a system paper as a shared task on automatic Spanish text adaptation, with one subtask targeting Plain Language and another targeting Easy-to-Read rewriting (Ayesh et al., 5 Aug 2025). The two uses share the shared-task format and an interest in language adaptation, but they are methodologically and substantively different.

The literature also contains a recurrent source of confusion with CLEF. The NFDI4DS overview notes that no task named CLEARS appears among its twelve shared tasks and identifies the closest CLEF-hosted neighbor as CheckThat!Lab Task 4: Scientific Web Discourse (Ahmad et al., 26 Sep 2025). A plausible implication is that references to “CLEARS” require venue- and year-level disambiguation rather than acronym matching alone.

2. SemEval-2022 CLEARS: task formulation

SemEval-2022 CLEARS is motivated by the observation that instructional texts frequently contain language that is implicit or underspecified. A step may omit a referring expression entirely, use a phrase whose intended head noun is left implicit, or otherwise require readers to infer what is being referred to. The official overview emphasizes that this matters particularly in instructional settings because misunderstanding can prevent goal completion and, in some cases, lead to serious errors (Roth et al., 2023).

The task is framed as a cloze-style evaluation. Each instance contains a sentence from an instructional text with a marked insertion site, surrounding discourse context, and a candidate filler. A “clarification” is the phrase inserted into the original text to make omitted or underspecified meaning more explicit. Systems do not generate clarifications from scratch in the shared task; they assess the plausibility of a given candidate.

Two subtasks are defined. Task 1: Classification predicts one of three labels—implausible, neutral, or plausible—for a candidate clarification. This is a 3-way classification setup evaluated primarily by accuracy. Task 2: Ranking predicts a continuous plausibility score, evaluated by Spearman’s rank correlation coefficient with human judgments (Roth et al., 2023).

A distinctive feature of the benchmark is that plausibility is explicitly non-unique. Multiple fillers for the same context may all be plausible. The overview illustrates this with a hair-salon guide sentence, “Call ____ and ask questions,” where the revised human insertion is “the salon,” but alternatives such as “the number” or “the owner” may also be plausible to varying degrees (Roth et al., 2023). This design opposes the common misconception that clarification recovery should be modeled as a single-correct-answer completion problem.

3. Data construction and annotation in SemEval-2022 CLEARS

The dataset was built from revision histories of wikiHow guides, specifically the wikiHowToImprove resource, which contains sentence-level revisions and context from more than 250,000 how-to guides (Roth et al., 2023). Data construction proceeded in three stages.

First, the organizers extracted clarification insertions from revision histories. They restricted attention to cases with a single contiguous insertion and no other change within a sentence. Diffs were computed with Python’s difflib, while preprocessing used spaCy for sentence splitting and tokenization, the Berkeley Neural Parser for constituency parsing, and Stanza for POS tagging, dependency parsing, and coreference resolution (Roth et al., 2023).

Second, the organizers focused on four clarification phenomena.

Phenomenon	Revision pattern	Illustrative characterization
Implicit reference	∅ → [[DET](https://www.emergentmind.com/topics/domain-elastic-transform-det)] NOUN	A non-verbalized referent is clarified by inserting a noun phrase
Fused head	DET/JJ ∅ → DET/JJ NOUN	A noun phrase with an implicit head noun is clarified by inserting the noun
Noun compound	∅ NOUN → NOUN NOUN	A dependent noun is inserted to create a more specific compound
Metonymy	Possessive or “of”-phrase addition	A noun is added to make explicit an aspect or component of another noun

Candidate filler sets were then constructed with language modeling. For each instance, the organizers generated the top-100 fillers for the insertion position. Unlike the pilot study, which had used GPT, the shared task used BERT (bert-base-uncased) because the insertions were one token long and BERT could directly condition on right context. After filtering out unsuitable outputs and wrong POS categories, they included the observed human insertion when possible and selected four additional fillers by k-means clustering with $k=4$ over BERT embeddings, yielding effectively five fillers per sentence (Roth et al., 2023).

Third, human plausibility judgments were collected with Amazon Mechanical Turk on a 1-to-5 scale. Annotators saw the clarification underlined in context and rated one clarification at a time. The training set contains 19,975 instances, corresponding to 3,995 sentences with 5 fillers each. These sentences are distributed as 1000 noun compounds, 1000 metonymy, 996 implicit references, and 999 fused heads. The development set contains 2,500 instances, and the test set also contains 2,500 instances; each corresponds to 500 sentences total, with 125 sentences per phenomenon (Roth et al., 2023).

Worker screening was relatively strict: workers had to be located in the United States or United Kingdom, have a HIT approval rate of at least 95%, and have completed at least 1000 approved HITs. They also had to pass a qualification test. The organizers later increased the number of qualification questions from 4 to 6, which they believe improved dev/test quality (Roth et al., 2023).

The official label mapping converts averaged human ratings into three classes: implausible if average score $\leq 2.5$, plausible if average score $\geq 4.0$, and neutral otherwise. The training label distribution is 5,474 implausible (27%), 7,162 neutral (36%), and 7,339 plausible (37%). An especially important statistic is the average number of plausible clarifications per sentence: 1.84 in training, 1.87 in development, and 1.84 in test (Roth et al., 2023). This confirms that the benchmark was intentionally designed around the possibility of multiple plausible clarifications.

4. Systems, baselines, and results in SemEval-2022 CLEARS

The official overview reports a naive majority-class baseline of 39% accuracy and an organizers’ BERT-based baseline of 45.7% on Task 1. A total of 21 users participated in the CodaLab competition, and 8 teams submitted system description papers. All described systems were based on Transformer architectures, including BERT, DeBERTa, ELECTRA, ERNIE, RoBERTa, S-BERT, T5, and XLM-R (Roth et al., 2023).

The strongest systems were predominantly ensemble-based. X-PuDu, the winning team, used an ensemble combining DeBERTa, ERNIE, and XLM-R, described as pattern-aware and using multi-loss training. HW-TSC used an ensemble with DeBERTa, RoBERTa, and S-BERT, including an unsupervised model. PALI used an ensemble with DeBERTa, RoBERTa, and XLM-R, also pattern-aware and multi-loss. Other notable variants included T5 with ordinal regression and multi-loss in Nowruz, and class weighting in DuluthNLP (Roth et al., 2023).

The official Task 1 results show a challenging but learnable benchmark. The human upper bound is 79.4% accuracy, while X-PuDu achieved 68.9%. The remaining described teams scored HW-TSC 66.1%, PALI 65.4%, Nowruz 62.4%, JBNU-CCLab 61.4%, DuluthNLP 53.3%, Stanford MLab 46.6%, and niksss 44.2% (Roth et al., 2023).

Team	Task 1 accuracy	Task 2 Spearman
X-PuDu	68.9%	0.807
HW-TSC	66.1%	0.774
PALI	65.4%	0.785
Nowruz	62.4%	0.707

For Task 2: Ranking, the top correlations were X-PuDu 0.807, PALI 0.785, HW-TSC 0.774, and Nowruz 0.707. The system ordering in Task 2 largely matched the classification ordering, except for the last two teams reported in the overview (Roth et al., 2023).

The post-hoc analyses are central to the task’s significance. When the organizers reevaluated systems excluding the neutral label, performance rose sharply for all systems. For X-PuDu, $F_1$ increased from 0.689 to 0.773. The paper interprets this as evidence that the neutral label is a major source of difficulty, plausibly because it conflates intermediate plausibility with annotator disagreement (Roth et al., 2023). In a further evaluation on identifying contexts with multiple plausible clarifications, the best system achieved 75.2% accuracy on the criterion $\#p \ge 2$, supporting the claim that top systems can help identify potentially ambiguous instructional contexts.

5. CLEARS-2025 at IberLEF: Spanish accessibility rewriting

A later use of the acronym appears in IberLEF 2025. A CardiffNLP system paper describes CLEARS-2025 as a shared task on Spanish text adaptation, with two subtasks targeting Plain Language (PL) and Easy-to-Read (E2R) rewriting (Ayesh et al., 5 Aug 2025). The task is motivated by the difficulty of official, administrative, and public-facing texts, and it focuses on transforming Spanish source texts into more accessible forms.

The paper stresses that the two subtasks are not identical. Plain Language targets a broad audience, including non-native speakers and readers with limited literacy or reading difficulties; its emphasis is on clarity, concision, active voice, common vocabulary, and the removal of ambiguity and jargon. Easy-to-Read goes further and is intended for readers with cognitive, intellectual, or learning disabilities, following the UNE 153101 EX guidelines. It therefore requires stronger structural simplification, reduced cognitive load, short sentences, direct phrasing, and often formatting decisions such as splitting information into separate lines (Ayesh et al., 5 Aug 2025). A common misconception rejected in the paper is that E2R is simply “more simplification”; the authors treat it as a partially different target style.

As described in that system paper, the shared-task corpus contained 3,000 municipal news items from Alicante province, with 2,100 training articles and 900 test articles, and each article had both a PL version and an E2R version (Ayesh et al., 5 Aug 2025). CardiffNLP approached both subtasks with prompt-based LLM rewriting, eventually selecting Gemma-3 after experiments with LLaMA-3.2 and other prompt variants.

The official semantic ranking metric is reported as the average of cosine similarity computed over TF-IDF embeddings and BERT embeddings. In Subtask 1, CardiffNLP scored 63% with TF-IDF and 77% with BERT, averaging to 70%, which placed the team third; the best team reached 75%. In Subtask 2, CardiffNLP scored 65% TF-IDF cosine and 77% BERT cosine, averaging to 71%, which earned second place behind 72% (Ayesh et al., 5 Aug 2025). This later CLEARS usage thus centers not on ambiguity detection in instructional text, but on controlled accessibility-oriented rewriting of Spanish public information. This suggests acronym reuse rather than direct continuity with SemEval-2022 CLEARS.

6. Broader methodology and related traditions

SemEval-2022 CLEARS belongs to a broader shared-task tradition that emphasizes phenomenon-sensitive evaluation rather than only aggregate i.i.d. held-out performance. A conceptually related precursor is Build It Break It, The Language Edition, which used builders and breakers, minimal pairs, and targeted perturbations to probe generalization beyond training distributions (Ettinger et al., 2017). CLEARS differs in format, but it shares the premise that evaluation should expose specific weaknesses—here, reasoning about implicit reference, fused heads, noun compounds, and metonymy—rather than treating all instances as homogeneous.

The organization of shared tasks is also a live methodological issue. Community survey work on shared-task transparency proposes a Shared Task Organisation Checklist spanning Transparency, Reporting and Replicability, System Ranking, and Metrics, and argues that organizers should make explicit matters such as whether organizers, annotators, or evaluators can participate; whether the task is open, closed, or both; whether evaluation metrics are fixed at announcement time; and whether code, data, and system descriptions will be released (Escartín et al., 2021). Those recommendations are directly pertinent to CLEARS-like tasks, especially where interpretation of ambiguity, evaluation design, and ranking incentives can materially shape what systems optimize.

Finally, the NFDI4DS overview is important as a disambiguation source. It explicitly states that the consortium’s twelve shared tasks do not include a task named CLEARS and identifies CheckThat!Lab Task 4: Scientific Web Discourse as the closest CLEF-hosted neighbor (Ahmad et al., 26 Sep 2025). A plausible implication is that “CLEARS Shared Task” should be treated as a context-dependent label whose meaning is fixed by venue, year, and paper lineage.

In the literature available here, the most authoritative and fully specified CLEARS benchmark remains SemEval-2022 Task 7: a shared task on plausibility judgments over clarifications in instructional text, notable for modeling ambiguity, disagreement, and the presence of multiple plausible clarifications in a principled benchmark design (Roth et al., 2023).