---
title: CLEARS Shared Task Overview
url: https://www.emergentmind.com/topics/clears-shared-task
type: topic
---

# CLEARS Shared Task Overview

“CLEARS Shared Task” refers to more than one shared-task usage in the recent literature. In the official overview of SemEval-2022 Task 7, it denotes a benchmark on “Identifying Plausible Clarifications of Implicit and Underspecified Phrases in Instructional Texts,” centered on plausibility judgments for candidate clarifications in instructional discourse [2309.12102]. In later IberLEF work, “CLEARS-2025” denotes a shared task on Spanish text adaptation for accessibility, with Plain Language and Easy-to-Read subtasks [2508.03240]. The term is therefore not a single stable benchmark family in the available sources. This is reinforced by the NFDI4DS survey of scholarly-document shared tasks, which explicitly lists twelve tasks and does not include any task named CLEARS [2509.22141].

## 1. Scope and nomenclature

The best-documented use of the term is the SemEval-2022 shared task officially overviewed as “Identifying Plausible Clarifications of Implicit and Underspecified Phrases in Instructional Texts,” which the source explicitly presents as the CLEARS shared task [2309.12102]. Its subject matter is not simplification, paraphrasing, or generation in the usual sense; instead, it evaluates whether a proposed clarification is plausible in the local and discourse context of an instructional sentence.

A second use appears in IberLEF 2025. There, CLEARS-2025 is described in a system paper as a shared task on automatic Spanish text adaptation, with one subtask targeting Plain Language and another targeting Easy-to-Read rewriting [2508.03240]. The two uses share the shared-task format and an interest in language adaptation, but they are methodologically and substantively different.

The literature also contains a recurrent source of confusion with **CLEF**. The NFDI4DS overview notes that no task named CLEARS appears among its twelve shared tasks and identifies the closest CLEF-hosted neighbor as CheckThat!Lab Task 4: Scientific Web Discourse [2509.22141]. A plausible implication is that references to “CLEARS” require venue- and year-level disambiguation rather than acronym matching alone.

## 2. SemEval-2022 CLEARS: task formulation

SemEval-2022 CLEARS is motivated by the observation that instructional texts frequently contain language that is **implicit** or **underspecified**. A step may omit a referring expression entirely, use a phrase whose intended head noun is left implicit, or otherwise require readers to infer what is being referred to. The official overview emphasizes that this matters particularly in instructional settings because misunderstanding can prevent goal completion and, in some cases, lead to serious errors [2309.12102].

The task is framed as a **cloze-style evaluation**. Each instance contains a sentence from an instructional text with a marked insertion site, surrounding discourse context, and a candidate filler. A “clarification” is the phrase inserted into the original text to make omitted or underspecified meaning more explicit. Systems do not generate clarifications from scratch in the shared task; they assess the plausibility of a given candidate.

Two subtasks are defined. **Task 1: Classification** predicts one of three labels—**implausible**, **neutral**, or **plausible**—for a candidate clarification. This is a **3-way classification** setup evaluated primarily by **accuracy**. **Task 2: Ranking** predicts a continuous plausibility score, evaluated by **Spearman’s rank correlation coefficient** with human judgments [2309.12102].

A distinctive feature of the benchmark is that plausibility is explicitly **non-unique**. Multiple fillers for the same context may all be plausible. The overview illustrates this with a hair-salon guide sentence, “Call ____ and ask questions,” where the revised human insertion is “the salon,” but alternatives such as “the number” or “the owner” may also be plausible to varying degrees [2309.12102]. This design opposes the common misconception that clarification recovery should be modeled as a single-correct-answer completion problem.

## 3. Data construction and annotation in SemEval-2022 CLEARS

The dataset was built from revision histories of wikiHow guides, specifically the **wikiHowToImprove** resource, which contains sentence-level revisions and context from more than **250,000** how-to guides [2309.12102]. Data construction proceeded in three stages.

First, the organizers extracted clarification insertions from revision histories. They restricted attention to cases with a **single contiguous insertion** and no other change within a sentence. Diffs were computed with Python’s `difflib`, while preprocessing used **spaCy** for sentence splitting and tokenization, the **Berkeley Neural Parser** for constituency parsing, and **Stanza** for POS tagging, dependency parsing, and coreference resolution [2309.12102].

Second, the organizers focused on four clarification phenomena.

| Phenomenon | Revision pattern | Illustrative characterization |
|---|---|---|
| Implicit reference | `∅ → [DET] NOUN` | A non-verbalized referent is clarified by inserting a noun phrase |
| Fused head | `DET/JJ ∅ → DET/JJ NOUN` | A noun phrase with an implicit head noun is clarified by inserting the noun |
| Noun compound | `∅ NOUN → NOUN NOUN` | A dependent noun is inserted to create a more specific compound |
| Metonymy | Possessive or “of”-phrase addition | A noun is added to make explicit an aspect or component of another noun |

Candidate filler sets were then constructed with language modeling. For each instance, the organizers generated the **top-100 fillers** for the insertion position. Unlike the pilot study, which had used GPT, the shared task used **BERT (`bert-base-uncased`)** because the insertions were one token long and BERT could directly condition on right context. After filtering out unsuitable outputs and wrong POS categories, they included the observed human insertion when possible and selected four additional fillers by **k-means clustering with \(k=4\)** over BERT embeddings, yielding effectively **five fillers per sentence** [2309.12102].

Third, human plausibility judgments were collected with **Amazon Mechanical Turk** on a **1-to-5 scale**. Annotators saw the clarification underlined in context and rated one clarification at a time. The **training set** contains **19,975 instances**, corresponding to **3,995 sentences** with **5 fillers each**. These sentences are distributed as **1000 noun compounds**, **1000 metonymy**, **996 implicit references**, and **999 fused heads**. The **development set** contains **2,500 instances**, and the **test set** also contains **2,500 instances**; each corresponds to **500 sentences total**, with **125 sentences per phenomenon** [2309.12102].

Worker screening was relatively strict: workers had to be located in the **United States or United Kingdom**, have a HIT approval rate of at least **95%**, and have completed at least **1000 approved HITs**. They also had to pass a qualification test. The organizers later increased the number of qualification questions from **4 to 6**, which they believe improved dev/test quality [2309.12102].

The official label mapping converts averaged human ratings into three classes: **implausible** if average score \(\leq 2.5\), **plausible** if average score \(\geq 4.0\), and **neutral** otherwise. The training label distribution is **5,474 implausible (27%)**, **7,162 neutral (36%)**, and **7,339 plausible (37%)**. An especially important statistic is the average number of plausible clarifications per sentence: **1.84** in training, **1.87** in development, and **1.84** in test [2309.12102]. This confirms that the benchmark was intentionally designed around the possibility of multiple plausible clarifications.

## 4. Systems, baselines, and results in SemEval-2022 CLEARS

The official overview reports a **naive majority-class baseline** of **39%** accuracy and an organizers’ **BERT-based baseline** of **45.7%** on Task 1. A total of **21 users** participated in the CodaLab competition, and **8 teams** submitted system description papers. All described systems were based on **Transformer architectures**, including **BERT, DeBERTa, ELECTRA, ERNIE, RoBERTa, S-BERT, T5, and XLM-R** [2309.12102].

The strongest systems were predominantly ensemble-based. **X-PuDu**, the winning team, used an **ensemble** combining **DeBERTa, ERNIE, and XLM-R**, described as **pattern-aware** and using **multi-loss** training. **HW-TSC** used an **ensemble** with **DeBERTa, RoBERTa, and S-BERT**, including an **unsupervised model**. **PALI** used an **ensemble** with **DeBERTa, RoBERTa, and XLM-R**, also **pattern-aware** and **multi-loss**. Other notable variants included **T5** with **ordinal regression** and **multi-loss** in **Nowruz**, and **class weighting** in **DuluthNLP** [2309.12102].

The official Task 1 results show a challenging but learnable benchmark. The **human upper bound** is **79.4%** accuracy, while **X-PuDu** achieved **68.9%**. The remaining described teams scored **HW-TSC 66.1%**, **PALI 65.4%**, **Nowruz 62.4%**, **JBNU-CCLab 61.4%**, **DuluthNLP 53.3%**, **Stanford MLab 46.6%**, and **niksss 44.2%** [2309.12102].

| Team | Task 1 accuracy | Task 2 Spearman |
|---|---:|---:|
| X-PuDu | 68.9% | 0.807 |
| HW-TSC | 66.1% | 0.774 |
| PALI | 65.4% | 0.785 |
| Nowruz | 62.4% | 0.707 |

For **Task 2: Ranking**, the top correlations were **X-PuDu 0.807**, **PALI 0.785**, **HW-TSC 0.774**, and **Nowruz 0.707**. The system ordering in Task 2 largely matched the classification ordering, except for the last two teams reported in the overview [2309.12102].

The post-hoc analyses are central to the task’s significance. When the organizers reevaluated systems **excluding the neutral label**, performance rose sharply for all systems. For **X-PuDu**, \(F_1\) increased from **0.689** to **0.773**. The paper interprets this as evidence that the **neutral** label is a major source of difficulty, plausibly because it conflates intermediate plausibility with annotator disagreement [2309.12102]. In a further evaluation on identifying contexts with **multiple plausible clarifications**, the best system achieved **75.2%** accuracy on the criterion \(\#p \ge 2\), supporting the claim that top systems can help identify potentially ambiguous instructional contexts.

## 5. CLEARS-2025 at IberLEF: Spanish accessibility rewriting

A later use of the acronym appears in IberLEF 2025. A CardiffNLP system paper describes **CLEARS-2025** as a shared task on **Spanish text adaptation**, with two subtasks targeting **Plain Language (PL)** and **Easy-to-Read (E2R)** rewriting [2508.03240]. The task is motivated by the difficulty of official, administrative, and public-facing texts, and it focuses on transforming Spanish source texts into more accessible forms.

The paper stresses that the two subtasks are not identical. **Plain Language** targets a broad audience, including non-native speakers and readers with limited literacy or reading difficulties; its emphasis is on clarity, concision, active voice, common vocabulary, and the removal of ambiguity and jargon. **Easy-to-Read** goes further and is intended for readers with cognitive, intellectual, or learning disabilities, following the **UNE 153101 EX guidelines**. It therefore requires stronger structural simplification, reduced cognitive load, short sentences, direct phrasing, and often formatting decisions such as splitting information into separate lines [2508.03240]. A common misconception rejected in the paper is that E2R is simply “more simplification”; the authors treat it as a partially different target style.

As described in that system paper, the shared-task corpus contained **3,000 municipal news items from Alicante province**, with **2,100 training articles** and **900 test articles**, and each article had both a PL version and an E2R version [2508.03240]. CardiffNLP approached both subtasks with prompt-based LLM rewriting, eventually selecting **Gemma-3** after experiments with **LLaMA-3.2** and other prompt variants.

The official semantic ranking metric is reported as the average of cosine similarity computed over **TF-IDF embeddings** and **BERT embeddings**. In **Subtask 1**, CardiffNLP scored **63%** with TF-IDF and **77%** with BERT, averaging to **70%**, which placed the team **third**; the best team reached **75%**. In **Subtask 2**, CardiffNLP scored **65%** TF-IDF cosine and **77%** BERT cosine, averaging to **71%**, which earned **second place** behind **72%** [2508.03240]. This later CLEARS usage thus centers not on ambiguity detection in instructional text, but on controlled accessibility-oriented rewriting of Spanish public information. This suggests acronym reuse rather than direct continuity with SemEval-2022 CLEARS.

## 6. Broader methodology and related traditions

SemEval-2022 CLEARS belongs to a broader shared-task tradition that emphasizes **phenomenon-sensitive evaluation** rather than only aggregate i.i.d. held-out performance. A conceptually related precursor is **Build It Break It, The Language Edition**, which used builders and breakers, **minimal pairs**, and targeted perturbations to probe generalization beyond training distributions [1711.01505]. CLEARS differs in format, but it shares the premise that evaluation should expose specific weaknesses—here, reasoning about implicit reference, fused heads, noun compounds, and metonymy—rather than treating all instances as homogeneous.

The organization of shared tasks is also a live methodological issue. Community survey work on shared-task transparency proposes a **Shared Task Organisation Checklist** spanning **Transparency**, **Reporting and Replicability**, **System Ranking**, and **Metrics**, and argues that organizers should make explicit matters such as whether organizers, annotators, or evaluators can participate; whether the task is **open**, **closed**, or **both**; whether evaluation metrics are fixed at announcement time; and whether code, data, and system descriptions will be released [2105.05020]. Those recommendations are directly pertinent to CLEARS-like tasks, especially where interpretation of ambiguity, evaluation design, and ranking incentives can materially shape what systems optimize.

Finally, the NFDI4DS overview is important as a disambiguation source. It explicitly states that the consortium’s twelve shared tasks do **not** include a task named CLEARS and identifies **CheckThat!Lab Task 4: Scientific Web Discourse** as the closest CLEF-hosted neighbor [2509.22141]. A plausible implication is that “CLEARS Shared Task” should be treated as a context-dependent label whose meaning is fixed by venue, year, and paper lineage.

In the literature available here, the most authoritative and fully specified CLEARS benchmark remains SemEval-2022 Task 7: a shared task on plausibility judgments over clarifications in instructional text, notable for modeling ambiguity, disagreement, and the presence of multiple plausible clarifications in a principled benchmark design [2309.12102].

Source: https://www.emergentmind.com/topics/clears-shared-task