---
title: CLEARS 2025 Subtask 1 Overview
url: https://www.emergentmind.com/topics/clears-2025-subtask-1
type: topic
---

# CLEARS 2025 Subtask 1 Overview

“CLEARS 2025 Subtask 1” denotes a recurring first-track designation across several heterogeneous 2025 evaluation settings rather than a single uniform benchmark. In the supplied literature, the label is attached to question answering over tabular data, Spanish plain-language rewriting, gastrointestinal endoscopic visual question answering, Bangla hate-speech modeling, spoken Arabic dialect identification, Arabic and Islamic culture multiple-choice evaluation, Islamic inheritance reasoning, and end-to-end autonomous driving [2505.22264] [2508.00762] [2509.17209] [2508.03240] [2507.14544] [2601.06306] [2511.18324] [2509.02038] [2509.02550] [2509.23793] [2606.06219]. This suggests that “Subtask 1” functioned as a stable organizational label for the primary track within distinct benchmark families, while the underlying task definition, output space, and scoring rule remained domain-specific.

## 1. Benchmark landscape and problem scope

Across the cited papers, Subtask 1 is consistently the main entry point of a benchmark family, but the underlying computational problem varies sharply. Some formulations are generative, some discriminative, some executable, and some closed-loop.

| Benchmark family | Subtask 1 formulation | Main reported metric |
|---|---|---|
| SemEval-2025 Task 8 / DataBench | Question-answering over tabular data | Accuracy |
| CLEARS Spanish adaptation | Adaptation of text to Plain Language in Spanish | SIM and Fernández-Huerta |
| ImageCLEFmed MEDVQA-GI 2025 | Visual question answering for GI endoscopy | BLEU, ROUGE, METEOR |
| BLP-2025 Task 1 | Bangla hate-speech type / target-group classification | micro-F1 |
| NADI 2025 | 8-way spoken Arabic dialect identification | Accuracy and $C_{avg}$ |
| PalmX 2025 | General Arabic Culture / General Islamic Culture MCQs | Accuracy |
| QIAS 2025 | Islamic Inheritance Reasoning MCQs | Accuracy |
| NAVSIM v1 setting in CLEAR | End-to-end autonomous driving trajectory planning | PDMS |

In the tabular QA papers, Subtask 1 is framed as “Question-Answering over Tabular Data,” where answers must be recovered from a provided table and matched to accepted output types rather than produced as unconstrained prose [2505.22264] [2508.00762]. In the Spanish adaptation papers, Subtask 1 is the Plain Language track, centered on rewriting municipal or public-administration texts while preserving meaning and improving readability [2509.17209] [2508.03240]. In MEDVQA-GI, Subtask 1 is a multimodal medical VQA problem over gastrointestinal endoscopy images [2507.14544].

The classification-oriented instantiations are equally varied. In BLP-2025 Task 1, the first task concerns Bangla hate-speech analysis, with papers addressing hate-type classification and target-group identification over Bangla social media text and YouTube comments [2601.06306] [2511.18324]. In NADI 2025, Subtask 1 is spoken dialect identification over eight country-level Arabic dialect labels [2509.02038]. In PalmX 2025, Subtask 1 is “General Arabic Culture,” a multiple-choice benchmark for culturally grounded knowledge in Modern Standard Arabic; the paired Islamic-knowledge track is structurally analogous [2509.02550]. QIAS 2025 defines Subtask 1 as Islamic inheritance reasoning over scholar-verified MCQs supported by a fatwa corpus [2509.23793]. CLEAR extends the designation to end-to-end autonomous driving, where the first task is reframed as multimodal trajectory generation, adaptive compute allocation, and semantic candidate selection on NAVSIM v1 [2606.06219].

## 2. Canonical task formulations and data regimes

The tabular QA instantiations are among the most explicitly formalized. MRT characterizes the benchmark as “Question-Answering over Tabular Data” on DataBench, a benchmark of 65 real-world datasets with more than 1,300 manually crafted QA pairs, and reports train/dev/test splits of 49/16/15 tables and 988/320/522 questions. It restricts final answers to Boolean, String, Number, List of Strings, or List of Numbers, and assumes each question must be answered from the provided table [2505.22264]. ITUNLP describes the same Subtask I as zero-shot reasoning over a single relational table per question; in train/dev, fields such as `answer`, `type`, `columns_used`, `column_types`, `sample_answer`, and `dataset` are available, whereas the test split contains only `question` and `dataset` [2508.00762].

The Spanish Plain Language track is described in two closely related but not identical dataset accounts. One paper reports 2,400 training text pairs and 607 source texts in the official test set, drawn from official municipal communications published in 2022–2023 by municipalities such as Madrid, Alicante, Benidorm, Orihuela, Torrevieja, and Elda [2509.17209]. Another describes 3,000 Alicante municipal news items paired with both a Plain Language reference and an Easy-to-Read reference, using a 70/30 split into 2,100 training and 900 test items [2508.03240]. Both papers agree that Subtask 1 targets Plain Language rather than the stricter Easy-to-Read format.

The MEDVQA-GI paper defines Subtask 1 as gastrointestinal endoscopy VQA over Kvasir-VQA, described as containing about 58,849 image–question–answer triplets associated with 6,500 high-resolution images, 20 unique question templates, and 502 unique answers. Because of computational constraints, the reported system uses only a 1% stratified subset, split first into 90% training and 10% testing, and then repartitioning the test portion into training and validation [2507.14544].

The Bangla hate-speech systems expose a shared low-resource and imbalanced setting. SyntaxMind reports 35,522 instances each for Subtask 1A and Subtask 1B, with severe skew toward the “None” class in both tasks [2601.06306]. Gradient Masters frames the same benchmark as Bangla hate-speech identification over YouTube comments, with separate labels for hate type in Subtask 1A and target group in Subtask 1B, and emphasizes spelling variation, transliteration, mixed scripts, and slang [2511.18324].

The Arabic speech and culture benchmarks are more tightly bounded. NADI 2025 defines Subtask 1 as an 8-way spoken dialect classification task over Algerian, Egyptian, Jordanian, Mauritanian, Moroccan, Palestinian, Emirati, and Yemeni speech, using an Adaptation set and a blind Test set with non-overlapping series [2509.02038]. PalmX 2025 defines Subtask 1 as four-option MCQ answering for “General Arabic Culture,” with final split sizes of 2,000 training, 500 development, and 2,000 test questions spanning all 22 Arab countries [2509.02550]. QIAS 2025 defines Subtask 1 as Islamic Inheritance Reasoning with 9,450 training questions, 1,500 validation questions, 1,000 test questions, and 32,000 fatwas as an external knowledge base [2509.23793].

## 3. Dominant methodological paradigms

A central methodological divide is between direct answer generation and constrained intermediate structure. The tabular QA systems both reject pure end-to-end text answering. MRT decomposes inference into a Column Descriptor, Explainer, Coder, Runner, Interpreter, and Formatter. The Column Descriptor computes datatype, unique values, missingness, min/max/mean/std when applicable, frequent values, and an LLM-generated column description; the Explainer turns the question and metadata into natural-language instructions; the Coder emits a single Pandas function with signature `def parse_dataframe(df: pd.DataFrame) -> str: ...`; the Runner executes the code with retries; the Interpreter infers the answer type; and the Formatter applies final conversions [2505.22264]. ITUNLP uses a lighter zero-shot code-generation pipeline: schema prompting, one Python code block, controlled execution, and up to two correction iterations with error messages and faulty code fed back to the model [2508.00762].

The Plain Language systems are prompt-centric rather than execution-centric. HULAT-UC3M compares zero-shot prompting and LoRA fine-tuning over Salamandra-7B-Instruct and RigoChat-7B-v2, with a normalization pipeline handling dates, times, money amounts, percentages, years, ranges, and large numbers. Its final selection is RigoChat-7B-v2 with a dedicated Plain-Language-oriented prompt and no fine-tuning [2509.17209]. CardiffNLP likewise avoids model fine-tuning, instead iterating through zero-shot, one-shot, and few-shot prompts over LLaMA-3.2 and then Gemma-3. Its most distinctive operational constraint is to force the model to emit the simplification as a Python dictionary value, combined with explicit sentence-level simplification instructions [2508.03240].

The multimodal and classification papers favor domain-specialized architectures. MEDVQA-GI fine-tunes `microsoft/Florence-2-base-ft`, freezing the ViT-L/14 vision tower and using autoregressive answer generation with domain-specific augmentations tailored to endoscopy images [2507.14544]. SyntaxMind uses BanglaBERT embeddings followed by parallel Bi-GRU and CNN branches, each with attention, then feature fusion and dense classification [2601.06306]. Gradient Masters uses transformer fine-tuning with k-fold cross-validation, light rule-based normalization, and FGSM adversarial training in embedding space, with $\epsilon = 0.1$ and $\alpha = 0.5$ [2511.18324].

The Arabic speech and knowledge benchmarks show another recurring pattern: domain adaptation over strong pretrained backbones. NADI’s baseline is SpeechBrain VoxLingua107 ECAPA-TDNN, while the winning system, ELYDATA-LIA, uses Whisper Large-v3 with two-stage fine-tuning, partial encoder freezing, and augmentations such as speed perturbation, added noise, frequency dropping, and chunk dropping [2509.02038]. PalmX participant systems are constrained to decoder-only models of at most 13B parameters and rely mainly on full fine-tuning or LoRA over models such as NileChat-3B, Fanar-1-9B-Instruct, and ALLaM-7B-Instruct [2509.02550]. QIAS adopts a hybrid RAG stack—Arabic preprocessing, BM25 retrieval, dense retrieval with Arabic-Triplet-Matryoshka-V2, reranking with `cross-encoder/mmarco-mMiniLMv2-L12-H384-v1`, and prompting of Fanar or Mistral with the top five passages [2509.23793]. CLEAR uses yet another structured pipeline: frozen Drive-JEPA features, a single-step conditional drift decoder in VAE latent space, Qwen 3.5 0.8B hidden states as semantic embeddings, an Adaptive Scheduler over predefined $(\alpha, N)$ schemes, and a Cross-Attention Scorer for trajectory selection [2606.06219].

## 4. Evaluation regimes and reported results

The literature reports task-specific metrics rather than a single CLEARS-wide score. Raw values are therefore not directly comparable across tracks, but they do show the operating range of each Subtask 1 formulation.

| Area | Metric | Reported result |
|---|---|---|
| DataBench table QA | Accuracy | MRT: 70.50%; ITUNLP best test: 84.67 with DeepSeek-V3 |
| Spanish Plain Language | SIM / FH | HULAT-UC3M: SIM 0.75, FH 69.72; CardiffNLP: final averaged cosine similarity 70%, FH 78.81 |
| MEDVQA-GI | BLEU / ROUGE-L / METEOR | Private test: 0.160 / 0.880 / 0.490 |
| Bangla hate speech | micro-F1 | SyntaxMind: 0.7345 in 1A and 0.7317 in 1B; Gradient Masters: 72.33 test in 1A and 73.44 test in the 1B table, with 73.28 cited in the abstract |
| Spoken Arabic dialect ID | Accuracy / $C_{avg}$ | ELYDATA-LIA: 79.8 / 17.88 |
| PalmX MCQs | Accuracy | 72.15% on General Arabic Culture; 84.22% on General Islamic Culture |
| Islamic inheritance reasoning | Accuracy | Fanar + Transformer Tafsir: 45.0% |
| Autonomous driving | PDMS | CLEAR: 93.7 |

Within DataBench, MRT reports a 70.50% test score for Subtask 1, while ITUNLP reports substantially higher open-source-model results, including 84.67 test accuracy for DeepSeek-V3 and 84.09 for DeepSeek-R1 on Subtask I [2505.22264] [2508.00762]. In the Spanish adaptation track, HULAT-UC3M ranks first in semantic similarity with SIM = 0.75 but fourth in readability with FH = 69.72, whereas CardiffNLP reports third place in Subtask 1 with a final averaged cosine similarity of 70% and Fernández-Huerta 78.81 [2509.17209] [2508.03240].

The multimodal medical and speech benchmarks report stronger task-specific ceilings. MEDVQA-GI reaches private-test BLEU 0.160, ROUGE-1 0.880, ROUGE-L 0.880, and METEOR 0.490 [2507.14544]. NADI’s top spoken-dialect system reaches 79.8% accuracy with $C_{avg}=17.88$, a large gain over the 61.1% / 34.22 baseline [2509.02038]. PalmX reports 72.15% on General Arabic Culture and 84.22% on General Islamic Culture, while QIAS reaches only 45.0% on inheritance reasoning, reflecting the much heavier symbolic burden of that benchmark [2509.02550] [2509.23793]. CLEAR reports PDMS 93.7 on NAVSIM v1, with particularly strong NC 99.1, DAC 98.8, and TTC 97.2 [2606.06219].

## 5. Recurrent failure modes and benchmark-specific bottlenecks

The table QA papers provide the most granular error accounting. MRT attributes 37.66% of errors to wrong instructions from the Explainer, 14.29% to wrong cell-value filtering, 14.29% to wrong code or exceptions, 6.49% and 3.90% to formatting issues depending on category, and 23.38% to “others” such as ambiguity, unclear expected answers, or incorrect ground truth. It further states that value matching failures such as “Biden” versus “Joe Biden” alone explain about 15% of the errors [2505.22264]. ITUNLP identifies runtime, syntax, and degenerate-loop errors, and reports that the correction loop roughly halves execution errors on average [2508.00762].

The Plain Language papers converge on a different bottleneck: unstable supervision and metric tension. HULAT-UC3M stresses that training adaptations often did not clearly follow ISO 24495-1:2023, that some adapted texts added information while others omitted content, and that improving Fernández-Huerta often reduced semantic similarity [2509.17209]. CardiffNLP reports prompt failures such as copying the original sentence, using English, producing the wrong format, hallucinating dates and numbers, or copying few-shot examples [2508.03240].

Reasoning-heavy MCQ tasks expose the limits of retrieval and similarity. QIAS identifies three main failure modes in inheritance reasoning: complex fractional reasoning, relevant but incomplete retrieved passages, and logical inference beyond retrieval [2509.23793]. PalmX observes that general Arabic culture is harder than Islamic knowledge because it is more heterogeneous, more region-specific, and less canonical, while the utility of data augmentation is domain-dependent [2509.02550].

The classification and speech benchmarks are dominated by imbalance and coverage effects. SyntaxMind notes severe class imbalance, especially for Sexism in Subtask 1A [2601.06306]. Gradient Masters shows that minority classes such as Sexism, Community, and Society are frequently collapsed into None, and that external data augmentation from public Bangla hate-speech datasets helped development performance but hurt test performance because of domain mismatch [2511.18324]. NADI explicitly warns that not all Arabic dialects are represented, that country-level labels simplify a continuous dialect space, and that real-world speech conditions such as background noise, disfluencies, accented speech, and code-switching remain underrepresented [2509.02038]. MEDVQA-GI reports that heavy augmentation, including vertical flip and random rotation, degrades performance relative to standard and fine-tuned augmentation, and that the model is stronger on spatial or binary questions than on abstract or procedural ones [2507.14544]. CLEAR identifies a different pair of limitations: the scheduler operates over a discrete grid of $(\alpha, N)$ schemes, and the overall system requires multi-stage training [2606.06219].

## 6. Cross-cutting significance and research directions

Several cross-task regularities recur despite the heterogeneity of Subtask 1. First, many successful systems replace unconstrained generation with an explicit intermediate representation: Pandas code in tabular QA, dictionary-valued outputs in simplification, cross-encoder-ranked passages in RAG, adjective bottleneck vectors in interpretable sexism detection, or predefined $(\alpha, N)$ schemes in autonomous driving [2505.22264] [2508.00762] [2508.03240] [2509.23793] [2606.06219]. This suggests that Subtask 1 settings repeatedly reward controllability and typed structure over free-form generation.

Second, domain-specific adaptation is consistently privileged over generic model scale. QIAS reports that Fanar is preferable to a generic model for Islamic inheritance reasoning, even though retrieval gains remain limited [2509.23793]. PalmX shows NileChat-3B reaching the top cultural score through task-specific fine-tuning, while LoRA emerges as the predominant and most effective strategy among participants [2509.02550]. SyntaxMind chooses BanglaBERT because general pretrained models can exhibit language-specific bias [2601.06306]. HULAT-UC3M favors RigoChat-7B-v2 because it is Spanish-specialized [2509.17209]. MEDVQA-GI emphasizes augmentations that preserve mucosal texture, bleeding points, lesions, and anatomical orientation rather than generic image perturbation [2507.14544].

Third, compute awareness is not incidental but architectural. MRT switches from per-question sequential execution to batch-by-step execution so that only the model needed for the current stage remains loaded on a single RTX A6000 (48 GB) [2505.22264]. HULAT-UC3M uses LoRA on a single NVIDIA L4 GPU with 23 GB VRAM [2509.17209]. Gradient Masters reports dual NVIDIA T4 constraints and explicitly rejects more expensive adversarial methods such as PGD or AWP [2511.18324]. CLEAR makes compute allocation itself a learned subproblem through adaptive selection of sample count $N$ and conditioning coefficient $\alpha$ [2606.06219].

Finally, the future-work statements are notably consistent in one respect: they call for stronger grounding than current automatic metrics or generic prompting can provide. MRT proposes reducing losses introduced between pipeline stages and moving toward a less code-centric, more linguistically robust approach [2505.22264]. The Spanish adaptation papers call for richer evaluation beyond cosine similarity and Fernández-Huerta, ideally with stronger user-centered validation and closer alignment with ISO 24495-1:2023 [2509.17209] [2508.03240]. MEDVQA-GI points to visual grounding, uncertainty handling, semantic medical reasoning, and multi-turn conversational VQA [2507.14544]. NADI points to broader dialect coverage and more robust dialect-aware speech technology [2509.02038]. Taken together, these directions indicate that CLEARS 2025 Subtask 1, across its many instantiations, served primarily as a testbed for controlled adaptation under domain-specific constraints rather than as a single monolithic task.

Source: https://www.emergentmind.com/topics/clears-2025-subtask-1