SG-UniBuc-NLP at SemEval-2026 Task 6: Multi-Head RoBERTa with Chunking for Long-Context Evasion Detection

Abstract: We describe our system for SemEval-2026 Task 6 (CLARITY: Unmasking Political Question Evasions), which classifies English political interview responses by coarse-grained clarity (3-way) and fine-grained evasion strategy (9-way). Since responses frequently exceed the 512-token limit of standard Transformer encoders, we apply an overlapping sliding-window chunking strategy with element-wise Max-Pooling aggregation over chunk representations. A shared RoBERTa-large encoder supplies two task-specific heads trained jointly via a multi-task objective, with inference-time ensembling over 7-fold stratified cross-validation. Our system achieves a Macro-F1 of 0.80 on Subtask 1 and 0.51 on Subtask 2, ranking 11th in both subtasks.

• The paper introduces a hierarchical RoBERTa model with overlapping sliding-window chunking that preserves critical contextual cues in long interview transcripts.
• It employs a multi-task learning framework with dual classification heads, achieving Macro-F1 scores of 0.80 for clarity and 0.51 for evasion detection.
• Error analysis highlights that class imbalance and semantic ambiguities remain challenging, underscoring the need for data augmentation and refined modeling techniques.

Multi-Head RoBERTa with Chunking for Long-Context Political Evasion Detection

Task Context and Technical Setting

This paper addresses the detection of political question evasion in long-form interview transcripts, as posed by SemEval-2026 Task 6 (CLARITY). The task is operationalized as classifying English question--answer (QA) pairs at two levels: a 3-way clarity taxonomy (Clear Reply, Clear Non-Reply, Ambivalent) and a 9-way fine-grained evasion taxonomy. The dataset consists of 3,756 QA pairs from White House interviews, with strong label imbalance and varying response lengths—more than a quarter of instances exceed the contextual limits of standard Transformer encoders, introducing a challenge for evidence preservation throughout the input.

Hierarchical Model Architecture

To address the long-context input regime, the authors design a hierarchical architecture centered on RoBERTa-large encoders, employing an overlapping sliding-window chunking strategy. Each Q+A input is tokenized and segmented into overlapping windows of 512 tokens (stride 256), such that each token except those at the extreme document ends appears in at least two chunks. Each chunk is represented via the encoder's embedding at the first token position, and all chunk representations are aggregated using element-wise Max-Pooling to form a global response-level vector. This approach strictly bounds memory utilization and avoids out-of-memory errors commonly encountered by long-sequence Transformer fine-tuning, while maintaining compatibility with standard pretrained encoders.

Figure 1: The majority of sample QA pairs fit within standard RoBERTa input lengths, but approximately 29% exceed the 512-token limit, necessitating a chunking strategy.

This Max-Pooling representation is then shared between two task-specific linear classification heads, which are trained jointly under a multi-task objective. The joint loss regularizes the more challenging evasion classification with the (stronger signal) coarse-grained clarity objective. Training is performed via 7-fold stratified cross-validation, with predicted probabilities ensembled at test time by arithmetic mean.

Empirical Results

On the SemEval-2026 blind test set, the proposed system achieves a Macro-F1 of 0.80 on Clarity and 0.51 on Evasion, ranking 11th out of 41 and 33 teams, respectively. These results validate the effectiveness of chunking with Max-Pooling aggregation, especially compared against two critical baselines:

• Pooling strategy: Max-Pooling consistently outperforms Mean-Pooling and a First-Chunk baseline by 2-3 points in Macro-F1 on both subtasks, indicating that critical evidential cues for evasion are often localized and may not be well represented by average pooling.
• Multi-task learning: Jointly training on the clarity task provides regularization and boosts Macro-F1 for fine-grained evasion detection by 3 points compared to the single-task variant.

Ensembling across more folds further improves robustness, though at proportional computational expense.

Error Landscape and Model Analysis

The analysis of both confusion matrices highlights class imbalance and label overlap as dominant sources of prediction error.

Figure 3: The Clarity confusion matrix reveals "Ambivalent" acts as a majority sink, absorbing a substantial fraction of Clear Reply and Clear Non-Reply instances.

Figure 2: The Evasion confusion matrix demonstrates high confusion between semantically proximate strategies, especially for minority and implicature-driven labels.

Performance correlates with the lexical explicitness of evasion strategy: classes such as Clarification, Explicit, and Declining to answer exhibit much higher recall and confidence than categories dependent on inferential or pragmatic cues (General, Implicit, Deflection, or Partial/half-answer). Minority classes (e.g., Partial/half-answer, Claims ignorance) are rarely predicted, and attempts at mitigating imbalance via reweighting or focal loss degrade Macro-F1 by sacrificing majority-class performance more than they recover minority recall. Qualitative inspection confirms that model errors are correlated with peaks of human annotator disagreement, and remaining predictive failures are often inherent ambiguities rather than isolated modeling defects.

Theoretical and Practical Implications

This study demonstrates that hierarchical chunking with Max-Pooling is an effective alternative to sparse-attention or extended-encoder approaches for long-context classification tasks where only moderate fractions of input exceed standard Transformer limits. The strict memory control and compatibility with standard finetuning pipelines are operational benefits, particularly in resource-constrained environments.

From an annotation and representational perspective, error analysis reinforces that fine-grained classification of political evasion exhibits fundamental ambiguity, with label boundaries that are not well-captured by surface lexical forms alone, nor always agreed upon by human annotators. The persistent failures on minority classes indicate that overcoming data sparsity and semantic overlap requires external knowledge integration or representation learning beyond current context-aggregation strategies.

Prospective Directions

Future work should pursue targeted data augmentation for minority classes, either via synthetic paraphrases or leveraging new annotation. Given that concatenating questions and answers as a flat input fails to explicitly model their relational structure, approaches drawn from NLI (e.g., late-interaction dual encoders such as ColBERT) could encode more precise answer relevance and evasion dynamics. Advances in retrieval-augmented and inference-focused architectures may facilitate progress on detecting fine-grained pragmatic strategies.

Conclusion

This system substantiates the viability of hierarchical chunking and Max-Pooling feature fusion for long-context political evasion detection. Ensemble-based multi-task learning is shown to provide regularization, especially for label-imbalanced problems, though performance remains circumscribed by data distribution and inherent ambiguity. Research that targets coreference, the QA dependency structure, and pragmatic discourse modeling holds promise for future advances in this domain.