Bridging the Gap: In-Context Learning for Modeling Human Disagreement (2506.06113v1)

Abstract: LLMs have shown strong performance on NLP classification tasks. However, they typically rely on aggregated labels-often via majority voting-which can obscure the human disagreement inherent in subjective annotations. This study examines whether LLMs can capture multiple perspectives and reflect annotator disagreement in subjective tasks such as hate speech and offensive language detection. We use in-context learning (ICL) in zero-shot and few-shot settings, evaluating four open-source LLMs across three label modeling strategies: aggregated hard labels, and disaggregated hard and soft labels. In few-shot prompting, we assess demonstration selection methods based on textual similarity (BM25, PLM-based), annotation disagreement (entropy), a combined ranking, and example ordering strategies (random vs. curriculum-based). Results show that multi-perspective generation is viable in zero-shot settings, while few-shot setups often fail to capture the full spectrum of human judgments. Prompt design and demonstration selection notably affect performance, though example ordering has limited impact. These findings highlight the challenges of modeling subjectivity with LLMs and the importance of building more perspective-aware, socially intelligent models.

LuaLaTeX and XeLaTeX Template Utilization in *ACL Style Files

The academic paper under consideration provides a technical guide on employing LuaLaTeX and XeLaTeX for the preparation of manuscripts adhering to the *ACL style guidelines. Targeted towards researchers engaging with document preparation systems, particularly in the computational linguistics domain, the paper outlines a methodical approach for implementing multilingual text, a critical feature given the diversity of languages in computational research.

LaTeX Integration with *ACL Styles

At its core, the document emphasizes the ease of integrating LuaLaTeX and XeLaTeX—both extensions of the TeX typesetting system—into the *ACL (Association for Computational Linguistics) style files. LuaLaTeX and XeLaTeX provide robust solutions for handling complex script and font requirements, essential for compiling documents in a linguistically diverse environment.

Multilingual Capability

The paper exemplifies the incorporation of different language scripts, specifically Hindi and Arabic, underscoring the necessity to accommodate various linguistic datasets native to the computational linguistics field. It utilizes the babel package, demonstrating how different typefaces (TeX Gyre Termes, Lohit Devanagari, and Noto Sans Arabic) are employed for the proper rendering of respective scripts. This feature is pivotal in ensuring that research papers maintain textual integrity across multiple languages and scripts, often a challenge in global academic dissemination.

References and Citation Management

Additionally, the paper discusses the handling of citations, referencing seminal works like Gusfield (1997) for algorithms on strings and Aho and ULLMan (1972) on parsing and compiling, among others. This exemplifies the structured approach these LaTeX systems offer for managing extensive bibliographic data—vital for academic scholarship in building upon previous work with accuracy and coherence.

Practical and Theoretical Implications

The practical implications of such a template are profound, simplifying the manuscript preparation process for researchers who aim to publish their findings while adhering to established stylistic conventions. It streamlines the integration of multiple languages into a single document, facilitating linguistic pluralism in the field's scholarly discourse. Theoretically, this template stands as an adaptive tool that encourages the submission of more varied and inclusive research work to *ACL conferences, potentially fostering a richer exchange of ideas.

Future Developments

Looking forward, the scholarly community can anticipate enhancements in LaTeX environments that further simplify language integration and document preparation, such as expanding script support and optimizing template efficiency. The continued evolution of LaTeX systems will likely play an essential role in the broader adoption and adaptability of multilingual research documentation standards within computational linguistics and neighboring fields.

In summary, this paper offers critical insights into leveraging LuaLaTeX and XeLaTeX within *ACL style frameworks, promoting improved manuscript preparation and encouraging the multilingual dissemination of computational research contributions. It serves as a valuable resource for researchers requiring comprehensive linguistic support in scholarly publishing.