Frameworks for flexible, dynamic label spaces in modeling social constructs

Develop modeling frameworks that accommodate flexible and dynamically adjusted label spaces during training and inference to represent ambiguous social constructs in Social-AI agents, rather than relying solely on static predefined discrete or continuous labels.

Background

The paper argues that many social constructs (e.g., rapport, tension) are ontologically subjective and inherently ambiguous, making it difficult to define a single ground truth label. Current modeling practices often aggregate annotations into static discrete or continuous labels, which can misalign with diverse actor and annotator interpretations that vary over time.

To better capture ambiguity, the authors suggest exploring richer and dynamically generated label spaces, potentially expressed in natural language and adjusted during training and inference, and identify the need for frameworks that can operationalize such flexible labeling in Social-AI models.

References

How to best design frameworks to accommodate flexible and dynamic label spaces when modeling social constructs remains an open question and research opportunity.

— Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions  (2404.11023 - Mathur et al., 2024) in Section 4, Subsection (C1) Ambiguity in Constructs, C1 Opportunities and Open Questions

Discerning between noise and valid HLV is not straightforward, but an active research question .

— Post-hoc Alignment of LLM-judges to Human Judgment Distribution  (2609.01073 - Steindl et al., 1 Sep 2026) in Section 2, subsection “Human Label Variation (HLV)”