Annotator-Specific Preference Modeling

• The paper introduces techniques that capture individual annotator biases using mathematical models such as mixed-effects, latent variable mixtures, and intuitionistic fuzzy sets.
• It details methods like query-based embeddings, dynamic weighting, and EM estimation to calibrate and operationalize individual preferences in complex annotation tasks.
• Aggregated strategies and fairness-focused ensembles improve LLM alignment, medical image segmentation, and multimodal evaluations by preserving annotator diversity.

Annotator-specific preference modeling encompasses statistical, algorithmic, and representation techniques to capture, analyze, and operationalize the differences in how individual annotators judge, score, or select among options in subjective or complex data annotation tasks. Unlike consensus-oriented aggregation, which seeks to recover a single “truth” from diverse human feedback, annotator-specific frameworks model systematic deviations, unique tendencies, and uncertainty that arise due to heterogeneous expertise, personal bias, task difficulty, or contextual factors. These models undergird state-of-the-art data annotation protocols, LLM alignment via reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO), medical image segmentation, multimodal evaluation, and beyond.

1. Mathematical Foundations of Annotator-specific Preference Models

The parameterization of individual annotator preferences spans classic mixed-effects models, latent variable mixture models, density estimation approaches, and structured neural architectures.

• Intuitionistic Fuzzy Sets (IFS): Each annotator’s judgment of an option $x$ is encoded as a triplet $(\mu_A(x),\nu_A(x),\pi_A(x))$ where $\mu_A(x)$ is the support (degree of preference), $\nu_A(x)$ is the opposition (degree of rejection), and $\pi_A(x)=1-\mu_A(x)-\nu_A(x)$ is the hesitation (uncertainty). Constraints enforce $\mu_A(x),\nu_A(x)\in[0,1]$ and $\mu_A(x)+\nu_A(x)\leq1$ (Du, 30 May 2025).
• Mixed-Effects Utility: In pairwise comparisons, the observed score is modeled as $$y_{ij}^u = \beta^\top x_{ij} + (u^u)^\top x_{ij} + \gamma^u + \epsilon_{ij}^u$$, with $\beta$ as global consensus, $u^u$ as annotator deviation, and $\gamma^u$ as position bias, regularized for parsimonious representation (Xu et al., 2018).
• Multi-task Decomposition: Personalized attribute ranking weights per user/task $W^{(i)}$ are decomposed as $W^{(i)} = \theta + G^{(i)} + P^{(i)}$, capturing consensus ($\theta$), co-cluster group structure ($G^{(i)}$), and fine-grained personalization ($P^{(i)}$), optimized against AUC-based loss (Yang et al., 2019).
• Mixture Models with Latent Types: Annotators possess latent “preference types” $Z_i$; each subgroup is indexed by $k$, generating preference data via group-specific policy $\pi_{\theta_k}$, and fit via EM over annotator responsibilities $\gamma_{i,k}$ (Chidambaram et al., 2024, Chidambaram et al., 17 Oct 2025).
• Graph-based User-Response Interaction: Annotators and responses are nodes in a bipartite, signed graph; message passing captures multi-hop relationships, enabling learned user and response embeddings for collaborative filtering of pairwise preferences (Choi et al., 3 Mar 2025).

2. Elicitation, Estimation, and Calibration of Annotator-specific Parameters

Preference modeling begins with direct elicitation, systematic calibration, and dynamic estimation protocols:

• Direct Elicitation: Annotators adjust sliders or scales to report $\mu_A(x)$ and $\nu_A(x)$ per option; interface enforces feasibility ($\mu+\nu\leq1$, compute $\pi=1-\mu-\nu$ instantly) (Du, 30 May 2025).
• Calibration on Gold Standards: Affine mappings $f_i$ ($\hat\mu_i = \alpha_i\,\mu_{\textrm{raw}}+\beta_i$) are fitted per annotator against benchmark examples to align raw preferences with reference judgments, minimizing IFS distance (Du, 30 May 2025).
• Dynamic Weighting: Each annotator is assigned a weight $w_i$ by normalized combination of consistency (variance of hesitation), expertise (accuracy vs. gold), and agreement (mean IFS distance to peers), $$w_i = a\,\textrm{consistency}_i + b\,\textrm{expertise}_i + c\,\textrm{agreement}_i$$ (Du, 30 May 2025).
• Principal–Agent Contract Modeling: Annotators' intrinsic preference for effort $\eta$ is monitored via continuous-action principal–agent analysis, allowing inference of annotation quality and incentivization via binary/linear contract optimization (Liu et al., 10 Feb 2025).
• Query-based Embedding: Each annotator is represented by a parameter-light learnable query vector $\mathbf{q}_k$ (in $\mathbb{R}^d$), which attends to sample features and to other queries (through self-attention), capturing both tendency and inter-annotator correlation (Zhang et al., 19 Mar 2025, Zhang et al., 23 Jul 2025).

3. Aggregation and Consensus under Annotator Heterogeneity

Consensus labels must reconcile diverse, sometimes conflicting, annotator-specific preferences:

Aggregation Method	Core Formula / Mechanism	Contexts of Use
Weighted IFS Averaging	$\mu_{\text{agg}}(x) = \sum_{i=1}^k w_i\,\mu^i(x)$; normalization when $\mu+\nu>1$	Side-by-side preference annotation for LLMs (Du, 30 May 2025)
EM-DPO Mixture Policies	Soft assignment $\gamma_{i,k}$, optimize $\theta_k$ on weighted data, mixture $w_k$	RLHF/DPO with latent type discovery (Chidambaram et al., 17 Oct 2025, Chidambaram et al., 2024)
Min-Max Regret Ensemble	$w^* = \arg\min_{w} \max_{k} \mathrm{regret}_k(w)$, regret via policy performance margin	Equitable aggregate over latent subtypes (fairness guarantee)
Self-attention Regularization	Implicit correlation alignment among query vectors in Transformer blocks	Multimodal behavior modeling, tendency preservation
Consensus Mask Fusion	Majority vote, STAPLE, Bayesian confusion-matrix fusion on binary/multi-class segmentation	Medical image annotation (Abhishek et al., 25 Dec 2025, Liao et al., 2021)

IFS aggregation supports nuanced consensus; EM-DPO and related mixture models enable provable identification of latent preferences given ternary or richer data, not mere binary choices. Min-max regret ensembles minimize worst-case policy degradation for minority preference clusters. Query-based self-attention enforces soft-sharing of preference structure, mitigating overfitting in sparse label regimes.

4. Metrics and Evaluation for Annotator-specific Quality and Diversity

Robust evaluation must address both prediction quality and preservation of annotator diversity:

• IFS-specific Metrics:
  • Annotation Confidence: $1-\frac{1}{n}\sum_{i=1}^n \pi(x_i)$
  • Preference Clarity: $\frac{1}{n}\sum_{i=1}^n|\mu(x_i)-\nu(x_i)|$
  • IFS Agreement: $1-\frac{1}{k(k-1)}\sum_{i<j} dIFS(A^i,A^j)$
• Difference of Inter-annotator Consistency (DIC): Measures change in Cohen's $\kappa$ agreement structure before vs. after modeling; $\mathrm{DIC} = \|M-M'\|_F$ (Zhang et al., 19 Mar 2025).
• Consensus vs. Personalization Metrics: Dice, IoU, Hausdorff Distance, calibration error stratified by annotator/tool/skill (Abhishek et al., 25 Dec 2025), per-annotator ROC-AUC, F1, macro-averaged accuracy (Plepi et al., 2022).
• Contract-theoretic Utility Gaps: Quantifies deviation from first-best principal-agent solutions, as $O(n^{-1/2}\log^{-1/2} n)$ for binary, $O(1/n)$ for linear contracts (Liu et al., 10 Feb 2025).
• Interpretability via Sparse Feature Weights: Annotator-specific vectors $w_a$ in SAE models reveal subjective preferences (e.g. formatting, prose style), enabling explicit analysis and targeted personalization (Movva et al., 30 Oct 2025).

High clarity and agreement scores correlate with annotator efficiency and label robustness; DIC provides a quantitative measure for tendency preservation, and dense annotation or rich multi-query attention boosts both individual and consensus performance.

5. Downstream Model Training and Fair Personalization

Preference annotations shaped by individual differences directly determine the quality and fairness of machine learning models:

• LLM Alignment via RLHF/DPO: IFS-aggregated labels are transformed into pairwise probabilities and used for reward-model learning in RLHF, or for policy objectives in DPO (Du, 30 May 2025). EM-DPO retains annotator type separation, minimizing identifiability issues (Chidambaram et al., 17 Oct 2025, Chidambaram et al., 2024).
• Mixture-of-Experts Personalization: Per-user (annotator) LoRA expert adapters are gated by user embeddings, separating global knowledge (shared adapter) from individual “twists” (specialist experts) for response ranking (Choi et al., 3 Mar 2025).
• Medical Image Segmentation: Preference-involved Annotation Distribution Learning (PADL) and EM-based bias/noise estimation decouple consensus and individual annotator segmentation, delivering robust meta segmentations and individualized masks (Liao et al., 2021, Abhishek et al., 25 Dec 2025).
• Query-based Tendency Learning: QuMATL/QuMAB assign a lightweight query embedding to each annotator, cross-attend it to image/video features, and output per-annotator predictions, preserving individualization while exploiting implicit regularization for scalability and robustness (Zhang et al., 19 Mar 2025, Zhang et al., 23 Jul 2025).
• Fair Policy Aggregation: Min-max regret ensembles ensure no minority preference type suffers policy performance collapse, as quantified by explicit regret objectives over mixture weights (Chidambaram et al., 17 Oct 2025, Chidambaram et al., 2024).

Per-annotator and consensus performance measures demonstrate that personalized (not just aggregated) modeling improves outcome accuracy in social norms (Plepi et al., 2022), emotion recognition (Zhang et al., 19 Mar 2025), and LLM alignment (Du, 30 May 2025, Choi et al., 3 Mar 2025).

6. Interpretability, Explainability, and Applicability

Modern approaches to annotator-specific modeling provide interpretable, explainable, and actionable characterizations:

• Sparse Autoencoders (SAE): WIMHF identifies human-interpretable difference features driving annotator decisions; fitting annotator-specific weight vectors $w_a$ enables fine-grained, transparent personalization and direct analysis of subjective features (Movva et al., 30 Oct 2025).
• Visualization of Attention Patterns: Query-based methods (QuMATL/QuMAB) produce interpretable heatmaps of annotator focus regions, illuminating preference-driven divergences in multimodal tasks such as perceptual impression and emotion labeling (Zhang et al., 19 Mar 2025, Zhang et al., 23 Jul 2025).
• Principal-Agent Contract Implications: Explicit modeling of effort, risk, and incentive structures allow principled monitoring and incentivization of annotator quality (Liu et al., 10 Feb 2025).
• Persona-based Prompting in LLMs: Defining strong (individual) and weak (aggregate) data perspectivism, and deploying prompts with natural-language persona description, enables comparative analysis of LLM and human annotator alignment, surfacing homogenization effects in model outputs and highlighting challenges of full preference diversity elicitation (Sarumi et al., 23 Aug 2025).

Explainable models clarify downstream behavior, flag risky or controversial preference signals, and support curation or targeted re-labeling in safety-sensitive datasets (Movva et al., 30 Oct 2025).

7. Limitations, Open Problems, and Future Directions

Despite major progress, several challenges persist:

• Identifiability Under Binary Feedback: Binary comparisons (n=2) are insufficient for general population preference recovery absent impractical numbers of per-user samples; ternary or higher-rank choices guarantee nonparametric identification (Chidambaram et al., 17 Oct 2025).
• Scalability in Large Annotator Pools: Query/self-attention cost grows as $O(K^2)$ with annotator count; sparse, hierarchical, or low-rank methods may be required for ultra-dense crowdsourcing (Zhang et al., 23 Jul 2025).
• Homogenization in LLM “Persona” Modeling: LLMs prompted with persona text tend toward aggregated views with high label agreement, struggling to replicate full spectrum of human preference diversity (Sarumi et al., 23 Aug 2025).
• Sparse Annotation Regimes: Sample coverage remains a core bottleneck; collaborative and graph-based sharing of signals offers mitigation, yet optimal query strategies and annotation policies are an open research area (Choi et al., 3 Mar 2025, Zhang et al., 23 Jul 2025).
• Measurement of Equity and Fairness: Min-max regret minimization formalizes fairness across types, but utility tradeoffs and selection of the regret-optimal mixture remain subject to further exploration (Chidambaram et al., 17 Oct 2025, Chidambaram et al., 2024).

A plausible implication is that future methods must closely couple elicitation protocol design, personalized model architecture, interpretable metric development, and fair aggregation for principled and scalable annotator-specific preference modeling.