User Willingness Annotations

• User Willingness Annotations are explicit or implicit signals representing a user's readiness to engage, share data, or act within digital platforms.
• They are quantified using both subjective measures like Likert scales and objective methods such as ensemble learning and game-theoretic models.
• Accurate annotations enhance system designs in search, recommendations, transport, and health applications by aligning digital interventions with real-world behaviors.

User willingness annotations represent explicit or implicit indications of a user's motivation, readiness, or propensity to engage with a system, share data, or take action within digital environments. These annotations model individual attitudes such as willingness to continue an activity, disclose behaviors, use new technologies, or fulfill recommendations, and are foundational in information retrieval (IR), recommender systems, technology acceptance analyses, human-computer interaction, transport simulations, public health information systems, and sales dialogue optimization. Accurate user willingness annotation is essential for predicting behavioral choices, improving interaction outcomes, and designing systems that align with real-world human preferences and variability.

1. Conceptual Foundations of User Willingness Annotation

User willingness is a multidimensional concept encompassing several behavioral attitudes. In IR models, willingness is the user's motivation to continue searching for relevant content; expectation, an allied concept, is the anticipation regarding future reward or satisfaction, such as encountering additional relevant items (Piwowarski et al., 2012). In recommender systems, willingness may refer to the user's readiness to disclose behaviors or engage with recommended items (Wang et al., 2022). Safety incidents and intervention models in transport and public health measure willingness as intent to use, engage, or share personal data, modulated by trust, risk, and demographic factors (He et al., 2020, Zakaria et al., 2022, Vosooghi et al., 2019).

Willingness annotations may be:

• Explicit, e.g., users rating their willingness at the utterance or session level (Hentona et al., 27 Dec 2024, Wang et al., 2022).
• Implicit, inferred from actions, behavioral metadata (e.g., speed of annotation, engagement patterns), or acceptance/propagation probabilities (Ng et al., 26 Mar 2025, Hou et al., 19 Aug 2025).

2. Methodologies for Annotating and Modeling Willingness

Annotation approaches range from subjective self-report via Likert scales, explicit labeling at dialogue/utterance levels, to modeling through game-theoretic formulations, influence maximization, and ensemble learning. The following summarize major methodological axes:

Approach	Domain	Core Mechanism
Satisfaction-stopping models	Information Retrieval (Piwowarski et al., 2012)	Stopping probability ($p_k$), expectation-satisfaction function
Disclosure vector annotation	Recommendation (Wang et al., 2022)	Binary selection vector ($o^u$), user-defined sensitivity ($\beta^u$)
Wizard-of-Oz + willingness labels	Dialogue/sales (Hentona et al., 27 Dec 2024)	Positive/neutral/negative rating for each utterance across multiple willingness dimensions
Meta-feature weighted ensembles	Annotation reliability (Ng et al., 26 Mar 2025)	Annotator speed/fatigue and agreement enriches prediction confidence
Trust and sociodemographic factors	Transport (Vosooghi et al., 2019), Health (Fan et al., 2023)	Utility adjustments by age, sex, income, digital literacy
RR Sets and Influence Maximization	Propagation/Recommendation (Hou et al., 19 Aug 2025)	Secondary spread profits and shared reachability modeling

Willingness annotation is often entwined with satisfaction, trust, privacy disposition, and user expectation, and is operationalized through mathematical constructs in structured models (e.g., $\overline{z}_u$, $\beta^u$, $I_h(u)$).

3. Impact on System Design and Evaluation

Embodied in ranking, recommendation, and interactive dialogue systems, willingness annotation allows for:

• More realistic modeling of abandonment and satisfaction in search engines (Piwowarski et al., 2012).
• Balancing privacy and recommendation quality, letting users explicitly shape their disclosed behavior profiles (Wang et al., 2022).
• Improving fleet sizing and dynamic demand response in ride-hailing/robo-taxi simulation by accounting for demographic heterogeneity and trust (Vosooghi et al., 2019).
• Enhancing propagation and engagement in social recommendation via spread/willingness integrated metrics (Hou et al., 19 Aug 2025).
• Targeting intervention strategies (e.g., sales talk, health app recommendation) based on demographic and attitudinal moderators (Fan et al., 2023, Hentona et al., 27 Dec 2024).

Models that explicitly capture willingness outperform traditional approaches that rely on simplified or static assumptions (see Table below summarizing improvements):

System/Model	Willingness Annotation	Improvement/Impact
MSWEEM ensemble (Ng et al., 26 Mar 2025)	Annotator meta-features	+14%/+12% F1 score vs. baseline
HeteroIR/HeteroIM (Hou et al., 19 Aug 2025)	Secondary influence + willingness	+8.5%/+10% in large-scale A/B tests
LLM annotation for UTAUT (Smolinski et al., 30 Jun 2024)	Structured attitude variables	LLM annotations match human experts
Recommender game-theory (Wang et al., 2022)	Disclosure vector, influence fn.	Maintained quality, greater privacy
Robo-taxi simulation (Vosooghi et al., 2019)	Trust and sociodemographic data	+1000 vehicles fleet after accounting for willingness
Sales dialogue (Hentona et al., 27 Dec 2024)	Utterance-level willingness labels	Models with annotation outperform baselines

4. Statistical and Mathematical Formulations

Formal modeling of willingness annotation is central for robust implementation and analysis. Key formulae include:

• Expectation-Satisfaction Metric (IR):

$$E[S] = \sum_{k=1}^\infty \left( \prod_{u=1}^{k-1} (1-p_u) \right) p_k s_k$$

where $p_k$ is the probability of stopping at rank $k$, $s_k$ satisfaction at stopping.

• Reward Function in Recommendation:

$$\overline{z}_u(o^u, o^{-u}) = -L_f(\mathcal{T}^u, \hat{\theta}(o)) - \lambda \sum_{k=1}^{|S^u|} o^u_k \, \beta^u_k$$

where $L_f$ is the validation loss, $\lambda$ the trade-off parameter, and $\beta^u_k$ willingness penalty.

• Ensemble Weighted Encoding:

$$y = H([M_1 \cdot p_{a_1}, M_2 \cdot p_{a_2}, \ldots])$$

where $M_l$ are annotator meta-features, $p_{a_j}$ posterior probabilities from auxiliary classifiers.

• Influence estimation and reranking (HeteroIR/HeteroIM):

Spread influence for user $u$:

$I_h(u) = \sum_{v \in N_u'} P_{uv} U_v$

Profit for recommendation of $v$ to $u$:

$B_{uv} = P_{uv} U_v + P_{uu} U_u I_h(v)$

• Structural Equation Model (SEM) for behavioral intention (Fan et al., 2023):

$$\mathrm{BI} = \gamma_1\,(\text{Digital Literacy}) + \gamma_2\,(\text{Online Sharing}) + \cdots$$

These representations facilitate objective quantification and optimization of willingness-dependent systems.

5. Moderators, Quality, and Challenges

User willingness annotations are sensitive to a variety of moderators including sociodemographic (age, gender, income, education, country of residence), psychological (trust, risk perception, privacy disposition), and operational (fatigue, annotation speed, ecological validity, system fidelity) factors.

Challenges include:

• Achieving consistency in subjective annotation (addressed via meta-feature weighted modeling (Ng et al., 26 Mar 2025)).
• Ecological validity of annotation scenarios (addressed via Wizard-of-Oz and realistic setup (Hentona et al., 27 Dec 2024)).
• Generalizability across domains (surveys, transport, dialog, health apps).
• Computational efficiency in exploring combinatorial spaces (influence functions, multi-anchor solutions (Wang et al., 2022)).
• Balancing privacy and model utility, especially where user willingness to disclose is critical.

Practical guidance from recent research recommends active annotation monitoring, dynamic adjustment of models to annotation quality, and the use of structured, user-facing elicitation protocols.

6. Applications and Future Directions

User willingness annotation research informs a broad spectrum of applications:

• Search evaluation and IR metric refinement (Piwowarski et al., 2012)
• Privacy-preserving and customizable recommender systems (Wang et al., 2022)
• Large-scale fleet sizing and urban transport design (Vosooghi et al., 2019)
• Sales dialogue optimization and personalized agent systems (Hentona et al., 27 Dec 2024)
• Automated, scalable technology acceptance measurement using LLMs and structured annotation (Smolinski et al., 30 Jun 2024)
• Influence propagation and viral marketing campaigns (Hou et al., 19 Aug 2025)
• Health app adoption strategies based on demographic and behavioral modeling (Fan et al., 2023)

Potential future research paths include empirical validation of dynamic willingness models using longitudinal logs, deeper integrations of ecological validity protocols, advancement of meta-feature guided annotation quality controls, and scalable multimodal annotation using advanced LLMs and real-time behavioral sensing.

User willingness annotations represent a critical scholarly foundation for optimizing the alignment between algorithmic systems and human behavioral diversity, ensuring that system outcomes reflect not only technical efficiency but also nuanced personal preferences, risk propensities, and engagement potential.