AI Recommendation Adherence

• AI recommendation adherence is a framework that measures how well users align their actions with algorithmic advice based on correct judgment and discrimination.
• Empirical studies and adherence-aware models show that integrating post-hoc explanation techniques and reinforcement learning can mitigate under- and over-reliance.
• Robust adherence frameworks are pivotal in high-stakes domains, balancing behavioral biases and optimizing decision outcomes by calibrating human-AI interaction.

AI recommendation adherence refers to the spectrum of ways in which humans and algorithmic agents respond to recommendations made by artificial intelligence systems, encompassing the rates, patterns, determinants, and consequences of adopting, ignoring, partially following, or selectively modifying algorithmic advice. Adherence is central to high-stakes domains including healthcare, finance, education, and autonomous systems, as ultimate responsibility for decision implementation frequently rests with human operators or secondary agents whose behavior may diverge from the algorithmic optimum. The adherence phenomenon introduces a critical feedback channel between system design, deployed policy, user psychology, and observed outcomes.

1. Conceptualization and Measurement of Appropriateness in AI Reliance

Appropriate reliance (AR) constitutes the normative target for AI recommendation adherence: humans should rely on AI advice when it is correct and reject it when it is likely incorrect. Schemmer et al. formalize AR as a two-dimensional construct composed of (a) discrimination—the human’s ability to identify when AI advice is likely correct versus incorrect, and (b) behavior—the human’s willingness to follow correct advice and reject incorrect advice on a case-by-case basis (Schemmer et al., 2022).

The operationalization of AR in binary decision settings is given by two indices: $$RAIR = \frac{\#(\text{Positive AI‐reliance})}{\#(\text{Positive AI‐reliance})+\#(\text{Negative self‐reliance})}$$

$$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$

Here, $RAIR$ quantifies the rate at which users appropriately accept AI advice when their initial judgment is incorrect, while $RSR$ quantifies the rate at which users appropriately trust their own correct judgment in the face of incorrect AI advice. The ideal is $(RAIR,RSR)=(1,1)$; values below $(0.5,0.5)$ on either axis denote systematic under-reliance or over-reliance, respectively.

Empirical findings demonstrate that even highly accurate AI can be underutilized when users are reluctant to defer to correct advice—an effect mitigatable to some extent by post-hoc explanations such as LIME-based feature highlights, which selectively improve $RAIR$ without significantly eroding $RSR$. This underlines the necessity of both discrimination (clarity in identifying quality of advice) and behavioral calibration (aligning user action with discrimination).

2. Adherence-Aware Modeling and Reinforcement Learning

Classical algorithmic recommendation typically assumes perfect adherence, optimizing recommendations with the expectation they will be implemented exactly. Grand-Clément and Pauphilet establish that such assumptions can produce arbitrarily large losses when actual adherence is partial, as seen in real-world expert-in-the-loop systems (Grand-Clément et al., 2022). Their adherence-aware optimization framework defines the effective policy as a convex combination of the recommendation policy $\pi_{alg}$ and the baseline human policy $\pi_{base}$, weighted by adherence $$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$0: $$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$1 The optimal recommendation then solves: $$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$2 where $$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$3 is the infinite-horizon discounted return of the resulting Markov decision process (MDP). Value iteration or linear programming can be employed on a surrogate MDP with convex-combined transitions and rewards.

Reinforcement learning algorithms extend adherence-awareness by estimating $$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$4 online and propagating its effect through Bellman updates and Q-learning: $$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$5 These algorithms provably converge and empirically outperform both adherence-agnostic methods and baseline policies, especially for moderate-to-high adherence levels (Faros et al., 2023, Sun et al., 28 Apr 2025).

The defer option further allows the algorithm to refrain from making recommendations in non-critical states, incorporating an advice cost penalty $$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$6 to ensure advice is provided only when maximizing net value (Chen et al., 2023).

3. Psychological, Behavioral, and Interactional Aspects

Behavioral studies reveal that adherence is not binary; users may ignore, trust, or negotiate with AI advice. In complex settings such as clinical sepsis management, negotiation emerges as a distinct interaction mode: clinicians may accept dosage but not modality, or temporally stagger acceptance across treatment components (Sivaraman et al., 2023). Interface design—such as surfacing multiple options, presenting confidence intervals, and aligning the AI’s state representation with clinician inputs—facilitates nuanced negotiation rather than mere acceptance or rejection.

Recommendation-driven designs can also systemically shift user evidence thresholds—quantified as the decision criterion $$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$7 in Signal Detection Theory. Even with identical discriminative accuracy ($$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$8), exposure to confident AI recommendations lowers users’ standards for confirming a hypothesis, resulting in latent bias towards acceptance (liberal criterion shift), increased false alarms, and reduced independent scrutiny. This effect is observed uniformly across both novices and experts (Benk et al., 16 Mar 2026).

To counteract such hidden biases, system designers are encouraged to decouple recommendation from decision threshold, provide hypothesis-driven exploration tools, introduce cognitive forcing functions, and explicitly monitor and recalibrate user criteria in deployment.

4. Adherence Metrics in Explainable Recommender Systems

Adherence is also a desideratum for explainable recommenders—especially those employing surrogate-based post-hoc explanations such as LIME-RS. Here, adherence refers to whether the explanation’s features genuinely match the attributes present in the recommended item, while constancy refers to the stability of explanations across multiple runs. Formal metrics include:

• Adherence ($$RSR = \frac{\#(\text{Positive self‐reliance})}{\#(\text{Positive self‐reliance})+\#(\text{Negative AI‐reliance})}$$9): Proportion of runs where at least one of the top-$RAIR$0 explanation features overlaps the item’s true features.
• Constancy ($RAIR$1): Averaged frequency of the same feature appearing in position $RAIR$2 across $RAIR$3 runs.

Empirically, adherence and constancy can vary substantially by recommender type and dataset. High adherence and constancy bolster user trust and system accountability, whereas low adherence (e.g., explanations referencing irrelevant genres) can degrade system credibility (Anelli et al., 2021). Recommendations to improve explanation adherence include verifying that candidate features are present in the item, limiting explanation length to positions where constancy remains high, and improving metadata granularity.

5. Group Differences, Biases, and Strategic Implications

Human-AI adherence is subject to both psychological bias and strategic interaction effects. Alon-Barkat and Busuioc study automation bias (overreliance) and selective adherence (greater acceptance of advice consistent with social stereotypes) in public-sector decision-making (Alon-Barkat et al., 2021). Their large-scale experiments find no significant automation bias among lay citizens (AR for algorithmic advice ≈ 11.1%, for human advice ≈ 10.5%) and a more complex pattern among civil servants, where publicized algorithmic failures decreased, rather than increased, adherence. Selective adherence is observed in stereotype-congruent scenarios but not amplified by algorithmic versus human sources.

In multi-advisor ecosystems, strategic interaction further complicates adherence patterns. When a personal AI influences a decision alongside an institutional advisor, the equilibrium loss and the advisor’s optimal counteraction intensity are hump-shaped functions of the AI’s adoption probability $RAIR$4 and its relative trust index $RAIR$5 (the ratio of its perceived precision to that of the institutional advisor). Vulnerability to misalignment peaks at intermediate adoption and influence levels but vanishes at extremes, pointing to the importance of calibrating relative trust and designing regulatory or interface levers that ensure transparent joint disclosure of all recommendations (Liu et al., 2 Mar 2026).

6. Applications, Monitoring, and Future Directions

Robust adherence-aware algorithms have been deployed in personalized education (Drushchak et al., 27 Feb 2025), healthcare (adherence forecasting for digital behavioral interventions (Côté-Allard et al., 2022)), and autonomous driving (lane-change recommendations with partial driver adherence (Sun et al., 28 Apr 2025)). Responsible AI adherence in these settings requires:

• Continuous logging and recalibration of adherence levels to re-optimize recommendations.
• Auditing for group-wise fairness and evidence-threshold drift.
• Providing transparent rationales and user feedback mechanisms for recommendation acceptance or rejection.

Future directions include extending AR measurement to multi-class, regression, or ranking tasks (Schemmer et al., 2022), integrating richer psychological and behavioral models, and developing training objectives and retrieval architectures that directly reward adherence to normative guidelines—particularly in safety-critical applications such as medicine, where observed guideline adherence rates for LLMs remain significantly below detection rates (21.8–63.2% vs. 71.1–89.6%) (Tan et al., 26 Mar 2026). There is a pressing need for deployment strategies that anticipate, measure, and remediate both under-adherence and over-adherence, while preserving both equity and user autonomy.