Green Shielding: A User-Centric Approach Towards Trustworthy AI

Abstract: LLMs are increasingly deployed, yet their outputs can be highly sensitive to routine, non-adversarial variation in how users phrase queries, a gap not well addressed by existing red-teaming efforts. We propose Green Shielding, a user-centric agenda for building evidence-backed deployment guidance by characterizing how benign input variation shifts model behavior. We operationalize this agenda through the CUE criteria: benchmarks with authentic Context, reference standards and metrics that capture true Utility, and perturbations that reflect realistic variations in the Elicitation of model behavior. Guided by the PCS framework and developed with practicing physicians, we instantiate Green Shielding in medical diagnosis through HealthCareMagic-Diagnosis (HCM-Dx), a benchmark of patient-authored queries, together with structured reference diagnosis sets and clinically grounded metrics for evaluating differential diagnosis lists. We also study perturbation regimes that capture routine input variation and show that prompt-level factors shift model behavior along clinically meaningful dimensions. Across multiple frontier LLMs, these shifts trace out Pareto-like tradeoffs. In particular, neutralization, which removes common user-level factors while preserving clinical content, increases plausibility and yields more concise, clinician-like differentials, but reduces coverage of highly likely and safety-critical conditions. Together, these results show that interaction choices can systematically shift task-relevant properties of model outputs and support user-facing guidance for safer deployment in high-stakes domains. Although instantiated here in medical diagnosis, the agenda extends naturally to other decision-support settings and agentic AI systems.

• The paper presents a user-centric approach—Green Shielding—that investigates reliability risks from benign user input variations in AI systems.
• It employs a structured CUE and PCS framework to assess context, utility, and elicitation, highlighting significant prompt sensitivity in clinical diagnostics.
• It introduces HCM-Dx, a realistic medical benchmark, and demonstrates tradeoffs between diagnostic plausibility and critical coverage through prompt neutralization.

User-Centric Trustworthy AI: The Green Shielding Paradigm

Motivation and Framework for Green Shielding

The paper introduces Green Shielding as a user-centric research agenda for trustworthy AI, specifically targeting reliability risks from benign, non-adversarial user inputs ("AI Safety II") rather than catastrophic adversarial threats ("AI Safety I"). Standard LLM evaluation pipelines and red-teaming focus on worst-case scenarios, neglecting routine instability that can arise in everyday interactions due to inadvertent user variation in prompt phrasing or content. Green Shielding aims to empirically characterize these effects to inform actionable deployment guidance.

Green Shielding operationalizes behavioral assessment via the CUE criteria:

• Context: Benchmarks that capture authentic deployment situations.
• Utility: Reference standards and evaluation metrics reflecting meaningful utility for end-users.
• Elicitation: Realistic perturbations that reflect genuine variation in user inputs.

The agenda leverages the PCS framework—Predictability, Computability, Stability—to ensure assessment procedures are methodologically sound, scalable, and robust to heterogeneity.

Figure 1: Overview of the CUE-guided empirical design for Green Shielding in medical diagnosis.

Prompt Sensitivity in Medical Benchmarks

A pilot study analyzes prompt sensitivity using MedQA and MedXpertQA, employing open-ended prompts and various input perturbations along content, format, and tone axes. Results demonstrate that even minor prompt alterations can cause substantial shifts in response correctness. Notably, ablation of objective test results or symptom history induces measurable declines in accuracy and transitions between correct and incorrect outputs. The injection of user beliefs, even those contradicting the default answer, systematically alters model predictions—substantiating strong prompt-level sensitivity.

Figure 2: Percentage of samples whose correctness changes under individual prompt-level perturbations on MedQA and MedXpertQA.

Benchmarks relying on single-answer accuracy fail to capture diagnostic uncertainty and the structure of differential diagnosis commonly encountered in authentic patient queries.

HCM-Dx: Realistic User Queries and Structured Evaluation

To address limitations of traditional benchmarks, the paper introduces HealthCareMagic-Diagnosis (HCM-Dx), a dataset of patient-authored queries filtered for explicit diagnosis requests. Reference diagnosis sets are constructed using ensembles from multiple frontier LLMs (GPT-5.2, Gemini-3-Pro, Claude-4.5-Opus), partitioned into plausible, highly likely, and safety-critical categories. Evaluation metrics include:

• Plausibility: Fraction of model-proposed differentials matching the plausible set.
• H-coverage: Coverage over highly likely diagnoses.
• S-coverage: Recall over safety-critical conditions.
• Breadth: Number of diagnoses proposed.

Semantic metrics such as evidence grounding, indirect inference rate, and epistemic uncertainty quantification are included to further dissect model behavior.

Prompt neutralization rewrites real patient queries into objective clinical case descriptions while annotating prominent input-level factors:

• Content Factors: Specific guesses, irrelevant details, missing objective data/symptom history.
• Format Factors: Unstructured requests, ambiguity.
• Tone Factors: Emotional/urgent language, first-person framing.

  Figure 3: Annotation criteria for user prompt-level factors and their frequencies in HCM-Dx patient queries.

Empirical Results: Prompt Neutralization and Tradeoffs

Five frontier LLMs are evaluated across raw and neutralized patient inputs. On raw prompts, model-generated differentials exhibit high plausibility but suboptimal coverage of highly likely and safety-critical diagnoses. Prompt neutralization induces a quantifiable tradeoff: increased plausibility and reduced breadth (more concise differentials), but lower coverage of critical diagnoses. Models shift toward clinician-like profiles in selectivity and communication style under neutralized prompts, though substantial discrepancies persist.

Figure 4: HCM-Dx results showing systematic effects of prompt neutralization on frontier LLMs, detailing precision-coverage tradeoffs.

Content-level ablation, when performed in isolation, produces little change relative to the full effect of joint neutralization. Format and tone ablations exert stronger impact, indicating that behavioral shifts are primarily mediated by non-content-related factors.

Figure 5: Effects of removing content-level factors only, as compared to raw patient inputs.

Expert Alignment and Reference Set Robustness

Automated evaluation pipelines leveraging LLM judges for both diagnosis extraction and semantic matching are validated against expert clinician annotation. High precision is observed for diagnosis pairs predicted as matches by unanimous LLM judges; however, positive matches are conservative, and borderline equivalents may be undercounted. Clinician-amended reference sets indicate low commission error rates under consensus criteria, but omission errors attributable to underspecified queries and inherent clinical ambiguity remain nontrivial.

Figure 6: Analysis of semantic-equivalence matching between LLM judges and expert clinicians.

Figure 7: Inter-annotator agreement rates and expert amendments on reference diagnosis sets, highlighting task ambiguity and disagreement.

Discussion, Limitations, and Future Directions

The formalization of Green Shielding reveals that routine prompt-level factors—beyond those addressed by adversarial testing—materially shape model outputs across clinically relevant axes. These operating points define explicit tradeoffs between precision and coverage, with context-dependent desirability. A party-neutralized prompt does not universally dominate the raw form; the optimal interaction strategy may be domain-, task-, and user-dependent.

The HCM-Dx evaluation pipeline enhances realism and granularity, but remains limited by task underspecification, single-turn interaction design, and abstraction away from conversational dynamics and downstream utility. Reference set ambiguity, as corroborated by expert disagreement, exposes the challenge of normatively evaluating decision-support systems in high-uncertainty domains.

Practical implications extend to regulatory alignment (e.g., EU AI Act, NIST RMF), evidence-backed deployment guidance, and interactive system design. The agenda is generalizable to other domains such as education, law, and agentic AI, where interaction sensitivity represents a primary axis of reliability and trustworthiness.

Conclusion

Green Shielding provides a structured, empirical foundation for user-centric, evidence-backed evaluation of LLMs under non-adversarial deployment scenarios. Systematic prompt variation produces measurable and interpretable shifts in clinically meaningful output properties, underscoring the necessity of nuanced, context-aware deployment strategies. The approach highlights explicit tradeoffs between plausibility and critical coverage, exposes the limits of single-score benchmarking, and informs future work targeting multi-turn, agentic, and conversational settings. This paradigm advances the methodological rigor and practical relevance of AI reliability research in real-world decision-support environments.

---