ResidencyRL: Reinforcement Learning in Simulated Clinical Environments
Abstract: In medical education, physicians convert academic knowledge into clinical expertise through residency: years of training across thousands of encounters, with diverse sources of feedback and progressively greater autonomy. Much of clinical reasoning relies on the patient encounter, a dialogue in which a clinician elicits history, refines diagnostic hypotheses, and decides management under uncertainty. While LLMs excel on static medical benchmarks, methods to optimize the full sequence of clinical decisions remain underdeveloped. We present ResidencyRL, a reinforcement learning (RL) method for training clinical AI agents through simulated multi-turn clinical encounters (up to 60 dialogue turns and 8 tool calls per trajectory). ResidencyRL pairs the policy agent with LLM simulators capable of complex, adversarial behaviors, training against a structured reward aligned to diagnostic accuracy, management quality, communication, documentation, and safety. On held-out evaluations, the ResidencyRL agent improves diagnostic accuracy by 7.0% under adversarial conditions (88.0% vs. 81.0%) and reduces missed red flag rates by 31%, demonstrating rigorous mitigation of premature closure. Blinded expert clinicians validated these gains, preferring the trained agent in 87.6% of side-by-side comparisons. The procedural competencies transfer to unseen benchmarks: the agent outperforms the base model across all six clinical axes of the AMIE multi-visit benchmark, and shows consistent directional improvements on AgentClinic and CRAFT-MD. Our findings demonstrate that sequential clinical decision-making can be effectively learned through multi-turn RL in simulation, yielding robust, generalizable capabilities, paving the way towards clinical mastery. Prospective validation with real-world workflows remains necessary to establish clinical utility.
Summary
- The paper introduces a POMDP-based training environment where an LLM clinician performs history-taking, diagnosis, management, documentation, communication, and termination across simulated encounters.
- The trained agent improves adversarial diagnostic accuracy from 81.0% to 88.0%, reduces missed red flags from 45.5% to 31.5%, and earns clinician preference in 87.6% of blinded comparisons.
- The results suggest reinforcement learning improves sequential clinical behavior and generalizes to longitudinal and specialist tasks, but synthetic environments, autorater bias, and the lack of real-patient validation limit clinical conclusions.
ResidencyRL: Reinforcement Learning for Sequential Clinical Reasoning
“ResidencyRL: Reinforcement Learning in Simulated Clinical Environments” (2608.07418) investigates whether clinical AI agents can acquire procedural competence through long-horizon reinforcement learning rather than merely improving performance on static medical question-answering benchmarks. The central claim is that diagnostic and management quality depends not only on encoded medical knowledge, but also on the sequential process by which a clinician elicits information, revises hypotheses, evaluates urgency, communicates uncertainty, documents the encounter, and determines when to conclude. ResidencyRL operationalizes this process as an interactive partially observable environment in which an LLM-based clinician conducts simulated patient encounters and receives trajectory-level rewards.
The empirical results are substantial but should be interpreted within the limitations of synthetic simulation and LLM-based evaluation. Relative to the Gemini 3.5 Flash base model, the trained agent improves adversarial diagnostic accuracy from 81.0% to 88.0%, reduces missed red flags from 45.5% to 31.5%, and is preferred by clinicians in 87.6% of blinded side-by-side comparisons. The reported gains extend to longitudinal disease management, oncology consultations, and external clinical dialogue benchmarks. At the same time, the authors explicitly acknowledge that the training signal is a proxy for patient outcomes, the autorater exhibits positive bias toward the optimized policy, and prospective real-patient validation remains necessary.
Clinical Motivation and Problem Formulation
The paper targets a specific limitation of current medical LLM evaluation: strong performance on static vignettes does not establish competence in interactive clinical decision-making. In a real consultation, relevant evidence is distributed across multiple turns and may not be volunteered by the patient. Patients can provide incomplete histories, misunderstand questions, conceal medications or substance use, resist recommendations, overstate symptoms, or present atypically. The clinician must therefore select questions under uncertainty and determine when the expected value of additional information justifies continued interaction.
ResidencyRL models this setting as a long-horizon POMDP. The hidden state includes the patient’s clinical condition, risk factors, behavioral characteristics, and undisclosed information. Observations arrive through patient responses and environment tools. The action space is heterogeneous: it includes natural-language questions, diagnostic and management submissions, urgency classification, patient-facing summaries, SOAP documentation, and encounter termination. This is materially different from single-turn RLHF, where a prompt-response pair is treated as an effectively one-step decision problem.
The paper positions its contribution alongside earlier clinical agents that use prompting, supervised self-play, or RL for narrower tasks. AgentClinic (Schmidgall et al., 2024) and related systems evaluate dynamic dialogue, while AMIE and AMIE Mx establish strong baselines for conversational diagnosis and multi-visit management. Other concurrent RL systems focus on history-taking, diagnostic test ordering, or consultation flow. ResidencyRL’s distinctive objective is to optimize the complete encounter, including information gathering, differential diagnosis, management, documentation, communication, and safety within one trajectory.
The ResidencyRL Training Environment
The training pipeline contains four tightly coupled components: scenario generation, interactive simulation, structured autorating, and online policy optimization.
Figure 1: ResidencyRL trains a clinical policy through generated patient scenarios, multi-turn simulation, structured evaluation, and GRPO updates.
The scenario generator produces 57,453 cases: 49,870 routine telehealth scenarios, 5,000 targeted history-taking scenarios, and 2,583 adversarial safety scenarios. Routine cases cover 81 presenting complaints across respiratory, cardiovascular, musculoskeletal, genitourinary, reproductive, gastrointestinal, and miscellaneous categories. Clinical complexity is varied through comorbidities, symptom overlap, medication interactions, social confounders, and differential-diagnosis breadth.
The authors use a multi-stage generation process. Demographic, geographic, behavioral, and clinical attributes are sampled first. An LLM then generates the structured medical scenario, including symptom evolution, relevant positives and negatives, medication history, ground-truth diagnosis, and ranked differential. An independent judge assesses consistency, realism, demographic appropriateness, differential quality, and completeness. Finally, character-trigram TF-IDF deduplication removes near-duplicate narratives. This procedure is intended to avoid both clinical incoherence and policy memorization.
Two extension packs are especially important. Targeted history-taking cases hide clinically decisive information unless the agent asks about the relevant domain: social and lifestyle factors, medication specifics, symptom qualifiers, or environmental exposures. Adversarial cases test nine safety categories, including concealed emergencies, unsafe telehealth contexts, atypical presentations, pharmacological boundaries, vulnerability and abuse, age verification, autonomy, mental-health crises, and patient manipulation.
Figure 2: A simulated constipation encounter conceals daily Kratom use, requiring targeted questioning to identify a medication- or substance-related diagnosis.
The patient simulator is conditioned on the complete scenario but responds only to the agent’s current message and dialogue history. Its behavioral layer adapts vocabulary to health literacy, distinguishes volunteered from elicited information, limits disclosure to directly relevant questions, and restricts responses to approximately 50 words. Adversarial prompts cause patients to minimize symptoms, resist escalation, deflect questions, or sustain misleading narratives. The simulator is prevented from observing the clinician’s tool calls, hidden reasoning, or evaluation rubric, reducing the risk that it will inadvertently validate the agent’s hypothesis.
The environment exposes a documentation API with tools for chart review, differential submission, primary diagnosis, urgency, management plan, patient-facing summary, SOAP note, and termination. This action space is significant: the agent is not trained merely to ask questions, but to transform an evolving history into structured clinical decisions and records.
Composite Reward and Optimization
ResidencyRL uses online Group Relative Policy Optimization. For each scenario, multiple parallel trajectories are sampled. Gemini 3.1 Pro evaluates completed transcripts and clinical artifacts using a structured rubric, and GRPO updates the policy using relative performance within the sampled group rather than a separately trained value function.
The primary reward covers six dimensions:
- diagnostic accuracy;
- management quality;
- clinical intake and screening completeness;
- patient-centered communication;
- documentation quality;
- conversational style.
Management receives the largest weight, followed by diagnostic accuracy. The management rubric evaluates urgency, investigations, treatment, follow-up, safety-netting, and overall plan quality. Documentation is assessed using PDQI-9-inspired criteria, while communication incorporates relationship building, information gathering, and emotional responsiveness.
Critical safety flags generate explicit penalties. These include hallucinated history, contraindicated actions, plan leakage, missed mandatory questions, under-triage, and missed red flags. System-level penalties address parsing errors, missing termination, and excessive conversational length. The length penalty begins after 30 conversational turns and reaches its maximum at 40 turns, imposing a direct efficiency constraint on a policy otherwise incentivized to ask more questions.
This reward design reflects a central tension in clinical dialogue: thoroughness is necessary to avoid premature closure, but indiscriminate questioning is inefficient and can burden patients. The policy must learn not only to ask more questions, but to ask questions with sufficient diagnostic and safety value.
Figure 3: Training increases encounter length, internal reasoning allocation, and total trajectory steps before these quantities stabilize under efficiency constraints.
The training dynamics support this interpretation. Median encounter length increases from approximately 18 to 23 turns by training step 100, while the conversational range expands from roughly 13–24 turns to 16–32 turns. Average thinking tokens per turn also increase, and total trajectory length stabilizes at approximately 52 agent steps. Intake completeness improves most rapidly, whereas documentation saturates early and communication continues improving more gradually. These trajectories suggest that RL changes the allocation of computation and interaction, not merely the wording of final responses.
In-Domain Clinical Results
On held-out routine telehealth cases, diagnostic performance improves modestly from 86.4% to 88.4% for encounters scoring at least 4/5. The adversarial condition produces the more informative result: accuracy rises from 81.0% to 88.0%, a 7.0 percentage-point improvement. This difference indicates that the policy benefits particularly from scenarios in which the correct diagnosis depends on resisting misleading surface presentations or eliciting concealed information.
Management quality increases from 3.98 to 4.52 on routine cases and from 3.94 to 4.51 on adversarial cases. Improvements occur across treatment, investigations, safety-netting, urgency assessment, and follow-up planning. The proportion of encounters receiving an urgency score of at least 4/5 increases from 90.0% to 96.5% under adversarial conditions.
The largest changes concern screening completeness. On routine cases, social and lifestyle history improves from 1.31 to 2.64, while exposure history rises from 2.27 to 3.39. Under adversarial conditions, medication and past-history screening rises from 2.63 to 3.43, and exposure screening from 3.73 to 4.40. These results are consistent with the training curriculum’s explicit emphasis on hidden information and targeted elicitation.
Communication also improves, particularly under adversarial conditions. The responding-to-emotions score rises from 2.74 to 3.57, while fostering relationship increases from 2.96 to 3.57. These gains suggest that the policy does not simply become more exhaustive and procedural; it also becomes more responsive to patient concerns.
Safety results are clinically more consequential than small changes in diagnostic accuracy. Missed critical questions fall from 65.5% to 43.5%, and missed red flags from 45.5% to 31.5%. The authors characterize these as approximately one-third relative reductions. Contraindicated actions decrease from 21.5% to 16.5%, although the confidence intervals overlap. Under-triage remains essentially unchanged at 4.0% versus 4.5%, indicating that the safety improvement is concentrated in information elicitation and red-flag recognition rather than urgency classification.
Generalization Beyond the Training Distribution
The strongest theoretical implication of the paper is that RL appears to improve clinical procedure rather than only case-specific medical recall. The authors evaluate the model on five out-of-domain frameworks with different scenarios, simulators, rubrics, and task structures.
AMIE Mx Multi-Visit Management
On the AMIE Mx benchmark, the model is evaluated across 120 longitudinal scenarios and 360 simulated encounters. ResidencyRL improves all six clinical axes. Management reasoning increases from 80.1% to 88.4%, patient communication from 83.7% to 92.2%, and clinical skills from 79.6% to 84.8%. Guideline adherence rises from 84.2% to 86.7%, while precision and inter-visit memory increase from 90.0% to 92.7%.
Figure 4: ResidencyRL improves across the six AMIE Mx clinical axes and most of the benchmark’s 60 sub-metrics.
The largest improvements occur in management reasoning and communication rather than diagnosis-management accuracy, which changes only from 90.4% to 90.7%. This pattern suggests that the learned behavior is most valuable when the task requires sustained monitoring, treatment adjustment, patient explanation, and integration of prior encounters.
Performance is also stable across visits. ResidencyRL scores 89.3%, 89.3%, and 89.1% at Visits 1–3, compared with 85.0%, 85.3%, and 83.7% for the baseline. The absence of progressive degradation is relevant because longitudinal settings expose memory and consistency failures that single-visit evaluation cannot detect.
Specialist Oncology Consultations
The oncology benchmark contains 300 expert-authored cases, including 240 solid-tumor cases across 21 cancer types and 60 hematological malignancy cases across 10 subtypes. Oncology workflows were absent from training, making this evaluation a test of procedural transfer rather than domain matching.
ResidencyRL wins significantly more often on completeness, clinical accuracy, actionability, and the overall composite. The overall win rate is 42.9% versus 18.6% for the baseline; clinical accuracy is 26.0% versus 9.1%, and completeness is 34.8% versus 10.5%. The models are more similar on case relevance and safety/triage.
Expert review identifies a consistent behavioral difference: the trained agent asks more targeted questions, identifies red flags earlier, explores exposure and family history, ranks differentials more explicitly, and distinguishes immediate actions from routine workup. A residual weakness is that the agent sometimes asks multiple questions in one turn, potentially reducing patient response completeness—an example of how improved information-seeking can still create interactional inefficiency.
AgentClinic and CRAFT-MD
On AgentClinic-MedQA, diagnostic accuracy increases from 81.4% to 85.6%; on AgentClinic-MIMIC-IV, it increases from 53.5% to 60.0%. Neither comparison reaches conventional statistical significance, with and , respectively. Nevertheless, the direction is consistent, and the MIMIC-IV consultations become longer, increasing from 11.2 to 15.8 turns.
CRAFT-MD shows the same pattern. The largest gain occurs in Consultation Free-Response, from 58.1% to 62.8%, while Visit Free-Response rises from 81.3% to 84.5%. Structured Visit Multiple Choice is the only metric favoring the baseline, marginally: 94.2% versus 93.7%. The results support the claim that ResidencyRL primarily improves active information gathering and free-form diagnostic synthesis, rather than static recognition.
Expert-Curated Telehealth in an Optimized Harness
The most stringent evaluation places both models in the same expert-optimized AMIE Telehealth harness and evaluates 299 clinician-curated scenarios. This controls for agentic scaffolding and tests whether RL adds value after substantial prompting, orchestration, and tool integration have already improved the base model.
Figure 5: In an out-of-domain telehealth harness, ResidencyRL improves overall clinical performance, safety, treatment, follow-up, style, and naturalness.
Eight of ten automated metrics improve significantly after FDR correction. Overall clinical rubric score increases from 3.93 to 4.43, while naturalness rises from 2.52 to 3.24. Treatment improves from 4.43 to 4.61, safety from 4.73 to 4.80, and follow-up from 4.78 to 4.88. Diagnostic appropriateness is close to ceiling, increasing from 4.98 to 5.00.
Human evaluation provides the more important validation. Across 97 valid cases, clinicians prefer ResidencyRL for overall impression in 87.6% of comparisons, compared with 10.3% for the baseline and 2.1% ties. The trained agent wins 90.7% of comparisons on completeness of information gathering, 75.3% on management-plan appropriateness, and 66.0% on diagnostic assessment.
Figure 6: Blinded clinicians consistently prefer the ResidencyRL-trained agent across clinical, safety, and conversational dimensions.
For management-plan safety, the trained model is preferred or tied in 96.9% of cases, with only 3.1% losses. Accuracy without hallucination is the exception: 77.3% of comparisons are ties, and the difference is not statistically significant. Thus, the training improves clinical process without evidence that it increases hallucination or factual degradation.
The autorater–clinician calibration analysis is informative but also exposes a methodological limitation. Cases preferred by clinicians show larger autorater deltas, with median differences of +0.61 versus +0.31. However, autorater deltas remain systematically positive even when clinicians do not prefer ResidencyRL. This indicates that the optimized evaluator favors the trained model in absolute terms, consistent with reward-model overoptimization. The authors appropriately treat relative discrimination, rather than absolute autorater scores, as the more credible signal.
Figure 7: Autorater deltas correlate with clinician preference but remain positively shifted toward the RL-trained model.
Why Reinforcement Learning Outperforms Demonstrations
An ablation compares ResidencyRL with in-context learning. One baseline receives five high-reward trajectories generated before RL training; another receives five trajectories sampled from the top 10% of RL rollouts. The latter improves over zero-shot prompting on management, communication, and screening completeness, but remains below the RL-trained policy on every reported clinical axis.
The distinction is theoretically important. In-context examples expose the model to desired behavior but do not optimize its policy over the consequences of sequential actions. The trained-demonstration condition produces the longest encounters, averaging 22.3 turns, yet remains inferior to ResidencyRL, which achieves better performance with approximately 19.9 turns in the ablation. This suggests that RL learns a policy for allocating questions and terminating encounters, whereas demonstrations primarily induce a stylistic tendency toward greater verbosity.
The result supports the paper’s broader argument that clinical competence involves policy-level adaptation across trajectories. Merely displaying thorough encounters does not reliably teach the model which questions are consequential, how to revise its differential, or when additional questioning is no longer justified.
Clinical Case Analyses
The case studies make the quantitative findings concrete. In one scenario, a patient demands exploratory laparotomy for a self-diagnosed fistula. The baseline accepts the framing and anchors its SOAP note on an intra-abdominal fistula. ResidencyRL instead probes the evidentiary basis of the request, discovers that no fistula has been diagnosed, identifies a pattern compatible with somatic symptom or illness-anxiety pathology, and persists with pregnancy screening despite patient deflection.
The second case involves transient dysphasia and facial asymmetry concerning for TIA. The baseline recognizes urgency but terminates after minimal history-taking and then fabricates hypertension, diabetes, and medications in the SOAP note. ResidencyRL asks about symptom duration, vascular risk factors, medications, anticoagulation, arrhythmia, and family history before directing the patient to emergency care. Its differential includes TIA, ischemic stroke, and hypoglycemia, and its management plan recommends emergency evaluation with ECG and glucose testing.
These cases illustrate a critical interaction between premature closure and hallucination. When a model terminates before obtaining necessary information but is still expected to complete structured documentation, it may fill missing fields with plausible but unsupported facts. RL that rewards both information elicitation and documentation grounded in the transcript can reduce this failure mode. However, the examples also raise a policy-design question: in emergencies, how much history should be gathered before escalation? ResidencyRL’s behavior is beneficial when questioning adds actionable context without delaying emergency referral, but this tradeoff requires explicit clinical validation.
Limitations and Evaluation Risks
The paper’s evidence is strongest for simulated procedural behavior and weaker for patient-level clinical effectiveness. Training is limited to English-language, US-centered, text-based telehealth encounters. The environment does not model physical examination, voice, facial expression, imaging interpretation, laboratory-result evolution, treatment response, clinician collaboration, or resource constraints. Management plans are evaluated against scenario-grounded rubrics rather than downstream patient outcomes.
The simulator is sophisticated but remains an LLM. Its disclosure policy, adversarial behavior, pacing, and clinical consistency are prompt-mediated. It may therefore produce challenges that are more systematic and controllable than those encountered with real patients. Real-world patients introduce cultural, linguistic, emotional, and contextual variability not captured by scenario parameters alone.
The reward is also intrinsically imperfect. A Gemini-based autorater assesses clinical quality, safety, communication, documentation, and style, but its judgments are not equivalent to outcomes or expert consensus. The paper documents positive autorater bias toward ResidencyRL and metric saturation near the upper end of the scale. These findings are particularly important because the policy is optimized directly against related evaluation signals. The blinded clinician results reduce concern about pure reward hacking, but the human sample contains only 97 valid cases and remains embedded in simulated encounters.
Reproducibility is constrained by proprietary Gemini models and internal infrastructure. Although several evaluation datasets are public, the full scenario-generation, simulator, training, and grading stack is not released. This limits independent replication and makes it difficult to determine which component—scenario diversity, adversarial prompting, reward decomposition, GRPO, or model scale—accounts for the observed gains.
Implications and Future Research
ResidencyRL provides evidence that post-training can improve an already capable medical foundation model by changing how it conducts clinical interactions. The most transferable improvements are procedural: asking targeted questions, probing hidden risk factors, resisting patient-provided diagnostic anchors, maintaining differential breadth, producing safer management plans, and documenting only elicited information.
Future systems should extend the environment along three dimensions. First, the state should become multimodal, incorporating laboratory results, imaging, physiological signals, speech, and video. Work on multimodal conversational diagnosis and real-time audio-visual clinical assistance illustrates the importance of these channels (Shah et al., 10 May 2026). Second, training should move from isolated visits to longitudinal patient trajectories in which interventions change future states, treatment response, adherence, and disease progression. Clinical environment simulators that model patient and hospital dynamics provide a relevant direction [Luo2026-oa]. Third, the agent should operate in multi-actor environments involving nurses, caregivers, interpreters, specialists, and institutional workflows.
Reward modeling also requires development. More robust approaches could combine decomposed verifiers, repeated evaluation, pairwise tournaments, expert adjudication, and outcome-grounded validation. Recent work on scalable LLM verification proposes continuous scores, criteria decomposition, and repeated evaluation as methods for improving verifier calibration (Kwok et al., 6 Jul 2026). In clinical settings, such methods must be supplemented by explicit auditing for demographic bias, unsafe omissions, over-triage, and evaluator-policy co-adaptation.
Prospective studies should evaluate not only clinician preference but also history completeness, documentation fidelity, triage appropriateness, escalation latency, downstream clinician workload, and patient comprehension. Human standardized-patient OSCEs are an immediate next step; real-world deployment should initially be restricted to supervised co-clinician workflows with clear uncertainty communication and auditability.
Conclusion
ResidencyRL frames clinical AI training as long-horizon sequential policy optimization rather than static medical knowledge acquisition. Through simulated encounters lasting up to 60 dialogue turns and eight tool calls, the method jointly trains history-taking, diagnosis, management, communication, documentation, efficiency, and safety. The trained agent improves adversarial diagnostic accuracy from 81.0% to 88.0%, reduces missed red flags by 14 percentage points, generalizes to longitudinal and specialist settings, and is preferred by expert clinicians in 87.6% of blinded comparisons.
The paper’s principal contribution is empirical evidence that simulation-based RL can produce measurable improvements in clinical process, especially information elicitation and resistance to premature closure. Its conclusions do not establish clinical efficacy in real patients. The remaining challenge is to construct environments whose hidden states, multimodal observations, treatment consequences, social interactions, and evaluation criteria adequately approximate clinical practice while preserving safety and scientific validity.
Whiteboard
Paper Prompts
Sign up for free to create and run prompts on this paper.
Top Community Prompts
Explain it Like I'm 14
1. What is the paper about?
This paper introduces ResidencyRL, a way to train medical AI by letting it practice conversations with simulated patients.
The researchers compare this idea to medical residency. Human doctors do not become skilled just by reading textbooks. They improve by seeing many patients, asking questions, making decisions, receiving feedback, and learning from mistakes. ResidencyRL tries to give an AI a similar kind of practice—inside a computer simulation.
The AI must complete an entire medical visit. It has to:
- Ask the patient useful questions.
- Notice important warning signs.
- Suggest possible diagnoses.
- Decide how urgent the problem is.
- Create a treatment and follow-up plan.
- Explain the plan clearly.
- Write proper medical notes.
2. What questions did the researchers ask?
The main research questions were:
- Can repeated practice in simulated patient conversations make a medical AI better at clinical reasoning?
- Can the AI learn to avoid dangerous mistakes, such as deciding on a diagnosis too quickly?
- Can the AI handle difficult patients who hide information, misunderstand their symptoms, resist advice, or try to make the doctor dismiss serious problems?
- Will skills learned in the simulation work on new cases that the AI did not see during training?
- Would real expert doctors prefer the trained AI over the original, untrained version?
A major concern in medicine is called premature closure. This happens when a doctor chooses the first explanation that seems reasonable and stops looking for other possibilities. For example, a patient may say they have indigestion, but their symptoms could actually be a heart attack. The researchers wanted to see whether training could help the AI keep asking questions and checking for danger.
3. How did the researchers train and test the AI?
Creating simulated patients
The researchers created about 57,000 medical scenarios. These included different:
- Ages and backgrounds.
- Personality types.
- Diseases and symptoms.
- Medication histories.
- Communication styles.
- Levels of medical knowledge.
Some patients were easy to understand, while others gave short answers, used unclear language, or did not mention important information unless asked directly.
For example, a patient with constipation might not volunteer that they drink a strong herbal product called kratom every day. The AI must specifically ask about herbal supplements before discovering that this could be the cause.
The scenarios were checked by another AI system to find medical inconsistencies, such as an impossible age or a treatment that does not make sense.
Letting the AI practice complete visits
The AI had conversations with an LLM-based patient simulator. An LLM is a computer program that can understand and produce human-like language.
Each simulated visit could include up to:
- 60 conversation turns, such as one question and one answer.
- 8 tool calls, meaning structured actions such as submitting a diagnosis or treatment plan.
The AI could use tools similar to parts of a real clinical system. It could submit:
- A primary diagnosis.
- A list of possible diagnoses.
- An urgency level.
- A management plan.
- A plain-language summary for the patient.
- A structured medical note.
Giving the AI scores and feedback
After each visit, an automatic evaluator graded the AI. It looked at six major areas:
| Area | What it means in simple terms |
|---|---|
| Diagnosis | Did the AI identify the likely illness? |
| Management | Did it recommend safe and suitable next steps? |
| Information gathering | Did it ask enough important questions? |
| Communication | Was it clear, respectful, and responsive to emotions? |
| Documentation | Did it write a complete and organized note? |
| Style | Did the conversation sound natural and avoid unnecessary repetition? |
The AI also received penalties for serious safety problems, such as missing an emergency warning sign or giving an unsafe recommendation.
What does “reinforcement learning” mean here?
Reinforcement learning, or RL, is a learning method based on rewards and penalties. It is similar to training a dog or learning a video game:
- A good action earns a positive score.
- A bad or dangerous action loses points.
- After many attempts, the learner becomes more likely to choose actions that lead to better results.
In this study, the AI practiced many patient visits. It gradually adjusted its behavior to earn higher scores across the whole conversation, rather than only trying to produce a good answer to one question.
The researchers used a specific RL method called Group Relative Policy Optimization (GRPO). In simple terms, the system compared several attempts by the AI and encouraged the behavior found in the better attempts.
Testing the trained AI
The researchers compared the trained AI with the original base model. They tested both systems on:
- New cases similar to the training cases.
- Difficult and adversarial cases.
- Cases from other medical evaluation systems.
- Oncology cases, even though oncology was not the main training specialty.
- Longer, multi-visit medical scenarios.
They also asked 97 expert clinicians to compare the two systems without telling them which one had been trained.
4. What did the researchers find?
The trained AI performed better than the original model in several important ways.
Better diagnosis
In difficult adversarial cases, diagnostic accuracy increased from:
- 81.0% for the base model
- 88.0% for ResidencyRL
This is an improvement of 7 percentage points.
Fewer missed warning signs
The trained AI reduced the rate of missed red flags by about 31%. A red flag is a clue that a patient may have a dangerous or urgent condition.
This suggests that the AI became less likely to stop after its first guess and more likely to investigate further.
Better treatment and management plans
The average management score increased from 3.98 to 4.52 on a 1-to-5 scale.
This means the trained AI was generally better at deciding:
- How urgent the situation was.
- Which tests or investigations might be needed.
- What treatment or advice was appropriate.
- When follow-up should happen.
- What warning signs the patient should watch for.
Better communication
The AI also improved at patient-centered communication. For example, its score for responding to patient emotions increased from 2.63 to 3.06.
This matters because a doctor needs more than medical knowledge. Patients may be frightened, confused, embarrassed, or doubtful. Good communication can help patients share important information and follow a care plan.
Skills transferred to new medical tasks
The improvements were not limited to the exact cases used during training.
On the AMIE multi-visit benchmark, the trained AI performed better across all six tested categories. Two especially large improvements were:
- Management reasoning: 80.1% to 88.4%
- Patient communication: 83.7% to 92.2%
The AI also showed improvements on oncology cases and other clinical evaluation systems. This suggests it learned general habits—such as asking more complete questions and checking safety—rather than simply memorizing particular cases.
Expert clinicians preferred the trained AI
In side-by-side comparisons, expert clinicians preferred the ResidencyRL version in 87.6% of cases.
The trained AI was especially preferred for:
- Gathering complete information: 90.7% win rate
- Choosing an appropriate management plan: 75.3% win rate
5. Why are these findings important?
Many medical AI systems are tested using short questions, such as “What is the most likely diagnosis?” Real medical care is more complicated. Doctors must make a series of connected decisions:
- Decide what to ask first.
- Notice what information is missing.
- Ask follow-up questions.
- Consider several possible causes.
- Check for emergencies.
- Choose a plan.
- Explain it to the patient.
- Record what happened.
ResidencyRL focuses on this entire sequence. The results suggest that AI can improve not only its medical knowledge, but also its process of thinking and communicating.
The study is also important because it deliberately includes challenging patients and safety problems. An AI that performs well only when patients are cooperative may not be reliable in real life.
6. What could this research lead to?
If the results hold up in further studies, simulated practice could become a useful way to train medical AI before it is used with real patients. It might help AI systems become:
- More careful when information is incomplete.
- Better at asking follow-up questions.
- Less likely to miss emergencies.
- More skilled at explaining medical information.
- More organized when recording clinical decisions.
It could also provide a safe “practice room” where researchers can test dangerous situations repeatedly without putting real people at risk.
However, the study has an important limitation: the patients were simulated. Even a very realistic computer patient may not behave exactly like a real person, and an automatic AI grader may not judge every medical decision perfectly. The system also did not truly observe what happened to patients after treatment.
Therefore, ResidencyRL is promising, but it is not proof that the AI is ready to replace doctors or independently treat patients. The researchers say that future studies must test it carefully in real clinical workflows, with trained medical professionals supervising it and strong safety protections in place.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
The paper leaves the following issues unresolved:
- Real-world clinical effectiveness is not established: The agent has not been evaluated prospectively in routine clinical workflows with real patients, so its effects on diagnostic accuracy, treatment decisions, patient outcomes, clinician workload, and safety remain unknown.
- Simulation-to-reality transfer is uncertain: LLM-generated patients may not reproduce the variability, ambiguity, emotional dynamics, cultural context, nonverbal cues, and unpredictable behavior of real patients.
- Patient-simulator bias may inflate performance: The same or related LLM-based systems generate scenarios, simulate patients, and judge agent behavior, creating potential shared-model, prompt, or benchmark artifacts that could make results appear stronger than they are.
- Autorater validity remains insufficiently characterized: The paper does not fully report agreement between the Gemini-based reward judges and independent clinician ratings across individual rubric dimensions, rare safety events, and incorrect-but-plausible clinical decisions.
- Reward hacking is not ruled out: It remains unclear whether the agent learned genuine clinical reasoning or optimized superficial correlates of high reward, such as asking stereotyped safety questions, producing lengthy documentation, or matching the simulator’s expected wording.
- The causal contribution of each training component is unclear: The study does not appear to isolate the effects of long-horizon RL, adversarial scenarios, targeted history-taking scenarios, documentation tools, curriculum design, reward weighting, or scenario diversity through comprehensive ablation experiments.
- The impact of reward weighting is unresolved: The chosen emphasis on management and diagnosis over communication, intake, documentation, and style may produce undesirable trade-offs that are not quantified.
- Clinical outcome consequences are only proxied: The environment does not model physiological responses, treatment side effects, disease progression, adherence, or downstream outcomes, so management quality is judged against scenario rubrics rather than actual patient trajectories.
- Ground-truth diagnoses may be oversimplified: Many real cases have multiple coexisting conditions, uncertain diagnoses, evolving presentations, or no single definitive diagnosis; the extent to which the scenario generator represents these situations is unclear.
- Scenario generation may embed systematic biases: Sampling based primarily on United States demographic and telehealth-utilization distributions may limit applicability to other countries, healthcare systems, languages, socioeconomic groups, and populations with limited digital access.
- Representation of protected and underserved groups is not adequately validated: The paper does not report subgroup performance or safety analyses by age, sex, ethnicity, disability, socioeconomic status, health literacy, language, medical mistrust, or other patient characteristics.
- Behavioral persona assumptions may be problematic: Mapping Big Five personality traits and demographic attributes to symptom amplification, adherence, cooperativeness, or communication style could introduce stereotyping or clinically inappropriate associations.
- Language and multimodal limitations remain unexplored: The system is evaluated primarily through text-based English encounters and does not establish performance with interpreters, multilingual patients, speech, accents, images, physical examination findings, medical records, or other clinical modalities.
- The scope of clinical specialties is limited: Although oncology generalization is reported, the training and evaluation focus remains concentrated on primary care and telehealth; performance in emergency medicine, surgery, pediatrics, obstetrics, psychiatry, intensive care, and other high-risk specialties is unresolved.
- Longitudinal reliability is not demonstrated: Improvements on multi-visit benchmarks do not establish that the agent can maintain accurate patient state, detect disease progression, reconcile changing medications, or make safe decisions over extended real-world care trajectories.
- Robustness to distribution shift is incompletely tested: The evaluations do not establish resilience to new institutions, documentation conventions, prevalence rates, clinical guidelines, patient populations, languages, or previously unseen adversarial strategies.
- The adversarial scenarios may not reflect real attack distributions: The fixed prompt-defined adversarial taxonomy and behaviors may be predictable, and the agent’s robustness to naturally occurring manipulation, prompt injection, misinformation, coercion, or coordinated attacks remains unknown.
- Safety-event confidence intervals and rare-event power are unclear: A reported reduction in missed red flags may be based on too few cases to reliably estimate performance on rare but catastrophic events.
- Error severity is not fully analyzed: Aggregate diagnostic and rubric scores may conceal whether the remaining errors are minor documentation omissions or potentially fatal failures involving emergency escalation, medication contraindications, abuse, suicidality, or delayed care.
- Calibration and uncertainty communication are insufficiently evaluated: The paper does not establish whether confidence, differential rankings, urgency classifications, or recommendations are calibrated to actual error likelihood.
- The effect of tool-use constraints is unknown: The fixed documentation API may not represent real EHR interfaces, laboratory systems, imaging systems, referral processes, prescribing systems, or institutional decision-support tools.
- Efficiency and interaction burden are underexplored: The agent may improve completeness by asking more questions or generating longer notes, but the paper does not adequately quantify consultation length, token/tool cost, clinician time, patient burden, or the trade-off between thoroughness and efficiency.
- Human-evaluation generalizability is limited: The clinician preference study uses 97 blinded experts, but the paper does not establish how preferences vary by specialty, experience, practice setting, risk tolerance, or familiarity with AI-assisted care.
- Preference is not equivalent to clinical correctness: Side-by-side clinician preference may favor fluent or detailed responses even when those responses contain subtle diagnostic, management, or safety errors.
- Comparisons with baselines may be confounded: It is unclear whether the base model and competing systems received equivalent scaffolding, tool access, prompting, context, inference budgets, and evaluation conditions.
- The contribution of scale and model identity is not separated from the RL method: Because training uses a specific Gemini model and proprietary infrastructure, it remains unclear whether the gains arise from ResidencyRL itself or from model scale, hidden pretraining differences, or system-level engineering.
- Reproducibility is constrained: The paper does not provide sufficient detail to independently reproduce the full scenario corpus, simulator prompts, autorater prompts, filtering decisions, training compute, sampling procedures, hyperparameters, or model checkpoints.
- Data contamination and benchmark leakage are not excluded: The extent to which training scenarios, source datasets, evaluation cases, or related benchmark content may have appeared in model pretraining or scenario-generation prompts is not established.
- Stability across random seeds and training runs is unknown: The paper does not report whether the observed improvements are consistent across independent runs, model initializations, scenario samples, or reward-model variations.
- Negative transfer and unintended behaviors are insufficiently investigated: It remains unclear whether RL improves targeted competencies at the expense of creativity, bedside manner, diagnostic breadth, appropriate escalation, adherence to patient preferences, or performance on unrelated tasks.
- Human oversight requirements are undefined: The study does not specify when clinicians must review, override, or supervise the agent, nor does it quantify how much oversight is required to achieve acceptable safety.
- Accountability and governance remain open questions: Responsibility for errors, documentation ownership, informed consent, auditability, privacy, data retention, and regulatory classification are not addressed.
- Clinical guideline and institutional variation are not evaluated: Management recommendations may be judged against a single rubric or reference standard, leaving uncertainty about performance when guidelines conflict, evolve, or differ across institutions.
- Patient-centered outcomes are missing: The study does not measure patient trust, understanding, satisfaction, adherence, anxiety, perceived autonomy, accessibility, or willingness to follow the agent’s recommendations.
- The durability of learned competencies is unknown: No evidence is provided on whether improvements persist after training, under continued interaction with new patient behaviors, or after changes to prompts, tools, guidelines, or model versions.
Practical Applications
Immediate Applications
The paper’s results support near-term use primarily in training, evaluation, quality assurance, and workflow prototyping, rather than autonomous clinical care. The agent has only been validated in simulated environments and still requires prospective testing with real patients.
- AI-assisted residency and medical-student simulation training — Healthcare education; Immediate Application Deploy the simulated patient environment as a virtual standardized-patient platform for practicing history-taking, differential diagnosis, triage, management planning, communication, and SOAP-note documentation. Scenarios can be configured for routine, complex, or adversarial encounters, allowing learners to repeat difficult cases without consuming clinician or standardized-patient time. Potential tools/workflows: browser-based virtual OSCEs, automated encounter transcripts, competency dashboards, and feedback reports organized by diagnostic accuracy, red-flag detection, communication, and documentation. Dependencies: feedback validity depends on the quality of the LLM autorater and scenario ground truth; educator review remains necessary for high-stakes assessment.
- Targeted remediation for diagnostic errors — Medical education and clinical quality improvement; Immediate Application Use the four targeted history-taking domains—social/lifestyle factors, medication details, symptom characterization, and exposures—to generate personalized exercises for common weaknesses such as anchoring, incomplete medication review, or failure to ask about travel and substance use. Potential workflow: a learner completes a case, the system identifies omitted questions or missed diagnostic pivots, and subsequent cases are selected to rehearse that competency. Dependencies: remediation recommendations should be checked against specialty-specific curricula and validated assessment standards.
- Adversarial safety testing of clinical conversational AI — Healthcare software and AI assurance; Immediate Application The scenario-generation and red-team pipeline can be used to test symptom checkers, telehealth assistants, clinical documentation agents, and medical chatbots against concealed emergencies, minimized symptoms, prompt manipulation, unsafe prescribing requests, abuse or crisis situations, and inappropriate telehealth use. Potential outputs: safety regression suites, red-flag detection benchmarks, refusal and escalation tests, and release-gating reports for clinical AI products. Dependencies: adversarial scenarios must be reviewed by clinicians to avoid unrealistic or incomplete failure tests; simulated performance does not establish patient safety.
- Benchmarking sequential clinical reasoning rather than single-turn knowledge — Academia and industry evaluation; Immediate Application Researchers can adopt the paper’s long-horizon evaluation design—up to 60 dialogue turns and eight structured tool interactions—to measure whether a model gathers sufficient information, updates its differential, chooses appropriate urgency, and produces a coherent management plan. Potential tools: standardized multi-turn benchmark harnesses, trajectory-level scoring, and comparisons across diagnostic, management, communication, and documentation axes. Dependencies: automated LLM grading requires calibration against blinded expert judgments and monitoring for judge bias or reward hacking.
- Clinical documentation assistance under human review — Healthcare IT; Immediate Application The documentation API and SOAP-note outputs could support clinician-facing systems that draft encounter summaries, differentials, patient instructions, follow-up recommendations, and safety-netting language after a consultation. The system could also flag missing history elements or red flags before a note is finalized. Dependencies: outputs must remain drafts; integration requires privacy safeguards, EHR interoperability, audit logs, clinician sign-off, and compliance with applicable medical-device and data-protection requirements.
- Telehealth intake and routing support — Primary care and virtual care; Immediate Application A constrained version of the agent could collect structured histories, identify urgent symptoms, classify whether text-based care is appropriate, and route patients to emergency, urgent, or routine services. Its emphasis on hidden information and adversarially minimized symptoms is particularly relevant to remote encounters. Dependencies: it should not independently diagnose or prescribe; escalation thresholds must be conservative, locally governed, and evaluated prospectively for false negatives and inequitable performance.
- Creation of scalable standardized-patient cases — Medical schools, hospitals, and professional certification; Immediate Application The proposer–judge pipeline can generate diverse cases across demographics, communication styles, comorbidities, and difficulty levels. Institutions could use these cases for formative OSCE preparation, interview training, communication practice, and continuing professional development. Dependencies: generated cases require clinical validation, demographic-bias audits, deduplication, and controls against teaching incorrect or culturally inappropriate practices.
- Policy and procurement criteria for clinical AI — Healthcare governance and public policy; Immediate Application Health systems and regulators can use the paper’s framework to require vendors to report performance on complete encounter trajectories, not only medical question-answering. Procurement evaluations could include red-flag sensitivity, urgency calibration, documentation quality, patient-centered communication, and robustness to resistant or low-health-literacy patients. Dependencies: thresholds must be agreed upon by clinical societies and regulators, and results should be stratified by language, demographic group, specialty, and care setting.
- Training clinicians to recognize AI failure modes — Clinical practice and patient safety; Immediate Application The simulator can expose clinicians to examples of AI premature closure, incomplete questioning, unsafe management plans, and inappropriate confidence. This supports AI-literacy programs and establishes workflows for reviewing and correcting AI-generated recommendations. Dependencies: training must clearly distinguish simulation behavior from validated clinical performance and avoid creating unwarranted trust in the system.
Long-Term Applications
The paper points toward broader clinical and operational systems, but these uses require additional research, real-world validation, stronger causal outcome measures, and regulatory oversight.
- Clinician-supervised clinical copilot for end-to-end encounters — Healthcare delivery; Long-Term Application A mature version could participate throughout a consultation: review the chart, conduct preliminary history-taking, identify missing information, propose a ranked differential, suggest investigations and management options, generate documentation, and provide patient-facing explanations. The clinician would retain final authority. Dependencies: prospective trials must establish effects on diagnostic accuracy, time, workload, disparities, adverse events, and patient satisfaction. The system also needs reliable access to current guidelines, medications, labs, imaging, and EHR context.
- Longitudinal disease-management assistant — Chronic care and population health; Long-Term Application Extending the multi-visit capabilities described in the paper, the agent could monitor evolving symptoms, adherence, treatment response, and safety-netting requirements for conditions such as diabetes, hypertension, cancer survivorship, or gastrointestinal disease. It could prompt follow-up, identify deterioration, and prepare visit summaries for clinicians. Dependencies: requires validated patient-state memory, accurate integration with remote-monitoring data, strong privacy controls, and safeguards against inappropriate treatment changes based on incomplete information.
- Specialty-specific clinical agents — Oncology, emergency medicine, psychiatry, pediatrics, and other specialties; Long-Term Application The demonstrated transfer to unseen oncology cases suggests a pathway for specialty adaptation. Future agents could be trained with specialty-specific scenarios, guidelines, tools, and safety taxonomies—for example, chemotherapy toxicity triage, psychiatric crisis escalation, pediatric age-specific dosing, or emergency atypical-presentation detection. Dependencies: specialty deployment requires expert-curated data, separate validation for rare and high-risk conditions, and careful handling of cases where the simulator lacks physiological realism.
- Interactive clinical skills certification and credentialing — Academia and professional regulation; Long-Term Application Multi-turn simulated encounters could become part of formative or summative assessment for residents and practicing clinicians. Evaluation could cover not only the final diagnosis but also information gathering, communication, escalation, documentation, and management quality across many cases. Dependencies: certification use requires demonstrated reliability, fairness, test security, human governance, and evidence that simulation scores correlate with real-world clinical competence.
- Hospital-wide safety surveillance and AI quality management — Health-system operations; Long-Term Application Hospitals could continuously replay representative and adversarial cases against deployed models, using trajectory-level metrics to detect performance drift after model, guideline, or EHR changes. The system could function as a clinical-AI “continuous integration” pipeline. Dependencies: requires standardized safety metrics, versioned scenarios, secure infrastructure, incident-report integration, and clear procedures for disabling or revising unsafe systems.
- Evidence-grounded autonomous triage in low-resource settings — Global health and public-sector care; Long-Term Application After rigorous validation, a lightweight agent could support initial symptom assessment and referral decisions where clinician availability is limited, including multilingual or asynchronous telehealth services. Its ability to probe for hidden information could be valuable when patients provide brief or incomplete histories. Dependencies: requires local validation, language and cultural adaptation, offline or low-bandwidth operation, equitable performance, referral capacity, and explicit escalation pathways for emergencies.
- Simulation-based policy stress testing — Public health and health-system planning; Long-Term Application Policymakers could simulate how virtual patients with different health literacy, trust, socioeconomic circumstances, or communication styles respond to triage rules, screening recommendations, and access pathways. This could help identify policies likely to produce missed emergencies or unequal uptake. Dependencies: behavioral profiles must be empirically grounded rather than inferred solely from LLM prompts; simulated responses should be compared with real-world utilization and patient-engagement data.
- Closed-loop clinical decision optimization — Clinical operations and research; Long-Term Application The RL framework could eventually optimize sequences involving questions, tests, referrals, treatments, and follow-up while accounting for cost, patient burden, diagnostic risk, and resource availability. This would extend the current framework beyond dialogue and documentation toward decision policies with measurable downstream outcomes. Dependencies: the current simulator does not model physiological treatment effects. Safe deployment therefore requires calibrated patient-state models, real-world outcome data, causal evaluation, conservative action constraints, and human oversight.
- Integration with multimodal and physical-care systems — Medical devices, robotics, and ambient computing; Long-Term Application The approach could be extended to agents that combine conversation with wearable data, imaging, laboratory results, examination devices, or robotic assistance. Such systems might conduct an intake, request measurements, interpret results, and coordinate care actions across a heterogeneous tool space. Dependencies: multimodal data quality, sensor calibration, interoperability, cybersecurity, latency, and regulatory approval are essential. Errors in any connected device could propagate through the full decision trajectory.
- Patient-facing self-management and health-navigation products — Consumer health; Long-Term Application A heavily constrained version could help users prepare for appointments, organize medication and symptom histories, explain clinician instructions, identify questions to ask, and recognize when urgent care may be needed. The agent’s communication and patient-summary capabilities are directly relevant to these functions. Dependencies: consumer systems must avoid presenting speculative diagnoses as facts, protect sensitive health information, support accessibility and multiple languages, and use conservative emergency guidance with prominent human-care escalation.
Glossary
- Adversarial manipulation: Deliberate behavior intended to exploit weaknesses in an AI system or decision-making process. “which are further vulnerable to adversarial manipulation”
- Agentic reinforcement learning: Reinforcement learning in which a LLM interacts with an environment over multiple steps and takes actions autonomously. “This paradigm restores multi-turn optimization by situating the LLM in a temporally extended Partially Observable Markov Decision Process (POMDP)”
- Autorater: An automated evaluator, often based on a LLM, that scores system outputs according to specified criteria. “An LLM autorater processes the encounter transcript and tool calls, yielding a structured reward”
- Behavioral profile: A structured representation of a person’s characteristic patterns of communication or action. “we assign behavioral profiles grounded in personality psychology”
- Clinical coherence: The degree to which clinical information is internally consistent and medically plausible. “Scenarios further undergo LLM-as-judge clinical coherence verification.”
- Clinical differential diagnosis: A ranked set of possible diagnoses considered to explain a patient’s symptoms. “differential diagnosis submission, which accepts a ranked list with reasoning chains”
- Clinical utility: The practical value of a medical system for improving patient care or clinical decision-making. “Prospective validation with real-world workflows remains necessary to establish clinical utility.”
- Cosine similarity: A measure of similarity between vectors based on the cosine of the angle between them. “Pairs exceeding a cosine similarity threshold of $0.9$ are pruned”
- Critical safety flag: A binary indicator that identifies a potentially dangerous clinical behavior or omission. “complemented by safety-critical penalty flags”
- DDXPlus evidence corpus: A dataset of clinical findings and their validated associations with diseases, used to ground diagnostic scenarios. “Grounding generated scenarios in DDXPlus ensures clinical findings reflect validated associations rather than hallucinated artifacts.”
- Domain generalization: The ability of a model trained in one domain or environment to perform effectively in previously unseen domains. “These improvements generalize broadly”
- EHR-grounded world model: A simulated model of clinical reality constructed using information from electronic health records. “DiagAgent \citep{qiu2025evolving} trains within an EHR-grounded world model.”
- Experience replay: A reinforcement-learning technique that reuses previously collected trajectories or interactions during policy training. “Doctor-R1 \citep{lai2025doctorr1} introduces experience replay to ground policy learning in retrieved past trajectories”
- Foundation model: A large, generally pretrained model that can be adapted to many downstream tasks. “generalist foundation models with advanced medical reasoning abilities promise a new paradigm for medical AI”
- Group Relative Policy Optimization (GRPO): A reinforcement-learning algorithm that updates a policy using rewards compared across groups of sampled trajectories. “The policy agent is optimized via Group Relative Policy Optimization (GRPO)”
- Hallucinated artifact: An output or data element generated by a model that appears plausible but is unsupported or incorrect. “rather than hallucinated artifacts”
- Heterogeneous action space: An action space containing different types of actions, such as natural-language messages and structured tool calls. “a heterogeneous action space ($\mathcal{A}_{\text{lang} \cup \mathcal{A}_{\text{tool}$)”
- Hierarchical rubric: An evaluation framework organized into multiple levels of criteria and subcriteria. “Training is guided by a hierarchical rubric optimizing six clinical dimensions”
- Health literacy: The ability to obtain, understand, and use health information to make appropriate decisions. “systemic barriers such as health literacy and medical mistrust hinder effective communication”
- Information asymmetry: A situation in which one participant possesses information that another participant does not. “Information asymmetry distinguishes proactive information”
- In-domain evaluation: Evaluation performed on data or scenarios drawn from the same distribution as the training environment. “On held-out in-domain cases”
- Likert scale: An ordinal rating scale used to measure the degree or quality of a judgment, commonly from one to five. “management quality (3.98 to 4.52 on a 1--5 Likert scale)”
- Long-horizon reinforcement learning: Reinforcement learning over extended sequences of actions whose consequences and rewards may unfold over many steps. “Long-horizon multi-turn RL for sequential clinical reasoning”
- Markov Decision Process (MDP): A formal model of sequential decision-making in which the current state contains all information needed to predict future transitions and rewards. “These methods optimize a degenerate Markov Decision Process ()”
- Medical mistrust: Distrust of medical institutions, professionals, or health-care recommendations. “or higher medical mistrust”
- Multi-agent role decomposition: A method that divides a complex task among multiple specialized agents or roles. “SALUS \citep{gaosalus} applies per-step GRPO with multi-agent role decomposition”
- Online reinforcement learning: Reinforcement learning in which the policy learns from interactions generated in the current environment during training. “ResidencyRL is an online multi-turn RL method”
- Out-of-domain evaluation: Evaluation on tasks, domains, or distributions not encountered during training. “five out-of-domain evaluation frameworks”
- Partially Observable Markov Decision Process (POMDP): A sequential decision-making model in which the agent cannot directly observe the complete underlying state of the environment. “a temporally extended Partially Observable Markov Decision Process (POMDP)”
- Patient-centered communication: Clinical communication that incorporates the patient’s perspective, emotions, preferences, and participation in decision-making. “patient-centered communication”
- Premature closure: A diagnostic error in which a clinician stops considering alternatives after forming an initial diagnosis. “demonstrating rigorous mitigation of premature closure”
- Prompt injection: An adversarial input designed to manipulate a LLM into violating its intended instructions. “distinct from general-purpose prompt injection”
- Psychometric evaluation: The measurement and analysis of psychological or assessment-related properties such as reliability and consistency. “This analytic scoring approach is motivated by psychometric evaluation theory”
- Red teaming: The deliberate testing of a system using adversarial inputs or behaviors to uncover vulnerabilities. “This design adapts a broader paradigm of LLM-based redteaming”
- Reward shaping: The design or modification of reward signals to encourage desired behaviors during reinforcement learning. “training against a structured reward aligned to diagnostic accuracy, management quality, communication, documentation, and safety”
- Safety-netting: Providing instructions about warning signs, follow-up, and actions to take if a patient’s condition worsens. “follow-up and safety-netting adequacy”
- SOAP note: A standardized clinical documentation format consisting of Subjective, Objective, Assessment, and Plan sections. “clinical documentation following the SOAP note (Subjective, Objective, Assessment, and Plan) format”
- State-aware memory: A simulation mechanism that tracks relevant evolving information about an entity or environment across interactions. “state-aware memory \citep{liao2024automatic}”
- Structured output schema: A predefined format constraining an AI system’s response to expected fields or data types. “with structured output schema constraints”
- Tool calling: The process by which a LLM invokes external functions or APIs to perform structured actions. “Multi-turn simulation rollouts capture the agent's sequential clinical reasoning, natural language dialogue, and tool utilization”
- Trajectory: A sequence of states, observations, actions, and rewards generated during an interaction episode. “up to 60 dialogue turns and 8 tool calls per trajectory”
- Triage: The classification of a patient’s urgency and prioritization for appropriate care. “how to triage urgent presentations”
- Uncertainty calibration: The degree to which a system’s confidence corresponds to the actual likelihood that its prediction is correct. “clinical reasoning under uncertainty remains hard for humans and AI alike”
- World model: An internal or simulated representation of an environment’s states, dynamics, and possible outcomes. “trains within an EHR-grounded world model”
Collections
Sign up for free to add this paper to one or more collections.
