ResidencyRL: Teaching AI to Think Like a Doctor
This presentation explores ResidencyRL, a groundbreaking reinforcement learning system that trains clinical AI agents through long-horizon simulated patient encounters. Unlike traditional medical AI that excels at answering static questions, ResidencyRL learns the sequential art of clinical reasoning: asking targeted questions, resisting premature closure, eliciting hidden risk factors, and producing safer management plans. The talk examines how procedural competence emerges through interaction, why demonstrations alone fail to teach clinical decision-making, and what evidence suggests about transferring simulation-trained skills to real clinical settings.Script
A model that aces every medical exam can still fail a real patient by asking the wrong questions. ResidencyRL tackles the gap between knowing medicine and practicing it, training AI through thousands of simulated clinical encounters where the right diagnosis depends not on recall, but on procedural judgment.
In adversarial simulations, patients conceal medication use, minimize symptoms, or resist escalation. The baseline model's diagnostic accuracy drops to 81 percent. After reinforcement learning, it climbs to 88 percent because the trained policy learned to probe what patients don't volunteer.
The training environment is not a question-answering task. It is a partially observable decision process spanning up to 60 dialogue turns, eight tool calls, differential ranking, urgency classification, SOAP documentation, and patient-facing summaries. The agent must decide not only what to ask, but when to stop.
Training changed how the model allocates interaction. Median encounter length grew from 18 to 23 turns. Screening for social history, medications, and exposures doubled. Missed red flags dropped by 14 percentage points, and management quality jumped from 3.94 to 4.51 on a 5 point scale.
The strongest evidence comes from out-of-domain transfer. ResidencyRL improved performance on longitudinal management, specialist oncology consultations, and external telehealth benchmarks it never saw during training. In blinded clinician comparisons, experts preferred the trained agent 87.6 percent of the time.
But simulation is not reality. The patient simulator is an Large Language Model, the reward is a proxy, and the evaluation remains embedded in text-based telehealth scenarios. Real clinical validation requires standardized patient studies, multimodal observation, treatment response, and outcome data. Visit EmergentMind.com to explore the full paper and create your own video summaries of the research shaping clinical AI.
