Specialist-level decision-making capacity of large language models

Determine whether large language models possess the nuanced understanding and intricate specialized medical knowledge required to effectively replicate the decision-making process of experts in highly specialized medical fields, such as subspecialty cardiology.

Background

The paper examines whether an LLM-based system (AMIE) can assist general cardiologists in cases typically requiring subspecialist expertise, focusing on inherited cardiomyopathies. Despite promising results, the broader question of whether LLMs can truly match subspecialists' decision-making across specialist domains is explicitly stated as unresolved.

The authors note that rigorous assessments in medical specialties remain scarce and datasets for evaluation are limited, underscoring the need to establish whether LLMs have the depth of understanding and knowledge base required for expert-level clinical decisions across specialized fields.