Generalizability of results to non-US healthcare settings

Ascertain how well the performance and assistive impacts reported for AMIE and general cardiologists in this single-center, US-based, English-only dataset extrapolate to healthcare systems and patient populations in non-US settings.

Background

The study uses retrospective, de-identified data from a single US center with English-language clinical text, which may introduce geographical and linguistic biases. The authors explicitly state uncertainty regarding applicability beyond the US context.

Evaluating generalizability is essential to understand whether AMIE-assisted workflows and observed benefits hold in diverse international healthcare environments with different patient populations, resource constraints, and clinical practices.

References

Further limitations of our work include a biased sample of patients - patients were selected from one US center, using only English text. It is unclear how well our results will extrapolate to other non-US settings.

— Towards Democratization of Subspeciality Medical Expertise  (2410.03741 - O'Sullivan et al., 2024) in Section 6 (Discussion) — Limitations

Moreover, as we used medical claims from insurers for the elderly in Japan, the study population is biased toward elderly individuals, meaning that the generalizability of our findings to populations with different genetic backgrounds or healthcare systems is unclear.

— Pretrained Medical Representations for the Practical Screening of Drug Repositioning Candidates  (2609.19865 - Fujioka et al., 17 Sep 2026) in Appendix, Section 'Limitations'