MatchMiner-AI: An Open-Source Solution for Cancer Clinical Trial Matching (2412.17228v1)

Abstract: Clinical trials drive improvements in cancer treatments and outcomes. However, most adults with cancer do not participate in trials, and trials often fail to enroll enough patients to answer their scientific questions. Artificial intelligence could accelerate matching of patients to appropriate clinical trials. Here, we describe the development and evaluation of the MatchMiner-AI pipeline for clinical trial searching and ranking. MatchMiner-AI focuses on matching patients to potential trials based on core criteria describing clinical "spaces," or disease contexts, targeted by a trial. It aims to accelerate the human work of identifying potential matches, not to fully automate trial screening. The pipeline includes modules for extraction of key information from a patient's longitudinal electronic health record; rapid ranking of candidate trial-patient matches based on embeddings in vector space; and classification of whether a candidate match represents a reasonable clinical consideration. Code and synthetic data are available at https://huggingface.co/ksg-dfci/MatchMiner-AI . Model weights based on synthetic data are available at https://huggingface.co/ksg-dfci/TrialSpace and https://huggingface.co/ksg-dfci/TrialChecker . A simple cancer clinical trial search engine to demonstrate pipeline components is available at https://huggingface.co/spaces/ksg-dfci/trial_search_alpha .

• The paper presents MatchMiner-AI, an open-source AI pipeline that leverages LLMs and other models to extract, summarize, and match patient and trial criteria from unstructured clinical data.
• Evaluation showed the pipeline significantly improves precision in matching patients to clinical trials compared to baseline methods, achieving up to 0.89 precision@10.
• MatchMiner-AI offers a scalable solution to address bottlenecks in clinical trial recruitment with potential applications beyond oncology, though further validation is needed in clinical settings.

MatchMiner-AI: Enhancing Cancer Clinical Trial Matching with AI

The academic paper titled "#MatchMiner-AI: An Open-Source Solution for Cancer Clinical Trial Matching" presents a comprehensive paper on the development and evaluation of the MatchMiner-AI pipeline. This open-source solution utilizes artificial intelligence to optimize patient-trial matching processes, a formidable bottleneck in clinical oncology research.

Overview and Methodology

The MatchMiner-AI framework is engineered to transform longitudinal patient data and associated clinical trial information into actionable insights to identify suitable matches between patients and trials. A core innovation of this work is the integration of a LLM, specifically Llama-3.1-70B, to extract and synthesize pertinent data points from unstructured Electronic Health Records (EHR) and trial documents.

The pipeline's architecture is multi-faceted:

1. Information Extraction: Utilizing a customized model for condensing patients' records, the framework collates pertinent medical history, focusing on cancer type, histology, disease extent, biomarkers, and prior treatments. The TinyBERT model facilitates sentence-level classification to spotlight relevant information.
2. Patient Summarization: Following the condensation of medical records, Llama-3.1-70B generates structured summaries encapsulating core clinical criteria. These are pivotal for determining eligibility with respect to trial space definitions.
3. Trial Space Definition and Extraction: Through the clinicaltrials.gov database, the framework extracts trial spaces—distinct combinations of clinical criteria a trial targets—and maps them using Llama-3.1-70B.
4. Model Training and Evaluation:
   • TrialSpace Model: A stella-en-1.5B text embedding model undergoes fine-tuning to effectively align patient summaries with the correct trial spaces through contrastive learning approaches.
   • TrialChecker Model: Using a Roberta-Large-based model, it further refines predictions by assigning probabilities that a pairing is a "reasonable consideration."

Evaluation and Performance

The pipeline's overall performance metrics reveal substantial efficacy. In the patient-centric model, utilizing TrialSpace followed by TrialChecker yielded a precision @ 10 of 0.89, improving significantly from 0.72 with the baseline TrialSpace model alone. Likewise, in a trial-centric view, precision rose from 0.65 to 0.91 with the added TrialChecker model.

Supplementary evaluations on synthetic data, while less precise than real data-driven models, still substantiate the feasibility of distribution without privacy concerns. These demonstrate the adaptable effectiveness of MatchMiner-AI for broader institutional use and across divergent trial datasets.

Implications and Future Directions

The implications of MatchMiner-AI are substantial in both theoretical and practical spectrums. Methodologically, it presents a scalable and adaptive solution for trial matching exceeding existing comprehensive criteria extraction systems by focusing on high-impact core criteria. This evidently facilitates a reduction in redundant or non-contributory eligibility determinations, optimizing the trial recruitment process.

Practically, these insights have broad applicability beyond oncology. By aptly structuring trial spaces around pivotal disease-specific contexts, such as those seen in cancer treatment, other domains can develop analogous systems for complex disease trial matching.

Future work should aim to validate and enhance the system's performance across diverse healthcare institutions, adapting it to capture a wider array of patient and trial space variables. Moreover, advancing the quality of synthetic training data could bridge the performance absorption seen in transitions from real to contrived datasets, potentially surging AI deployment in healthcare-reviewed research contexts.

In summation, MatchMiner-AI delineates a robust pathway to augment clinical trial matching processes, with promising cross-over potential to other medical fields. Its capabilities require further empirical validation in clinical settings but provide a sound framework to address the longstanding challenges in patient-trial recruitment.