Eligibility-Aware Evidence Synthesis: An Agentic Framework for Clinical Trial Meta-Analysis

Abstract: Clinical evidence synthesis requires identifying relevant trials from large registries and aggregating results that account for population differences. While recent LLM-based approaches have automated components of systematic review, they do not support end-to-end evidence synthesis. Moreover, conventional meta-analysis weights studies by statistical precision without considering clinical compatibility reflected in eligibility criteria. We propose EligMeta, an agentic framework that integrates automated trial discovery with eligibility-aware meta-analysis, translating natural-language queries into reproducible trial selection and incorporating eligibility alignment into study weighting to produce cohort-specific pooled estimates. EligMeta employs a hybrid architecture separating LLM-based reasoning from deterministic execution: LLMs generate interpretable rules from natural-language queries and perform schema-constrained parsing of trial metadata, while all logical operations, weight computations, and statistical pooling are executed deterministically to ensure reproducibility. The framework structures eligibility criteria and computes similarity-based study weights reflecting population alignment between target and comparator trials. In a gastric cancer landscape analysis, EligMeta reduced 4,044 candidate trials to 39 clinically relevant studies through rule-based filtering, recovering all 13 guideline-cited trials. In an olaparib adverse events meta-analysis across four trials, eligibility-aware weighting shifted the pooled risk ratio from 2.18 (95% CI: 1.71-2.79) under conventional Mantel-Haenszel estimation to 1.97 (95% CI: 1.76-2.20), demonstrating quantifiable impact of incorporating eligibility alignment. EligMeta bridges automated trial discovery with eligibility-aware meta-analysis, providing a scalable and reproducible framework for evidence synthesis in precision medicine.

• The paper introduces EligMeta, a framework combining LLMs for natural language understanding with deterministic methods to achieve reproducible, eligibility-aware evidence synthesis.
• It demonstrates an eligibility-weighted meta-analysis using an extended Mantel-Haenszel estimator, aligning pooled effect estimates with target clinical populations.
• Empirical evaluations in oncology reveal improved guideline relevance and sensitivity compared to conventional meta-analysis techniques.

Eligibility-Aware Meta-Analysis of Clinical Trials: The EligMeta Framework

Introduction

The rapid expansion of clinical trial data, particularly in complex domains such as oncology and precision medicine, exacerbates challenges in systematic evidence synthesis. Existing methods for meta-analysis typically weight studies by statistical precision, disregarding heterogeneity in patient populations and study eligibility criteria. This practice can yield effect estimates that are misaligned with the clinical context of interest, undermining the applicability of synthesized evidence. The paper "Eligibility-Aware Evidence Synthesis: An Agentic Framework for Clinical Trial Meta-Analysis" (2604.02678) introduces EligMeta—a hybrid agentic framework that integrates LLMs with deterministic computation to perform transparent, reproducible, and eligibility-aware meta-analyses.

Framework Architecture and Methodology

EligMeta operationalizes a two-stage process: rule-based, LLM-assisted discovery and structuring of relevant trials, followed by deterministic, eligibility-aware meta-analysis. LLMs are utilized exclusively for natural language understanding (NLU), rule generation, and parsing of unstructured clinical text. All data filtering, eligibility scoring, and statistical computation are conducted deterministically to guarantee reproducibility and auditability.

Figure 1: Overview of the EligMeta framework for clinical trial evidence synthesis, illustrating the conversion of a free-text query into structured, auditable rule sets, deterministic trial filtering, and eligibility-aware, population-aligned meta-analysis.

Trial Selection and Rule Structuring

Free-text clinical queries are transformed into explicit, human-interpretable inclusion and exclusion criteria, leveraging LLM reasoning constrained to rule specification and parsing of trial metadata. Each criterion is mapped to a function plan that restricts LLM calls to format-constrained, schema-driven information extraction. This separation allows for expert review and adjustment prior to deterministic filtering over structured trial registries.

Figure 2: Pipeline for parsing clinical queries into structured rules and deterministic function plans, restricting the LLM's scope to interpretive tasks and parsing, with all selection logic executed deterministically.

Eligibility-Based Similarity and Meta-Analytic Weighting

The second stage quantifies population similarity. EligMeta structures both candidate and target trial eligibility criteria, then applies LLM-generated penalty rules to identify specific, clinically meaningful mismatches. Penalty scores—interpretable and attributable to individual criteria—are deterministically computed and then transformed into eligibility weights that modulate each trial's contribution to meta-analytic pooling.

Figure 3: Workflow for eligibility-aware meta-analysis. Free-text criteria are standardized, mismatch penalties are applied, and final meta-analytic weights are derived and used in pooled statistical inference.

The principal innovation is the eligibility-weighted extension of the classical Mantel-Haenszel estimator for binary outcomes. The eligibility-weighted Mantel-Haenszel (EW-MH) estimator incorporates study-specific eligibility weights, resulting in pooled effect estimates more representative of the population of clinical interest.

Empirical Evaluation

Rule-Based Filtering in Gastric Cancer

EligMeta's filtering capability was evaluated on a prototypical query for gastric and gastroesophageal junction (GEJ) cancer trials. From an initial 4,044 candidates retrieved from ClinicalTrials.gov, the systematic, sequential application of six LLM-specified and deterministically implemented rules yielded 39 eligible studies. Notably, 13 of these were cited in National Comprehensive Cancer Network (NCCN) guidelines, demonstrating both sensitivity and specificity of the approach to guideline-relevant evidence.

Figure 4: Stepwise attrition throughout the rule-based filtering cascade for the gastric cancer use case, highlighting accountability and transparency at every stage of study exclusion.

Comparative evaluations with state-of-the-art general-purpose LLM (GPT-5.4 Deep Research) and agentic coding (OpenAI Codex) pipelines demonstrated superior coverage and alignment of EligMeta with clinical guidelines, as well as stricter adherence to prespecified criteria. GPT-5.4, for example, retrieved only 11 studies (8 guideline-cited), while Codex identified 28 (7 guideline-cited), including ineligible studies and missing key pivotal trials.

Eligibility-Aware Meta-Analysis: Olaparib Adverse Events

The impact of eligibility-aware synthesis was assessed using a meta-analysis of adverse events (risk ratio for all-grade vomiting) from olaparib maintenance trials. Using Golan et al. as the reference population, penalty rules were derived for each comparator study—quantifying discrepancies in lines of therapy, cancer type, timing of randomization, and patient inclusion logic.

Eligibility-based weights were assigned: Golan (0.52), Moore (0.13), Ledermann (0.21), Pujade-Lauraine (0.13). Application of the EW-MH estimator yielded a pooled risk ratio of 1.97 (95% CI: 1.76–2.20), compared with 2.18 (95% CI: 1.71–2.79) under conventional precision-based weighting. This realignment is attributable to increased influence assigned to the Golan study due to perfect eligibility matching, and down-weighting of studies with fundamental population mismatches.

Figure 5: Differential influence of trials and pooled risk ratio estimates for olaparib-maintenance–related vomiting, as a function of classical versus eligibility-aware weighting.

Implications and Theoretical Considerations

EligMeta establishes a blueprint for integrating LLMs into high-stakes biomedical statistical workflows without conceding transparency or reproducibility. The explicit modularization—LLMs for interpretation, deterministic modules for computation—reduces susceptibility to LLM hallucination and isolates modeling choices to components open for domain expert review. This approach fulfills a core unmet need in evidence synthesis: the ability to tailor effect estimates to target clinical populations rather than "average" study populations.

The eligibility-weighted meta-analysis framework is both theoretically sound (with properties such as large-sample unbiasedness and explicit variance estimation) and broadly extensible. While demonstrated using the Mantel-Haenszel estimator for binary outcomes, the weighting principle is directly applicable to other effect measures (e.g., continuous outcomes, time-to-event data) and meta-analytic models (random/fixed effects, Bayesian frameworks).

Practical implications include more precise, auditable support for guideline development, regulatory assessment, and design of precision medicine interventions. The explicit modeling of eligibility alignment enables quantifiable sensitivity analysis with respect to study selection and population definition. Moreover, the systematic audit trail—from raw free-text query to study inclusion and effect estimation—facilitates regulatory compliance and reproducibility critical to biomedical research.

Future Directions

Potential expansions include (1) enrichment of the eligibility structuring framework to handle more complex logical structures and temporal criteria, (2) integration of publication databases to address registry incompleteness and outcome reporting biases, (3) rule calibration by iterated domain expert feedback and sensitivity analysis, and (4) extension of eligibility weighting methods to network meta-analysis, longitudinal outcomes, and living systematic reviews. Generalization to other healthcare domains and regulatory contexts is straightforward due to the modular, schema-driven architecture.

Conclusion

EligMeta presents a hybrid agentic architecture for clinical evidence synthesis that bridges the strengths of LLM-facilitated natural language understanding with deterministic, transparent computational modules. By making eligibility criteria a first-class citizen in trial selection and statistical pooling, EligMeta enables synthesis of evidence that is both clinically interpretable and population-specific. This framework transforms traditional meta-analysis from a purely statistical exercise into a transparent, auditable, and customizable process—setting a reference standard for evidence synthesis in the era of AI-augmented clinical research.