TrialCompass: Visual Analytics for Eligibility

• TrialCompass is a visual analytics system that refines clinical trial eligibility criteria by integrating raw EHR data, outcome metrics, and provenance tracking.
• It supports iterative, dual-mode workflows that combine knowledge-driven and outcome-driven exploration to balance enrollment, safety, and statistical power.
• The system integrates with methods like SECRET and target-trial frameworks to enhance trial retrieval and systematic refinement of eligibility criteria.

Searching arXiv for the primary TrialCompass paper and closely related papers on SECRET and target-trial integration. TrialCompass is a visual analytics system for enhancing the eligibility criteria design of clinical trials. It integrates a workflow that supports iterative exploration of the large space of eligibility criteria through knowledge-driven and outcome-driven approaches, and it incorporates history-tracking so clinicians can trace the evolution of adjustments and decisions while exploring eligibility criteria, outcome metrics, and detailed characteristics of original electronic health record (EHR) data. Using a real-world dataset, it was demonstrated for septic shock and sepsis-associated acute kidney injury, and it was presented as a system for systematic refinement of eligibility criteria rather than as a standalone statistical estimator or retrieval engine (Sheng et al., 16 Jul 2025).

1. Clinical role and problem setting

Eligibility criteria, as inclusion and exclusion rules, define exactly which patients may enter a trial. In TrialCompass, their importance is framed in terms of direct effects on sample size, the balance between treatment and control groups, and ultimately the statistical power, safety, and generalizability of the trial. The system is motivated by the observation that overly strict criteria can lead to insufficient enrollment, whereas overly loose criteria can increase the risk of adverse events or confounding (Sheng et al., 16 Jul 2025).

The design problem is presented as structurally difficult for three reasons. First, there is a combinatorial explosion of candidate criteria. The example given is that manually exploring every combination of thresholds such as age $<60, 65, 70$ or surgery history in the last $1, 3, 6$ months is impractical. Second, existing approaches often lack integration with raw EHR temporal data: they may report aggregate outcomes such as enrolled count versus hazard ratio while hiding time-series changes in lab values or organ-risk trajectories. Third, they provide no systematic history-tracking, so clinicians cannot easily retrace which criteria were tested, how outcomes evolved, or how insights from separate exploration paths should be merged (Sheng et al., 16 Jul 2025).

A common misconception is that eligibility design can be reduced to selecting a favorable aggregate endpoint. TrialCompass is explicitly organized against that view: the system combines eligibility criteria, outcome metrics, and detailed characteristics of original EHR data, including temporal displays of laboratory-derived risk trajectories. This suggests that its core contribution lies in coordinated exploration across multiple analytical objects rather than in optimizing a single scalar objective.

2. System architecture and iterative workflow

TrialCompass is organized as a three-layer architecture. The data layer is specified as raw EHR $\rightarrow$ cohort construction and preprocessing $\rightarrow$ feature extraction. The computation layer takes each criterion configuration $\theta$, builds treatment and control cohorts, computes propensity-score matching, computes outcome metrics, and organizes time-series data. The visualization layer then exposes three coordinated views—Criterion Specification, Criterion–Outcome Exploration, and Detailed Characteristic Exploration—together with a history-tracking widget (Sheng et al., 16 Jul 2025).

Its workflow has two complementary modes. In knowledge-driven exploration, a clinician adjusts sliders for known criteria, such as requiring mechanical ventilation, and immediately sees updated outcomes including $|C|$, hazard ratio, and organ-risk curves. In outcome-driven exploration, the clinician browses a precomputed scatterplot of candidate configurations, lassos clusters of points with desirable trade-offs such as high enrollment and low hazard ratio, and then drills down into the criteria underlying each cluster (Sheng et al., 16 Jul 2025).

The workflow is formalized as generation and evaluation over a set of candidate configurations,

$\Theta = \{\theta_1,\dots,\theta_N\}.$

For each $\theta_j$, TrialCompass performs cohort construction and matching, computes metrics and time series, and feeds the results into the outcome and detailed characteristic views. The clinician then returns to criterion adjustment, making the overall workflow explicitly iterative. This dual-mode organization mirrors real clinical workflows, where prior domain knowledge and outcome-oriented search are alternated rather than separated.

3. EHR-based cohort construction and quantitative metrics

TrialCompass denotes each patient record by $x \in X$ and defines scalar feature functions

$f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$

with examples including patient age and baseline creatinine. A criterion configuration $1, 3, 6$0 specifies lower and upper bounds $1, 3, 6$1 on a subset of features, yielding the cohort

$1, 3, 6$2

This formulation makes the relation between interface-level criterion editing and back-end cohort construction explicit (Sheng et al., 16 Jul 2025).

To control confounding, each patient has background covariates $1, 3, 6$3, and the propensity score is estimated via logistic regression,

$1, 3, 6$4

where $1, 3, 6$5 indicates treatment. TrialCompass then performs $1, 3, 6$6 nearest-neighbor matching within a caliper $1, 3, 6$7, often set to the median absolute deviation of $1, 3, 6$8 (Sheng et al., 16 Jul 2025). This does not by itself resolve all causal-identification issues, but it defines the comparison cohorts underlying the interactive outcome displays.

The system computes several outcome metrics. Cohort size is

$1, 3, 6$9

For diversity, using a categorical attribute with category fractions $\rightarrow$0, Shannon entropy is

$\rightarrow$1

For survival analysis, TrialCompass fits a Cox proportional-hazards model,

$\rightarrow$2

with hazard ratio

$\rightarrow$3

It also displays the Kaplan–Meier survival estimate,

$\rightarrow$4

where $\rightarrow$5 is the number of events at time $\rightarrow$6 among $\rightarrow$7 at risk (Sheng et al., 16 Jul 2025).

For organ-risk ratios, if $\rightarrow$8 and $\rightarrow$9 denote average daily kidney or liver risk levels in treatment and control, the overall risk ratio is

$\rightarrow$0

or the average of $\rightarrow$1. Sensitivity and specificity are also defined when a binary adverse-event detector with threshold $\rightarrow$2 is used: $\rightarrow$3 The significance of this metric set is not merely breadth. It enables simultaneous inspection of enrollment, treatment effect, diversity, and temporal organ-risk behavior, which is central to TrialCompass’s criterion-refinement logic (Sheng et al., 16 Jul 2025).

4. Visual analytics design and provenance tracking

The Criterion Specification View supports structured entry of inclusion and exclusion rules, including “AND/OR” logic and aggregation operators such as min, max, and count $\rightarrow$4. Any bound can be marked as adjustable and assigned discrete steps, which automatically generates the candidate set $\rightarrow$5. This is the mechanism by which the system operationalizes the exploration of many alternative eligibility definitions without requiring manual enumeration (Sheng et al., 16 Jul 2025).

The Criterion–Outcome Exploration View has three panels. Its criterion panel exposes one slider per bound, with tick marks colored by the count of satisfying configurations, and it can highlight two selected configurations in contrasting colors. Its outcome panel is a scatterplot in which any two metrics can be chosen as axes, and each $\rightarrow$6 becomes a point whose size can encode cohort size or obesity fraction; zoom, pan, and lasso selection are supported. Its exploration panel records stages as thumbnails in sequence, showing both criterion values and metric evolution over time (Sheng et al., 16 Jul 2025).

The Detailed Characteristic Exploration View has two modes: group average $\rightarrow$7 SD and individual configurations. It displays histograms for static distributions such as age and gender and line charts for serum creatinine and AST risk over time for treatment and control, with side-by-side comparison of two groups or two candidates. This corrects a limitation of tools that expose only aggregate endpoints, because it retains access to the detailed characteristics of original EHR data during criterion refinement (Sheng et al., 16 Jul 2025).

History-tracking is implemented through stage objects $\rightarrow$8 containing criteria settings $\rightarrow$9, outcome selections such as axis choices or lasso picks, and optional clinician annotations including notes or importance rating. The stages form an ordered list,

$\theta$0

where each $\theta$1 references a pointer to the precomputed results for $\theta$2. In each thumbnail, a small matrix of circle glyphs encodes numeric criterion bounds and a miniature line chart shows trajectories of five outcome metrics across stages. Clinicians can expand or collapse groups via stage controls (Sheng et al., 16 Jul 2025). This provenance structure is important because it externalizes reasoning stages rather than treating exploration as a sequence of ephemeral interface actions.

5. Demonstrated use in septic shock and sepsis-associated acute kidney injury

In the septic shock case study, TrialCompass was applied to a hydrocortisone trial identified as NCT01448109. The initial historical criteria included age $\theta$3 with no upper bound, which corresponded to $\theta$4 and $\theta$5 $\theta$6. Requiring mechanical ventilation dropped $\theta$7 to $\theta$8. The expert then added two axes for exploration: “No cardiac surgery in past $\theta$9 months” with $|C|$0, and “BMI \ge \theta)” to test an obesity paradox. The exploration found that omitting the ventilation requirement yielded $|C|$1 and kidney and liver ratios $|C|$2, so those candidates were rejected. Adding “no surgery for $|C|$3 months” stabilized $|C|$4 and reduced risk metrics. Outcome-driven lasso selection grouped candidates with $|C|$5; those candidates had higher minimum age $|C|$6 and included obese patients. In the temporal detail view, the older treatment cohort had a mid-term AST spike but recovered later. The final criteria achieved $|C|$7, $|C|$8, and $|C|$9, with $\Theta = \{\theta_1,\dots,\theta_N\}.$0 moderately reduced (Sheng et al., 16 Jul 2025).

In the sepsis-associated acute kidney injury case study, the initial criteria were AKI stage $\Theta = \{\theta_1,\dots,\theta_N\}.$1, age $\Theta = \{\theta_1,\dots,\theta_N\}.$2, SOFA $\Theta = \{\theta_1,\dots,\theta_N\}.$3, BMI $\Theta = \{\theta_1,\dots,\theta_N\}.$4, and no GCS bound, giving $\Theta = \{\theta_1,\dots,\theta_N\}.$5, $\Theta = \{\theta_1,\dots,\theta_N\}.$6 (significant), $\Theta = \{\theta_1,\dots,\theta_N\}.$7, and $\Theta = \{\theta_1,\dots,\theta_N\}.$8. Outcome-driven selection then lassoed four rim-regions balancing $\Theta = \{\theta_1,\dots,\theta_N\}.$9 in the range $\theta_j$0–$\theta_j$1 against $\theta_j$2 in the range $\theta_j$3–$\theta_j$4. Exploration history suggested that the AKI upper bound could be relaxed to $\theta_j$5 without large hazard-ratio change, that the SOFA upper bound could expand to $\theta_j$6, and that the GCS lower bound could be set to $\theta_j$7. A detailed comparison of two subgroups showed one with $\theta_j$8, $\theta_j$9, and rising liver risk, and another with $x \in X$0, $x \in X$1, and stable risk curves; the latter was chosen. Knowledge-driven final tweaks then included AKI stage $x \in X$2 while keeping $x \in X$3, and allowed higher BMI, resulting in final $x \in X$4, $x \in X$5, $x \in X$6, and $x \in X$7 (Sheng et al., 16 Jul 2025).

These case studies clarify the intended analytical semantics of TrialCompass. It is not limited to choosing a single “best” criterion set; rather, it supports tracing trade-offs among enrollment, hazard ratio, and organ-risk trajectories, while preserving a record of how those trade-offs were discovered and justified.

6. Integration with trial similarity search and target-trial methodology

A related line of work describes how TrialCompass can incorporate SECRET, a semi-supervised clinical trial document similarity search framework built from three modules: a summarization module that converts long clinical-trial protocols into compact question–answer representations, a local-contrastive learning module at the Q/A level, and a global-contrastive learning and retrieval module at the trial level (2505.10780). In the integration description, TrialCompass periodically ingests new or updated trial protocols, runs a Q/A summarization pipeline, computes and stores trial embeddings $x \in X$8, and then processes a user-provided trial design or partial specifications by generating Q/A, embedding to $x \in X$9, and performing vector-nearest-neighbor search over stored embeddings. It can return top-$f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$0 similar trials together with key Q/A overlaps to justify similarity. During patient screening, it can convert a patient EHR summary into Q/A form, retrieve top trials, and integrate retrieved trial eligibility Q/A against patient data to show explicit match(Yes/No) on each criterion. The same integration note states that the separation of summarization and encoding makes each component independently deployable, the embedding store can leverage approximate nearest-neighbor engines for sub-second retrieval, and all retrieval calls reduce to a single vector similarity operation plus metadata lookups (2505.10780).

SECRET’s reported retrieval gains provide the empirical rationale for this extension. It achieved $f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$1 versus a best baseline of $f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$2, corresponding to a $f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$3 relative gain, and $f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$4 versus $f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$5, corresponding to a $f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$6 relative gain. It also outperformed baselines in partial trial similarity search by approximately $f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$7 recall@2, and in zero-shot patient–trial matching with $f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$8 precision and $f_1(x), f_2(x), \dots, f_p(x) : X \to \mathbb{R},$9 recall over the best unsupervised baseline on TREC2021 (2505.10780). Within a TrialCompass setting, this suggests a complementary relation between interactive eligibility refinement and protocol-level retrieval of historical trials.

A separate methodological extension uses the target trial framework for combining information from multiple, diverse sources in TrialCompass (Ung et al., 23 May 2026). In that formulation, investigators define a super-population $1, 3, 6$00, an inclusion indicator $1, 3, 6$01, and source-specific sampling indicators $1, 3, 6$02, with sampling probabilities $1, 3, 6$03 or stratum probabilities $1, 3, 6$04. Eligibility is formalized as an eligibility set $1, 3, 6$05, treatment is represented by $1, 3, 6$06 or $1, 3, 6$07, and causal contrasts include

$1, 3, 6$08

The framework emphasizes explicit alignment of time zero, follow-up window $1, 3, 6$09, outcome definition, censoring, assignment mechanism, and the mapping of data elements across sources. It also highlights potentially irreconcilable misalignments in eligibility criteria, treatment assignment, and treatment receipt, and it recommends reporting structures including a target trial protocol, source-mapping table, DAG or SWIG, identification formula, estimation code, and sensitivity analyses (Ung et al., 23 May 2026).

This suggests an important boundary condition for interpreting TrialCompass. Interactive refinement of eligibility criteria, however detailed, is distinct from causal identification when evidence must be combined across trials and observational sources. The target-trial extension makes that distinction explicit by requiring a sampling model, formal estimands, and transparent documentation of assumptions and misalignments.

7. Limitations, open questions, and research prospects

TrialCompass is presented with several forward-looking research prospects. Provenance and cognitive support are one such prospect: clinicians reportedly benefited from defining their own reasoning stages, and future systems are proposed to support more flexible annotations, hierarchical histories, and collaborative sharing. A second prospect concerns hybrid reasoning patterns: the alternation between knowledge-driven and outcome-driven modes is said to mirror real clinical workflows, and extending this dual-mode paradigm may help in other multi-objective decisions such as resource allocation and precision public health. A third prospect is scalability: with thousands of candidate configurations, initial uniform sampling in the scatterplot followed by on-demand refinement is proposed as a way to maintain interactivity. Additional directions include incorporating cardiovascular risk, neurological outcomes, and patient-reported endpoints as new axes, and linking real-time trial data from wearables or simulating adaptive trial designs such as group-sequential adjustments (Sheng et al., 16 Jul 2025).

The related SECRET integration description adds a different set of limitations and open questions relevant to TrialCompass deployments (2505.10780). Q/A coverage is limited to six protocol sections, and adding “Description” or “Study Design” may improve recall at scale. Reliance on an external LLM for Q/A introduces variability and cost, so a smaller fine-tuned summarization model is proposed as a way to reduce expense. Labeled data scarcity outside Cochrane-derived pairs means that unsupervised positives based on dropping one Q/A may not always reflect realistic protocol modifications. Domain drift as new therapeutic areas emerge requires periodic re-evaluation of hard negatives and Q/A templates. Interpretability remains an open issue because end users may require clearer explanations of why two trials are similar beyond cosine similarity; attention-based highlighting of matching Q/A is identified as future work (2505.10780).

Taken together, these limitations indicate that TrialCompass should be understood as an extensible analytic environment for eligibility design, provenance capture, and, potentially, trial retrieval and source harmonization. Its research significance lies in combining EHR-driven cohort construction, multiple outcome metrics including $1, 3, 6$10, $1, 3, 6$11, diversity, and organ-risk ratios, temporal detail extraction, and a flexible visual interface with history-tracking for systematic refinement of eligibility criteria (Sheng et al., 16 Jul 2025).