Octozi: AI-Assisted Clinical Data Cleaning
- Octozi is an AI-assisted clinical data cleaning platform that integrates LLMs and domain-specific heuristics to detect and resolve trial data discrepancies.
- It achieves a 6.03-fold increase in cleaning throughput, reduces errors from 54.67% to 8.48%, and minimizes false positives by 15.48-fold in oncology adverse event reviews.
- The system employs a human-in-the-loop review model with full audit trails to ensure regulatory compliance and standardized clinical data evaluation.
Searching arXiv for the specified paper and closely related entries to ground the article. Octozi is an AI-assisted clinical data cleaning platform for pharmaceutical clinical trials that combines LLMs with domain-specific heuristics to support clinical reviewers in detecting, classifying, and resolving discrepancies in trial data. Its reported use case is oncology-focused adverse event review, where data must be synthesized across adverse events, concomitant medications, medical history, procedures, vital signs, labs, and dosing. In a controlled experimental study, Octozi increased data-cleaning throughput by 6.03-fold, reduced cleaning errors from 54.67% to 8.48%, and reduced false positive queries by 15.48-fold, while maintaining human oversight and an audit trail suitable for regulated environments (Purri et al., 7 Aug 2025).
1. Clinical role and problem setting
Octozi is situated in the clinical data review workflow that follows data capture in Electronic Data Capture systems through case report forms. In the workflow described for modern trials, clinical data managers write edit checks to catch basic inconsistencies such as missing values and invalid ranges, but medical reviewers still manually read adverse event narratives, labs, vital signs, dosing records, concomitant medications, and related listings to find clinically meaningful discrepancies. When an issue is detected, a query is sent to the site requesting clarification or correction (Purri et al., 7 Aug 2025).
The platform addresses a workflow characterized as slow and labor-intensive, highly variable from reviewer to reviewer, poorly scalable to large multi-source datasets, and prone both to false positives and to missed cross-domain patterns. The operational consequence is delayed database lock, which in turn delays regulatory submissions and market access. The underlying study states that each day of delay is estimated to cost on the order of $500,000 across a product’s lifecycle. Within that framing, Octozi is presented not merely as a review aid but as an intervention at a recognized bottleneck in clinical trial operations (Purri et al., 7 Aug 2025).
Its target users are clinical data managers, medical monitors, and clinical scientists. The central design premise is not full automation of clinical judgment, but assistance: the system prioritizes the most important discrepancies, suggests clinically coherent rationales and classifications, and reduces erroneous or unnecessary queries while leaving reviewers in control.
2. Architecture, data model, and discrepancy taxonomy
The described architecture has three integrated layers. The first is data ingestion and unification. Octozi ingests clinical trial data from Electronic Data Capture and associated case report forms, integrates oncology-relevant domains including adverse events, concomitant medications, medical history, procedures, vital signs, labs, and dosing, and constructs a synthetic but medically coherent patient-centric view in which each patient’s timeline and cross-domain relationships can be analyzed (Purri et al., 7 Aug 2025).
The second layer is a hybrid discrepancy detection engine. This engine combines proprietary LLMs derived from Llama 4, fine-tuned specifically for oncology clinical trial data cleaning, with heuristic algorithms and clinical rules codified with expert reviewers. The heuristics cover temporal alignment between adverse event onset or offset and concomitant medication administration, consistency between adverse event severity and associated labs or vitals, plausibility of dosing changes in relation to adverse events, and causality assessments relating adverse events to study drug. The LLM and heuristics jointly determine whether a data point is discrepant or clean and classify the discrepancy into clinically meaningful categories (Purri et al., 7 Aug 2025).
The third layer is a human-in-the-loop review interface. Suspected discrepancies are presented in context, including the relevant adverse event, labs, medications, and dosing, together with a predicted discrepancy category and justification. Reviewers can accept, modify, or reject suggestions, and the system maintains an audit trail suitable for regulated environments.
For the controlled study, the source was a phase III oncology trial with Electronic Data Capture data across 51 separate case report form datasets for more than 150 patients. The evaluation focused on 8 case report forms directly relevant to adverse event assessment and documentation. To preserve anonymity while maintaining realism, the authors built libraries of all unique adverse events, concomitant medications, procedures, and medical history entries; created synthetic patients by sampling from these libraries with weights proportional to original frequencies; and had expert annotators refine the synthetic patients for clinical coherence. Discrepancies were then introduced into 10% of data points using stratified randomization (Purri et al., 7 Aug 2025).
The planted discrepancies were organized into six primary categories: inappropriate concomitant medication for the adverse event; misaligned timing between concomitant medication and adverse event; incorrect adverse event severity given the event description; mismatched dosing change relative to the adverse event; incorrect causality assessment; and adverse event lacking supporting data. These categories map directly to the discrepancy types the system is trained to recognize and classify.
On the annotated synthetic dataset, the hybrid classifier is reported to achieve 83.6% accuracy, 97.5% recall, and 77.2% precision. This metric profile indicates tuning toward high recall, with the heuristics and refinement mechanisms intended to increase precision and thereby minimize unnecessary queries (Purri et al., 7 Aug 2025).
3. Experimental design and quantitative results
The comparative evaluation used a within-subjects controlled design with 10 clinical research professionals who had at least 2 years of experience in clinical data review, proficiency in adverse event adjudication, and familiarity with oncology clinical data cleaning. Each participant completed one baseline session using traditional spreadsheet review and one AI-assisted session using Octozi. The order is described as baseline first and then AI-assisted, with training before each phase. In both conditions, the review window was 30 minutes (Purri et al., 7 Aug 2025).
In the baseline condition, participants received a 10-minute briefing on dataset structure and discrepancy types and then used their own spreadsheet environment. In the AI-assisted condition, they received an interactive tutorial, a live demonstration, and hands-on practice with a training dataset containing known discrepancies before completing a matched-complexity review session using the full platform. Post-task assessment included NASA-TLX and SUS (Purri et al., 7 Aug 2025).
The principal quantitative outcomes are summarized below.
| Measure | Traditional | Octozi |
|---|---|---|
| Correctly cleaned data points in 30 minutes | Mean 3.4 | Median 20.5 |
| Overall accuracy | 45.3% | 91.5% |
| Precision | 44.6% | 93.2% |
| Recall | 85.2% | 88.9% |
| F1 score | 58.5 | 91.0 |
| Cleaning error rate | 54.67% | 8.48% |
| False positives among classifications | 48.0% | 3.1% |
The throughput improvement was 6.03-fold, with a per-participant improvement range of 3.7-fold to 15-fold and no participant worsening. Throughput was defined operationally as the number of data points correctly cleaned within the fixed 30-minute session. The traditional condition yielded a mean of 3.4 correctly cleaned data points, with a range of 1 to 6, whereas Octozi yielded a median of 20.5, with a range of 7 to 35 (Purri et al., 7 Aug 2025).
The controlled task’s confusion-matrix analysis reported 36 false positives out of 75 classifications in the manual condition, compared with 7 false positives out of 224 classifications in the Octozi condition. On that basis, false positive query generation was reduced by 15.48-fold. The study also reports very large effect sizes, including Cohen’s for accuracy improvements and Cohen’s for throughput, with paired two-sided -tests for primary measures reported as . A post-hoc power analysis reported observed for throughput, based on improvement of 17.1 records per session and standard deviation 8.16, corresponding to greater than 99.9% power (Purri et al., 7 Aug 2025).
Usability and workload findings are reported in parallel. SUS scores were 86.3 for Octozi and 56 for traditional spreadsheets. NASA-TLX indicated significant reductions in perceived mental demand, time pressure, effort, and frustration. Qualitative feedback emphasized prioritized findings, reduced cognitive burden from not repeatedly cross-referencing multiple listings, and exposure to discrepancy types some reviewers had not previously recognized (Purri et al., 7 Aug 2025).
4. Human-AI collaboration and workflow restructuring
Octozi is explicitly framed as a human-in-the-loop system rather than an autonomous query generator. Operationally, reviewers log into a unified interface in which patient data from multiple domains are integrated. The platform surfaces a queue of candidate discrepancies with discrepancy type, patient and visit context, relevant supporting data, and a machine-generated rationale. The reviewer then accepts the AI classification, modifies it, or rejects it if the data are judged clinically acceptable. Each action is recorded in a full audit trail, including what was flagged automatically, what decisions were made, and timestamped logs suitable for inspection (Purri et al., 7 Aug 2025).
The study emphasizes that the performance gains were observed for every participant and were consistent across experience levels. A plausible implication is that the combination of LLM reasoning, expert-encoded heuristics, and contextualized presentation standardizes lower-level checks that are otherwise applied inconsistently in manual review. The reported result that the lowest performer on Octozi outperformed the best spreadsheet user is consistent with that interpretation, as is the observation that manual baseline performance was both slower and more varied (Purri et al., 7 Aug 2025).
The system’s guardrails are described implicitly rather than as a standalone safety module. They include advisory rather than self-executing AI output, visible case-level evidence from the underlying clinical data, explicit heuristic logic alongside model inference, and quantified benchmark behavior through precision, recall, and confusion-matrix analysis. Within the paper’s framing, these design choices align Octozi with expectations for clinical tools that influence safety-critical judgments.
5. Regulatory position, safety context, and compliance
Octozi is presented within a regulatory context defined by Good Clinical Practice and by FDA and EMA expectations for data quality. The relevant regulatory themes are transparent and auditable processes, clear accountability for clinical decisions, and validation of systems that influence safety-critical judgments (Purri et al., 7 Aug 2025).
The platform’s alignment with those expectations is argued primarily through auditability, traceability, and human accountability. Each AI-proposed discrepancy and the reviewer’s final decision can be logged, allowing the data-cleaning process to be reconstructed for inspection readiness. The controlled experimental study is itself part of the validation narrative, because it documents improved accuracy and reduced errors. The paper also states that metrics such as sensitivity, specificity, precision, and F1 provide regulators and quality-assurance groups with an evidence-based view of system behavior. AI is positioned as an assistant, with human reviewers retaining responsibility for final judgments; the paper characterizes this as “augmented intelligence,” consistent with current regulatory guidance on AI in GxP settings (Purri et al., 7 Aug 2025).
At the same time, the deployment claims are qualified. The evidence comes from a synthetic dataset derived from a single phase III oncology trial rather than from multiple live production trials. Regulatory and quality-assurance teams are said to require study- and system-specific validation, documentation of model training and version control, and evidence that system changes do not compromise validated behavior. Accordingly, Octozi is presented as a framework and proof of concept for integrating AI into safety-critical workflows rather than as a fully certified GxP solution out of the box (Purri et al., 7 Aug 2025).
6. Limitations, generalizability, and distinction from similarly named systems
The reported evidence has several explicit limitations. The dataset scope is confined to a single therapeutic area, oncology, and to adverse event-related discrepancies rather than all clinical data domains. The source material derives from a single trial. The synthetic patients were clinically refined, but synthetic data may not capture all idiosyncrasies of real-world data entry errors, site behaviors, or Electronic Data Capture design quirks. The participant sample is , although the within-subjects design and reported effect sizes mitigate some statistical concerns (Purri et al., 7 Aug 2025).
Potential failure modes are also outlined. The system may identify that something is wrong while misclassifying the discrepancy type; some clean data points are still flagged, even though false positives drop to 3.1%; and some discrepancies remain undetected, given reported recall of about 88.9% in the AI-assisted condition. The text identifies ambiguous or incomplete source data, domain drift across different treatment modalities or grading scales, and protocol-specific nuances not captured in generic heuristics as plausible triggers for failure (Purri et al., 7 Aug 2025).
The authors regard the underlying pattern—LLMs plus clinical heuristics plus a unified data pipeline plus a human-in-the-loop interface—as generalizable to other trials, indications, sponsors, and Electronic Data Capture systems, but only with additional training and tuning. They specifically identify alignment with CDISC, including CDASH and SDTM, and incorporation of additional domain-specific rules as relevant for more standardized cross-trial deployment. This suggests a broader architectural template, while leaving robust external validity to future validation studies (Purri et al., 7 Aug 2025).
A likely source of confusion is the similarity of names across unrelated systems. Octozi is distinct from “Octo,” an open-source transformer-based generalist robot policy for manipulation (Team et al., 2024), and from “Octopi-1.5,” a visual-tactile-LLM centered on touch, vision, and language interaction (Yu et al., 14 Jul 2025). Those systems operate in robotics and multimodal tactile reasoning, respectively, whereas Octozi is a clinical data cleaning platform for pharmaceutical trials.
