MedNuggetizer: Query-Driven Evidence Extraction

• MedNuggetizer is a tool that employs large language models and semantic clustering to extract concise, high-confidence clinical recommendations from heterogeneous medical documents.
• The system uses repeated LLM extractions combined with confidence scoring to mitigate stochasticity and ensure the reliability of extracted information nuggets.
• Its workflow integrates OCR processing, sentence embeddings, and cross-document clustering to offer a reproducible and transparent synthesis of medical evidence.

MedNuggetizer is a tool for high-confidence, query-driven extraction and clustering of information nuggets from medical documents. It is engineered to support clinicians and researchers in navigating, synthesizing, and validating medical evidence across heterogeneous sources such as clinical guidelines, systematic reviews, and randomized controlled trials (RCTs). Leveraging LLMs, sentence embeddings, and scalable clustering, MedNuggetizer offers a reproducible and transparent workflow for harvesting concise, contextually coherent recommendations. Its design addresses the challenges posed by LLM stochasticity and the sheer length and diversity of medical reference sources (Donabauer et al., 17 Dec 2025).

1. Problem Scope, Use Cases, and Design Objectives

MedNuggetizer is motivated by the clinical need to efficiently distill precise, evidence-based recommendations from extensive and often heterogeneous medical documentation. The tool directly targets scenarios in which clinicians must compare recommendations (e.g., antibiotic prophylaxis regimens) sourced from guidelines and primary literature. Its core design imperatives include:

• Transparent Query-Driven Extraction: Users pose free-text queries, prompting the system to extract short, self-contained statements (“information nuggets”) directly relevant to the query.
• Reliability of Outputs: By performing repeated extractions and assigning confidence scores based on nugget recurrence across runs, the system mitigates the inherent non-determinism of LLM outputs.
• Evidence Consolidation: Clusters of semantically similar nuggets, both within and across documents, yield concise summaries that aid comparison and synthesis.
• Clinical Usability: Presented via an accessible web interface, results are organized to enable rapid evidence exploration, reproducibility, and auditability for clinical end-users.

The methodology is explicitly constructed to provide high-confidence, query-focused evidence extraction for scenarios such as antibiotic prophylaxis before prostate biopsy, using both clinical guidelines and contemporary PubMed studies as data sources (Donabauer et al., 17 Dec 2025).

2. System Workflow and Core Algorithms

The MedNuggetizer pipeline comprises the following sequenced modules:

2.1 User Query and Preprocessing

Uploaded PDF documents are OCR-processed, split into coherent passages or text blocks, and indexed. Users specify their information need via a free-text query.

2.2 LLM Extraction with Repetition and Prompt Diversity

The backend employs Google Gemini 2.5 Flash, selected for context scalability (up to 1,000 pages) and cost efficiency. For each document, the prompt window includes both passage text and the user query. The main extraction prompt requires the LLM to "Extract all short, self-contained statements (information nuggets) from the following text that directly answer the query: '<user query>'." Extraction temperature is set to 0.7. Top-k sampling is used to induce response diversity, with default parameters n=5 extraction loops and conf=0.8 as the minimum confidence threshold.

2.3 Confidence-Based Filtering

Each nugget $p$ is tracked across $n$ extraction runs. Let $c_p$ denote the number of runs where $p$ is extracted. The confidence score is:

$\text{confidence}(p,n) = \frac{c_p}{n}$

A nugget survives if $\text{confidence}(p,n) \geq \text{conf}$; equivalently, only if $$| \{ \text{runs containing } p \} | \geq \lceil n \times \text{conf} \rceil$$.

2.4 Semantic Clustering

Each nugget is embedded using a pre-trained sentence-transformer encoder (e.g., all-MiniLM-L6, $d \approx 384$), then clustered via the BERTopic methodology—cosine similarity, class-based TF-IDF, and HDBSCAN hierarchical density clustering. Only clusters with size $\geq n \times \text{conf}$ are retained.

2.5 Summarization

For each high-confidence within-document cluster, the LLM is prompted: "Write a single concise statement that captures the common content of these cluster members." The result is a cluster-level "unified nugget."

2.6 Cross-Document Evidence Aggregation

Unified nuggets from all input documents are clustered using the same pipeline. Each cross-document cluster receives an auto-generated heading, encapsulating a piece of multi-source medical evidence.

3. Nugget Representation, Embedding, and Similarity Metrics

All candidate nuggets undergo standardized text normalization, including lowercasing, de-hyphenation, and boilerplate removal. Each nugget is encoded via a sentence-transformer embedding, yielding a fixed-dimensional vector. Clustering operates over this embedding augmented by class-based TF-IDF features derived from the cluster vocabulary. Cosine distance, defined as $1 - \text{sim}(u, v)$ for two embedding vectors $u$, $v$, is the principal similarity metric in both within- and cross-document clustering steps. This design aims to ensure semantically coherent grouping and minimization of redundancy within the evidence synthesis workflow (Donabauer et al., 17 Dec 2025).

4. Evaluation: Protocols, Datasets, and Empirical Results

4.1 Datasets and Clinical Queries

MedNuggetizer's performance was assessed on a multi-source corpus comprising four major urological guidelines (EAU Prostate Cancer 2025, EAU Urology Infections 2025, AWMF S3 Prostate Cancer 2025, AWMF S3 Peri-interventional Antibiotic Prophylaxis 2024) and ten PubMed articles (six systematic reviews, four recent RCTs, dated Oct 2024–Sep 2025). Five clinically relevant queries on antibiotic prophylaxis were selected by urologist domain experts.

4.2 Expert Annotation and Metrics

With default extraction parameters ($n=5$, $\text{conf}=0.8$), outputs were evaluated for cluster coherence and nugget relevance:

• Clusters $(N_C = 155)$: Expert-rated on coherence (1–5 scale).
• Nuggets $(N_N = 406)$: Expert-rated on query relevance (1–5 scale).
• Reliability: Inter-annotator agreement measured via Cohen’s $\kappa$; results were $\kappa=0.81$ (nugget relevance), $\kappa=0.78$ (cluster coherence).

Table: Per-Query Evaluation Statistics

Query	Clusters ($n_C$)	$C_\text{mean}$	$C_\text{median}$	Nuggets ($n_N$)	$N_\text{mean}$	$N_\text{median}$
q0	28	4.00	4	66	4.00	4
q1	34	4.24	4	97	4.23	4
q2	44	4.77	5	103	4.64	5
q3	25	4.84	5	65	4.75	5
q4	24	4.75	5	75	4.72	5

All ratings on 1–5 Likert scale.

A plausible implication is that the high mean ratings ($\geq$4.0) signal both coherent clustering and relevance to clinical queries. Clusters were observed to decompose information into context, recommendation, and limitations—a desirable property for nuanced evidence synthesis.

5. Clinical Case Example: Antibiotic Prophylaxis for Prostate Biopsy

Application to the question of antibiotic prophylaxis before prostate biopsy generated high-confidence clusters corresponding to consensus and controversy points:

• “Single-dose fluoroquinolone is recommended for transrectal biopsy in patients without fluoroquinolone-resistant organisms.”
• “Targeted prophylaxis based on rectal swab cultures reduces post-biopsy infection rates compared to empirical prophylaxis.”
• “Transperineal biopsy without antibiotics shows similar cancer detection and lower infective complications.”

The tool highlighted both broad agreement (e.g., the utility of targeted prophylaxis) and areas of ongoing debate (choice between fluoroquinolone, fosfomycin, aminoglycoside regimens), providing clinically actionable synthesis across document types (Donabauer et al., 17 Dec 2025).

6. Limitations and Prospective Development

Identified limitations include extraction of undefined abbreviations (e.g., “TRUS”), missing context qualifiers in some nuggets (“high-risk patients” without specifying criteria), retention of method-focused text in clusters, and partial redundancy when semantically close recommendations diverge lexically.

Future work is planned to expand backend configurability (e.g., k-means, spectral clustering), incorporate alternative LLMs (OpenAI GPT-4, LLaMA 3), automate abbreviation expansion and context window management, and establish gold-standard corpus evaluations for precision and recall (Donabauer et al., 17 Dec 2025). MedNuggetizer thus represents a hybrid architecture that combines confidence-calibrated, LLM-powered extraction, semantic clustering, and LLM-driven summarization to advance high-confidence, reproducible medical evidence synthesis.