MedFact: Factuality in Medical AI

• MedFact is a term used for various systems and benchmarks focusing on enhancing factual reliability in medical AI by reducing hallucinations and ensuring clinician-grade accuracy.
• MedFact-R1 employs a two-stage training framework that first uses pseudo-label supervised fine-tuning to ground models in external expertise and then applies GRPO reinforcement learning to refine reasoning.
• The framework demonstrates up to a 22.5% absolute improvement in factual accuracy on benchmarks, highlighting its significance in improving diagnosis reliability in medical vision-language tasks.

MedFact is a name used in recent medical-AI literature for several distinct systems and benchmarks concerned with factual reliability. In the usage most directly associated with medical vision-language reasoning, MedFact-R1 is a two-stage training framework for medical vision-LLMs that integrates external knowledge grounding through pseudo-label supervised fine-tuning and reinforcement learning through Group Relative Policy Optimization, with the stated aim of improving factual medical reasoning, reducing hallucinations, and producing factually consistent, clinically reliable, and self-justifying answers on medical question answering tasks (Li et al., 18 Sep 2025).

1. Scope and nomenclature

The name MedFact is not attached to a single research object. Recent papers use closely related names for different technical agendas, ranging from factual medical reasoning and benchmark construction to EHR representation learning.

Usage	Function	Reference
MedFact-R1	Two-stage framework for factual medical reasoning in medical vision-LLMs	(Li et al., 18 Sep 2025)
MedFact	Benchmarking the fact-checking capabilities of LLMs on Chinese medical texts	(He et al., 15 Sep 2025)
MedFact	Evidence-based Chinese medical fact-checking dataset of LLM-generated medical content	(Chen et al., 22 Sep 2025)
MedFACT	Modeling medical feature correlations in patient health representation learning via feature clustering	(Ma et al., 2022)

Within this set of usages, MedFact-R1 is the formulation that treats factuality as a training objective for multimodal reasoning. By contrast, the two Chinese MedFact papers define benchmark and dataset resources for medical fact-checking, and the earlier MedFACT paper addresses patient health representation learning rather than hallucination or claim verification. The shared naming nevertheless reflects a recurrent concern across the literature: medical AI systems must be evaluated and optimized not only for fluency or task accuracy, but for factual correctness, grounding, and clinically meaningful consistency.

2. Problem setting: factual unreliability in medical AI

MedFact-R1 targets a specific failure mode of medical vision-LLMs: they may produce hallucinated findings, rely on spurious correlations, give inconsistent reasoning, or miss subtle but clinically important signs. The paper argues that this is especially problematic in medicine because even small factual mistakes can lead to wrong diagnoses, while reinforcement learning alone can be unstable without a good starting policy and medical data are scarce and expensive to annotate (Li et al., 18 Sep 2025).

This problem formulation aligns with a broader shift in medical NLP and multimodal learning. Recent work on free-form answer evaluation argues that standard decompose-then-verify factuality pipelines are a poor fit for medical text because medical answers are often condition-dependent, conversational, hypothetical, and structurally diverse, making valid claim decomposition difficult (Huang et al., 24 May 2025). Evidence-based medical fact-checking datasets likewise frame retrieval and verification as central bottlenecks, not auxiliary tasks; HealthFC, for example, grounds claim verification in systematic reviews and clinical trials rather than generic web evidence (Vladika et al., 2023). In patient-specific settings, VeriFact reformulates factuality as support against the electronic health record, showing that medical verification is often chart-grounded rather than world-knowledge-grounded (Chung et al., 28 Jan 2025).

A plausible implication is that “medical factuality” is not a single metric. In current literature it includes at least four separable properties: correctness of the final answer, grounding in medically relevant concepts, coherence between reasoning and conclusion, and recoverability of evidentiary support from an external source such as retrieved knowledge, radiology reports, or the EHR.

3. MedFact-R1 as a two-stage training architecture

MedFact-R1 is organized as a two-stage training framework. The first stage is a cold-start supervised fine-tuning step based on pseudo-labels; the second stage is GRPO reinforcement learning over sampled response groups (Li et al., 18 Sep 2025).

In the first stage, pseudo-label supervised fine-tuning injects external factual expertise into the vision-LLM. The paper states that pseudo-labels are produced via calibrated retrieval to mitigate factuality risks and preference alignment to harmonize external priors with internal knowledge. In the reported experiments, the pseudo-label source is the RULE system’s outputs, treated as factual pseudo-diagnoses. The SFT objective is standard next-token prediction under maximum likelihood:

$$\mathcal{L}_{\text{SFT}} = - \sum_{t=1}^{T} \log p_\theta(y_t \mid y_{<t}, I, Q),$$

where $I$ is the image, $Q$ is the question, and $y_t$ is the target token. This stage is described as grounding the model in external medical expertise, teaching it to imitate reliable factual diagnoses, improving recall and disease identification, and providing a strong cold start for reinforcement learning (Li et al., 18 Sep 2025).

The second stage applies Group Relative Policy Optimization. For each input, the model samples a group of responses; their rewards are compared relatively, and policy updates encourage responses with higher advantage. The paper characterizes this as group-based Monte Carlo advantage estimation plus policy gradient optimization, with a GRPO-style objective that includes reward maximization and a KL term against a reference policy. The essential design point is that MedFact-R1 does not rely purely on supervised targets after initialization: it uses relative rewards over multiple completions to refine reasoning behavior (Li et al., 18 Sep 2025).

This architecture encodes a strong asymmetry between knowledge acquisition and policy improvement. The ablation results are presented as evidence that knowledge grounding must precede policy optimization: GRPO without pseudo-label SFT is reported to be unstable and to underperform the full pipeline.

4. Reward design and self-consistent factual reasoning

The GRPO phase uses a normalized sum of four factual reward components: Accuracy Reward, Format Reward, Fact Reward, and Consistency Reward (Li et al., 18 Sep 2025).

The Accuracy Reward is the direct supervision signal. The predicted binary answer is compared with the pseudo-label; the reward is 1.0 if the answer exactly matches the pseudo-label and 0 otherwise. The Format Reward enforces a structured output in which reasoning appears inside > ... and the final answer appears inside <answer> ... </answer>. The reward is 1 if all four tags appear exactly once and no extra content appears outside the designated regions, and 0 otherwise. The paper presents this as a way to stabilize generation and make reasoning inspectable.

The Fact Reward encourages inclusion of clinically relevant concepts. For each question, a small set of domain-specific concepts is extracted using GPT-4; each concept correctly reflected in the answer contributes 0.2 points, counted once. The Consistency Reward checks whether the reasoning supports the final answer. Here too the paper uses GPT-4, guided by curated examples of medically sound and unsound outputs; the reward is 1 if the answer is logically supported by the reasoning and a penalty of -0.5 otherwise (Li et al., 18 Sep 2025).

The four rewards target different failure modes. Accuracy rewards exact diagnostic correctness; format rewards structured reasoning; fact reward promotes grounding in medical terminology; consistency reward enforces alignment between explanation and conclusion. The ablation study reports that Accuracy + Format already produces a strong improvement, that adding Fact reward yields further gains, and that adding Consistency reward gives the final best results. This suggests that MedFact-R1 treats factual reasoning as a composite property rather than as answer matching alone.

5. Benchmarks, baselines, and reported performance

MedFact-R1 is evaluated on three public medical QA benchmarks: IU-Xray, Harvard-FairVLMed, and MIMIC-CXR. The reported metrics are Accuracy, Precision, Recall, and F1. Baselines include general and medical vision-LLMs such as Qwen2.5-VL-3B, LLaVA-Med v1.5 (7B), MedDr, RULE, MMed-RAG, and FactMM-RAG, as well as decoding or post-hoc methods on LLaVA-Med such as Greedy Search, Beam Search, DoLa, OPERA, and VCD (Li et al., 18 Sep 2025).

Benchmark	Accuracy	F1
IU-Xray	97.63	95.55
Harvard-FairVLMed	96.54	98.03
MIMIC-CXR	95.36	96.40

The paper reports up to 22.5% absolute improvement in factual accuracy over previous methods and states that the method reaches scores exceeding 95% on all evaluation metrics across benchmarks (Li et al., 18 Sep 2025). It also reports a pseudo-label choice study in which pseudo-labels generated from RULE perform strongly; human labels are slightly better, but pseudo-label-based SFT remains very competitive. The central ablation claims are that the base Qwen2.5-VL-3B performs poorly, pseudo-label SFT alone already yields a large boost, and the full two-stage setup performs best.

The empirical message is not only that MedFact-R1 improves benchmark scores, but that its gains are attributed to a specific division of labor between the two stages. Stage 1 supplies reliable pseudo-expertise; Stage 2 performs reward-guided self-refinement.

6. MedFact-style factuality beyond MedFact-R1

A broader ecosystem has developed around problems that are closely aligned with MedFact-R1, even when the methods target evaluation rather than training. AtomiMed treats medical report evaluation as hierarchical fact-checking over Atomic Clinical Facts, separating disease-level entities from attribute-level descriptors such as location, size, morphology, severity, quantity, and temporal change; its Agentic Cross-Verification loop is explicitly presented as a multi-radiologist peer-review simulation, and it reports higher correlation with radiologist judgment than traditional metrics on ReXVal, ReFiSco-v0, and RadEvalX (Wang et al., 30 Jun 2026). FActBench similarly decomposes generated medical text into atomic facts and finds that Unanimous Voting between Chain-of-Thought prompting and NLI correlates best with domain expert evaluation in grounded generation settings (Afzal et al., 2 Sep 2025).

Other work has specialized the same factuality agenda to different modalities and failure modes. In radiology report generation, FactMM-RAG mines factually informed report pairs with RadGraph and reports improvements of +6.5% absolute on F1CheXbert and +2.0% absolute on F1RadGraph on MIMIC-CXR (Sun et al., 2024). FactReranker reframes faithful summarization as selection of the candidate with the highest estimated factual consistency score, reaching RadGraph 53.17 and $F_1$CheXbert 76.36 on MIMIC-CXR (Xie et al., 2023). Fact-Flow inserts an explicit multi-label finding representation between image and report generation, using LLM-bootstrapped finding labels to guide the MLLM and reporting substantial factual gains on tuberculosis and ophthalmology datasets (Yang et al., 28 Feb 2026). At the detection end of the spectrum, MedFabric and ETHER focus on fabrication as a subtype of medical hallucination under high structural similarity, reporting that ETHER outperforms the state of the art on word-level fabrication benchmarks by more than 15% overall (Kwok et al., 5 May 2026).

Two additional lines of work broaden the meaning of MedFact toward benchmark design. One MedFact benchmark evaluates Chinese medical text fact-checking through veracity classification and error localization, and finds that models are generally better at deciding whether a text contains an error than at identifying the exact erroneous span, while also documenting an over-criticism failure mode (He et al., 15 Sep 2025). Another MedFact dataset targets evidence-based verification of LLM-generated medical content in Chinese with five-way labels—supported, partially supported, refuted, uncertain, not applicable—and shows that task-specific fine-tuning can outperform larger untuned models on macro F1 (Chen et al., 22 Sep 2025).

7. Limitations and open directions

MedFact-R1 states several remaining challenges: scaling to complex real-world clinical scenarios, handling rare or ambiguous cases robustly, improving safety and fairness, and extending reward design toward richer rewards and video reasoning (Li et al., 18 Sep 2025). These limitations are consistent with the broader literature.

Related evaluation work suggests that omission remains a major unresolved problem. MedFactEval reports that in discharge summary generation, contradiction rates are lower than omission errors and that omissions are the main failure mode, even when factuality is assessed against clinician-defined key facts and an LLM Jury (Grolleau et al., 7 Sep 2025). MedScore further argues that decompose-then-verify pipelines remain precision-oriented and do not adequately measure omission or recall in patient-facing medical answers (Huang et al., 24 May 2025). A plausible implication is that future MedFact-style systems will need to couple factual precision with explicit coverage modeling, especially in settings where missing clinically salient content can be as harmful as fabricated content.

The literature therefore points toward a convergent research program rather than a finished solution. In that program, factual medical reasoning is optimized through external grounding, structured rewards, atomic or hierarchical fact decomposition, evidence-aware verification, and modality-specific evaluation. MedFact-R1 occupies the training side of that program: it operationalizes factuality as a property that can be learned through pseudo-label cold start and reward-guided policy refinement, rather than treated only as an after-the-fact evaluation criterion.