MedCritical: Self-Collaborative Medical SLM Framework

• MedCritical is a dual-stage framework combining teacher-guided chain-of-thought fine-tuning with self-collaborative direct preference optimization to enhance medical reasoning.
• It leverages structured reasoning templates and contrasts between correct and incorrect trajectories to reduce the cost and time of external supervision.
• On the CMExam benchmark, MedCritical achieves state-of-the-art accuracy of 70.62%, outperforming baseline models by significant margins.

MedCritical is a two-stage fine-tuning and self-collaborative optimization framework for medical small LLMs (SLMs), introduced to equip a 7B-parameter student model with large-model–style medical reasoning while reducing the cost, time, and efficiency burdens associated with repeated teacher-guided correction. Its design couples supervised fine-tuning on teacher-generated long chain-of-thought templates with a self-learning direct preference optimization stage in which the student model learns from contrasts between its own correct and incorrect reasoning trajectories. On the CMExam benchmark, MedCritical reports 70.62% total accuracy, exceeding Taiyi by 3.04% absolute and Huatuo-o1-7B by 10.12% absolute, and is presented as new state-of-the-art performance among 7B-class small models (Su et al., 27 Sep 2025).

1. Problem formulation and design objective

MedCritical is situated in the setting of medical reasoning tasks such as clinical diagnosis, treatment planning, and medical knowledge integration, where small LLMs often underperform relative to larger models such as GPT-4 and Deepseek. The framework is explicitly motivated by a limitation of recent knowledge distillation-based methods: teacher-guided error correction via an “LLM as judge” paradigm is described as challenging in terms of cost, time, and efficiency. MedCritical addresses this by using a large teacher model for initial guidance, then shifting the main corrective signal into self-collaborative training by the student itself (Su et al., 27 Sep 2025).

The framework’s central claim is not that the teacher can be removed entirely, but that the most expensive part of repeated external supervision can be reduced after the initial transfer stage. This produces a specific conception of medical SLM improvement: first induce a structured expert-style reasoning format, then optimize the model to prefer its own success trajectories over its failures. In that sense, MedCritical combines elements of distillation, chain-of-thought supervision, and preference learning, but its defining feature is the use of a fine-tuned student model “to play against itself.”

2. Stage 1: template-guided supervised fine-tuning

The first stage of MedCritical is supervised fine-tuning with long chain-of-thought templates. For each medical question $q$ in an original medical Q&A corpus $D=\{(q,a)\}$, a teacher prompt $P_{tea}$ elicits both a generic structured reasoning template and a fully detailed reasoning chain $c$. The template specifies a response format such as using > ... for step-by-step reasoning and <answer>...</answer> for the final answer. The resulting supervised dataset is

$D_{sft}=\{(q,a,c)\}.$

The student policy $\pi$, initialized from Qwen2.5-7B-Instruct, is then optimized to reproduce both the reasoning chain and the answer under a student prompt $P_{stu}$ that mirrors the teacher format. The objective is

$$L_{sft}(\pi)=E_{(q,a,c)\sim D_{sft}}\bigl[\log \pi(c,a\mid q,P_{stu})\bigr].$$

This stage has two distinct functions. First, it transfers substantive medical reasoning content from the teacher’s detailed chain of thought. Second, it transfers a reusable high-level reasoning schema, described in the paper as “thinking templates,” that constrains the student toward a more structured deductive style. The output of this stage is a refined policy $\pi_{ref}$, which becomes the reference model for the second stage (Su et al., 27 Sep 2025).

3. Stage 2: self-collaborative direct preference optimization

The second stage is model self-collaborative DPO. A held-out set $D_{test}$ is used to sample candidate answers and reasoning chains from $D=\{(q,a)\}$0 by running it $D=\{(q,a)\}$1 times per question. A teacher model, prompted for correctness judgment, partitions the candidate set into correct and incorrect subsets:

$D=\{(q,a)\}$2

These form the “cor-err” dataset

$D=\{(q,a)\}$3

The optimization target is a DPO loss that uses $D=\{(q,a)\}$4 as a reference policy and trains a new student policy $D=\{(q,a)\}$5 to increase preference for its own correct trajectories over its incorrect ones:

$D=\{(q,a)\}$6

This formulation is the core of MedCritical’s “self-collaborative correction.” Rather than relying on stepwise teacher intervention during training, the system uses the teacher primarily to generate initial templates and to judge sampled outputs as correct or incorrect; the actual preference optimization is driven by the student’s own contrasting trajectories. The framework characterizes this as a form of self-correction through adversarial iteration. A plausible implication is that MedCritical treats reasoning quality as an internal distributional preference problem rather than only a supervised imitation problem (Su et al., 27 Sep 2025).

4. Model configuration, data regime, and CMExam results

The teacher model in the reported experiments is QwQ-32B, used only for data generation and judgment. The student base model is Qwen2.5-7B-Instruct, a decoder-only model with byte-pair encoding, sequence length 2,048 tokens, and 16-bit mixed precision. The source benchmark is CMExam, described as originally containing 68 K multi-choice medical exam questions. After filtering out ill-posed, non-medical, or too-long items, 13,887 eligible samples remain, with 3,000 used for Stage 1 fine-tuning and Stage 2 self-optimization and 10,887 held out for final evaluation. Stage 1 uses learning rate $D=\{(q,a)\}$7, batch size 16, 3 epochs, AdamW, weight decay 0.01, and prompt length approximately 100 tokens; Stage 2 uses learning rate $D=\{(q,a)\}$8, batch size 8, 2 epochs, and $D=\{(q,a)\}$9, with early stopping on a held-out development split (Su et al., 27 Sep 2025).

Model	Base	Accuracy (%)
MedCritical	TemplateSFT	70.62
Taiyi	Qwen-7B	67.58
Huatuo-o1-7B	Qwen2.5-7B	60.50

The empirical result reported for MedCritical is 70.62% total accuracy on CMExam. Relative to the listed baselines, this corresponds to a +3.04% absolute gain over Taiyi and a +10.12% absolute gain over Huatuo-o1-7B. The ablation study further isolates the contribution of each stage: RawSFT reaches 58.47%, ThoughtSFT reaches 68.53%, SelfDPO with standard CoT reaches 69.50%, and the combined template-based SFT plus self-collaborative DPO configuration reaches 70.62%. Within the paper’s experimental frame, these ablations indicate that both long-CoT template supervision and self-collaborative preference optimization are additive rather than redundant (Su et al., 27 Sep 2025).

5. Position within the medical reasoning literature

MedCritical addresses a problem that is visible across several adjacent benchmarks. In specialized critical care medicine question answering, Meta-Llama 3.1 models were evaluated on 871 non-public CCM board-style questions, where Llama3.1:70B achieved 60% average accuracy and Llama3.1:8B approximately 30%, with substantial domain variation: Research/Ethics reached 68.4%, whereas Renal fell to 47.9% (Alwakeel et al., 16 Sep 2025). This establishes that even comparatively large models show uneven subspecialty expertise, providing context for why explicit medical reasoning enhancement remains necessary.

Its preference-learning and self-correction design also sits near other reflection-oriented methods. Med-REFL uses a tree-of-thought decomposition, fine-grained step and reflection scoring, and automatically generated DPO data; it reports roughly 33,000 preference pairs and an average accuracy increase from 63.22% to 67.33% on MedQA-USMLE across four 7B/8B models, an overall gain of +4.11% (Yang et al., 11 Jun 2025). The resemblance is methodological rather than identical: both frameworks use preference optimization and self-improvement signals, but Med-REFL emphasizes fine-grained reflection quality, whereas MedCritical emphasizes template extraction and self-collaborative correction between correct and incorrect student trajectories.

The broader evaluation landscape also shows that exam-style multiple-choice accuracy is only one dimension of medical reasoning. CareMedEval, built from 534 French biomedical critical-appraisal questions linked to 37 scientific articles, reports that even strong models remain limited on article-grounded reasoning: GPT-4.1 reaches Exact Match 0.49, F$P_{tea}$0 0.84, and LCA 0.68, and performance is especially weak on statistics and limitations; generating intermediate reasoning tokens substantially improves Exact Match for multiple models (Bonzi et al., 5 Nov 2025). Likewise, MedEinst demonstrates a different failure mode in diagnosis: on 5,383 paired counterfactual differential-diagnosis cases across 49 diseases, high baseline accuracy can coexist with severe Bias Trap Rates, and ECR-Agent on Qwen3-32B reports 69.49% baseline accuracy, 24.21% robust accuracy, and 33.75% BTR (Chen et al., 10 Jan 2026). Taken together, these results suggest that MedCritical addresses one important axis of the field—small-model medical exam reasoning—but does not by itself resolve article-grounded critical appraisal or counterfactual diagnostic robustness.

6. Limitations, extensions, and deployment relevance

MedCritical’s reported limitations are explicit. It depends on a powerful teacher LLM for both template extraction and correct-versus-incorrect judgments, so teacher biases or errors may propagate into the student. Its feedback signal is binary, which the paper notes may miss nuanced partial credit in multi-step reasoning. Its validation is confined to 7B models and a single medical exam dataset. These constraints define the present scope of the framework more narrowly than the term “medical reasoning” might imply (Su et al., 27 Sep 2025).

The proposed extensions are correspondingly concrete: adapting the method to larger SLMs in the 13 B–70 B range, expanding to broader medical corpora such as clinical notes and guidelines, enriching feedback granularity by identifying which reasoning step fails rather than using only whole-chain verdicts, automating template adaptation to sub-domains such as radiology versus pharmacology, and incorporating multi-agent collaboration so that multiple SLMs critique one another without a centralized LLM judge. These proposed directions indicate that MedCritical is currently a training framework for benchmarked medical reasoning, not yet a complete clinical decision-support stack.

A plausible implication is that any movement from exam benchmarks toward deployment-oriented critical care use would require stronger external validation substrates. METRE is relevant in this regard because it extracts harmonized static, vital, and intervention tables from MIMIC-IV and eICU, supports direct cross-database validation, and reports cross-database AUC changes that can be as small as +0.019 or -0.015 on some tasks (Liao et al., 2023). Such infrastructure does not implement MedCritical itself, but it exemplifies the kind of multicenter validation environment that would be needed if MedCritical-style reasoning models were extended from question answering to operational clinical modeling.