Zhongjing: Chinese Medical LLM Framework

• Zhongjing is a comprehensive framework that integrates medical pre-training, supervised fine-tuning, and reinforcement learning from human feedback to excel in Chinese clinical dialogue.
• It leverages large-scale authentic doctor–patient dialogues, knowledge graphs, and rigorous data curation to ensure high factual reliability and adherence to clinical ethics.
• Experimental results demonstrate competitive performance against larger models, with marked improvements in dialogue professionalism, fluency, and safety in medical applications.

Zhongjing is a comprehensive framework for advancing the medical capabilities of LLMs in the Chinese clinical context by systematically integrating domain-specific pre-training, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF). Built upon a LLaMA-style 13B parameter backbone, Zhongjing defines a full-stack methodology that leverages large-scale authentic doctor–patient dialogues, rigorous knowledge-graph-based data curation, and clinical expert feedback to produce a conversational assistant that demonstrates high knowledge, proactive inquiry, and alignment with medical ethics and safety standards (Yang et al., 2023).

1. Model Foundation and Continuous Pre-training

Zhongjing utilizes Ziya-LLaMA-13B-v1, an autoregressive LLaMA-based model with 13 billion parameters, as its initialization. The continuous pre-training corpus totals approximately 1 GB, comprising:

• 20 MB of Chinese medical textbooks,
• 436 MB of ChatMed medical Q&A,
• 158 MB of in-domain CMtMedQA dialogue,
• Wikipedia-style medical entries (106 MB),
• Knowledge graphs (CMeKG: 28 MB; pediatrics: 5 MB; obstetrics: 7 MB; cardiology: 8 MB),
• Proprietary hospital sources (health records: 73 MB, clinical reports: 140 MB, EHRs: 105 MB).

The corpus is shuffled and mixed to ensure maximal coverage of domain content. Pre-training optimizes the standard causal language modeling loss: $$\mathcal{L}_\text{PT} = -\sum_{t=1}^T \log p_\theta(w_t \mid w_{<t})$$ Specialized medical knowledge is infused using a "KG–Instruction collaborative-filtering" procedure: entities extracted from the CMeKG knowledge graph are inserted as constraints within instruction-style prompts, and examples that fail consistency checks with CMeKG are filtered out. This strategy ensures high factual reliability and medical specificity (Yang et al., 2023).

2. Supervised Fine-Tuning with Mixed Dialogue

The supervised fine-tuning (SFT) phase builds on the pre-trained model using a mixture of:

• Single-turn medical QA from ChatMed,
• 70,000 de-identified multi-turn doctor–patient dialogues from CMtMedQA (~400,000 utterances),
• Medical NLP tasks re-cast into dialogue form (e.g., symptom extraction, report generation),
• General dialogue data to mitigate catastrophic forgetting.

A 7:1 single-turn to multi-turn ratio is maintained. SFT data preparation includes stylistic normalization of all doctor responses (via self-instruct using GPT-3.5-turbo) into a consistent and friendly "doctor voice," while preserving original patient utterances verbatim. Concise single-turn exchanges are occasionally expanded into multi-turn sequences to reflect authentic clinical probing. KG–Instruction filtering is applied to remove entity-inconsistent cases. All dialogues are fully de-identified, and the dataset encompasses 14 specialties with systematic inclusion of proactive inquiries (e.g., allergies, prior medication timing).

Each example is structured as a user-assistant exchange. Training utilizes LoRA with a rank of 16, a learning rate of $7 \times 10^{-5}$, 3 epochs, batch size 16, gradient accumulation 4, dropout 0.1, cosine learning rate decay, fp16, and ZeRO-2 optimization (Yang et al., 2023).

3. Reinforcement Learning from Human Feedback

To further align model outputs with clinical expert standards, Zhongjing applies a bespoke RLHF pipeline:

• 10,000 in-distribution and 10,000 out-of-distribution prompts are sampled.
• For each prompt, $K=4$ candidate responses are generated using the SFT model.
• Six clinical experts independently rank these responses using a three-dimensional rubric:
  1. Safety (accuracy, no harmful advice, ethics),
  2. Professionalism (comprehension, clarity, initiative),
  3. Fluency (coherence, consistency, warm tone).

• Inter-expert disagreements are resolved by an additional reviewer.

A reward model is instantiated from the backbone (pre-SFT), with a linear head to score response pairs $(x, y)$: $$L_\text{RM}(\theta) = -\frac{1}{\binom{K}{2}} \sum_{(x,y_h,y_l)\in D} \log\sigma\bigl(r_\theta(x,y_h)-r_\theta(x,y_l)\bigr)$$ where $\sigma$ is the sigmoid and $y_h \succ y_l$ indicates preference by human evaluation.

Policy optimization is performed with PPO, maximizing the reward-regularized clipped surrogate: $$L_\text{PPO}(\phi) = \mathbb{E}_t\Bigl[\min\bigl(\rho_t \hat{A}_t,\, \mathrm{clip}(\rho_t,1-\epsilon,1+\epsilon)\hat{A}_t\bigr)\Bigr] - \beta\,\mathrm{KL}\bigl[\pi_\phi(\cdot\mid x_t)\Vert\pi_{\text{ref}}(\cdot\mid x_t)\bigr]$$ where $\rho_t = \frac{\pi_\phi}{\pi_{\rm ref}}$ and $\hat{A}_t$ is the estimated advantage. PPO is run for 2 epochs, with LoRA rank 16, learning rate $7 \times 10^{-5}$0, batch size 8, and gradient accumulation 4.

4. CMtMedQA: Multi-Turn Clinical Dialogue Dataset

CMtMedQA is a large-scale dataset comprising 70,000 real-world, de-identified multi-turn doctor–patient dialogues (approximately 400,000 utterances) spanning 14 clinical specialties and over 10 consultation scenarios (diagnosis, drug advice, health counseling). Dialogues are characterized by authentic turn-taking, with frequent proactive physician questioning (medical history, allergies, comorbidities), enabling learning of context tracking and initiative.

The dataset is curated with self-instruct normalization for uniformity in "doctor voice," and KG–Instruction filtering to ensure entity-level consistency with CMeKG. All patient-identifying details are rigorously anonymized. This resource uniquely positions Zhongjing to excel in multi-turn, context-sensitive dialogue and to emulate proactive information gathering characteristic of clinical encounters (Yang et al., 2023).

5. Evaluation Protocols and Benchmarks

Zhongjing introduces a three-dimensional evaluation rubric prioritizing Safety > Professionalism > Fluency, with each axis subdivided into three abilities:

• Safety: Accuracy, Harmlessness, Ethics
• Professionalism: Comprehension, Clarity, Initiative
• Fluency: Coherence, Consistency, Warm Tone

Evaluation employs the following testbeds:

• CMtMedQA-test: 1,000 withheld multi-turn cases and 200 adversarial safety probes
• Huatuo-26M-test: 6,000 single-turn cases with curated gold answers

Scoring is conducted as follows: human medical experts assess Safety, while Professionalism and Fluency are auto-scored by GPT-4 using a pairwise win/tie/loss metric.

6. Experimental Findings and Component Analysis

Zhongjing demonstrates superior or competitive performance compared to baselines Ziya-LLaMA, BenTsao (6K instructions), DoctorGLM, and HuatuoGPT, matching ChatGPT (175B parameters) on some axes despite a ≈100× smaller parameter count. Performance gains are particularly pronounced in multi-turn Professionalism and Fluency, attributed to the scale and coverage of CMtMedQA.

Ablation studies document stage-wise contributions:

Removed Stage	Effect on Performance
Continuous pre-training	≈20 pp drop in Professionalism; modest Safety/Fluency decline
RLHF	≈15 pp loss in Safety; ~40% reduction in response length; minor on others

Each component (pre-training, SFT, RLHF) is found to be complementary: pre-training imparts medical knowledge, SFT yields dialogic coherence and style, and RLHF enforces expert-aligned initiative and safety.

7. Summary and Positioning

The Zhongjing framework operationalizes a full-stack, clinical-domain LLM pipeline using targeted medical pre-training, large-scale multi-turn SFT, and expert-driven RLHF. The initiative advances the state of Chinese medical conversational AI, yielding an assistant with enhanced knowledge, proactive dialogic initiative, contextual coherence, and alignment with clinical safety and ethical standards. Output code, models, and datasets are accessible at https://github.com/SupritYoung/Zhongjing (Yang et al., 2023).