MedCodex: Medical Translation Framework

• MedCodex is a hybrid framework that systematically integrates domain-specific biomedical knowledge with large language models to enhance medical translation.
• It uses a three-stage pipeline—concept extraction, chain-of-dictionary prompting, and LoRA fine-tuning—to accurately render rare and polysemous medical terms.
• Empirical evaluation on a MedlinePlus corpus shows significant performance gains over baseline models, ensuring improved safety and precision in clinical contexts.

MedCOD (Medical Chain-of-Dictionary) is a hybrid framework developed to enhance English-to-Spanish medical translation by systematically integrating structured biomedical knowledge into LLMs. It is architected around two core mechanisms—domain-specific knowledge injection and lightweight, parameter-efficient model adaptation—to meet the unique lexical and safety requirements of medical translation. MedCOD is explicitly designed to address rare terminology, complex disambiguation, and critical error minimization by leveraging both curated ontologies and the latent capabilities of LLMs (Salim et al., 31 Aug 2025).

1. Framework Design and Pipeline

MedCOD’s pipeline comprises three sequential stages: preprocessing and concept extraction, chain-of-dictionary prompting, and optional LoRA-based fine-tuning.

• Preprocessing and Concept Identification: From each English sentence in medical texts (e.g., MedlinePlus articles), MedCOD identifies “concepts”—distinct domain-specific terms—using a pre-trained LLM (GPT-4o-mini) as a knowledge base. This extraction targets clinically relevant lexical items warranting precise translation.
• Chain-of-Dictionary Prompting: For each concept, MedCOD concatenates multilingual variants, synonyms, and Unified Medical Language System (UMLS) dictionary entries into a structured prompt appended to the translation input.
• Parameter-Efficient Model Adaptation: Selected open-source models (Phi-4, Qwen2.5-14B/7B, LLaMA-3.1-8B) are optionally adapted via Low-Rank Adaptation (LoRA), updating only a constrained set of parameters to minimize computational overhead.

This approach is motivated by the observation that clinical translation often encounters rare or polysemous terms (e.g., “transposition of the great vessels”) where unaugmented LLMs may hallucinate or misrender meaning, risking clinical inaccuracies.

2. Domain Knowledge Integration

MedCOD operationalizes domain knowledge injection via two sources:

• UMLS Dictionary Layer: For each identified concept, canonical Spanish entries, English and Spanish synonyms, and contextual UMLS definitions are retrieved. This component establishes a “hard dictionary” informed by a curated biomedical ontology, ensuring terminological alignment.
• LLM-as-Knowledge-Base (LLM-KB): GPT-4o-mini is systematically queried using templated prompts to solicit:
  1. Multilingual term translations (e.g., French, German).
  2. Language-specific synonym sets.

LLM-KB thus fills lexical gaps in static ontologies by surfacing neologisms or colloquial usages, complementing the static UMLS vocabulary. The combined product forms a multi-source, context-specific lexicon for each translation instance.

3. Chain-of-Dictionary Prompting Structure

MedCOD’s chain-of-dictionary prompt systematically embeds three layers of knowledge for each relevant sentence:

• Concept List Identification: Each domain-specific term is explicitly listed (e.g., immune system, AIDS, cancer, transplant, corticosteroids).

• LLM-KB Output: Multilingual translations and synonyms are provided for each concept, structured as:

$W$5

• UMLS-Dict Layer: Definitions and standardized mappings (e.g., “shortness of breath” means “dificultad para respirar”).

The structured prompt is prepended to the translation query using a fixed template, as documented in Table 1 of the source. By comparison, baseline prompts omit knowledge augmentations, resulting in ambiguity and terminology inconsistency.

4. Model Adaptation via LoRA Fine-Tuning

LoRA fine-tuning in MedCOD is applied to open-source LLMs, restricting updates to a low-rank factorization of weight matrices. The process can be described as:

$$\Delta W = BA, \quad A \in \mathbb{R}^{n \times r}, \quad B \in \mathbb{R}^{r \times m}$$

where $W$ denotes the original weight matrix and $\Delta W$ the learned update. This method maintains the full model’s weights unchanged, learning only the low-rank correction. Key hyperparameters include:

• max_seq_length = 2048 (with RoPE scaling)
• LoRA rank $r$ = 16; $\alpha$ = 16; dropout = 0
• per_device_train_batch_size = 2; gradient_accumulation_steps = 4
• learning_rate = $2 \times 10^{-4}$; max_steps = 60

Standard cross-entropy loss is retained for all experiments.

5. Evaluation Regime and Metrics

MedCOD is evaluated on a parallel corpus comprising 2,999 MedlinePlus article pairs (yielding 143,760 aligned sentences) and a 100-sentence test set covering a spectrum of syntactic complexity. Evaluation employs three metrics:

• SacreBLEU: $$BLEU = \exp\left( \sum_{n=1}^N w_n \log p_n \right) \times BP$$ with $p_n$ being modified n-gram precisions and $BP$ the brevity penalty.
• chrF++: $$chrF++ = \frac{(1+\beta^2) \cdot P \cdot R}{\beta^2 \cdot P + R}$$ where $W$0 and $W$1 are n-gram-level precision and recall at the character level.
• COMET: $W$2 with $W$3 a neural regressor trained on human annotation.

6. Empirical Results and Component Analyses

Experimental comparison (see Figure 1 and Table 3 in the source) demonstrates that MedCOD—both as a prompt augmentation and with LoRA fine-tuning—substantially improves translation performance over base models and outperforms strong proprietary baselines (e.g., GPT-4o). Performance for the Phi-4 model:

Model Version	BLEU	chrF++	COMET
Base	18.77	8.79	0.643
+MedCOD only	32.71★	21.86★	0.819★
+LoRA only	42.52★	28.35★	0.862★
MedCOD + LoRA	44.23★	28.91	0.863

(★: statistically significant versus baseline at $W$4)

Parallel gains are observed with Qwen2.5-14B and LLaMA-3.1-8B. Ablation confirms that both prompting and fine-tuning independently deliver improvements, with their combination producing optimal results.

Additional analysis (Table 4) assesses variants of the prompting recipe: LLM-KB multilingual, LLM-KB synonyms, and UMLS dictionary only. The best BLEU and COMET scores are generally achieved with the multilingual variant, though results may shift based on architecture and fine-tuning state.

7. Limitations and Future Outlook

MedCOD’s design and results underscore the value of structured ontological knowledge and LLM-derived lexical flexibility in medical translation. Future research directions outlined include:

• Expansion to more heterogenous corpora, e.g., clinical notes and discharge summaries.
• Application to additional language pairs, prioritizing low-resource and morphologically complex languages.
• Development of dynamic, context-adaptive glossaries, potentially via RLHF-based error correction.
• Human-in-the-loop validation for high-stakes contexts.

These suggest that leveraging complementary external knowledge with parameter-efficient adaptation offers a scalable approach for enhancing LLMs in specialized, safety-critical translation scenarios (Salim et al., 31 Aug 2025).