FocusMed: Focused Medical Question Summarization

• FocusMed is a framework that condenses consumer health questions into concise, FAQ-style summaries by explicitly modeling medically important entities.
• It employs a three-stage pipeline—focus extraction, focused fine-tuning, and multi-dimensional quality evaluation—to ensure faithfulness, conciseness, and comprehensive coverage.
• It demonstrates significant improvements in ROUGE-L, BERTScore, and SummaC_ZS metrics on MEDIQA and MeqSum benchmarks compared to prior methods.

Searching arXiv for the FocusMed paper and closely related MQS context papers. FocusMed is a LLM-based framework for medical question summarization (MQS) that transforms a long, noisy consumer health question (CHQ) into a concise FAQ-style summary by explicitly modeling the question’s core focus and using that focus to guide both supervision and inference (Liu et al., 6 Oct 2025). In this context, “focus” denotes the medically important entities or concerns that represent the main question intent, rather than a generic summary topic. The framework is motivated by two recurrent MQS failure modes—poor focus identification and hallucination—and organizes their mitigation into three linked stages: question focus extraction, focus-enriched fine-tuning, and multi-dimensional quality evaluation with selection.

1. Task definition and problem setting

Medical question summarization maps an input CHQ $C$ to a short standardized question $y$ that preserves the patient’s true intent while removing redundancy, colloquial phrasing, and irrelevant detail. FocusMed situates this task in the setting of online medical platforms, where CHQs often contain multiple symptoms, medications, timelines, and extraneous context. The paper identifies two central difficulties. First, existing models may summarize the wrong aspect of the question or omit the actual core issue; the paper characterizes this as poor focus identification. Second, LLMs may generate medically plausible but unsupported content, including hallucinated medications, side effects, or treatments (Liu et al., 6 Oct 2025).

The framework is explicitly positioned against prior MQS families. Early Seq2Seq systems with attention are described as struggling with medical-domain complexity and often generating irrelevant or inaccurate summaries. PLM-based approaches such as BART-like models improve performance through pretrained biomedical knowledge, but still rely heavily on maximum likelihood estimation and do not explicitly model semantic focus well. Reinforcement-learning and contrastive-learning methods improve semantic rewards or hard-negative discrimination, yet the paper argues that they still do not directly solve focus extraction. Direct LLM fine-tuning is also presented as insufficient, because it remains prone to focus identification bias and unfaithful generation.

A common misconception is to treat FocusMed as ordinary instruction tuning for MQS. The framework is more specific: it treats summarization as a focus-guided, faithfulness-aware generation problem. This suggests that the system’s novelty lies less in the choice of base LLM than in the explicit structuring of supervision and decoding around core-focus preservation.

2. Framework organization and representational design

FocusMed is a three-stage pipeline comprising Question Focus Extraction, Model Fine-tuning on an enhanced dataset, and Multi-dimensional Quality Evaluation and Selection (Liu et al., 6 Oct 2025). Its central design choice is to augment the original CHQ with an extracted focus representation before supervised fine-tuning, then to treat generation as a candidate-selection problem rather than a single-pass output.

Stage	Operation	Output
1	Prompt-driven core focus extraction	$f_i$
2	Focus-augmented supervised fine-tuning	Summary candidates
3	Faithfulness/conciseness/coverage scoring	Final selected summary

The enhanced model input is defined as

$x = (I, C, K)$

where $I$ is the task instruction, $C$ is the consumer health question, and $K$ is the extracted key focus. For each original CHQ-FAQ pair $(x_i, y_i)$, the pipeline extracts a focus $f_i$, verifies that focus for faithfulness, and constructs an augmented example

$e_i = x_i \cup f_i.$

The resulting pair $y$0 is added to the enhanced fine-tuning set, which the paper describes as a silver-standard dataset.

This organization has two implications. First, the framework does not assume that latent attention in a pretrained model is sufficient for reliable question-focus recovery. Second, it separates two forms of control that are often conflated in summarization systems: semantic control over what the summary should be about, and quality control over which generated candidate should be retained.

3. Core focus extraction and faithfulness filtering

The first stage extracts the core focus of a CHQ using a prompt template designed to extract only medications and symptoms, to limit the output to at most two entities per category, and to use JSON + CoT format (Liu et al., 6 Oct 2025). The use of structured JSON is motivated as a way to improve extraction reliability, while the Chain-of-thought-style justification is used to improve reasoning and accuracy during extraction.

The extracted focus is not accepted unconditionally. To reduce hallucination during focus extraction, FocusMed applies a semantic faithfulness validation procedure. It first applies TextRank to the extracted focus output to obtain key phrases. It then extracts noun phrases from the original CHQ and computes similarity between each extracted key phrase and the original noun phrases. If any similarity falls below a threshold $y$1, the extracted focus is treated as unfaithful and the sample is rejected. The paper reports that a threshold in the range $y$2–$y$3 works best on the development set.

Operationally, the dataset-construction logic is described as follows: for each input $y$4, a model $y$5 extracts a focus $y$6; TextRank(f_i) yields key phrases $y$7; noun phrases $y$8 are extracted from the source text; similarity $y$9 is computed; if $f_i$0, the sample is discarded and retried; otherwise the augmented example $f_i$1 is retained. This mechanism is the framework’s primary defense against focus hallucination at the data-construction stage.

The emphasis on focus extraction addresses a specific failure pattern illustrated in the paper: a question involving methotrexate was summarized as a generic drug side-effect question, thereby missing the actual relation being asked about. In FocusMed, such errors are treated as focus-identification failures rather than generic summarization noise.

4. Focus-aware supervised fine-tuning

After focus extraction, FocusMed fine-tunes base LLMs with QLoRA to reduce memory and computation cost (Liu et al., 6 Oct 2025). The output sequence is written as

$f_i$2

with each token $f_i$3 sampled from $f_i$4, where the state is

$f_i$5

The supervised fine-tuning objective minimizes negative log-likelihood: $f_i$6 In this formulation, the distinctive element is not the loss itself, but the fact that training examples are focus-enriched through $f_i$7, so the model learns to condition summarization on an explicit statement of the question’s core concern.

The paper experiments with Qwen2.5-7B, LLaMA3.1-8B, DeepSeek-R1-Distill-Qwen-7B, and GPT-4o. The fine-tuning stage specifically uses Qwen2.5-7B and LLaMA3.1-8B in four extraction/fine-tuning combinations. Training settings are reported as 10 epochs, learning rate $f_i$8, batch size 4, weight decay 0.01, max output length 512, AdamW optimizer, gradient clipping 0.3, and a single NVIDIA 4090 24GB GPU.

This stage clarifies another point often overlooked in summaries of the method: FocusMed does not claim that focus extraction alone solves MQS. The extracted focus becomes part of the supervised input representation, so the model is explicitly trained to use focus-aware conditioning during generation.

5. Multi-dimensional quality evaluation and candidate selection

Even after focus-aware fine-tuning, FocusMed does not accept a single generated summary by default. Instead, it generates multiple candidates from four extraction/fine-tuning model combinations and reranks them using a three-dimensional quality model (Liu et al., 6 Oct 2025). The three scoring dimensions are faithfulness, conciseness, and coverage.

Faithfulness measures factual consistency between the generated summary and the source CHQ. The paper decomposes the generated summary into atomic facts using DeepSeek-R1 and computes the fraction of those facts entailed by the original input: $f_i$9

Conciseness measures how efficiently the summary conveys essential content. TextRank is applied to extract key phrases, and the ratio of total key-phrase length to total summary length is computed: $x = (I, C, K)$0

Coverage measures how much source information is preserved in the summary. Here the source CHQ is decomposed into atomic facts, and the fraction entailed by the summary is computed: $x = (I, C, K)$1

The final selection score is a weighted combination: $x = (I, C, K)$2

The weights are dataset-specific and selected by grid search. For MEDIQA, $x = (I, C, K)$3, $x = (I, C, K)$4, and $x = (I, C, K)$5. For MeqSum, $x = (I, C, K)$6, $x = (I, C, K)$7, and $x = (I, C, K)$8. The final system output is the candidate with the highest overall score.

This reranking stage is significant because it treats summary generation as a structured decision under multiple criteria rather than as a single autoregressive sample. It also makes explicit that “better” MQS output is not reducible to lexical overlap alone.

6. Evaluation, ablations, and reported significance

FocusMed is evaluated on two MQS benchmarks: MeqSum and MEDIQA (Liu et al., 6 Oct 2025). MeqSum contains 1,000 patient health questions, split into train 400, dev 100, and test 500. MEDIQA, from the MEDIQA 2021 shared task, uses train 1,000, dev 50, and test 100. Average CHQ/FAQ lengths are reported as 59.4/10.0 for MeqSum and 66.2/11.3 for MEDIQA. Evaluation metrics are ROUGE-1, ROUGE-2, ROUGE-L, BERTScore, and SummaC$x = (I, C, K)$9 for faithfulness.

The full framework is reported to achieve the best results on all metrics among the listed baselines. On MEDIQA, FocusMed reports ROUGE-L $I$0, BERTScore $I$1, and SummaC$I$2 $I$3. The best prior baseline listed is MEDAL with ROUGE-L $I$4, BERTScore $I$5, and SummaC$I$6 $I$7. On MeqSum, FocusMed reports ROUGE-L $I$8, BERTScore $I$9, and SummaC$C$0 $C$1, compared with the listed prior best QFCL at ROUGE-L $C$2, BERTScore $C$3, and SummaC$C$4 $C$5. The paper highlights ROUGE-L improvements of $C$6 on MeqSum and $C$7 on MEDIQA over the prior state of the art.

The ablation studies attribute gains to both major components. In MEDIQA, the selection ablation reports: LLMs Selection at $C$8, FocusMed (w/o Selection) at $C$9, FocusMed (w/o Extract) at $K$0, and full FocusMed at $K$1 for ROUGE-L, BERTScore, and SummaC$K$2, respectively. Focus extraction also improves performance across model combinations, and larger extraction models improve results, with Qwen2.5 extraction models showing the trend $K$3.

These findings support a narrower and more technically precise interpretation of the framework’s contribution. FocusMed is not simply an LLM applied to medical summarization; it is a pipeline that externalizes semantic focus, filters that focus for faithfulness, injects it into supervised fine-tuning, and then performs candidate selection using explicitly defined factual and structural criteria. A plausible implication is that its strongest contribution lies in converting MQS from an undifferentiated generation problem into a staged control problem over question intent, factual support, and summary economy.