---
title: Medical Question Summary (MQS)
url: https://www.emergentmind.com/topics/medical-question-summary-mqs
type: topic
---

# Medical Question Summary (MQS)

Searching arXiv for the target paper and closely related work on medical question summarization and medical QA benchmarks.
I’m looking up the cited arXiv records to ground the article in the relevant MQS and medical QA literature.
Medical Question Summary (MQS) is the task of transforming a long, noisy consumer health question into a short, medically salient question that preserves the core information need. In the MQS literature, this is commonly formalized as converting a consumer health question (CHQ) into a concise, doctor-like frequently asked question (FAQ), while preserving the **question focus** and the **question type**—that is, the central medical entity or entities and the aspect being asked about them, such as treatment, testing, cause, symptoms, or susceptibility [2106.00219][2304.07437][2510.04671][2512.23637]. The task sits between natural language understanding and downstream medical QA: it reduces peripheral narrative, normalizes lay phrasing, and produces a compact query that is more suitable for retrieval, answer ranking, entailment, or clinical review [2304.07437][2512.23637].

## 1. Definition, scope, and core semantics

MQS is defined in closely related ways across the literature. One formulation describes it as converting lengthy and complex CHQs into concise, standardized questions that are easier for doctors to comprehend [2510.04671]. Another defines it as generating a short, abstractive question that preserves the key medical information needed to answer the original CHQ [2512.23637]. In both cases, the source text is typically a free-form patient question containing redundancy, emotional content, colloquialisms, spelling variation, and multiple sub-questions, while the target is a brief, medically focused question in professional or semi-professional style [2106.00219][2304.07437].

Two semantic notions recur throughout the field. The first is **question focus**, usually the main disease, symptom, drug, procedure, test, or clinical relation that the summary must preserve. The second is **question type**, the information need attached to that focus, such as treatment, test, cause, information, physician, ingredients, or other coarse categories in early work, or a richer schema in later datasets [2106.00219][2512.23637]. This distinction is not merely descriptive. The question-aware transformer literature explicitly argues that MQS models must preserve both focus and type to remain useful for consumer health QA, because a summary that retains the right disease but changes a treatment query into a symptoms query is semantically incorrect even if it looks fluent [2106.00219].

The practical motivation is consistent across papers. CHQs often dilute the medically relevant content inside personal history and peripheral narrative, and patients use non-professional vocabulary that does not align well with FAQ collections, evidence bases, or clinician-authored resources [2304.07437][2512.23637]. MQS therefore functions as an intent distillation layer. A plausible implication is that MQS is best understood not as generic summarization in a medical domain, but as a specialized form of query reformulation whose output is optimized for downstream medical information access.

## 2. Datasets, annotation practice, and corpus design

MQS research has been shaped by a small number of annotated datasets, each reflecting a different balance between expert curation, realism, and scale. Early work centered on MeQSum, while later work introduced CHQ-Summ and reused MEDIQA 2021 HQS and MeqSum in broader LLM evaluations [2106.00219][2512.23637][2510.04671].

| Resource | Size / split | Distinctive property |
|---|---|---|
| MeQSum (augmented setup) | 5,155 training pairs and 500 test samples | Long consumer questions with short question summaries [2106.00219] |
| CHQ-Summ | 1,507 CHQs; train 1000, test 400, validation 107 | Expert summaries plus focus, type, and MeSH-linked analysis [2512.23637] |
| MEDIQA 2021 HQS / MeqSum in FocusMed | MEDIQA: 1000/50/100; MeqSum: 400/100/500 | Used for LLM-based MQS experiments and ablations [2510.04671] |

MeQSum was originally introduced as a benchmark of real-world consumer health questions with manually written short summaries, and one influential transformer study treated “the MeQSum dataset” as an augmented resource with 5,155 training pairs and 500 test samples after adding clinically similar question–summary pairs from Ely et al. [2106.00219]. CHQ-Summ later expanded the annotation space by introducing 1,507 expert-annotated consumer healthcare questions from Yahoo! Answers, along with explicit annotations for question focus and question type, and MeSH-based corpus characterization [2512.23637]. The CHQ-Summ annotation protocol involved six experts, with inter-annotator agreement reported as F1 83.22 for question focus, F1 85.44 for question type, and ROUGE-L 52.8 for question summary on 30 common questions [2512.23637].

These corpora differ not only in scale but in what they make observable. CHQ-Summ explicitly formalizes focus and type as annotation targets and reports that summaries are usually one sentence, occasionally two, with mean summary length about 13.2 words [2512.23637]. The question-aware transformer work instead operationalizes focus and type through automatic medical entity recognition and question-type conditioning, without requiring new human annotations in the base corpus [2106.00219]. FocusMed uses MEDIQA 2021 HQS and MeqSum as MQS benchmarks but constructs an additional focus-enhanced training signal automatically, rather than through manual focus labels [2510.04671].

Dataset quality has also become a substantive methodological issue. The entity-driven contrastive learning study reports serious leakage in some MQS datasets, especially a 33% duplicate rate in iCliniq and 523 duplicates in HealthCareMagic, and reorganizes those datasets into deduplicated splits for fair evaluation [2304.07437]. This is significant because MQS corpora are relatively small, so duplicate contamination can materially distort reported gains.

## 3. Modeling paradigms

Three broad modeling paradigms dominate MQS research: sequence-to-sequence summarization, question-aware representation learning, and LLM-centered focus-guided generation.

The first paradigm treats MQS as abstractive summarization with standard encoder–decoder models. BART, PEGASUS, ProphetNet, T5, and related architectures are fine-tuned directly on CHQ–summary pairs [2106.00219][2512.23637]. This approach is straightforward and often strong, but several papers argue that maximum-likelihood training alone does not reliably preserve question focus. The ECL paper, for example, emphasizes that pure Seq2Seq objectives optimize token prediction rather than semantic alignment between CHQ and FAQ, often omitting key entities or changing the question type [2304.07437].

The second paradigm explicitly injects question semantics into the model. In question-aware transformer models, this takes two forms. **Question-Focus Aware** training uses multiple Cloze objectives—sequence-to-sequence masking, n-gram masking, and medical entity masking—to force the model to recognize and regenerate medically salient entities [2106.00219]. **Question-Type Aware** training conditions the decoder on question type either explicitly via learned type embeddings or implicitly via multi-task question-type prediction [2106.00219]. This line of work treats focus and type as structural signals that should shape hidden representations rather than emerge indirectly.

Entity-driven contrastive learning extends this idea by making medical entities in expert-written FAQs the operative definition of question focus [2304.07437]. It uses BART as the base summarizer, a MoCo-style contrastive architecture, and two forms of negatives: simple negatives from other FAQs and hard negatives created by replacing FAQ medical entities with other medical entities extracted by Stanza biomedical and clinical NER [2304.07437]. The combined objective is written as
$$
\mathcal{L} = \mathcal{L}_{ce} + \mathcal{L}_{scl} + \mathcal{L}_{hcl},
$$
so token-level generation is trained jointly with representation-space separation between the CHQ and both unrelated FAQs and entity-perturbed hard negatives [2304.07437]. This design is intended to force sensitivity to the exact medical focus.

The third paradigm is LLM-based focus guidance. FocusMed argues that direct LLM fine-tuning for MQS is prone to focus identification bias and unfaithful generation, and therefore decomposes the task into three stages: question focus extraction, focus-enhanced fine-tuning, and multi-dimensional candidate selection [2510.04671]. The framework extracts up to two medication entities and two symptom entities plus a concise explanation, validates that extraction through TextRank and semantic similarity filtering with threshold $\tau \in [0.8, 0.9]$, then fine-tunes LLMs on inputs consisting of instruction, CHQ, and extracted focus [2510.04671]. At inference time it generates four candidate summaries from different extraction/fine-tuning model combinations and selects the best using weighted faithfulness, conciseness, and coverage scores [2510.04671].

CHQ-Summ broadens the LLM perspective by benchmarking prompting strategies without task-specific fine-tuning. It evaluates standard prompting, element-aware prompting, hierarchical coarse-to-fine prompting, and Chain-of-Density prompting, showing that prompt design itself can materially alter MQS behavior [2512.23637]. This suggests that MQS has become a testbed not only for summarization architectures but also for controllable medical generation.

## 4. Evaluation methodology and empirical findings

MQS evaluation is unusually heterogeneous because lexical overlap, semantic faithfulness, and downstream usefulness do not always coincide. Early work relied primarily on ROUGE, but later studies added BERTScore, SummaC\(_{ZS}\), entailment, semantic coherence, human evaluation, and downstream QA retrieval metrics [2106.00219][2510.04671][2512.23637][2005.09067].

In the question-aware transformer literature, the best explicit question-type-aware model achieved ROUGE-1 45.20, ROUGE-2 28.38, and ROUGE-L 48.76 on the augmented MeQSum setup, outperforming the previous MeQSum state of the art by 10.2 ROUGE-L points [2106.00219]. Manual evaluation on 50 questions with three experts also found large qualitative gains over a MiniLM baseline, and Fleiss’ kappa values of 72.67, 71.09, and 67.39 were reported for the proposed variants [2106.00219]. The same paper also showed that automatic MQS improved downstream IR-based QA on LiveQA 2017 questions with MedQuAD retrieval, raising avgScore from 0.673 for original questions to 0.875 for automatically summarized questions, while manual reference summaries reached 1.125 [2106.00219].

Entity-driven contrastive learning reports consistent ROUGE gains over BART and QFCL across four datasets. On MeQSum it achieves ROUGE-1 52.85, ROUGE-2 36.06, and ROUGE-L 50.48, compared with QFCL’s 51.48, 34.16, and 49.08 [2304.07437]. The same study also reports markedly higher focus identification rates than overlap-based focus detection, including 0.96 versus 0.53 on MeQSum and 0.94 versus 0.75 on CHQ-Summ, reinforcing the claim that entity-grounded focus definitions are more robust [2304.07437].

FocusMed emphasizes faithfulness as much as overlap. On MEDIQA, full FocusMed reaches ROUGE-L 0.386, BERTScore 0.782, and SummaC\(_{ZS}\) 0.603; on MeqSum it reaches ROUGE-L 0.589, BERTScore 0.859, and SummaC\(_{ZS}\) 0.473 [2510.04671]. The ablations are especially informative: a best single focus-aware model without selection yields ROUGE-L 0.364 on MEDIQA, and a selection system without explicit focus yields 0.347, indicating that both focus extraction and multi-dimensional selection contribute to performance [2510.04671].

CHQ-Summ makes the evaluation landscape broader still. Among fine-tuned seq2seq systems, ProphetNet-Large-Uncased gives the best ROUGE-Lsum at 0.3596 and METEOR at 0.3034, BART-Large gives the best BERTScore at 0.9142, T5-Base gives the best Semantic Coherence at 0.7307, and PEGASUS-Large gives the best Entailment at 0.8294 [2512.23637]. Among prompted LLMs, Llama-3.2-3B-Instruct with Hierarchical 5-shot prompting reaches ROUGE-Lsum 0.3628 and METEOR 0.3073, while Qwen2-7B-Instruct with Element-Aware 0-shot prompting reaches the highest Entailment score, 0.8623 [2512.23637]. Human evaluation on 30 questions rates LLaMA-3.2-3B at 4.43 factual correctness, 4.51 informativeness, and 4.91 fluency, which is notably high for a small instruction-tuned model [2512.23637].

A recurring methodological lesson is that question conditioning matters. In question-driven answer summarization, BART+Q—formed by concatenating the question to the answer text—consistently outperforms plain BART across pages and passages, with statistically significant differences, which reinforces the general MQS principle that explicit conditioning on the information need improves content selection [2005.09067]. At the same time, several papers note that ROUGE can understate semantic adequacy when distinct summaries are both clinically valid, so overlap metrics alone are insufficient [2106.00219][2005.09067].

## 5. Relation to broader medical QA and multilingual benchmark ecosystems

MQS is adjacent to, but distinct from, several other strands of medical QA research. Those neighboring tasks matter because MQS is often used as an upstream normalization step for retrieval, reading comprehension, or answer generation rather than as an endpoint.

One neighboring line is span-extraction QA over clinical text. The emrQA-msquad dataset restructures emrQA into the SQuAD v2.0 framework with 253 contexts, 163,695 questions, and 4,136 manually obtained answer spans, enabling standard extractive MRC fine-tuning in a medical setting [2404.12050]. This is not MQS, but it addresses a complementary problem: once a question is understood or normalized, the system must locate an exact textual answer in EHR-derived context. A plausible implication is that MQS and span extraction can be composed in a pipeline, with MQS improving the query and MRC improving answer localization.

A second neighboring line is grounded consumer-facing QA. MeQA, an official Spanish system for questions about medicines for human use, answers only from AEMPS-authorized leaflets using IR modules and a bi-LSTM section predictor, and reports about 87% F1 on a manually annotated evaluation corpus [2111.02760]. MeQA does not perform MQS explicitly, but it does perform a form of intent-to-section mapping, which resembles categorical question summarization at the level of leaflet sections.

The multilingual benchmark literature further broadens the context. MediQAl introduces a French benchmark with 32,603 questions across 41 medical subjects and explicit Understanding versus Reasoning labels [2507.20917]. IMB contributes two Italian medical benchmarks: IMB-QA with 782,644 patient–doctor conversations from 77 medical categories and IMB-MCQA with 25,862 examination questions [2510.18468]. PediatricsMQA adds an age-stratified multimodal pediatric benchmark with 3,417 text-based MCQs and 2,067 vision-based MCQs [2508.16439]. These are not MQS datasets, but they show the broader environments into which MQS systems would be deployed: multilingual, multimodal, clinically heterogeneous, and often reasoning-heavy. This suggests that future MQS systems may need to summarize not only English adult CHQs but also pediatric, image-grounded, and multilingual inputs.

The distinction between MQS and answer summarization is equally important. MEDIQA-AnS focuses on question-driven summarization of answers, with 156 questions, 348 unique articles, and 552 passages from CHiQA-derived consumer health QA, and shows that including the question in the summarizer input improves performance [2005.09067]. MQS, by contrast, summarizes the question itself. The two tasks are complementary: MQS clarifies the user’s need, while question-driven answer summarization condenses the retrieved answer.

## 6. Limitations, misconceptions, and future directions

Several limitations recur across the literature. The first is **focus preservation**. Direct fine-tuning can produce fluent summaries that miss critical entities or distort the question type, and FocusMed explicitly identifies focus identification bias and unfaithful content as central failure modes of direct LLM fine-tuning [2510.04671]. Entity-driven contrastive learning was proposed largely because prior models often omitted entities such as “vaccine,” “dressings,” or “allopurinol” even when those entities were central to the user’s intent [2304.07437].

The second is **evaluation mismatch**. ROUGE and related lexical metrics remain standard, but multiple papers show that they correlate imperfectly with semantic correctness and practical usefulness [2106.00219][2005.09067]. CHQ-Summ therefore adds semantic coherence and entailment, and FocusMed adds faithfulness-oriented SummaC\(_{ZS}\) plus internal faithfulness, conciseness, and coverage scores [2512.23637][2510.04671]. A common misconception is that higher overlap necessarily means a better MQS system; the literature does not support that conclusion.

The third is **dataset reliability and transferability**. Data leakage in iCliniq and HealthCareMagic required explicit cleaning before fair comparison [2304.07437]. CHQ-Summ is English and Yahoo!-derived, while MeQSum is relatively small and often augmented with non-consumer material in experiments [2512.23637][2106.00219]. This suggests that MQS performance on one benchmark should not be treated as domain-general clinical robustness.

A fourth limitation is **detail sensitivity**. FocusMed notes that its models and underlying LLMs can under-attend to numerical and temporal details, such as preserving “13 days” versus using vague duration language [2510.04671]. In medical settings, such details can be clinically decisive. This suggests that future MQS systems will need stronger mechanisms for retaining dosages, durations, age markers, and temporal qualifiers.

Finally, the broader QA literature shows that domain adaptation, retrieval, and human oversight remain necessary. Italian medical QA benchmarks show that specialized adaptation strategies and RAG can outperform larger general-purpose models in non-English medical QA [2510.18468]. French reasoning benchmarks show that even strong zero-shot models remain far from human-level clinical reasoning and require human oversight [2507.20917]. It follows that MQS should be treated as an assistive layer for medical information systems, not as a substitute for clinical judgment. The most plausible near-term direction is therefore a modular architecture in which MQS, retrieval, evidence grounding, and answer generation or extraction are optimized jointly but evaluated separately for faithfulness and safety.

Source: https://www.emergentmind.com/topics/medical-question-summary-mqs