CMED: Contextualized Medication Event Dataset

• The CMED topic is a clinical NLP dataset that captures medication mentions along with rich contextual attributes such as change type, temporality, certainty, actor, and negation.
• It was constructed from 500 de-identified clinical notes with detailed multi-dimensional annotations, supporting tasks like medication timeline and event-based reconciliation.
• Recent transformer-based models, including GatorTron and ensemble strategies, have leveraged CMED to improve extraction accuracy despite challenges like class imbalance and subtle contextual cues.

The Contextualized Medication Event Dataset (CMED) is a clinical NLP corpus for extracting medication mentions together with the contextual information that determines whether a medication change is being discussed and, if so, what kind of change it is, when it occurs, who initiates it, and how certain or negated it is. CMED was introduced to move beyond surface medication mention detection in electronic health records (EHRs), because applications such as medication timeline generation and medication reconciliation require more than recognition of drug names alone (Mahajan et al., 2020). The dataset later served as the basis of the n2c2 2022 shared tasks on contextual medication information extraction from clinical narratives (Sarker et al., 29 Jun 2025).

1. Origins and conceptual basis

CMED was developed from the premise that medication change extraction in clinical narratives is not adequately characterized by simple start/stop labeling. The original dataset paper presents a conceptual framework that organizes the context of medication change events into orthogonal dimensions, with the goal of capturing clinically relevant distinctions needed for downstream reasoning over medication histories (Mahajan et al., 2020). In this formulation, the critical issue is not merely whether a medication is mentioned, but whether a change is discussed and how that change should be interpreted in context.

The source material is clinical text drawn from the 2014 i2b2/UTHealth shared task corpus, with 500 clinical notes randomly selected for annotation; the original description specifies that these were selected from 2–5 notes per patient (Mahajan et al., 2020). Later CMED descriptions situate the corpus within the n2c2 2022 Track 1 challenge and frame it as a benchmark for extracting contextual medication information from unstructured EHR notes (Abdul-Quddoos et al., 23 Sep 2025).

This design makes CMED a bridge between entity extraction and event interpretation. A plausible implication is that the dataset was intended not only as a benchmark for sequence labeling and classification, but also as a substrate for temporally and causally meaningful medication history reconstruction.

2. Corpus construction and annotation process

The original CMED description reports 9,013 medication mentions annotated over 500 clinical notes (Mahajan et al., 2020). A later comparative analysis reports 9,012 across all notes, indicating a minor discrepancy in published corpus totals (Abdul-Quddoos et al., 23 Sep 2025). Across publications, CMED is consistently described as a 500-note corpus built from de-identified clinical narratives and annotated for medication-centered event understanding.

The annotation process involved three annotators, led by a physician, with 120 notes double-annotated and adjudicated for agreement (Mahajan et al., 2020). Medication mentions were pre-annotated by an extraction model, then corrected during manual annotation. Annotators followed explicit rules and disambiguation procedures, using note context such as section structure and surrounding sentences; the original report also notes that medical background knowledge was used where necessary (Mahajan et al., 2020).

Published split conventions vary across task papers. One summary reports 80% (400 EHRs, 7,230 mentions) for training/evaluation (Sarker et al., 29 Jun 2025). A transformer study describes 350 training notes, 50 validation notes, and 100 test notes (Chen et al., 2023). A later comparison reports 7,229 medication mentions (train) and 1,783 (test) (Abdul-Quddoos et al., 23 Sep 2025). These differences reflect different reporting conventions across CMED-based studies rather than a change in the underlying corpus.

Inter-annotator agreement in the original dataset paper was high for event type detection and variable across contextual dimensions. On 2,495 mentions, kappa for distinguishing Disposition, NoDisposition, and Undetermined was 0.88. For agreed Disposition events $(n=367)$, agreement was 0.87 for Action, 0.94 for Temporality, 0.75 for Certainty, and 0.72 for Actor (Mahajan et al., 2020). The lower agreement for Certainty and Actor was attributed to language ambiguity and to cases in which the initiator was unspecified.

3. Annotation schema and task decomposition

At the event level, each medication mention is assigned one of three event categories: Disposition, NoDisposition, or Undetermined. In the original framework, a Disposition mention is then contextualized along four orthogonal dimensions: Action, Temporality, Certainty, and Actor (Mahajan et al., 2020). Later challenge formulations and CMED system papers operationalize contextual classification across five dimensions by including Negation alongside Action, Temporality, Certainty, and Actor (Chen et al., 2023).

The original label inventories are explicitly enumerated. Action includes Start, Stop, Increase, Decrease, OtherChange, UniqueDose, and Unknown. Temporality includes Past, Present, Future, and Unknown. Certainty includes Certain, Hypothetical, Conditional, and Unknown. Actor includes Physician, Patient, and Unknown (Mahajan et al., 2020). Later task descriptions add Negation with classes Negated and Not Negated (Abdul-Quddoos et al., 23 Sep 2025).

A distinctive property of CMED is that a single mention can be assigned more than one change event if the context supports it, for example when a patient starts and stops a drug within a narrative (Mahajan et al., 2020). This makes the dataset more expressive than corpora that assume one event label per mention.

CMED supports a staged task decomposition that became standard in the n2c2 Track 1 literature:

Task	Annotation unit	Output
Task 1	Medication mention	Entity span
Task 2	Each medication mention	Disposition, NoDisposition, Undetermined
Task 3	Disposition mentions	Action, Temporality, Certainty, Actor, Negation

For Tasks 1 and 2, several CMED studies convert labels to BIO tag format for token-level sequence labeling, with tags such as B-Disposition, I-Disposition, B-NoDisposition, and O (Sarker et al., 29 Jun 2025). For Task 3, only medications with a Disposition event are retained and labeled for the contextual dimensions (Abdul-Quddoos et al., 23 Sep 2025).

The class distribution is skewed. In the original train-set statistics, the event counts are 1,413 Disposition, 5,260 NoDisposition, and 557 Undetermined (Mahajan et al., 2020). A later summary emphasizes that the corpus is imbalanced and notes that only 7% of the data are labeled as Undetermined (Sarker et al., 29 Jun 2025). This imbalance is one of the central methodological constraints in CMED modeling.

4. Evaluation protocol and early baselines

CMED evaluations use standard information extraction metrics, typically reported as micro- and macro-averaged precision, recall, and F-score, under both strict and lenient matching criteria. In the strict setting, a prediction must have an exact offset match with the gold annotation; in the lenient setting, overlap in offsets is sufficient (Sarker et al., 29 Jun 2025).

The metric definitions reported in CMED studies include:

$$\mathrm{MacroF} = \frac{1}{C} \sum_{c=1}^{C} Fscore_c$$

$$\mathrm{MacroP} = \frac{1}{C} \sum_{c=1}^{C} Precision_c$$

$$\mathrm{MacroR} = \frac{1}{C} \sum_{c=1}^{C} Recall_c$$

$$Fscore = \frac{2 \times Precision \times Recall}{Precision + Recall}$$

$Precision = \frac{TP}{TP+FP}$

$Recall = \frac{TP}{TP+FN}$

where $C$ is the number of event types (Sarker et al., 29 Jun 2025).

The original CMED paper reports preliminary experiments using a feature-based linear SVM across five related classification tasks: event detection plus the four contextual dimensions. Features included n-grams, lexico-syntactic information, windowed context, syntactic dependency structure, note-section, and RxNorm-derived information, evaluated with 5-fold cross-validation on the train set (Mahajan et al., 2020). Reported F1-scores for the full feature set were 0.65 for Disposition, 0.88 for NoDisposition, and 0.38 for Undetermined. For contextual dimensions, performance was stronger on majority labels such as Past (0.85), Certain (0.92), and Physician (0.96), and substantially weaker on minority labels such as Decrease (0.10), Future (0.39), Hypothetical (0.41), Conditional (0.37), and Patient (0.43) (Mahajan et al., 2020).

These results established two persistent themes in CMED research. First, note-section information and local lexical context are highly informative. Second, the hardest CMED problems are not medication mention detection, but minority-label and fine-grained context classification under severe imbalance.

5. Transformer-era modeling and benchmark results

Subsequent CMED work shifted from feature-based baselines to pretrained transformer architectures. Reported model families include BERT$_{base}$, RoBERTa$_{base}$, RoBERTa$$\mathrm{MacroF} = \frac{1}{C} \sum_{c=1}^{C} Fscore_c$$0, BioBERT, Clinical BERT, Discharge BERT, BioClinical BERT, BioDischarge BERT, BioReddit BERT, Clinical Longformer, and GatorTron variants, with pretraining corpora spanning Wikipedia, BookCorpus, PubMed abstracts, PMC journals, Reddit, and MIMIC clinical notes (Sarker et al., 29 Jun 2025, Abdul-Quddoos et al., 23 Sep 2025).

A transformer pipeline study reported that GatorTron achieved the best results among six compared models, with 0.9828 F1 for medication extraction, 0.9379 for event classification, and 0.9126 micro-average accuracy for context classification; the reported end-to-end accuracy, requiring all stages to be correct jointly, was 0.6178 (Chen et al., 2023). This result showed that large clinical LLMs could leverage CMED’s rich annotations effectively, while also indicating that full end-to-end contextual extraction remained substantially harder than the isolated subtasks.

An ensemble study focused on subtask 2 and proposed a pipeline of pretraining, fine-tuning on CMED, multiple predictions on the test data, and ensemble integration with voting strategies. The study evaluated hard voting, soft voting, and a weighted voting variant in which weights were proportional to Expected Calibration Error (ECE) computed on the training set; it reports that the study primarily employed majority soft voting, and that weighted ensembling did not outperform majority voting in that setup (Sarker et al., 29 Jun 2025). Relative to $$\mathrm{MacroF} = \frac{1}{C} \sum_{c=1}^{C} Fscore_c$$1, the Majority Voting Ensemble improved strict Micro-F from 0.8003 to 0.8401 and strict Macro-F from 0.7176 to 0.7744, corresponding to about 5% and 6% gains respectively (Sarker et al., 29 Jun 2025).

Another direction used LLMs for data augmentation rather than direct prediction. A ChatGPT-based study sampled 10% of training data, generated paraphrased versions under manually designed prompts, monitored class balance and bias manually, and merged augmented instances with the original CMED training set (Sarker et al., 2023). In that setting, Enhanced RoBERTa$$\mathrm{MacroF} = \frac{1}{C} \sum_{c=1}^{C} Fscore_c$$2 improved strict MicroF for medication event classification from 0.8003 to 0.8104, and lenient MacroF from 0.7317 to 0.7356; medication identification strict F1 improved from 0.8844 to 0.8946 (Sarker et al., 2023).

Comparative evidence on pretraining domain is not entirely uniform. One CMED ensemble study states that BERT models pretrained on biomedical/clinical corpora outperformed those pretrained only on general-domain data (Sarker et al., 29 Jun 2025). By contrast, a later systematic comparison reports that models pretrained on clinical data were more effective for medication and medication-event detection, but BERT Base, pretrained on general-domain data, was the most effective model for fine-grained context classification, achieving overall context F1 0.7387 versus an average 0.5986, and the highest combined-context F1 0.3006 against an average 0.1754 (Abdul-Quddoos et al., 23 Sep 2025). This divergence is one of the more important empirical findings in the CMED literature.

6. Applications, limitations, and significance

CMED was designed to support clinically meaningful downstream tasks. The original paper identifies medication timeline generation and medication reconciliation as target applications (Mahajan et al., 2020). Later work broadens the utility claim to medication timeline construction, adverse event detection, safety alerts, and related forms of context-aware EHR analytics that require knowledge of what was done, by whom, when, and with what certainty (Ramachandran et al., 2022). Because CMED links medication mentions to explicit contextual attributes, it functions as a benchmark for methods that must move from mention detection toward event interpretation.

The dataset is also important as a methodological stress test. The reported challenges are consistent across papers: class imbalance, especially for Undetermined and other minority classes; ambiguity in distinguishing factual, planned, hypothetical, and conditional events; underspecified actors; subtle cues for temporality and negation; heavy reliance on section headers and nearby sentences; and the difficulty of cross-sentence context linkage (Mahajan et al., 2020, Ramachandran et al., 2022). These properties make CMED difficult for both classical models and transformers, and they explain why per-dimension performance is usually much stronger on majority labels than on rare ones.

A common misconception is to treat CMED as merely a medication NER resource. The published task structure contradicts that interpretation: medication extraction is only the first stage, followed by event-status classification and then multi-dimensional contextual classification (Abdul-Quddoos et al., 23 Sep 2025). Another potential misconception is that domain-specific pretraining is uniformly superior for every CMED subtask. The comparative findings are more nuanced: some studies show clear advantages for biomedical and clinical pretraining, whereas another reports the strongest fine-grained context classification from general-domain BERT Base (Sarker et al., 29 Jun 2025, Abdul-Quddoos et al., 23 Sep 2025).

CMED’s broader significance lies in its combination of span-level annotation, contextual event typing, and clinically interpretable dimensions. This suggests a general annotation pattern for other clinical event domains in which event identity, temporality, certainty, agency, and negation must all be modeled jointly rather than inferred post hoc from isolated mentions.