---
title: 'CMED: Contextualized Medication Event Dataset'
url: https://www.emergentmind.com/topics/contextualized-medication-event-dataset-cmed
type: topic
---

# CMED: Contextualized Medication Event Dataset

The Contextualized Medication Event Dataset (CMED) is a clinical NLP corpus for extracting medication mentions together with the contextual information that determines whether a medication change is being discussed and, if so, what kind of change it is, when it occurs, who initiates it, and how certain or negated it is. CMED was introduced to move beyond surface medication mention detection in electronic health records (EHRs), because applications such as medication timeline generation and medication reconciliation require more than recognition of drug names alone [2011.08835]. The dataset later served as the basis of the n2c2 2022 shared tasks on contextual medication information extraction from clinical narratives [2506.23315].

## 1. Origins and conceptual basis

CMED was developed from the premise that medication change extraction in clinical narratives is not adequately characterized by simple start/stop labeling. The original dataset paper presents a conceptual framework that organizes the context of medication change events into orthogonal dimensions, with the goal of capturing clinically relevant distinctions needed for downstream reasoning over medication histories [2011.08835]. In this formulation, the critical issue is not merely whether a medication is mentioned, but whether a change is discussed and how that change should be interpreted in context.

The source material is clinical text drawn from the 2014 i2b2/UTHealth shared task corpus, with 500 clinical notes randomly selected for annotation; the original description specifies that these were selected from 2–5 notes per patient [2011.08835]. Later CMED descriptions situate the corpus within the n2c2 2022 Track 1 challenge and frame it as a benchmark for extracting contextual medication information from unstructured EHR notes [2509.19224].

This design makes CMED a bridge between entity extraction and event interpretation. A plausible implication is that the dataset was intended not only as a benchmark for sequence labeling and classification, but also as a substrate for temporally and causally meaningful medication history reconstruction.

## 2. Corpus construction and annotation process

The original CMED description reports **9,013 medication mentions annotated over 500 clinical notes** [2011.08835]. A later comparative analysis reports **9,012 across all notes**, indicating a minor discrepancy in published corpus totals [2509.19224]. Across publications, CMED is consistently described as a 500-note corpus built from de-identified clinical narratives and annotated for medication-centered event understanding.

The annotation process involved **three annotators**, led by a physician, with **120 notes double-annotated and adjudicated for agreement** [2011.08835]. Medication mentions were **pre-annotated by an extraction model**, then corrected during manual annotation. Annotators followed explicit rules and disambiguation procedures, using note context such as section structure and surrounding sentences; the original report also notes that medical background knowledge was used where necessary [2011.08835].

Published split conventions vary across task papers. One summary reports **80% (400 EHRs, 7,230 mentions)** for training/evaluation [2506.23315]. A transformer study describes **350 training notes, 50 validation notes, and 100 test notes** [2303.08259]. A later comparison reports **7,229 medication mentions (train)** and **1,783 (test)** [2509.19224]. These differences reflect different reporting conventions across CMED-based studies rather than a change in the underlying corpus.

Inter-annotator agreement in the original dataset paper was high for event type detection and variable across contextual dimensions. On **2,495 mentions**, kappa for distinguishing `Disposition`, `NoDisposition`, and `Undetermined` was **0.88**. For agreed `Disposition` events \((n=367)\), agreement was **0.87** for Action, **0.94** for Temporality, **0.75** for Certainty, and **0.72** for Actor [2011.08835]. The lower agreement for Certainty and Actor was attributed to language ambiguity and to cases in which the initiator was unspecified.

## 3. Annotation schema and task decomposition

At the event level, each medication mention is assigned one of three event categories: **`Disposition`**, **`NoDisposition`**, or **`Undetermined`**. In the original framework, a `Disposition` mention is then contextualized along four orthogonal dimensions: **Action**, **Temporality**, **Certainty**, and **Actor** [2011.08835]. Later challenge formulations and CMED system papers operationalize contextual classification across **five dimensions** by including **Negation** alongside Action, Temporality, Certainty, and Actor [2303.08259].

The original label inventories are explicitly enumerated. **Action** includes `Start`, `Stop`, `Increase`, `Decrease`, `OtherChange`, `UniqueDose`, and `Unknown`. **Temporality** includes `Past`, `Present`, `Future`, and `Unknown`. **Certainty** includes `Certain`, `Hypothetical`, `Conditional`, and `Unknown`. **Actor** includes `Physician`, `Patient`, and `Unknown` [2011.08835]. Later task descriptions add **Negation** with classes `Negated` and `Not Negated` [2509.19224].

A distinctive property of CMED is that **a single mention can be assigned more than one change event if the context supports it**, for example when a patient starts and stops a drug within a narrative [2011.08835]. This makes the dataset more expressive than corpora that assume one event label per mention.

CMED supports a staged task decomposition that became standard in the n2c2 Track 1 literature:

| Task | Annotation unit | Output |
|---|---|---|
| Task 1 | Medication mention | Entity span |
| Task 2 | Each medication mention | `Disposition`, `NoDisposition`, `Undetermined` |
| Task 3 | `Disposition` mentions | Action, Temporality, Certainty, Actor, Negation |

For Tasks 1 and 2, several CMED studies convert labels to **BIO tag format** for token-level sequence labeling, with tags such as `B-Disposition`, `I-Disposition`, `B-NoDisposition`, and `O` [2506.23315]. For Task 3, only medications with a `Disposition` event are retained and labeled for the contextual dimensions [2509.19224].

The class distribution is skewed. In the original train-set statistics, the event counts are **1,413 Disposition**, **5,260 NoDisposition**, and **557 Undetermined** [2011.08835]. A later summary emphasizes that the corpus is imbalanced and notes that only **7%** of the data are labeled as `Undetermined` [2506.23315]. This imbalance is one of the central methodological constraints in CMED modeling.

## 4. Evaluation protocol and early baselines

CMED evaluations use standard information extraction metrics, typically reported as micro- and macro-averaged precision, recall, and F-score, under both **strict** and **lenient** matching criteria. In the strict setting, a prediction must have an **exact offset match** with the gold annotation; in the lenient setting, **overlap in offsets** is sufficient [2506.23315].

The metric definitions reported in CMED studies include:

$$
\mathrm{MacroF} = \frac{1}{C} \sum_{c=1}^{C} Fscore_c
$$

$$
\mathrm{MacroP} = \frac{1}{C} \sum_{c=1}^{C} Precision_c
$$

$$
\mathrm{MacroR} = \frac{1}{C} \sum_{c=1}^{C} Recall_c
$$

$$
Fscore = \frac{2 \times Precision \times Recall}{Precision + Recall}
$$

$$
Precision = \frac{TP}{TP+FP}
$$

$$
Recall = \frac{TP}{TP+FN}
$$

where \(C\) is the number of event types [2506.23315].

The original CMED paper reports preliminary experiments using a **feature-based linear SVM** across five related classification tasks: event detection plus the four contextual dimensions. Features included **n-grams, lexico-syntactic information, windowed context, syntactic dependency structure, note-section, and RxNorm-derived information**, evaluated with **5-fold cross-validation on the train set** [2011.08835]. Reported F1-scores for the full feature set were **0.65** for `Disposition`, **0.88** for `NoDisposition`, and **0.38** for `Undetermined`. For contextual dimensions, performance was stronger on majority labels such as **Past (0.85)**, **Certain (0.92)**, and **Physician (0.96)**, and substantially weaker on minority labels such as **Decrease (0.10)**, **Future (0.39)**, **Hypothetical (0.41)**, **Conditional (0.37)**, and **Patient (0.43)** [2011.08835].

These results established two persistent themes in CMED research. First, note-section information and local lexical context are highly informative. Second, the hardest CMED problems are not medication mention detection, but minority-label and fine-grained context classification under severe imbalance.

## 5. Transformer-era modeling and benchmark results

Subsequent CMED work shifted from feature-based baselines to pretrained transformer architectures. Reported model families include **BERT\(_{base}\)**, **RoBERTa\(_{base}\)**, **RoBERTa\(_{large}\)**, **BioBERT**, **Clinical BERT**, **Discharge BERT**, **BioClinical BERT**, **BioDischarge BERT**, **BioReddit BERT**, **Clinical Longformer**, and **GatorTron** variants, with pretraining corpora spanning Wikipedia, BookCorpus, PubMed abstracts, PMC journals, Reddit, and MIMIC clinical notes [2506.23315][2509.19224].

A transformer pipeline study reported that **GatorTron** achieved the best results among six compared models, with **0.9828** F1 for medication extraction, **0.9379** for event classification, and **0.9126** micro-average accuracy for context classification; the reported end-to-end accuracy, requiring all stages to be correct jointly, was **0.6178** [2303.08259]. This result showed that large clinical language models could leverage CMED’s rich annotations effectively, while also indicating that full end-to-end contextual extraction remained substantially harder than the isolated subtasks.

An ensemble study focused on **subtask 2** and proposed a pipeline of **pretraining**, **fine-tuning on CMED**, **multiple predictions on the test data**, and **ensemble integration with voting strategies**. The study evaluated **hard voting**, **soft voting**, and a **weighted voting** variant in which weights were proportional to **Expected Calibration Error (ECE)** computed on the training set; it reports that the study primarily employed **majority soft voting**, and that weighted ensembling did **not** outperform majority voting in that setup [2506.23315]. Relative to **\(Roberta_{large}\)**, the **Majority Voting Ensemble** improved **strict Micro-F** from **0.8003** to **0.8401** and **strict Macro-F** from **0.7176** to **0.7744**, corresponding to about **5%** and **6%** gains respectively [2506.23315].

Another direction used large language models for **data augmentation** rather than direct prediction. A ChatGPT-based study sampled **10% of training data**, generated paraphrased versions under manually designed prompts, monitored class balance and bias manually, and merged augmented instances with the original CMED training set [2306.07297]. In that setting, **Enhanced RoBERTa\(_{large}\)** improved **strict MicroF** for medication event classification from **0.8003** to **0.8104**, and **lenient MacroF** from **0.7317** to **0.7356**; medication identification strict F1 improved from **0.8844** to **0.8946** [2306.07297].

Comparative evidence on pretraining domain is not entirely uniform. One CMED ensemble study states that **BERT models pretrained on biomedical/clinical corpora outperformed those pretrained only on general-domain data** [2506.23315]. By contrast, a later systematic comparison reports that models pretrained on clinical data were more effective for medication and medication-event detection, but **BERT Base**, pretrained on general-domain data, was the most effective model for fine-grained context classification, achieving overall context F1 **0.7387** versus an average **0.5986**, and the highest combined-context F1 **0.3006** against an average **0.1754** [2509.19224]. This divergence is one of the more important empirical findings in the CMED literature.

## 6. Applications, limitations, and significance

CMED was designed to support clinically meaningful downstream tasks. The original paper identifies **medication timeline generation** and **medication reconciliation** as target applications [2011.08835]. Later work broadens the utility claim to **medication timeline construction**, **adverse event detection**, **safety alerts**, and related forms of context-aware EHR analytics that require knowledge of what was done, by whom, when, and with what certainty [2208.08417]. Because CMED links medication mentions to explicit contextual attributes, it functions as a benchmark for methods that must move from mention detection toward event interpretation.

The dataset is also important as a methodological stress test. The reported challenges are consistent across papers: **class imbalance**, especially for `Undetermined` and other minority classes; **ambiguity** in distinguishing factual, planned, hypothetical, and conditional events; **underspecified actors**; subtle cues for **temporality** and **negation**; heavy reliance on **section headers and nearby sentences**; and the difficulty of **cross-sentence context linkage** [2011.08835][2208.08417]. These properties make CMED difficult for both classical models and transformers, and they explain why per-dimension performance is usually much stronger on majority labels than on rare ones.

A common misconception is to treat CMED as merely a medication NER resource. The published task structure contradicts that interpretation: medication extraction is only the first stage, followed by event-status classification and then multi-dimensional contextual classification [2509.19224]. Another potential misconception is that domain-specific pretraining is uniformly superior for every CMED subtask. The comparative findings are more nuanced: some studies show clear advantages for biomedical and clinical pretraining, whereas another reports the strongest fine-grained context classification from general-domain BERT Base [2506.23315][2509.19224].

CMED’s broader significance lies in its combination of span-level annotation, contextual event typing, and clinically interpretable dimensions. This suggests a general annotation pattern for other clinical event domains in which event identity, temporality, certainty, agency, and negation must all be modeled jointly rather than inferred post hoc from isolated mentions.

Source: https://www.emergentmind.com/topics/contextualized-medication-event-dataset-cmed