---
title: 'Term2Note: Privacy-Preserving Clinical Note Synthesis'
url: https://www.emergentmind.com/topics/term2note
type: topic
---

# Term2Note: Privacy-Preserving Clinical Note Synthesis

Searching arXiv for the Term2Note paper and closely related context.
Term2Note is a methodology for synthesising full-length clinical notes under differential privacy by separating clinically salient **content** from document **form**, then generating section-wise text conditioned on medical terms and filtering candidate outputs with a **DP quality maximizer** [2509.10882]. It is designed for the setting in which real clinical notes are difficult to share because of privacy leakage risk, yet downstream machine learning still requires training corpora with realistic statistical and semantic structure. The method addresses the particular difficulty of long-form clinical text generation, where notes may be thousands of tokens long, contain many specialised medical terms and abbreviations, and exhibit a challenging privacy-utility trade-off under strong DP constraints [2509.10882].

## 1. Problem formulation and design rationale

The Term2Note problem setting is the construction of a mechanism \(\mathcal{M}\) that maps a private dataset \(D^{\text{src}}\) of clinical notes to a synthetic dataset \(D^{\text{syn}}\) satisfying \((\epsilon,\delta)\)-differential privacy [2509.10882]. The paper adopts the standard definition
\[
\mathbb{P}[\mathcal{M}(D) \in S] \leq e^\epsilon \mathbb{P}[\mathcal{M}(D')\in S] + \delta,
\]
for neighboring datasets \(D,D'\) differing in one record and all measurable sets \(S\) [2509.10882]. It also explicitly relies on the standard DP facts of **post-processing** and **parallel composition**.

The motivation for the method is that clinical note synthesis under DP is substantially harder than short-text DP generation. The reasons given are threefold: **long-form generation complexity**, because notes are often thousands of tokens long and errors compound over long generation chains; **domain specificity**, because medically plausible text depends on specialised terminology and abbreviations; and the **privacy-utility trade-off**, because stronger DP noise harms semantic fidelity, term accuracy, and downstream usefulness [2509.10882]. The paper further argues that existing DP text generation methods mostly target short, easy-to-model text such as reviews and are therefore not suitable for long clinical notes.

The central architectural decision is to separate **content** from **form**. In Term2Note, content is represented by section-wise clinical terms, while form is represented by the section-wise note structure and narrative realisation [2509.10882]. This decomposition replaces direct end-to-end long-note generation with a staged procedure in which terms are extracted, optionally privatized, used to generate section-wise term content, then used again to condition section-wise note generation. This suggests that the method treats clinical notes not as monolithic sequences but as structured compositions whose semantic anchors can be handled separately from stylistic and discourse realisation.

## 2. Pipeline architecture

The paper describes four main modules: **Format and Term Identification**, **Clinical Terms Generation**, **Clinical Note Generation**, and **DP Quality Maximizer** [2509.10882]. The overall workflow is summarized in Algorithm 1 of the paper.

| Module | Function | Key implementation detail |
|---|---|---|
| Format and Term Identification | Split notes and extract section-wise terms | Rule-based regex segmentation; QuickUMLS; SNOMED CT |
| Clinical Terms Generation | Reconstruct or synthesize term lists from embeddings | Public term dataset; \(DPRP^*\) for privatized embeddings |
| Clinical Note Generation | Generate each section conditioned on instruction, prior sections, and terms | FastDP training under \((\epsilon_n,\delta_n)\) |
| DP Quality Maximizer | Select best note among \(k\) candidates | Perplexity-based post-processing |

The first stage applies
\[
[\text{SEC}_1^{\text{src}}, \dots, \text{SEC}_m^{\text{src}}] = \text{SecSplit}(X^{\text{src}})
\]
to split a private note into up to \(m\) standardized sections [2509.10882]. The splitting is implemented with **rule-based regex segmentation** over common clinical section titles. The resulting sections are grouped into six broad categories: **Patient Information**, **Clinical Course / History**, **Examinations / Findings**, **Laboratory / Imaging Results**, **Hospital Stay / Treatment**, and **Medications / Discharge Plan** [2509.10882].

Term extraction then applies
\[
[T_1^{\text{src}}, \dots, T_m^{\text{src}}] = \text{TermExt}([\text{SEC}_1^{\text{src}}, \dots, \text{SEC}_m^{\text{src}}])
\]
using **QuickUMLS**, while keeping only concepts mapped to **SNOMED CT** [2509.10882]. These section-wise term lists act as content anchors. This is an important distinction from the baselines described in the paper, which condition on **ICD control codes** rather than extracted section-wise clinical terms.

The term-generation stage is trained on a **public dataset of clinical terms** \(D_{\text{public}}\). The paper introduces an encoder \(\text{Emb}\), projection layer \(\text{Proj}_{\theta_p}\), and term generator \(\text{TermGen}_{\theta_t}\), and defines
\[
E' =
\begin{cases}
\text{Emb}(D_{\text{public}}) + \mathcal{N}(0, \sigma_{\text{emb}}), & \text{if training}, \\
\text{DPRP}^*\!\left(\text{Emb}(T^{\text{src}}), \frac{\epsilon_t}{m}, \frac{\delta_t}{m}\right), & \text{otherwise}.
\end{cases}
\]
The synthetic terms are then sampled as
\[
T^{\text{syn}} \sim \text{TermGen}_{\theta_t}(\text{Proj}_{\theta_p}(E')).
\]
Because privatization is applied section by section, the term privacy budget is divided evenly across sections as \(\left(\frac{\epsilon_t}{m}, \frac{\delta_t}{m}\right)\) [2509.10882].

The note-generation stage synthesizes each section conditionally:
\[
\text{SEC}_i^{\text{syn}} \sim \text{NoteGen}_{\theta_n}(I, [\text{SEC}_1^{\text{syn}}, \dots, \text{SEC}_{i-1}^{\text{syn}}], T_i),
\]
and assembles the full note as
\[
X^{\text{syn}} = [\text{SEC}_1^{\text{syn}}, \dots, \text{SEC}_m^{\text{syn}}].
\]
The note generator is trained with **FastDP** under \((\epsilon_n,\delta_n)\) [2509.10882]. The conditioning term list \(T_i\) may be either the original extracted terms \(T_i^{\text{src}}\) or the privatized synthetic terms \(T_i^{\text{syn}}\), yielding the paper’s distinction between **partial privacy** and **full privacy**.

## 3. Differential privacy mechanisms and guarantees

A notable technical component of Term2Note is the optional term-level privacy mechanism \(DPRP^*\), which adapts **DPRP** (Differentially Private Random Projections) to term embeddings [2509.10882]. The appendix pseudocode splits the privacy budget into
\[
(\epsilon_1,\delta_1), (\epsilon_2,\delta_2) \gets 0.85(\epsilon,\delta), 0.15(\epsilon,\delta),
\]
then adds Gaussian noise both to embeddings and to covariance:
\[
E' = E + \mathcal{N}(0,\sigma_1^2),
\]
\[
E_C' = E^T E + \mathcal{N}(0,\sigma_2^2).
\]
After this, the method performs
\[
V'\Sigma'{V'}^T = \text{SVD}(E_C'),
\]
keeps the top \(k\) components,
\[
V'_k = V'[1,\dots,k], \quad k = 0.6 \cdot E_{\text{hdim}},
\]
and reconstructs privatized embeddings as
\[
E_{\text{DP}} = E'{V'_k}^{T+}{V'_k}^{T}.
\]
The paper states that this yields DP term embeddings that preserve enough semantic structure to support generation [2509.10882].

The privacy guarantee of the full pipeline is summarized as
\[
(\epsilon, \delta) =
\begin{cases}
(\epsilon_n, \delta_n), & \text{if } T_i = T_i^{\text{src}}, \\
(\max(\epsilon_n,\epsilon_t), \max(\delta_n,\delta_t)), & \text{if } T_i = T_i^{\text{syn}}.
\end{cases}
\]
Thus, if only the note generator is private, the system is \((\epsilon_n,\delta_n)\)-DP; if both the term generator and note generator are private, the total guarantee follows from **parallel composition** plus **post-processing** and becomes \((\max(\epsilon_n,\epsilon_t), \max(\delta_n,\delta_t))\)-DP [2509.10882].

The **DP quality maximizer** is used only at inference time. The system generates \(k\) candidate notes,
\[
X^{\text{syn}[1]}, \dots, X^{\text{syn}[k]},
\]
scores them by perplexity,
\[
\text{PPL}(X^{\text{syn}[i]}) = \exp\left(-\frac{1}{t}\sum_{i=1}^{t}\log LLM_{\text{ppl}}(d_i \mid d_{<i})\right),
\]
and selects
\[
X^{\text{syn}} = \arg\min_{i\in[k]} \text{PPL}(X^{\text{syn}[i]}).
\]
Because this step is described as pure **post-processing**, it does not alter the DP guarantee [2509.10882]. The paper also reports that simpler reference-free metrics were explored, and that **maximum sentence character length** aligned best with human judgments in some ablations.

## 4. Experimental setting and evaluation protocol

The experimental study uses the **MIMIC** family of datasets. **MIMIC-III** serves as the public dataset for training the term generator, while **MIMIC-IV notes** are used as the private dataset for training the note generator; evaluation is conducted on a held-out **SNOMED-annotated subset** from MIMIC-IV [2509.10882]. After filtering, the dataset sizes are approximately \(D_{\text{public}}: 52{,}722\) notes, \(D_{\text{train}}^{\text{src}}: 122{,}202\) notes, and \(D_{\text{test}}^{\text{src}}: 204\) notes, with average note lengths roughly \(2800\)–\(3300\) tokens.

The evaluated privacy settings are
\[
\epsilon \in \{2,5,8\}, \qquad \delta = \frac{1}{N \log N},
\]
where \(N\) is the private dataset size [2509.10882]. The paper lists the model stack as follows: **MedEmbed-large** for term embeddings, **GPT2-Large** for term generation, **Llama-3.2-1B** or **Gemma-3-1B** for note generation, and **Asclepius-Llama3-8B** as the perplexity scorer.

Two baselines are used. **AUG-PE** is described as a private-evaluation-based text synthesis method adapted to clinical notes using ICD control codes, and **FastDP** is a DP-SGD / FastDP fine-tuning baseline also conditioned on ICD codes [2509.10882]. The comparison is therefore not only between privacy mechanisms but also between two conditioning paradigms: coarse label control via ICD codes and section-wise term conditioning via extracted medical terms.

Evaluation is divided into **fidelity**, **utility**, and **human evaluation**. Fidelity metrics are **Length KL divergence**, **Unary/binary clinical term Jaccard similarity**, **KL divergence over unary and binary term distributions**, and **MAUVE** [2509.10882]. Utility is assessed through **multi-label ICD code prediction** over 20 chapter-level ICD groups from discharge notes, using **Micro F1**, **Macro F1**, **Micro AUC**, **Macro AUC**, and **Precision@k** for \(k=3,5\) [2509.10882]. Human evaluation is based on pairwise comparisons by **three licensed physicians**, with rankings fit by a **Bradley–Terry** model.

The paper emphasizes the utility experiment as a direct test of whether the synthetic notes preserve **clinically actionable information**, rather than merely superficial plausibility [2509.10882]. This is especially relevant because the method is intended to preserve semantically meaningful content while providing formal privacy guarantees.

## 5. Empirical results

Term2Note is reported to consistently give the best fidelity among DP methods [2509.10882]. The paper reports **Length KL divergence** as low as **0.15–0.25**, the best or near-best term overlap scores, and the best **MAUVE** scores among DP methods, up to around **0.59** on **Llama-3.2-1B** in the main table and up to **0.80** in **Gemma** settings without note privacy [2509.10882]. A specific observation is that Term2Note preserves note length distributions much better than the baselines, which often generate notes that are too short.

For downstream ICD prediction, Term2Note is generally described as the best DP method across privacy settings [2509.10882]. At \(\epsilon=8\), Term2Note with private terms achieved approximately **Micro F1: 49.71**, **Macro F1: 21.28**, **Micro AUC: 80.03**, and **Macro AUC: 52.80** [2509.10882]. The real-data upper bound reported in the paper is **Micro F1 57.03**, **Macro F1 30.80**, **Micro AUC 82.01**, and **Macro AUC 58.88**. The synthetic notes therefore remain below real data but are described as meaningfully close to it.

Compared with **AUG-PE** and **FastDP**, Term2Note is said to achieve substantial improvements in both fidelity and utility while operating under fewer assumptions [2509.10882]. The baseline distinction matters: both baselines use ICD control codes rather than section-wise clinical terms, so the reported gains are associated not only with DP optimisation but also with the content/form decomposition of the synthesis task.

In human evaluation, physicians preferred Term2Note over AUG-PE by a large margin, with a Term2Note win rate of about **52.2%–53.5%** and an AUG-PE win rate of about **12.4%** [2509.10882]. The paper also reports that imposing full privacy on terms has only a **marginal** impact on human preference. This suggests that the term privatization layer can be added without severely degrading perceived clinical plausibility.

A recurring experimental theme is the comparison between **note-only privacy** and **full privacy**. The reported main takeaway is that full privacy slightly reduces fidelity and utility in some settings, but the drop is often modest, and the method remains substantially better than baselines even under stronger privacy constraints [2509.10882].

## 6. Scope, limitations, and significance

The paper identifies several assumptions underlying the method. It assumes that notes can be reliably segmented into sections, that clinical terms can be extracted with **QuickUMLS/SNOMED CT**, that public term corpora are available for pretraining the term generator, and that DP guarantees are meaningful at the note and term generation levels [2509.10882].

It also explicitly lists limitations. Some outputs still exhibit **missing or misordered sections**. Clinical accuracy issues remain, including **inconsistent medications**, **vague phrasing**, and **occasional reasoning errors** [2509.10882]. Under stronger privacy settings, repetition increases. The evaluation is limited to **MIMIC discharge notes** and **ICD prediction**, privacy evaluation is described as preliminary, and the **DP quality maximizer** is characterized as simple and potentially improvable with better reference-free filters [2509.10882].

Within the paper’s own framing, the main contributions are sixfold: a **new task formulation** as the first work to study full-length clinical note synthesis under DP; a **content/form decomposition** based on section-wise generation conditioned on medical terms; a **term-level privacy mechanism** via \(DPRP^*\); a **privacy-preserving generation pipeline** combining DP note generation with FastDP and parallel-composition guarantees; a **DP quality maximizer** based on post-processing selection via perplexity; and **strong empirical results** in fidelity, utility, and human preference relative to DP baselines [2509.10882].

In broader methodological terms, Term2Note occupies a specific niche within privacy-preserving clinical NLP: it is not simply a DP fine-tuning scheme for generic language generation, but a structured synthesis pipeline tailored to long clinical narratives and grounded in section-wise medical concepts [2509.10882]. A plausible implication is that its main significance lies in making long-form DP generation tractable by imposing document structure and medically salient intermediate representations. The paper further suggests practical relevance for institutions that need synthetic notes for training or evaluation when real notes cannot be shared, while still requiring formal privacy guarantees.

## 7. Conceptual interpretation within privacy-preserving clinical NLP

Term2Note can be understood as an instance of structured text synthesis in which latent document organisation is externalized into explicit intermediate objects: standardized sections, section-wise medical terms, and candidate-level quality scores [2509.10882]. The method’s key claim is not merely that synthetic notes can be made private, but that the clinical note synthesis problem becomes more manageable and more faithful when decomposed into section-specific subproblems.

The contrast with ICD-code-conditioned baselines is central. The paper argues that conditioning on high-level labels only can produce outputs that are too coarse or unrealistic, whereas extracted section-wise clinical terms provide finer-grained semantic anchors [2509.10882]. This suggests that the method’s gains arise from aligning the control signal with the internal discourse structure of clinical notes rather than from DP machinery alone.

At the same time, the paper does not present Term2Note as eliminating the privacy-utility tension. Instead, it presents a configuration in which **formal privacy**, **structural fidelity**, **semantic realism**, and **downstream utility** are jointly optimized more effectively than in the evaluated baselines [2509.10882]. The remaining issues—section ordering failures, clinical inconsistencies, and only preliminary privacy evaluation—indicate that the method is best viewed as a technically specific step in privacy-preserving clinical text generation rather than a complete resolution of the problem.

Source: https://www.emergentmind.com/topics/term2note