Wass-to-Data: OT in Data Transformation

• Wass-to-Data is a framework that uses Wasserstein geometry to compare, generate, and restructure data by aligning observed patterns with confirmed-correct examples.
• It applies to diverse tasks such as detecting hallucinations in neural machine translation and synthesizing clinically relevant text from label semantics.
• Quantitative analyses reveal that this approach enhances detection accuracy, supports privacy-preserving data generation, and enables direct valuation via optimization dual variables.

“Wass-to-Data” appears in recent arXiv literature in more than one closely related sense. In "Layer-Resolved Optimal Transport for Hallucination Detection in NMT and Abstractive Summarization" it denotes an optimal-transport detector that measures how close a sentence’s cross-attention geometry is to the attention patterns of confirmed-correct translations (Onyshchuk et al., 11 Jun 2026). In "Two Directions for Clinical Data Generation with LLMs: Data-to-Label and Label-to-Data" it is framed as a practical way to use LLMs not only to assign labels to clinical text, but also to create new labeled clinical text that can improve downstream modeling (Li et al., 2023). This suggests a broader research motif: Wasserstein geometry, or data-generation procedures organized around label semantics, is used to compare against, generate, valuate, or restructure data itself.

1. Canonical optimal-transport formulation

In its most explicit optimal-transport sense, Wass-to-Data (WTD) is an unsupervised detector built on cross-attention distributions in sequence generation. For decoder layer $\ell$ and decoding step $t$, the source-side attention distribution is the head-averaged cross-attention

$$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$

and Wasserstein-1 distance is computed on token positions with ground metric $|x-y|$. On a 1D token grid, the paper computes this exactly via the CDF formula rather than with Sinkhorn approximation. WTD differs from Wass-to-Unif (WTU) in its reference: WTU compares attention to a uniform distribution over source positions, whereas WTD compares attention to confirmed-correct translations (Onyshchuk et al., 11 Jun 2026).

Operationally, WTD retrieves $k=4$ nearest neighbors from a reference set of confirmed-correct sentences, with a length proximity filter $\delta=0.1$, and computes the mean Wasserstein-1 distance to their step-averaged attention distributions. The paper gives the operational definition

$$\text{WTD}(x) = \frac{1}{k}\sum_{i=1}^{k} W_1\!\left(\bar{\pi}(x), \bar{\pi}(x_i^{\text{correct}})\right),$$

with $k=4$, subject to the length filter. By contrast, WTU is

$$s^{(\ell)}_{\text{WTU}} = \frac{1}{T}\sum_{t=1}^{T} W_1\!\left(\pi^{(\ell,t)}, \mathbf{u}\right),$$

where $\mathbf{u} = (1/S,\ldots,1/S)^\top$.

The difference is methodological rather than cosmetic. WTU is a concentration detector: it asks whether cross-attention is diffuse or sharply concentrated relative to a flat baseline. WTD is a data-referenced detector: it asks whether attention resembles the empirical manifold of correct routing behavior. The paper explicitly characterizes them as complementary detectors specialized across hallucination types.

Detector	Reference	Sensitivity
WTU	Uniform source distribution	Absolute concentration
WTD	Confirmed-correct translations	Similarity to correct patterns

The reported aggregate AUROCs make the complementarity concrete. For full-unsupport hallucinations, WTU reaches 0.937 while WTD reaches 0.801. For strong-unsupport, WTD reaches 0.770 versus 0.629 for WTU. For repetitions, WTD reaches 0.790 versus 0.568 for WTU. A common misconception is therefore that WTD is merely another concentration score; the paper states the opposite by defining it against real correct examples rather than against a uniform baseline.

2. Layer specialization, decoding dynamics, and principled limits

The NMT study is conducted on the Fairseq DE$t$0EN hallucination corpus with $t$1 examples, including 324 hallucinated outputs and 2,882 confirmed-correct translations, while excluding 208 sentences with other error labels (Onyshchuk et al., 11 Jun 2026). The model has six decoder layers, plus the decoder entry layer referred to as L0 in the plots. Pairwise Wasserstein analysis reveals three regimes: L0 is transitional, L1–L4 form a tight cluster, especially L2–L4, and L5 is structurally isolated and most concentrated. Mean WTU concentration rises broadly from L1: 0.21, through L0: 0.25 and L4: 0.26, to L5: 0.33, with the sharpest jump between L4 and L5.

Hallucinated translations differ from confirmed-correct translations in four reported summary statistics: higher mean concentration, 0.296 versus 0.249; lower step-to-step OT, 0.082 versus 0.124; lower standard deviation of concentration, 0.090 versus 0.104; and lower mean layer OT, 0.055 versus 0.059. All differences are significant with $t$2 by Mann-Whitney U. The interpretation given is that hallucinated decoding is more static and more locked onto fixed source positions.

Layer-resolved detectability is sharply nonuniform. For full-unsupport hallucinations, WTU peaks at L2 with AUROC 0.946, remains above 0.93 on L1–L4, and drops to 0.750 at L5. For strong-unsupport, the maximum per-layer AUROC is only 0.672; for repetitions, only 0.667. L5 becomes anti-predictive for the harder types, with 0.461 for strong-unsupport and 0.374 for repetitions. The paper concludes that the most informative layers are the middle layers, especially L2–L4, and that the final layer is not uniformly useful.

The same paper identifies a two-phase decoding dynamic in correct translations. During roughly the first 25% of the generated sequence, attention shifts rapidly between source positions and concentration is low; during the remaining 75%, attention becomes more focused and step-to-step shifts decay. Hallucinated translations lack this exploratory attention phase and instead begin in a statically concentrated state from the first decoding step. This is central to the paper’s claim that hallucination is not merely a late-stage failure.

Transfer to abstractive summarization establishes a principled boundary. On AggreFact, with 558 CNN/DailyMail and 558 XSum examples, the unsupervised OT detector on T5-base achieves 57.2% and 57.6% balanced accuracy, respectively, above the 50.0% random baseline but substantially below MiniCheck-Flan-T5-L at 69.9% and 74.3%. The paper argues that this is not just an empirical shortfall: unlike NMT hallucinations, unfaithful summaries can attend correctly to source tokens while misrepresenting their content. Because the OT detector is built on cross-attention geometry, it is principled for source disengagement and fundamentally limited when the failure mode occurs downstream of attention.

3. Clinical data generation: from labels to synthetic notes

A second use of the term is developed for clinical text mining in "Two Directions for Clinical Data Generation with LLMs: Data-to-Label and Label-to-Data" (Li et al., 2023). The motivating task is detection of Alzheimer’s disease-related signs and symptoms from electronic health records, where labeled data are scarce, expensive, sensitive, and difficult to annotate. The paper defines nine AD-related categories in unstructured notes: cognitive impairment, notice/concern by others, requires assistance, physiological changes, cognitive assessment, cognitive intervention/therapy, diagnostic tests, coping strategy, and neuropsychiatric symptoms. This taxonomy guides both annotation and generation.

The paper distinguishes two synthetic-data directions. In data-to-label, the model takes text as input and outputs a label; this is the silver path. In label-to-data, the model takes a label definition or instruction as input and outputs text that exemplifies that label; this is the bronze path. The distinction is methodological: data-to-label relies on existing real text and attempts to assign labels to it, whereas label-to-data starts from label semantics and asks the model to create new text satisfying those semantics. The appendix prompt templates make the distinction concrete. Prompt 1 asks the LLM to annotate provided text and output JSON with "sentence" and "class". Prompt 2 asks it to check an annotation and explain the reason, outputting "sentence", "class", "decision", and "reason". Prompt 3 asks it to “Generate a clinical note and annotate the text based on the provided annotation guideline.”

Dataset	Size/source	Construction
Gold	16,194 sentences; 5,112 longitudinal EHR notes of 76 AD patients	Human-annotated
Silver	16,069 sentences from MIMIC-III discharge summaries	Llama 65B labels real text
Bronze	16,047 generated sentences	GPT-4 generates and annotates notes

The gold data were annotated by two medical professionals under physician supervision, with Cohen’s $t$3. The silver data were labeled by Llama 65B from guideline plus sentence, followed by a second prompt asking whether the annotation is correct and why. The bronze data were generated by GPT-4 using only the annotation guidelines. The paper explicitly states that the generated notes contain “no Protected Health Information (PHI) or (Personally Identifiable Information) PII.”

All three datasets are tokenized, lowercased, deduplicated, and split 80/10/10. Bronze is more balanced than gold and silver; gold is heavily skewed toward cognitive impairment, while categories such as notice/concern by others and coping strategy are relatively rare. Mean sentence lengths are reported as 17.60 tokens (SD 12.69) for gold, 19.60 (SD 15.44) for silver, and 16.58 (SD 4.69) for bronze. The classification system fine-tunes BERT, RoBERTa, and ClinicalBERT and combines them in a voting ensemble. Evaluation uses accuracy, precision, recall, and F1 on the human-annotated gold test set for both binary classification and nine-way multi-class classification.

The paper reports that synthetic data improve performance over the gold-only baseline. In binary classification, accuracy rises from 0.90 with gold only to 0.93 with bronze, 0.91 with silver, and 0.94 with bronze+silver; the positive-class F1 rises from 0.74 to 0.78, 0.75, and 0.80, respectively, with bronze+silver yielding a statistically significant gain in accuracy. In multi-class classification, accuracy rises from 0.68 with gold only to 0.73 with bronze, 0.69 with silver, and 0.72 with bronze+silver. Bronze alone gives the largest overall accuracy improvement, while bronze+silver yields especially large gains on minority classes: coping strategy F1 rises from 0.44 to 0.58, notice/concern by others from 0.38 to 0.46, and physiological changes from 0.64 to 0.78 with bronze alone.

The paper emphasizes that the improvement is not just from extra data volume. Silver is about the same size as gold, yet it helps less than bronze in several settings. Human experts judged a random sample of 100 silver and 100 bronze examples and found annotation accuracy of about 55% for silver and about 85% for bronze. Silver errors were mainly due to over-inference and failure to handle negation. At the same time, the privacy claim is presented as a tradeoff rather than as perfect realism: bronze is free of PHI/PII and publicly shareable, but generated data may not fully reflect the complexity of real clinical documentation and have smaller length variance than the real gold notes.

4. Data quality, valuation, and the economics of Wasserstein uncertainty

Another precise Wasserstein-to-data logic appears in "Data Valuation from Data-Driven Optimization" (Mieth et al., 2023). Here the central move is not generation or retrieval of data patterns, but valuation of data quality in stochastic optimization. Data quality is defined as how accurately the true distribution $t$4 of an uncertain variable can be estimated from a dataset $t$5, and the paper uses the $t$6-Wasserstein distance to formalize this:

$t$7

Because the true distance is unknown, the paper uses a confidence upper bound

$t$8

so that small $t$9 denotes high-quality data and large $$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$0 poor-quality data.

The main modeling contribution is a multi-source Wasserstein ambiguity set with one radius per feature or data source:

$$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$1

This extends standard Wasserstein DRO by assigning heterogeneous quality budgets across sources rather than a single ball around one joint empirical distribution.

The paper’s distinctive claim is that the marginal value of data quality can be read directly from the optimization dual variables. In the optimal power flow application, the sensitivity of the optimal objective to data quality is

$$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$2

where $$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$3 is the direct value of improving data quality in the expected-cost term and $$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$4 is the indirect value through the security constraint. This replaces repeated leave-one-out or Shapley-style recomputation with a direct optimization-based valuation principle.

The case study uses a modified IEEE 5-bus system with two uncertain wind injections and varies $$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$5. When $$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$6, both $$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$7 and $$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$8 are zero, the model ignores the data, reserves are sized conservatively, and the system cost is highest. As $$\pi^{(\ell,t)} = \frac{1}{H}\sum_{h=1}^{H} \alpha^{(h,\ell,t)} \in \Delta^{S-1},$$9 decreases, the optimizer trusts the data more, reserve requirements fall, and total cost decreases substantially. The paper also reports that out-of-sample tests with 1000 samples keep violation probabilities below the 5% target whenever $|x-y|$0. In this formulation, “Wass-to-Data” denotes a move from Wasserstein ambiguity radii to the operational and economic value of the datasets themselves.

5. Metric rewriting, dimensionality reduction, and distribution alignment

Several adjacent papers use Wasserstein geometry to transform the structure of data rather than to score hallucinations or value information. "The Wasserstein transform" replaces each point by a local probability distribution and defines a new pseudo-metric

$|x-y|$1

thereby turning a dataset into a new dataset whose geometry reflects local density and neighborhood structure (Mémoli et al., 2018). The paper emphasizes enhancement and denoising, connects the construction to mean shift, proves stability under perturbations, and shows that iterative application can separate blob-and-chain structures and preserve a noisy circle better than mean shift with the same truncation kernel and scale.

"Wasserstein Discriminant Analysis" applies regularized Wasserstein distances between empirical class distributions in a supervised linear dimensionality-reduction problem (Flamary et al., 2016). Its central criterion

$|x-y|$2

replaces classical scatter matrices with entropy-regularized OT geometry. The paper states that this allows WDA to capture both global interactions at distribution scale and local interactions at sample scale, and reports strong performance on multimodal toy data, MNIST, Caltech deep features, and UCI benchmarks.

"Wassmap: Wasserstein Isometric Mapping for Image Manifold Learning" treats images as probability measures and uses pairwise Wasserstein distances to produce a low-dimensional approximately isometric embedding (Hamm et al., 2022). It proves exact recovery for some image manifolds generated by translations or dilations of a fixed generating measure, extends the argument to discrete measures through a functional-to-discrete bridge, and reports good embeddings compared with Euclidean MDS, Isomap, and Diffusion Maps.

A more recent distribution-alignment formulation appears in "Spectral Efficiency-Aware Codebook Design for Task-Oriented Semantic Communications" (Zhang et al., 6 Aug 2025). There, Wasserstein distance regularizes the codeword activation distribution so that it better matches a desirable channel input distribution:

$|x-y|$3

The proposed WS-DC scheme combines task loss with a Wasserstein term against a hybrid target distribution

$|x-y|$4

and the paper reports improvements in both classification accuracy and codebook efficiency on CIFAR-10. Across these works, a plausible implication is that “Wass-to-Data” can denote not only comparison to data exemplars, but also the rewriting of data geometry or the transport of learned representations toward target data distributions.

6. Non-parallel, heterogeneous, and private data regimes

The term also has adjacent relevance in settings where data are heterogeneous, weakly aligned, or private. "Learning from Multiple Sources for Data-to-Text and Text-to-Data" studies D2T and T2D jointly in a multi-source, non-parallel setting, using a variational autoencoder built on T5 with disentangled style and deterministic content variables, cycle training via back-translation, denoising, and MMD regularization (Duong et al., 2023). The paper explicitly states that if “Wass-to-Data” means converting loosely matched or weakly aligned web text and structured sources into data representations, then its model provides a template: one shared system across formats, dual generation directions, cycle consistency, deterministic content, latent style, and learning from non-parallel corpora. The main caution stated there is that structured sources are linearized into triples, so richer schemas would require a more expressive representation or decoder.

Privacy-sensitive Wasserstein computation is addressed by "Private Wasserstein Distance" (Li et al., 2024). TriangleWad introduces a defense dataset or virtual agent and forms two local interpolating measures, $|x-y|$5 and $|x-y|$6, from private datasets $|x-y|$7 and $|x-y|$8 to the same defense anchor. Its core approximation is

$|x-y|$9

with an error bound

$k=4$0

The workflow is one-shot: choose a defense dataset, compute local interpolants on each side, compare interpolants, and rescale. The paper reports that TriangleWad is about 20× faster than FedWad on average, has almost no approximation gap in many settings with a better defense choice, and yields interpolants that look like noise rather than revealing image classes or meaningful words. At the same time, the paper does not present it as absolute cryptographic privacy; the stated limitation is that the security argument is heuristic/theoretical rather than a formal cryptographic protocol.

Taken together, these adjacent directions indicate that Wass-to-Data is not a single closed technique. In the literature summarized here, it names or motivates a family of moves from Wasserstein or optimal-transport structure toward data-centered objectives: detecting deviations from correct attention manifolds, synthesizing labeled clinical text from label semantics, attaching economic value to data quality through DRO dual variables, rewriting the intrinsic geometry of datasets, aligning latent distributions to target operating conditions, and comparing private datasets without revealing raw samples.