MEDAL: Manifold Embedding Distillation via Autoencoder Learning

Abstract: Low-dimensional embeddings are widely used as visual summaries of high-dimensional data and to enable downstream scientific discoveries. Yet, popular nonlinear dimension reduction methods, such as t-SNE and UMAP, are often selected based on visual appeal alone and without rigorous quantitative validation. A major reason is that manifold embeddings typically do not provide an out-of-sample map nor an inverse back to the original feature space; this makes held-out validation, the gold standard in supervised learning, all but impossible. To address these challenges, we develop a novel framework, MEDAL (Manifold Embedding Distillation via Autoencoder Learning), which distills a fitted manifold embedding into a reusable encoder--decoder model. MEDAL trains a constrained autoencoder whose bottleneck exactly matches any teacher embedding while the decoder reconstructs the original input; this yields an explicit map for new samples, an approximate inverse, and a pointwise reconstruction-based measure of distortion in the manifold space. This converts static manifold embeddings into models that can be evaluated on held-out data, enabling quantitative validation including comparing different dimension reduction methods as well as hyperparameter tuning. Across multiple benchmark and scientific case studies, we show that MEDAL enables held-out validation to determine optimal manifold embeddings and hyperparameters, reveals biologically coherent regions that are difficult to preserve in two dimensional embeddings, and detects distribution shift when new samples are mapped into a fixed reference manifold. MEDAL provides a general validation wrapper to any existing dimension reduction technique that will improve the rigor and

• The paper introduces MEDAL, which distills any precomputed teacher embedding into a constrained autoencoder to provide out-of-sample validation and an interpretable inverse mapping.
• It employs a combined objective to enforce near-exact teacher matching and minimal reconstruction error, with empirical evidence showing orders of magnitude improvement in distillation loss.
• MEDAL enables precise hyperparameter tuning, local distortion analysis, and cross-method comparisons, offering actionable insights for evaluating manifold embeddings.

MEDAL: Principled Validation and Comparison of Manifold Embeddings via Distilled Autoencoders

Introduction: Problem Context and Motivation

High-dimensional data analysis workflows in genomics, neuroscience, and physical sciences frequently leverage nonlinear dimension reduction (DR) methods such as t-SNE and UMAP for visualization and downstream analysis. Despite the critical role of these embeddings in scientific inference, the selection of DR methods and their hyperparameters has predominantly relied on qualitative, visual inspection. This leads to fragile and potentially irreproducible scientific conclusions since such embeddings often lack out-of-sample extension and do not possess an interpretable inverse mapping to the input space. Existing validation tools are generally DR-specific or operate post-hoc, lacking the universal, rigorous evaluation protocols standard in supervised learning.

The MEDAL (Manifold Embedding Distillation via Autoencoder Learning) framework addresses these limitations by distilling any precomputed, arbitrary “teacher” embedding into a constrained autoencoder model, providing a reusable encoder–decoder (student) pair. This approach endows any DR embedding with an explicit out-of-sample map, an approximate inverse, and a quantitative, pointwise reconstruction-based measure of information preservation, thus enabling held-out validation, DR method comparison, hyperparameter tuning, and distortion localization.

MEDAL Framework: Distillation Objective and Protocol

The core protocol of MEDAL is to train a student autoencoder $(e_\theta, d_\phi)$ such that the encoder maps inputs to points that match pre-existing teacher embedding coordinates, while the decoder reconstructs the original high-dimensional input. The distillation process minimizes a combined objective: $$\min_{\theta, \phi} \; \lambda_d \mathcal{L}_\mathrm{dist}(\theta) + \mathcal{L}_\mathrm{rec}(\theta, \phi)$$ where $\mathcal{L}_\mathrm{dist}$ enforces near-exact bottleneck alignment with the teacher embedding and $\mathcal{L}_\mathrm{rec}$ measures input-space MSE reconstruction loss. The alignment penalty parameter $\lambda_d$ is chosen to drive the encoder to nearly perfect teacher recovery, with empirical ablations showing sharp improvement (several orders of magnitude) in distillation loss without materially increasing reconstruction error as $\lambda_d$ increases (Figure 1).

Figure 1: Effect of the distillation penalty $\lambda_d$ on tradeoff between teacher matching and input-space reconstruction error across datasets.

The framework further enforces a linear bottleneck in the encoder to allow for signed, real-valued teacher coordinates, with non-linear activations reserved for hidden layers. Provided sufficient network capacity—empirically, moderate-depth, moderate-width MLPs suffice—exact bottleneck matching is possible even for complex, nonlinear teachers, consistent with the universal approximation properties of ReLU networks.

Validation and Downstream Tasks Enabled by MEDAL

After distillation, MEDAL enables several critical DR workflows:

• (1) Hyperparameter Tuning: Embeddings generated under different hyperparameter settings (e.g., UMAP $n_\mathrm{neighbors}$, t-SNE perplexity) are distilled and compared by held-out reconstruction loss. This quantifies information loss for unseen data, obviating reliance on visual appeal.
• (2) Pointwise Distortion Analysis: Per-sample reconstruction error identifies local regions of the manifold where input-space structure is poorly preserved, facilitating biological or scientific interpretation of embedding distortion.
• (3) Distribution Shift Detection: Out-of-sample points from novel distributions or domains exhibit elevated reconstruction error when projected through the learned model, enabling detection and localization of distribution shift.
• (4) Cross-Method Comparison: Fitted embeddings from different DR algorithms are placed on an equal footing by distilling and evaluating their reconstruction scores using the same student architecture/protocol.

Empirical Assessment Across Scientific Datasets

1. MNIST Benchmark: Proof-of-Concept and DR Method Comparison

MEDAL’s efficacy is illustrated using the MNIST benchmark, showing that reconstruction-based validation selects UMAP $n_\mathrm{neighbors}$ values balancing local separation and global coherence. High error regions correspond to ambiguous, hard-to-reconstruct digits, mirroring regions where manifold compression discards semantically relevant variation.

Figure 2: MEDAL, when distilled from a PCA teacher, yields linear autoencoder performance identically matching PCA; with nonlinear students, competitive or superior reconstruction is achieved at moderate-to-high ranks.

Figure 3: Comparison of MEDAL with EMBEDR, scDEED, and neMDBD for t-SNE hyperparameter selection and local distortion diagnostics; only MEDAL enables out-of-sample validation.

2. Single-Cell RNA-seq: Hydra and Mouse Neocortex

Application to the Hydra single-cell atlas demonstrates that MEDAL-selected hyperparameters yield embeddings consistent with biological lineage distinctions and that high-error regions localize to cell populations with complex, high-variance transcriptomes (e.g., nematocytes, basal disk epithelial cells). Comparison against in-sample diagnostics highlights that only MEDAL’s held-out-based protocol prevents selection of hyperparameters at the boundary extremes (e.g., overly high or low perplexity) favored by in-sample metrics.

Figure 4: MEDAL-selected t-SNE perplexity preserves major cell classes and allows precise localization of high distortion across cell types in the mouse neocortex.

3. Subject-Level Distribution Shift in Primate Retina

On macaque retina, MEDAL detects out-of-distribution samples (cells from unseen subjects) via elevated reconstruction error when projecting these samples onto a reference manifold distilled from another subject’s data. The approach localizes subject mismatch to specific cell types (e.g., microglia, Müller glia, endothelial cells), indicating that reconstruction error is not uniform but reflects meaningful biological variation.

Figure 5: Embedding of out-of-distribution retina cells from a second subject onto a reference manifold reveals elevated, cell-type-specific reconstruction errors strongly concentrated in vascular and glial populations.

Ablations, Robustness, and Practical Considerations

Ablation studies confirm that near-zero distillation does not require excessive architectural complexity; moderate-depth MLPs achieve exact teacher matching for batch sizes and input dimensions typical of modern scientific datasets (Figures 13, 14, 15). The requirement for a linear bottleneck is confirmed by substantial error increases when nonlinear bottleneck activations are imposed.

Additional experiments demonstrate that reconstruction remains a directly interpretable, meaningful metric under strict bottleneck alignment. When distilled from a PCA teacher, linear MEDAL exactly reproduces PCA reconstruction curves; for nonlinear teachers, the nonlinear student closely tracks unconstrained autoencoder performance at higher ranks (Figure 2).

Assessment Versus Competing Diagnostics

Direct comparison with EMBEDR, scDEED, and neMDBD shows that competing diagnostics often select hyperparameters at regime boundaries, resulting in fragmented or over-compressed embeddings; their local distortion signals are less interpretable than the input-space errors provided by MEDAL. MEDAL also offers substantial computational advantages, completing full hyperparameter sweeps an order of magnitude faster than permutation- or null-based approaches.

Theoretical and Practical Implications

MEDAL reframes DR model evaluation as a quantitative, testable procedure, closing the gap between qualitative unsupervised workflow and the principled validation standard in supervised learning. The flexibility arising from its model-agnostic approach enables direct comparison of method and hyperparameter choices on a shared information-preservation scale.

Contradictory to prevailing practice, strong empirical evidence shows that visually optimal or most detailed embeddings are not those that best preserve high-dimensional information out of sample. Out-of-distribution and high-complexity biological subpopulations are quantitatively—and locally—flagged by elevated MEDAL reconstruction error, providing actionable targets for further investigation or model refinement.

Future Directions

Open avenues include (i) theoretical analysis of reconstruction error as a proxy for information loss, (ii) extension of MEDAL to advanced student architectures (e.g., transformers for multimodal data), and (iii) integration with downstream atlas mapping, spatial analysis, and multiomics workflows. Embedding validation as an explicit, quantitative modeling step aligns with broader trends toward statistical rigor and inference principles in unsupervised learning.

Conclusion

MEDAL delivers a robust, universal protocol for validating, comparing, and auditing manifold embeddings via autoencoder distillation. By transforming static visualizations into testable mappings with explicit reliability signals, it addresses the reproducibility and rigor crisis pervasive in modern scientific DR applications. The out-of-sample validation enabled by MEDAL constitutes a substantial methodological advance for both theoretical and pragmatic practice in unsupervised representation learning (2605.24244).