---
title: Uniform Manifold Approximation & Projection (UMAP)
url: https://www.emergentmind.com/topics/uniform-manifold-approximation-and-projection-umap
type: topic
---

# Uniform Manifold Approximation & Projection (UMAP)

Uniform Manifold Approximation and Projection (UMAP) is a nonlinear dimension reduction and manifold learning technique designed for scalable visualization and representation of high-dimensional data. UMAP constructs a topological representation by modeling local similarities with fuzzy simplicial sets, then seeks a low-dimensional embedding that preserves these structures by optimizing a cross-entropy objective. It provides tunable trade-offs between local and global structure preservation, computational efficiency, and the ability to adapt to a wide array of dissimilarity metrics and data modalities.

## 1. Theoretical Foundations: Manifold Learning and Fuzzy Simplicial Sets

UMAP’s mathematical formulation is rooted in Riemannian geometry and algebraic topology. The method assumes the high-dimensional data $\{x_1,\ldots,x_n\} \subset \mathbb{R}^d$ is sampled from a manifold $\mathcal{M}$ with an unknown local metric $g$. To approximate local geometry, UMAP rescale local distances by finding, for each $x_i$, a local connectivity parameter $\rho_i$ (the distance to the closest neighbor) and a local smoothness/bandwidth $\sigma_i$ satisfying:
$$
\sum_{j: x_j \in \mathrm{NN}_k(i)} \exp\bigl(-\max(0, d_{ij} - \rho_i)/\sigma_i\bigr) = \log_2(k)
$$
where $k$ is the neighborhood size, and $d_{ij}$ is the chosen high-dimensional metric (often Euclidean, but alternatives are supported).

From these neighborhoods, UMAP constructs a weighted, directed $k$-nearest neighbor graph, assigning directed fuzzy membership strength:
$$
\mu_{i|j} =
\begin{cases}
1, & d_{ij} \leq \rho_i \\
\exp\bigl(- (d_{ij} - \rho_i)/\sigma_i \bigr), & d_{ij} > \rho_i
\end{cases}
$$
Symmetrization via probabilistic fuzzy union yields undirected edge weights:
$$
\mu_{ij} = \mu_{i|j} + \mu_{j|i} - \mu_{i|j} \cdot \mu_{j|i}
$$
Interpreting this weighted graph as the 1-skeleton of a fuzzy simplicial set, UMAP encodes local manifold structure suitable for downstream optimization [2502.11036, 1802.03426].

## 2. Objective Function and Optimization Dynamics

The low-dimensional embedding $\{y_1, \ldots, y_n\} \subset \mathbb{R}^{d'}$ is constructed so that its own fuzzy simplicial set best matches the high-dimensional one. Pairwise similarities in the target space are modeled by:
$$
\nu_{ij} = \frac{1}{1 + a\|y_i - y_j\|^{2b}}
$$
with $a$, $b$ chosen by curve fitting to express a desired minimum inter-point distance (typical values: $a\approx1.929$, $b\approx0.7915$).

The optimization minimizes cross-entropy between the two sets of affinities:
$$
C = \sum_{i \neq j} \Bigl[ \mu_{ij}\log\frac{\mu_{ij}}{\nu_{ij}} + (1-\mu_{ij})\log\frac{1-\mu_{ij}}{1-\nu_{ij}} \Bigr]
$$
or, equivalently (up to constants),
$$
C = \sum_{i \neq j} \left[ \mu_{ij}\log \nu_{ij} + (1-\mu_{ij})\log(1-\nu_{ij}) \right]
$$
UMAP employs stochastic gradient descent (SGD) with negative sampling: for each positive edge (neighbor pair with $\mu_{ij}>0$), several negative pairs (non-neighbors) are sampled to enforce repulsion. This leads to a non-convex optimization landscape, requiring careful tuning of step sizes and annealing schedules to ensure convergence and stability [2503.09101].

## 3. Algorithmic Pipeline, Hyperparameters, and Complexity

The algorithmic workflow for UMAP proceeds as follows:

| Stage                                | Core Operations                                                     |
|--------------------------------------|---------------------------------------------------------------------|
| High-dimensional graph construction  | k-NN search, local entropy calibration, fuzzy set membership        |
| Symmetrization                       | Fuzzy union (probabilistic t-norm) over directed memberships        |
| Embedding initialization             | Random draw, spectral method, or PCA                                |
| SGD optimization (w/ neg. sampling)  | Attractive updates for neighbors, repulsive for random negatives    |

Critical hyperparameters:

- **n_neighbors (k):** Controls locality/globality. Small $k$ emphasizes fine structure; large $k$ encourages global topology but may blur local features.
- **min_dist:** Governs the minimum spacing in the embedding. Small values enable tight grouping; large values enforce even distribution.
- **metric:** Choice of input space geometry (e.g., Euclidean, cosine, Manhattan), essential for meaningful neighborhood identification [2512.09032, 2209.04933].

With approximate NN-descent, neighbor graph construction is $O(n\log n)$; optimization is $O(nk\, n_\mathrm{epochs})$ with $k \ll n$ and moderate $n_\mathrm{epochs}$ ($200$–$500$), yielding practical scalability to millions of samples [2502.11036, 2011.06103].

## 4. Variations, Extensions, and Computational Enhancements

UMAP is extensible along methodological, computational, and application-driven axes:

- **Variants with alternative input metrics:** Swapping the Euclidean metric for problem-adapted measures (e.g., elastic shape distances) produces embeddings that respect intrinsic data invariances, critical for shape analysis and time-series [2209.04933].
- **Parametric UMAP:** Introduces a deep network $f_\theta(x)$ for the mapping, enabling batched updates and instant embedding of new/unseen points ($z=f_\theta(x)$). This is crucial for integration into autoencoders, semi-supervised classifiers, and real-time pipelines [2009.12981, 2109.02508].
- **Approximate UMAP (aUMAP):** After standard UMAP training, new points are embedded via weighted averaging of k-nearest neighbors in the original space. This provides order-of-magnitude speedups for streaming and online applications, trading small fidelity losses for reduced latency [2404.04001].
- **Distributed and GPU-accelerated UMAP:** Count Sketch–based approaches (“Sketch-and-Scale”) minimize memory and communication in geo-distributed contexts, while highly optimized GPU implementations (RAPIDS cuML) exploit fused kernels, memory pooling, and atomic reductions for up to 100$\times$ speedups on large datasets [2008.00325, 2011.06103].
- **Preprocessing variants:** Data-domain decompositions (e.g., correlated clustering and projection for scRNA-seq) serve as pre-UMAP dimensionality reduction, shaping the effective geometry and cluster separation in the embedding [2306.13750].

## 5. Analysis of Embedding Forces and Practical Tuning

The embedding process in UMAP is characterized by explicit attractive and repulsive forces derived from the gradient of the cross-entropy loss:

Attractive updates drive neighboring points together, while repulsive updates force random pairs apart. The force shapes,
$$
f_a^U(\zeta) = -\frac{2ab \zeta^{2(b-1)}}{1 + a\zeta^{2b}} \;\;\;\text{(attraction)}
$$
$$
f_r^U(\zeta) = \frac{2b}{\zeta^{2b} (1 + a\zeta^{2b})} \;\;\;\text{(repulsion)}
$$
govern cluster formation, inter-cluster spacing, and global layout [2503.09101]. Notably, the attractive force can be expansive at very small distances, necessitating learning-rate annealing to prevent overshoot and oscillatory behavior. For enhanced consistency under random initialization, a linear tail can be added to the attraction to improve convergence to reproducible embeddings. Manipulating these force shapes—by adjusting $a$, $b$, or adding global attraction/repulsion—offers direct control over the compactness, separation, and stability of clusters in the output space.

## 6. Empirical Performance and Domain Applications

UMAP has established itself as a leading tool for visualization, exploratory analysis, and representation learning across multiple domains:

- **Visualization of complex manifolds:** UMAP recovers both distinct clusters (e.g., digits in MNIST, classes in single-cell RNA-seq) and global topological features with robustness to subsampling [1802.03426, 2109.02508].
- **Quantitative improvements:** On classification and clustering tasks, UMAP often matches or exceeds t-SNE in local neighborhood preservation, while providing superior global structure retention and significantly faster runtimes. In scRNA-seq, CCP-UMAP improves ARI by ~19% over standard UMAP, while in astrophysical color–redshift mapping, UMAP-based regression yields lower outlier rates and improved fidelity relative to SOM grids [2306.13750, 2512.09032].
- **Scalability:** Leveraging ANN-Descent for neighbor search and negative sampling optimization, UMAP scales to millions of data points (e.g., SDSS stars, hyperspectral imaging) with cost-effective memory/communication usage, especially with sketching or GPU implementations [2008.00325, 2011.06103].
- **Real-time/online adaptation:** aUMAP and parametric UMAP enable high-rate online embedding suitable for streaming data and model introspection in dynamic settings (e.g., BCI signals) [2404.04001].

## 7. Strengths, Limitations, and Method Selection

UMAP’s core strengths include:

- Scalability via $O(n \log n)$ construction/optimization, with tuning options for target fidelity, cluster compactness, and global topology [2502.11036].
- Flexibility in metric choice, accommodating problem-driven geometry (e.g., shape-invariant, elastic, or custom distances) [2209.04933].
- Expressive embeddings that can be incorporated as regularizers in downstream deep-learning pipelines (autoencoders, semi-supervised classifiers) [2009.12981].

Limitations:

- Absence of an analytic inverse transform; mapping embedded points back to the original space is non-trivial [2502.11036].
- Sensitivity to hyperparameter settings; improper calibration of $k$ or min_dist can fragment or oversmooth the manifold.
- Non-convexity of the loss; stochasticity in initialization and optimization may lead to variability in embeddings.
- For settings where interpretability via explicit loadings or projection axes is required, or pure linearity is sufficient, PCA/KPCA may be more appropriate.

A practical workflow often leverages UMAP for rapid manifold discovery and initial visualization, followed by clustering or quantitative analysis in the low-dimensional space.

---

In summary, UMAP provides a mathematically rigorous, scalable, and versatile framework for nonlinear dimensionality reduction. By faithfully approximating manifold geometry via fuzzy simplicial sets and optimizing a cross-entropy objective, it achieves high-quality embeddings with rich structure preservation, broad applicability across domains, and adaptability to contemporary computational requirements [2502.11036, 2512.09032, 2404.04001, 2009.12981, 2306.13750].

Source: https://www.emergentmind.com/topics/uniform-manifold-approximation-and-projection-umap