---
title: Classical Multidimensional Scaling (CMDS)
url: https://www.emergentmind.com/topics/classical-multidimensional-scaling-cmds
type: topic
---

# Classical Multidimensional Scaling (CMDS)

Classical Multidimensional Scaling (CMDS) is a foundational Euclidean embedding technique that reconstructs the configuration of $n$ objects in a vector space based solely on their pairwise squared distances. The central algorithmic ingredient is the transformation of distance data into an inner product (Gram) matrix via double centering, followed by a spectral decomposition that yields coordinates for the embedded points. Recent developments have introduced a dual basis framework that represents the Gram matrix as a linear combination of rank-2 matrices corresponding to each object pair, providing both theoretical insight and practical tools for analyses involving incomplete or noisy data and explicit connections to metric constraint optimization.

## 1. Mathematical Foundations and the Classical CMDS Algorithm

Given a collection of objects with a (possibly incomplete) matrix of squared Euclidean distances, $D \in \mathbb{R}^{n \times n}$, CMDS forms the centered inner product (Gram) matrix via the double centering operation:
\[
X = -\frac{1}{2} J D J,
\]
where $J = I - \frac{1}{n} 11^\top$ is the centering projection. The spectral decomposition of $X$ yields the principal coordinates:
\[
X = V \Lambda V^\top,
\]
where $V = [v_1, \ldots, v_n]$ and $\Lambda = \text{diag}(\lambda_1, \ldots, \lambda_n)$. For rank $r$, the embedding is given by $Y = V_r \Lambda_r^{1/2}$.

Standard CMDS, while efficient and optimal in the strain metric for Euclidean-consistent $D$, does not natively facilitate sparse constraints, explicit local distance handling, or detailed spectral connection to related optimization problems (e.g., enforcing the triangle inequality for metric nearness).

## 2. Dual Basis Framework: Construction and Explicit Formulas

A major development is the construction of a dual basis tailored to the space of zero-centered, symmetric $n \times n$ matrices. For the unordered index set $I = \{(i, j) : 1 \leq i < j \leq n\}$, cardinality $L = n(n-1)/2$, one defines:
\[
w_{(i,j)} = \mathbf{e}_{i,i} + \mathbf{e}_{j,j} - \mathbf{e}_{i,j} - \mathbf{e}_{j,i},
\]
where $\mathbf{e}_{a,b}$ has a $1$ at position $(a, b)$, zeros elsewhere.

The dual basis matrices $\{v_{(i,j)}\}$ satisfy the property $\langle v_\alpha, w_\beta \rangle = \delta_\alpha^\beta$ (Kronecker delta). The paper provides a unified and explicit formula for all dual basis matrices:
\[
v_{(i,j)} = -\frac{1}{2}(a b^\top + b a^\top),
\]
with $a = \mathbf{e}_i - \frac{1}{n}\mathbf{1}$ and $b = \mathbf{e}_j - \frac{1}{n}\mathbf{1}$. Here, each $v_{(i,j)}$ is a symmetric, rank-2, zero-centered matrix, and the coordinate representation for any zero-centered Gram matrix $X$ can be expanded as
\[
X = \sum_{(i, j) \in I} D_{i,j}\, v_{(i,j)}.
\]
This recovers the classical double-centering formula and enables expressing $X$ as a linear expansion in terms of observed distances.

## 3. Spectrum of the Essential Inner Product Matrix

Define the Gramian matrix for the basis:
\[
H_{\alpha, \beta} = \langle w_\alpha, w_\beta \rangle,
\]
where, for $\alpha = (i,j)$ and $\beta = (k, l)$,
\[
H_{\alpha,\beta} =
\begin{cases}
4 & \alpha = \beta \\
1 & |\{i, j\} \cap \{k, l\}| = 1 \\
0 & \text{otherwise}.
\end{cases}
\]
$H$ is shown to be $4I_L + A(T)$, where $A(T)$ is the adjacency matrix of the triangular graph $T$ on the $n$ points.

The complete spectrum of $H$ is characterized:
- Eigenvalue $2$, multiplicity $L-n$
- Eigenvalue $n$, multiplicity $n-1$
- Eigenvalue $2n$, multiplicity $1$

This spectral structure governs the conditioning and invertibility of local expansions and clarifies the algebraic geometry underlying CMDS, as well as its links to spectral graph theory.

Notably, each dual basis matrix $v_{(i,j)}$ has a simple spectral structure: two nonzero eigenvalues, $\frac{1}{2}$ and $-\frac{1}{2} + \frac{1}{n}$, associated respectively to the vectors $a - b$ and $a + b$.

## 4. Stability and Applications to Noisy and Incomplete Distance Data

The dual basis approach provides an analytic mechanism for tracking the propagation of noise in the input distance matrix $D$ to the Gram matrix $X$, due to the explicit and linear dependence of $X$ on each $D_{i,j}$. Because each $v_{(i,j)}$ is explicitly known, perturbation bounds can be established for $X$ under arbitrary additive noise in $D$, with clear consequences for the stability of the reconstructed configuration.

Furthermore, because the expansion works for any subset $S \subset I$ of the distances, the dual basis method naturally extends to problems with missing data, sparse constraints, or partial observations—a regime where classical global eigendecomposition is not directly applicable.

## 5. Analytic Link to the Metric Nearness Problem

The metric nearness problem requires projecting an arbitrary dissimilarity matrix onto the convex cone of metric (distance) matrices, enforcing all triangle inequalities. The relevant constraint matrix $A$ encodes all triangle inequalities as linear inequalities.

The paper establishes that $A^\top A$ is, up to scaling and a shift, essentially the negative of the essential matrix $H$ from the dual basis framework:
\[
A^\top A = (3n-2)I - H.
\]
Consequently, the singular values of $A$ in the metric nearness literature are directly determined by the spectrum of $H$. This analytic link provides not only a theoretical foundation for empirical observations in metric projection algorithms but also a direct conduit for transferring advances in spectral CMDS analysis to constrained and regularized embedding problems.

## 6. Comparison to Classical Approach and Advantages of the Dual Basis Formulation

The standard CMDS framework relies exclusively on the double centering and eigendecomposition of $X$, viewing the embedding as a global operation. In contrast, the dual basis approach:
- Enables explicit linear expansion of $X$ in terms of the observed $D_{i,j}$ and analytically tractable basis elements $v_{(i,j)}$.
- Facilitates local analysis, inclusion of sparsity, and natural handling of missing or incomplete data.
- Supports direct computation of spectral quantities relevant for noise sensitivity and for embedding under partial constraints.
- Establishes a natural bridge between CMDS, distance geometry, and convex metric approximation problems.

| Aspect                      | Standard CMDS                        | Dual Basis Approach                      |
|-----------------------------|--------------------------------------|------------------------------------------|
| Gram matrix representation  | Double centering + eigendecomposition| Explicit sum over dual basis             |
| Dual basis vectors          | Not used                             | Uniform formula: $-\frac{1}{2}(ab^\top + ba^\top)$ |
| Spectrum analysis           | Focus on Gram matrix $X$             | Complete spectrum of matrix $H$          |
| Applicability to constraints| Not natural                          | Supports triangle inequalities, missing data |
| Metric nearness connection  | No explicit link                     | Direct analytic connection via $H$ and $A^\top A$   |

## 7. Implications and Future Directions

The dual basis approach offers enhanced interpretability for CMDS, analytical tractability for both full and partial data regimes, and an explicit connection to relevant quadratic forms in metric nearness problems. This methodology generalizes naturally to the analysis of structured incomplete data, constrained distance completion, and noisy geometric inference. The completed spectral analysis provides a foundation for further work on the geometry of embedding under linear constraints, stability to noise, and efficient convex optimization in metric spaces.

Source: https://www.emergentmind.com/topics/classical-multidimensional-scaling-cmds