---
title: Supervised Multi-D Scaling (SMDS)
url: https://www.emergentmind.com/topics/supervised-multi-dimensional-scaling-smds
type: topic
---

# Supervised Multi-D Scaling (SMDS)

Supervised Multi-Dimensional Scaling (SMDS) denotes a family of multidimensional-scaling methods in which the target low-dimensional geometry is constrained by side information rather than inferred solely from an unsupervised dissimilarity matrix. In recent arXiv literature, the acronym spans several distinct but structurally related usages: a localization framework based on complex or quaternion edge kernels with anchor supervision and heterogeneous range–angle measurements; a confirmatory ordination framework in which a two-dimensional embedding is constrained to preserve a label-derived pseudo-\(F\)-ratio; and a linear probing method for language-model activations in which a learned projector matches pairwise distances induced by label-defined manifold hypotheses [2509.01223], [2308.00354], [2510.01025]. The common theme is metric control by external structure, but the object being embedded, the form of supervision, and the optimization machinery differ substantially across domains.

## 1. Terminological scope and nomenclature

A recurrent source of ambiguity is that the acronym **SMDS** is not used uniformly. In wireless localization, “SMDS” commonly refers to **Super MDS** or **Super Multidimensional Scaling**, a kernel-based extension of MDS introduced for localization from heterogeneous measurements and known anchors. In microbiome ordination and language-model interpretability, the same acronym is used in the more literal sense of **supervised** or **confirmatory** MDS, where labels define the geometry to be recovered [2509.01223], [2308.00354], [2510.01025].

| Lineage | Supervision source | Embedded object |
|---|---|---|
| Localization SMDS | Anchor positions; known inter-node structure; ranges and angles | Edge vectors and edge kernels |
| \(F\)-informed MDS | Group labels through a pseudo-\(F\)-ratio | Low-dimensional ordination \(\mathbf{z}\) |
| LM-manifold SMDS | Label-derived ideal distances \(d(y_i,y_j)\) | Linear projector \(W\) and projected activations |

This multiplicity of usage matters because the phrase “supervised MDS” can refer either to classical side-information-driven dimensionality reduction or to the localization-specific “Super MDS” tradition. A plausible implication is that comparisons across papers require attention not only to the loss function but also to what counts as supervision: anchors, rigid-body conformation, class labels, or externally specified test statistics.

## 2. Shared mathematical structure and departure from classical MDS

Classical MDS is the common starting point. In the unified MDS framework, one seeks an embedding \(X=\{x_1,\dots,x_n\}\) in a target space \(T\) minimizing
\[
C(X,D)=\sum_i\sum_j Err\big(f(x_i,x_j)-d_{ij}\big),
\]
where \(f\) is the target-space distance and \(Err\) measures discrepancy [1003.0529]. In Euclidean metric MDS for graph drawing, the corresponding stress is
\[
\sigma(X)=\sum_{i<j} w_{ij}(\|X_i-X_j\|-d_{ij})^2,
\]
and spherical MDS replaces Euclidean distance by spherical geodesic distance while preserving the stress-minimization structure [2209.00191].

SMDS variants depart from this template in three principal ways. First, they often replace unsupervised distances by **label-derived ideal distances**, as in the LM-manifold setting, where the objective is
\[
\mathcal{L}=\sum_{i<j}\left(\|W(x_i-x_j)\|^2-d(y_i,y_j)^2\right)^2.
\]
Second, they may augment classical stress with a **confirmatory term**, as in \(F\)-informed MDS, where raw stress is combined with a penalty enforcing consistency between the embedding and a pseudo-\(F\)-ratio computed from group labels [2308.00354], [2510.01025]. Third, in localization SMDS the basic variables are not node coordinates but **edge vectors**, and the central object is a Gram-like edge kernel rather than a double-centered distance matrix [2509.01223].

The role of supervision is therefore heterogeneous. In one branch, supervision fixes unknown geometry through anchors and rigid-body structure; in another, it calibrates the embedding to a multivariate test statistic; in a third, it specifies the manifold hypothesis itself. This suggests that “SMDS” is better understood as a design principle—pairwise geometry constrained by external structure—than as a single algorithm.

## 3. Complex-edge SMDS for localization and rigid-body estimation

In the localization lineage, SMDS is formulated in terms of **complex edge vectors** in 2D. For \(T=M+N\) nodes, with anchors and targets represented in complex form, each unordered pair \(p=(i,j)\) yields an edge
\[
v_p=(a_{x_j}-a_{x_i})+j(b_{x_j}-b_{x_i})=d_p(\cos\theta_p+j\sin\theta_p),
\]
where \(d_p\) is Euclidean distance and \(\theta_p\) is the bearing. Stacking all edges gives \(\mathbf{v}\), linearly related to node positions by \(\mathbf{v}=\mathbf{C}\mathbf{x}\). The associated complex edge kernel is
\[
\boldsymbol{\mathcal{K}}=\mathbf{v}^*\mathbf{v}^\top,
\]
which is rank-1 in the noiseless case because it is an outer product of \(\mathbf{v}\) with itself [2509.01223].

For rigid body localization, the method exploits two forms of exact structure. Anchor–anchor edges \(\mathbf{v}_{\text{AA}}\) are known from anchor coordinates, and target–target edges \(\mathbf{v}_{\text{TT}}\) are known from the rigid-body conformation matrix \(\mathbf{C}\). The only unknown block is anchor–target edges \(\mathbf{v}_{\text{AT}}\). Rather than forming the full \(P\times P\) kernel, the method uses only the kernel minor
\[
\begin{bmatrix}
\mathcal{K}_1\\
\mathcal{K}_3\\
\mathcal{K}_4^\top
\end{bmatrix}
=
\begin{bmatrix}
\mathbf{v}_{\text{AA}}^*\\
\mathbf{v}_{\text{AT}}^*\\
\mathbf{v}_{\text{TT}}^*
\end{bmatrix}
\mathbf{v}_{\text{AT}}^\top,
\]
which is informative for estimating the unknown anchor–target block.

The algorithm proceeds in two stages. The first stage performs SMDS-based localization: it constructs known edge subvectors, initializes \(\mathbf{v}_{\text{AT}}\) from cross-kernel terms, and then applies an iterative “Turbo” refinement,
\[
\hat{\mathbf{v}}_{\text{AT}}^{(n+1)}
=
\frac{
\begin{bmatrix}
\mathcal{K}_1^\top & \mathcal{K}_3^\top & \mathcal{K}_4
\end{bmatrix}
\mathbf{w}^{(n)}
}{
\|\mathbf{w}^{(n)}\|^2
},
\]
with \(\mathbf{w}^{(n)}=[\mathbf{v}_{\text{AA}},\hat{\mathbf{v}}_{\text{AT}}^{(n)},\mathbf{v}_{\text{TT}}]^\top\). Node coordinates are then recovered by a pseudo-inverse or constrained inversion of the linear edge-to-node map. The second stage estimates rigid-body rotation and translation by a weighted Procrustes fit between estimated global landmark positions and the known body-frame conformation, with
\[
\mathbf{Q}=\mathbf{V}\,\operatorname{diag}\!\Big([1,\dots,1,\det(\mathbf{V}\mathbf{U}^\top)]^\top\Big)\mathbf{U}^\top,
\qquad
\mathbf{t}=\bar{\mathbf{s}}-\mathbf{Q}\bar{\mathbf{c}}.
\]

This formulation differs from classical metric MDS in three explicit respects: it uses complex edges incorporating distance and angle, it works directly with the edge kernel \(\boldsymbol{\mathcal{K}}=\mathbf{v}^*\mathbf{v}^\top\), and it uses strong supervision from anchor–anchor and rigid-body target–target geometry. Simulation results in the rigid-body setting show that full SMDS using distance and angle measurements attains performance very close to the CRLB across a wide range of range-error \(\sigma\), while a pure MDS-based rigid-body method performs worst among the reported baselines [2509.01223].

## 4. Quaternion-domain SMDS and rank-1 3D Gram edge kernels

In 3D localization, recent work reformulates SMDS in the quaternion domain. A 3D node coordinate \(\bm{x}_n=[a_n\;b_n\;c_n]^\top\) is mapped to
\[
\chi_n=a_n+\mathbf{i}b_n+\mathbf{j}c_n+\mathbf{k}\cdot 0,
\]
and an edge vector becomes the quaternion
\[
\nu_m=\grave{a}_m+\mathbf{i}\grave{b}_m+\mathbf{j}\grave{c}_m+\mathbf{k}\cdot 0.
\]
All edges are stacked as \(\bm{\nu}=\bm{C}\bm{\chi}\in\mathbb{H}^{M\times 1}\) [2504.17890], [2507.17645].

The key construction is the quaternion Gram Edge Kernel
\[
\bm{K}=\bm{\nu}\bm{\nu}^{\mathsf H}.
\]
Unlike the real-domain GEK, which has rank 3 in 3D, the quaternion-domain GEK is rank 1 because it is the outer product of a single quaternion vector with itself. Each entry \(\nu_m\nu_p^*\) simultaneously encodes the 3D inner product \(d_md_p\cos\alpha_{mp}\) and projected cross-product terms on the \(xy\), \(xz\), and \(yz\) planes, so the scalar part carries cosine information while the \(\mathbf{i}\), \(\mathbf{j}\), and \(\mathbf{k}\) parts carry sine-like angular information.

With noisy measurements, QD-SMDS performs quaternion SVD and retains only the dominant singular component,
\[
\hat{\bm{\nu}}=\sqrt{\lambda}\,\bm{u},
\]
where \(\lambda\) is the largest singular value and \(\bm{u}\) the corresponding singular vector. The resulting estimated edge vector is then converted back to a real edge matrix and inverted, using anchors, to recover node coordinates. The stated rationale is that rank-1 truncation maximizes the denoising effect because all non-dominant singular values are pure noise in the ideal model [2504.17890].

A later variant, QD-MRC-SMDS, removes the need for SVD by exploiting the block structure of the quaternion GEK. Partitioning the edge vector into anchor–anchor and anchor–target components yields a block matrix
\[
\bm{K}_{\mathrm q}=
\begin{bmatrix}
\bm{K}_1 & \bm{K}_2\\
\bm{K}_2^{\mathsf H} & \bm{K}_3
\end{bmatrix},
\]
from which the anchor–target block can be estimated directly by matrix multiplications in the quaternion domain. An iterative version further uses \(\bm{K}_3\) in a power-iteration-like refinement and is reported to converge essentially in one iteration in practice [2507.17645].

In simulations, quaternion-domain methods improve localization accuracy relative to the original real-domain SMDS, especially under substantial measurement errors. The SVD-free variant achieves comparable localization accuracy without requiring SVD. The papers also report that the advantage of QD-SMDS is stronger when angular noise is large and, in the missing-data setting, when low-rank completion benefits from the rank-1 or rank-2 quaternion/complex structure [2504.17890], [2507.17645].

## 5. Confirmatory and \(F\)-statistic-driven SMDS in ordination

A distinct branch of supervised MDS appears in microbiome ordination. Here the goal is not localization but a 2D configuration \(\mathbf{z}\in\mathbb{R}^{N\times 2}\) that preserves both pairwise sample dissimilarities and the group structure captured by a PERMANOVA-style pseudo-\(F\)-ratio. For binary labels \(y_i\in\{0,1\}\), the pseudo-\(F\)-ratio is
\[
F=
\frac{\sum_{i,j} d_{ij}^2 - 2\sum_{i,j}\{y_i=y_j\}d_{ij}^2}
{2\sum_{i,j}\{y_i=y_j\}d_{ij}^2}\cdot (N-2).
\]
The embedding objective augments classical raw stress with an \(F\)-informed confirmatory term:
\[
O(\mathbf{z})
=
\frac12 \sum_{i,j} \big(d_{ij}-\|\mathbf{z}_i-\mathbf{z}_j\|_2\big)^2
+
\lambda\cdot
\frac12\left|
\sum_{i,j}\Big[1-2\epsilon_{ij}\Big(1+\frac{f_{\mathbf{z}}(F)}{N-2}\Big)\Big]
\|\mathbf{z}_i-\mathbf{z}_j\|_2^2
\right|.
\]
When \(\lambda=0\), the method reduces exactly to classical MDS [2308.00354].

A central technical device is the mapping \(f_{\mathbf{z}}\), obtained by computing permutation distributions of pseudo-\(F\) values in the original distance matrix and in the current embedding, sorting the two distributions, and fitting a LOESS regression from one to the other. This lets the method constrain the 2D configuration so that its empirical PERMANOVA \(p\)-value aligns with the original high-dimensional test.

Optimization is performed by a Majorize–Minimize scheme analogous to SMACOF, starting from a classical MDS initialization and iteratively updating each point \(\mathbf{z}_k\) with a label-dependent quadratic surrogate. The method is computationally heavier than standard MDS because the permutation-based mapping is recomputed during the MM iterations.

Within the paper’s taxonomy, this method is explicitly a **confirmatory** and **supervised** MDS variant rather than a discriminative one. It is contrasted with Witten’s supervised MDS, which uses a label-ordering term, whereas the \(F\)-informed method aims instead to preserve the same multivariate hypothesis test outcome in low dimension. Empirically, the paper reports that the method is robust to hyperparameter selection, maintains statistical significance throughout the ordination process, and preserves local and global structures at levels comparable to state-of-the-art methods. It is also designed to be “statistically honest”: in the reported examples, the 2D PERMANOVA \(p\)-values remain aligned with those from the original data rather than exaggerating group separation [2308.00354].

## 6. SMDS as manifold discovery in language-model representations

In recent interpretability work, SMDS is used to discover latent feature manifolds in LM activations under explicit geometric hypotheses. Let \(X\in\mathbb{R}^{n\times d}\) be hidden states and \(y=(y_1,\dots,y_n)\) labels for the feature of interest. The assumption is that there exists a low-dimensional linear subspace such that projected distances approximate label-defined ideal distances \(d(y_i,y_j)\). The formal objective is
\[
\mathcal{L}
=
\sum_{i<j}
\left(
\|W(x_i-x_j)\|^2-d(y_i,y_j)^2
\right)^2,
\]
with \(W\in\mathbb{R}^{m\times d}\) and \(m\ll d\) [2510.01025].

The procedure has two stages. First, a manifold hypothesis is defined entirely by the label-distance function \(d\). Representative examples include linear,
\[
d(y_i,y_j)=|y_i-y_j|,
\]
log-linear,
\[
d(y_i,y_j)=|\log y_i-\log y_j|,
\]
circular,
\[
d(y_i,y_j)=2\sin\!\left(\pi\min(\delta_{ij},1-\delta_{ij})\right),
\qquad
\delta_{ij}=|y_i-y_j|,
\]
and cluster,
\[
d(y_i,y_j)=
\begin{cases}
0 & \text{if } y_i=y_j,\\
1 & \text{otherwise}.
\end{cases}
\]
Classical MDS is then applied to the label-distance matrix \(D_{ij}=d(y_i,y_j)^2\), producing an ideal manifold embedding \(Y\). Second, a linear regression aligns centered activations to centered manifold coordinates,
\[
W=Y_c^\top X_c\left(X_c^\top X_c+\alpha I\right)^{-1},
\]
with \(\alpha=0.1\) in the reported experiments.

Generalization is evaluated by a normalized stress on held-out data,
\[
S=
\frac{
\sum_{i<j}
\left(
\|W\hat{x}_i-W\hat{x}_j\|-\hat{d}_{ij}
\right)^2
}{
\sum_{i<j}\hat{d}_{ij}^2
}.
\]
The method is used as a model-selection device: candidate manifold hypotheses are compared by test stress, and the lowest-stress geometry is taken as the best explanation of the feature representation.

Applied to temporal reasoning, the paper reports that dates are consistently circular, times of day are circular or semicircular, durations and recurrence intervals are log-linear or log-semicircular, notable twentieth-century years are semicircular, and categorical tasks such as seasons or time-of-day phases are cluster-like. These geometries are reported as stable across model families and sizes. The paper further argues for a functional role of the discovered manifolds: perturbing hidden states with noise specifically aligned to the SMDS subspace substantially degrades task accuracy, whereas perturbing a random low-dimensional subspace of the same dimension does not for \(m<100\). Among models with nontrivial task performance, manifold quality also correlates positively with downstream accuracy [2510.01025].

## 7. Methodological limits, misconceptions, and relation to broader MDS research

The first misconception is that SMDS names a single canonical method. The literature instead contains multiple frameworks sharing a supervised geometric premise but differing in data type, supervision source, and optimization. In localization, supervision comes from anchors and known rigid-body structure; in microbiome ordination, from labels through a pseudo-\(F\)-ratio; in LM analysis, from label-defined ideal distances [2509.01223], [2308.00354], [2510.01025].

The second misconception is that supervision necessarily implies class discrimination. The \(F\)-informed microbiome method is explicitly described as **confirmatory but not directly discriminative**, and the LM-manifold method is closer to a metric-aware probe than to a classifier. Conversely, localization SMDS may be supervised without labels at all: anchor coordinates and structural priors provide the supervision.

Each branch also has distinct limitations. The rigid-body localization paper assumes the necessary cross measurements for the kernel blocks are available, and missing-kernel handling is discussed in related work but not implemented there. The quaternion localization papers require rich angular information and, in the full QSVD formulation, numerical routines for quaternion SVD; the SVD-free variant addresses part of that computational burden [2509.01223], [2507.17645]. The \(F\)-informed method is currently formulated for binary labels and is computationally expensive because it repeatedly estimates permutation distributions and fits LOESS mappings [2308.00354]. The LM-manifold approach depends on having well-defined labels and a plausible library of manifold hypotheses, and its projector is linear, so strongly nonlinear feature geometries may not be fully captured [2510.01025].

Broader MDS research supplies a useful backdrop. A unified algorithmic framework shows that many MDS variants can be treated through a modular per-point optimization subroutine over general costs and target geometries, including spherical ones [1003.0529]. Spherical graph drawing similarly generalizes stress minimization by replacing Euclidean distances with spherical geodesic distances and notes that pairwise weights can be chosen according to application importance [2209.00191]. This suggests, though the cited papers are unsupervised, that supervised extensions can often be understood as modifications of the discrepancy term, the target metric, or the pairwise weighting scheme.

Taken together, these developments show that SMDS is best understood as a broad family of metric-learning-by-geometry methods. What unifies them is the use of side information to prescribe or constrain pairwise structure; what differentiates them is whether that structure is expressed as a low-rank edge kernel, a confirmatory statistical constraint, or an ideal manifold in label space.

Source: https://www.emergentmind.com/topics/supervised-multi-dimensional-scaling-smds