---
title: Supervised Locality Preserving Projection (SLPP)
url: https://www.emergentmind.com/topics/supervised-locality-preserving-projection-slpp
type: topic
---

# Supervised Locality Preserving Projection (SLPP)

Supervised Locality Preserving Projection (SLPP) is a subspace learning algorithm designed to find a discriminative, low-dimensional embedding that preserves local data structure while leveraging class label supervision. SLPP is grounded in graph-based dimensionality reduction and extends the classical locality preserving projections (LPP) by constructing the similarity graph using class labels, fully connecting all within-class pairs. This technique yields a closed-form transformation via a generalized eigenproblem, efficiently capturing both discriminative and geometric information for tasks such as domain adaptation, zero-shot recognition, and face identification [1903.10601][1311.1279].

## 1. Data Formulation and Problem Setup

SLPP operates on labeled datasets, specifically seeking a linear transformation that maps high-dimensional samples into a lower-dimensional subspace where intraclass distances are minimized. Let $X^{(s)}\in\mathbb{R}^{d\times n^s}$ denote $n^s$ labeled source samples and, in transfer learning scenarios, $X^{(tl)}\in\mathbb{R}^{d\times n^{tl}}$ the $n^{tl}$ labeled target samples. All available labeled data are concatenated into $X^l = [X^{(s)}, X^{(tl)}]\in\mathbb{R}^{d\times n^l}$, with $n^l = n^s + n^{tl}$. Each sample $x_i$ in $X^l$ is associated with a class label $y_i\in\{1,\ldots,C\}$.

The algorithm seeks a linear projection matrix $P\in\mathbb{R}^{d\times d^{sub}}$ ($d^{sub}\ll d$), such that in the projected space, discriminative and class-consistent locality structure is preserved [1903.10601][1311.1279].

## 2. Construction of the Supervised Similarity Graph

SLPP constructs a weighted adjacency graph $W\in\mathbb{R}^{n^l\times n^l}$ where the edge weight $W_{ij}=1$ iff samples $i$ and $j$ share the same class label ($y_i=y_j$), and $0$ otherwise. This fully connects same-class pairs and ignores inter-class pairs, focusing on capturing intra-class localities. The diagonal degree matrix $D$ is given by $D_{ii}=\sum_j W_{ij}$, and the unnormalized graph Laplacian is $L=D-W$.

Unlike unsupervised LPP, which often restricts connections to $k$-nearest neighbors with Gaussian heat-kernel weights, SLPP's adjacency reflects explicit class membership and thus introduces supervision directly into the learned subspace [1903.10601][1311.1279].

## 3. Objective Function and Optimization

SLPP optimizes a criterion minimizing projected distances between all same-class point pairs:

\[
\mathcal{J}(P) = \sum_{i,j=1}^{n^l}\|\,P^T x_i - P^T x_j\|^2\,W_{ij} = 2\,\mathrm{Tr}\bigl(P^T X^l L (X^l)^T P\bigr)
\]

To avoid degenerate solutions and promote global spread among classes, the problem is cast as a Rayleigh quotient with an added regularization parameter $\alpha > 0$:

\[
\max_P\ \frac{\mathrm{Tr}\left(P^T X^l D (X^l)^T P\right)}{\mathrm{Tr}\left(P^T [\,X^l L (X^l)^T + \alpha I_d\,] P\right)}
\]

- The numerator $\mathrm{Tr}\left(P^T X^l D (X^l)^T P\right)$ ensures that high-degree, same-class nodes (i.e., points from populous classes) spread out, enhancing discrimination.
- The denominator penalizes within-class local variation; $\alpha I_d$ serves as a regularizer, mitigating overfitting.
  
The stationary points of this quotient are obtained by solving the generalized eigenproblem:

\[
X^l D (X^l)^T p = \lambda \left(X^l L (X^l)^T + \alpha I_d\right) p
\]

The $d^{sub}$ eigenvectors associated with the largest eigenvalues are stacked to construct $P$ [1903.10601].

## 4. Algorithmic Implementation and Hyperparameter Considerations

The canonical SLPP workflow comprises the following steps:

1. (Optional) $\ell_2$-normalize each column of $X^l$.
2. Build the supervised adjacency $W_{ij} = 1[y_i = y_j]$. Compute the Laplacian $L$ and degree $D$.
3. Compute $M_1 = X^l D (X^l)^T$ and $M_2 = X^l L (X^l)^T + \alpha I_d$.
4. Solve $M_1 p = \lambda M_2 p$ for top $d^{sub}$ eigenvectors.
5. Form $P=[p_1,\dots,p_{d^{sub}}]$.

Key hyperparameters and their tuning guidance include:

- Subspace dimension $d^{sub}$: Cross-validated; $d^{sub}=128$ empirically stable.
- Regularization $\alpha$: Small positive value ($10^{-3}$ to $1$); increase if overfitting occurs.
- In unsupervised domain adaptation, $W$ can be based on $k$-NN ($k=5$–$10$) with a heat kernel; $k$ and bandwidth must be tuned [1903.10601].

## 5. SLPP in Domain Adaptation and Zero-Shot Learning

In domain adaptation, SLPP is used to learn a joint subspace that aligns both source and (if available) labeled target samples. In zero-shot or semi-supervised settings, only labeled target samples are included; in fully unsupervised adaptation, pseudo-labels are produced for the target and high-confidence points are added to $X^l$ using the CAPLS heuristic.

By iteratively updating pseudo-labels and re-solving for $P$, SLPP effectively places both source and target points of the same class close together in the subspace, resulting in domain-invariant and discriminative representations. Notably, even the baseline SLPP (without CAPLS) achieves competitive results on benchmarks, with CAPLS yielding a further 5–7 percentage points improvement [1903.10601].

## 6. Post-projection Classification and Empirical Results

After projection, features are normalized and mean-centered. Test samples are classified using nearest-class-mean in the SLPP subspace:

\[
\hat y = \arg\min_{c\in\{1...C\}} \|z - \bar z_c\|_2,\quad \bar z_c = \frac{1}{N_c}\sum_{i:y_i=c}z_i
\]

On benchmark datasets (Office31, Office-Home), SLPP combined with CAPLS achieves average accuracies of $\sim88.2\%$ and $\sim70.6\%$, respectively, outperforming several deep-adversarial techniques. SLPP is robust to increases in $d^{sub}$ above 64 and stable for iteration counts above 10. In generalized zero-shot adaptation, SLPP attains harmonic means ($H\approx68\%$) between known and unseen classes [1903.10601].

## 7. Extensions: Globality-Locality Preserving Projections (GLPP)

A principal extension is Globality-Locality Preserving Projection (GLPP), which adds preservation of between-class (global) structure by simultaneously minimizing within-class distances and between-class mean distances. GLPP introduces a class mean graph with adjacency $B_{rs}$ and constructs the Laplacian $K$, leading to a composite minimization:

\[
O_{\rm GLPP} = 2w^T (U K U^T + \beta X L X^T)w
\]

where $U$ is the matrix of class means and $\beta$ trades off the importance of global versus local terms. The resulting eigenproblem is unconstrained and solved directly for the smallest eigenvalues. Empirically, GLPP yields consistent performance boosts (1–5 points in controlled, >5 in uncontrolled datasets) over SLPP, particularly when class count is large. The implementation remains computationally efficient and stable, especially if preceded by PCA for high $m$ [1311.1279]. The recommended $\beta$ is in $[10^3, 10^5]$.

### Recognition Accuracy Comparison on Face Datasets

| Dataset              | PCA (%) | LDA (%) | LPP (%) | DLPP (%) | GLPP (%) |
|----------------------|---------|---------|---------|----------|----------|
| ORL (leave-one-out)  | 94.3    | 99.0    | 98.0    | 98.3     | 99.5     |
| Yale (leave-one-out) | 89.8    | 97.0    | 99.4    | 99.4     | 100.0    |
| FERET (leave-one-out)| 87.7    | 94.4    | 94.4    | 95.1     | 96.3     |
| LFW-a (top-1)        | 27.4    | 57.0    | 58.3    | 52.3     | 63.9     |

GLPP outperforms SLPP and other baselines, especially under challenging conditions (e.g., LFW-a) [1311.1279].

## References

- "Unifying Unsupervised Domain Adaptation and Zero-Shot Visual Recognition" [1903.10601]
- "Face Recognition via Globality-Locality Preserving Projections" [1311.1279]

Source: https://www.emergentmind.com/topics/supervised-locality-preserving-projection-slpp