---
title: Unsupervised Feature Selection Methods
url: https://www.emergentmind.com/topics/unsupervised-feature-selection-methods
type: topic
---

# Unsupervised Feature Selection Methods

Unsupervised feature selection methods aim to extract a subset of informative features from unlabeled high-dimensional data while preserving essential structure for downstream analysis such as clustering, classification, or visualization. These methods address the limitations posed by redundant, irrelevant, or noisy features and operate without supervision—a setting that refrains from using true label information. They span a wide landscape of principled approaches, including filter, wrapper, embedded, and deep learning paradigms, with mathematical underpinnings in spectral analysis, graph theory, subspace learning, sparsity regularization, and information geometry.

## 1. Theoretical Foundations and Motivations

Unsupervised feature selection tackles the problem of selecting a minimal set of features that retains the intrinsic data information required for learning tasks in the absence of class labels [1306.1326]. Unlike supervised approaches that leverage label correlation, unsupervised techniques must exploit alternative criteria such as manifold geometry, variance, local neighborhood structure, or self-expressiveness [2010.05454], [1912.05458]. The chief motivations are:

- **Dimensionality reduction**: Reducing computational and storage burden, and improving algorithmic robustness against the curse of dimensionality.
- **Model interpretability and generalization**: By discarding irrelevant or redundant features, models can generalize better and offer clearer insight into data sources.
- **Noise and outlier resilience**: Robust selection approaches mitigate sensitivity to data anomalies [2512.18720].
- **Structure preservation**: Methods aim to maintain relevant sample relationships and latent cluster structures [2112.07227], [2411.00270].

## 2. Core Methodologies, Models, and Criteria

Unsupervised feature selection methods can be categorized by their principles and mathematical structures:

**A. Filter Methods**

These compute feature scores based on intrinsic properties or relationships (such as variance, correlation, or graph Laplacian), independent of downstream learning:

- **Principal Component Analysis (PCA)** projects data into uncorrelated directions capturing maximal variance, reducing feature dimensionality but sacrificing original-variable interpretability [1306.1326].
- **Empirical Distribution Ranking (EDR)** orders features by statistics derived from their empirical distribution functions [1306.1326].
- **Compactness Score (CSUFS)** directly scores features by evaluating local compactness, though specific algorithms require direct access to the cited paper [2201.13194].
- **Markov Multi-step Feature Selection (MMFS)** uses multi-hop graph transition probabilities to capture both local and global data structures, offering both “negative” (structure-breaking) and “positive” (structure-preserving) selection rules [2005.14359].

**B. Graph- and Manifold-based Embedded Methods**

These yield feature importance via optimizing embedding or clustering objectives:

- **Laplacian Score**, **MCFS**, and **NDFS** select features maximizing the preservation of local manifold structures built from k-NN graphs [1306.1326].
- **Dual Manifold Re-ranking (DMRR)** integrates sample-sample, feature-feature, and sample-feature manifold affinity matrices to jointly update sample and feature importances with biconvex optimization [2410.20388].
- **Graph Filtering Self-Representation (GFASR)** leverages high-order graph filters (exp(-ηL)) for both smoothing and enforcing self-representation regularizers, combined with ℓ_{2,1}-norm feature sparsity [2411.00270].

**C. Sparsity-Regularized Subspace Learning**

These methods select features through subspace projection and group sparsity constraints:

- **Structured Sparsity with Adaptive Graph (JASFS)** enforces ℓ_{2,0}-norm on the transformation matrix and learns an adaptive similarity graph for robust feature selection with automatic determination of the number of features [2010.05454].
- **Nonnegative Orthogonal Constrained Minimization (NOCRM)** jointly embeds group-sparse regression and nonnegative spectral clustering with inexact ALM and PAM optimization and guaranteed KKT convergence [2403.16966].
- **Class Margin Optimization (UFCM)** incorporates maximum margin criterion (between-cluster scatter) and within-cluster K-means compactness with a nonconvex ℓ_{2,p} sparsity penalty [1506.01330].

**D. Block-Alternating and Bi-Level Frameworks**

- **Self-Paced Learning and Redundant Regularization (SPLR)** employs self-paced sample weighting, subspace learning, explicit manifold and feature redundancy regularizers, and a nonconvex ℓ_{2,1/2}-norm to enhance robustness [2112.07227].
- **Bi-Level Framework (BLUFS)** combines spectral clustering pseudo-label embedding with exact ℓ_{2,0} feature selection in a unified PAM algorithm [2505.20563].

**E. Kernel and Autoencoder-based Methods**

- **Kernel Alignment UFS (KAUFS/MKAUFS)** optimizes matrix factorization jointly with kernel and redundancy alignment, enabling both single and multiple kernel learning to capture nonlinear feature interactions [2403.14688].
- **Robust Autoencoder and Adaptive Graph Learning (RAEUFS)** couples a nonlinear autoencoder, robust subspace recovery, graph-regularized clustering, and group sparsity via alternating block minimization [2512.18720].
- **Autoencoder Feature Selection (AEFS)** (architecture and experiments require full paper access) employs a regression autoencoder and group-lasso for unsupervised selection.

**F. Group Structure Models**

- **GroupFS** discovers and sparsely selects latent feature groups via Laplacian smoothness on both sample and feature graphs, stochastic group gates (STG), and fully differentiable loss minimization [2511.09166].

**G. Subspace Clustering with Self-Expressiveness**

- **SCFS** integrates self-expressive subspace clustering (joint learning of adaptive similarity) and row-sparse feature regression in a nonconvex alternating framework [1912.05458].

**H. Hypergraph-Based Models**

- **Point-Weighting Hypergraph Feature Selection (HPWL)** constructs soft hypergraphs using data centroids, applies point- and hyperedge-weighting schemes, and optimizes local/global structure and low-rank correlations via block-coordinate descent [1808.08414].

## 3. Optimization Algorithms and Convergence

Optimization strategies have evolved to accommodate nonconvexity, combinatorial sparsity, and manifold constraints:

- **Alternating minimization** or block-coordinate frameworks are universal, updating projection matrices, sample/feature weights, and affinity graphs in cycles ([2112.07227], [2010.05454], [2411.00270], [2403.16966], [1912.05458]).
- **Augmented Lagrangian Method (ALM)** and **proximal alternating minimization (PAM)** are rigorously used with convergence guarantees to KKT points, ensuring stable and monotonic decrease of nonsmooth objectives ([2403.16966], [2505.20563]).
- **Accelerated Matrix Homotopy Iterative Hard-Thresholding (AMHIHT)** and coordinate descent are adopted for structured sparsity and ℓ_{2,0} constraints ([2010.05454], [2411.15197]).
- **Multiplicative update rules**, spectral clustering eigen decompositions, and ADMM are employed in kernel and autoencoder factorization models ([2403.14688], [2411.00270]).
- When group structure is unknown, fully differentiable frameworks with stochastic gates and Gumbel-softmax sampling allow group sparsity in continuous relaxation ([2511.09166]).

## 4. Evaluation Protocols and Empirical Benchmarks

Universal evaluation protocol involves feature ranking and selection followed by clustering or classification on benchmark datasets, measuring clustering accuracy (ACC), normalized mutual information (NMI), purity, and sometimes redundancy rate ([2512.18720], [2010.05454], [2410.20388], [1912.05458], [2505.20563], [1306.1326]):

| Paper (Method)      | Datasets (examples)     | Major Baselines                   | ACC/NMI Highlights                  |
|---------------------|------------------------|-----------------------------------|-------------------------------------|
| SPLR [2112.07227]   | USPS, Isolet, COIL20   | LS, MCFS, UDFS, RNE, SGFS         | Best ACC on 7/9 datasets            |
| JASFS [2010.05454]  | Brain, MNIST, Jaffe    | L-score, UDFS, RUFS, AUFS, UGFS   | Wins on 5/8 datasets in ACC/NMI     |
| DMRR [2410.20388]   | WARPAR10P, LUNG        | LapScore, MCFS, GRM, AGRM         | ACC ↑ by ~12-14% over filters       |
| SCFS [1912.05458]   | Lung, ORL, Isolet      | LS, UDFS, NDFS, LDSSL             | Best on every dataset in ACC        |
| NOCRM [2403.16966]  | lung, Isolet, COIL20   | LS, MCFS, UDPFS                   | Outperforms all baselines           |
| BLUFS [2505.20563]  | Isolet, COIL20, lung   | LapScore, MCFS, UDFS, SOGFS       | ↑ Clustering and classification     |
| RAEUFS [2512.18720] | lung, COIL20, USPS     | URAFS, NNSE                       | Highest ACC/NMI; robust to outliers |


## 5. Feature Subset Redundancy, Robustness, and Interpretability

Effective methods extend beyond selection to:

- **Redundancy minimization**: Explicit penalties for feature-feature similarity or inner-product regularization (e.g. KAUFS/MKAUFS [2403.14688], SPLR [2112.07227], GroupFS [2511.09166]).
- **Outlier robustness**: Use of ℓ₁ loss (RAEUFS [2512.18720]), self-paced sample weighting (SPLR [2112.07227]), entropy-regularized graphs (JASFS [2010.05454]).
- **Adaptive graph learning**: Re-assigns neighborhood structure iteratively, providing both global and local structure alignment (GFASR [2411.00270], HPWL [1808.08414], SCFS [1912.05458]).
- **Group interpretability**: GroupFS [2511.09166] yields spatially and semantically coherent feature groups for analysis.

## 6. Algorithmic Complexity and Scalability

Complexity varies with method and target scale:

- Most alternating minimization methods scale as O(d^3), O(nd^2), or O(n^2d) per iteration, depending on whether matrix inverses, eigen-decompositions, or graph-building operations dominate [2010.05454], [2403.16966], [2411.00270].
- Methods utilizing low-rank factorization or centroids (HPWL [1808.08414]) minimize computational overhead, reaching convergence in ≈2 outer iterations.
- For deep models (RAEUFS [2512.18720], AEFS [1710.08310]), per-epoch complexity depends on autoencoder depth and chosen optimization algorithms.
- Graph construction steps (GroupFS [2511.09166], DMRR [2410.20388]) can become prohibitive when n≫10⁴; random-projection or anchor-graph approximations are proposed for scalability.

## 7. Limitations and Future Directions

Recognized constraints include:

- **Parameter sensitivity**: Several hyperparameters require grid-search tuning for optimal performance, e.g. regularization weights, graph construction parameters.
- **Scalability**: Large n or d may require approximate or randomized methods, particularly for graph-based constructions.
- **Local minima and nonconvexity**: Nonconvex regularizations (ℓ₂,₀, ℓ_{2,1/2}) and deep learning models admit only local convergence guarantees.
- **Group discovery and dynamic selection**: Adaptive group selection remains a challenge; methods (GroupFS [2511.09166]) suggest directions for dynamic or context-sensitive grouping.
- **Integration with deep, multi-view, and semi-supervised models**: Kernel alignment, multi-task meta-learning, and hybrid models are emerging for improved structure capture and sample efficiency [2107.00816], [1710.08310].

Future research will likely focus on scalable, adaptive graph construction, deep manifold modeling, and unsupervised meta-learning paradigms [2107.00816]. Integration of interpretable grouping and dynamic selection mechanisms is anticipated to benefit applications in vision, genomics, and social science data analysis, where reliable structure must be inferred absent labels.

Source: https://www.emergentmind.com/topics/unsupervised-feature-selection-methods