GroupFS: Unsupervised Group Feature Selection

• GroupFS is a differentiable framework for unsupervised feature selection that jointly discovers latent feature groups without relying on predefined partitions.
• It leverages sample and feature graphs with Laplacian smoothness to ensure coherent group assignments and employs stochastic gating for group selection.
• Empirical results demonstrate that GroupFS improves clustering accuracy and noise reduction across diverse high-dimensional datasets.

GroupFS is an end-to-end, fully differentiable framework for unsupervised feature selection that jointly discovers latent feature groups and selects the most informative groups among them, without relying on fixed a priori groups or label supervision. It is designed for high-dimensional learning settings in which labels are unavailable and feature selection is expected to reduce noise, improve generalization, and enhance interpretability. The framework is motivated by the observation that many informative signals emerge from groups of related features rather than from isolated coordinates, including adjacent pixels, functionally connected brain regions, and correlated financial indicators (Lifshitz et al., 12 Nov 2025).

1. Problem setting and motivation

Unsupervised feature selection is essential for high-dimensional learning tasks where labels are not available. In this setting, the central difficulty is that most existing unsupervised FS methods evaluate features in isolation, even though informative signals often emerge from groups of related features. GroupFS addresses this mismatch by making group discovery part of the feature-selection procedure itself rather than assuming that group structure is known in advance (Lifshitz et al., 12 Nov 2025).

The motivation is explicitly domain-agnostic. Spatial structure, functional organization, and correlated measurements are treated as instances of the same underlying phenomenon: features “act together” and should be grouped and selected as units. Prior group-aware FS methods are described as typically relying on predefined partitions or label supervision, which limits applicability when group structure is unavailable, uncertain, or biased. GroupFS therefore targets a stricter problem formulation: unsupervised selection with latent, data-driven group structure (Lifshitz et al., 12 Nov 2025).

A plausible implication is that GroupFS repositions feature selection from a coordinate-wise ranking problem to a structured representation problem. In the formulation given in the paper, selection quality is tied not only to informativeness but also to the coherence of the discovered groups.

2. Framework architecture

At a high level, GroupFS takes a data matrix $X \in \mathbb{R}^{N \times d}$, constructs both a sample graph and a feature graph, learns soft feature-to-group assignments, and attaches differentiable stochastic gates to groups so that only a sparse subset remains active. The entire pipeline is trained jointly via gradient descent (Lifshitz et al., 12 Nov 2025).

The framework has three coupled design elements. First, a sample graph encodes relationships among rows of $X$, allowing the model to preserve structure on the sample manifold. Second, a feature graph encodes relationships among columns of $X$, allowing correlated or otherwise similar features to be assigned coherently. Third, group-wise stochastic gating turns group discovery into group selection, so that the output is not merely a clustering of features but a compact subset of informative groups.

Component	Formulation / Purpose
Sample Graph	Affinity as per $W_{ij}$, Laplacian smoothness via $\mathcal{L}_s$
Feature Graph	Correlations among features, Laplacian via $\mathcal{L}_f$
Grouping (Soft)	Gumbel-Softmax $M_{ij}$; temp anneals to 0 for discrete assignment
Group Selection	Group-wise stochastic gates $z_j$ aggregate to per-feature weights
Sparsity Reg.	$\mathcal{L}_{\mathrm{reg}}$, penalizes number/size of active groups
Optimization	Differentiable, end-to-end, via Adam

This architecture makes group discovery endogenous to the optimization problem. Instead of selecting features first and analyzing their relationships afterward, GroupFS uses the relationship structure to determine which groups should be retained.

3. Mathematical formulation

The sample graph is built using the self-tuning kernel

$$W_{ij} = \exp\left(-\frac{\|\mathbf{x}_i - \mathbf{x}_j\|_2^2}{\gamma_i \gamma_j}\right),$$

where $X$0 is the distance from $X$1 to its $X$2-th nearest neighbor. For both sample and feature graphs, the symmetric normalized Laplacian is

$X$3

The feature graph is constructed analogously, with features as nodes and edges derived from empirical correlations (Lifshitz et al., 12 Nov 2025).

Feature-to-group assignment is represented by a matrix $X$4, where $X$5 is the soft probability that feature $X$6 belongs to group $X$7. GroupFS uses Gumbel-Softmax relaxation: $X$8 where $X$9 are logits, $X$0, and $X$1 is a temperature parameter. This provides a differentiable path from soft assignments toward discrete assignments as annealing proceeds (Lifshitz et al., 12 Nov 2025).

Group selection is implemented through stochastic gates: $X$2 The effective feature weight is then

$X$3

and the masked data matrix is

$X$4

This means that a feature is retained insofar as it belongs to an active group, with activity determined stochastically but trained end-to-end (Lifshitz et al., 12 Nov 2025).

The objective combines sample-wise smoothness, feature-wise smoothness, and group sparsity: $X$5 The feature-side regularization uses a projected representation $X$6, where $X$7 is learned, and includes

$X$8

The group sparsity term is

$X$9

The paper characterizes this design as enforcing Laplacian smoothness on both feature and sample graphs while applying a group sparsity regularizer to learn a compact, structured representation (Lifshitz et al., 12 Nov 2025).

4. Optimization and selection dynamics

All terms in GroupFS are differentiable through reparameterization tricks for Gumbel-Softmax and stochastic gates. The model, including group assignments, gates, and the group projection, is trained jointly via stochastic optimization, with Adam given as an example (Lifshitz et al., 12 Nov 2025).

The optimization procedure is not merely a convenience for implementation. It is central to the method’s claim of being end-to-end. The learned assignments $W_{ij}$0 and the learned gates $W_{ij}$1 are not post hoc quantities; they are optimized simultaneously against the same objective. This couples grouping quality to downstream unsupervised utility.

The resulting behavior is structurally different from classical feature scoring methods. Sample-wise Laplacian smoothness promotes retention of features that preserve the sample manifold after masking. Feature-wise Laplacian smoothness encourages strongly related features to receive similar group assignments. The group sparsity term penalizes activating too many groups and, ideally, encourages groups themselves to remain compact. This suggests that GroupFS is designed to trade off manifold preservation, group coherence, and parsimony within a single differentiable objective (Lifshitz et al., 12 Nov 2025).

5. Empirical evaluation

The empirical study includes both synthetic and real-world datasets. On synthetic data, the paper uses a “two-moons” dataset extended to 20–24 features, with 2 informative groups consisting of highly correlated features per moon coordinate and additional independent noise features. GroupFS is reported to consistently identify the true groups and reject the noise, achieving perfect group similarity metrics, true positive rate, and low false discovery rates, and remaining robust to both intra-group correlation weakening and substantial additive noise (Lifshitz et al., 12 Nov 2025).

On real-world benchmarks, the evaluation covers 9 datasets from biomedical and vision domains, with two comparison settings. In the fixed feature budget scenario, all methods must select the same number of features as GroupFS. In the adaptive budget scenario, each method can select the number of features that yields the highest clustering score. The reported evaluation metrics are $W_{ij}$2-means clustering accuracy and ARI. GroupFS achieved the best or tied-for-best clustering accuracy on 6/9 real datasets in both fixed and adaptive budget scenarios (Lifshitz et al., 12 Nov 2025).

The baseline set includes LS, MCFS, CAE, DUFS, MGAGR, and CompFS. Within the reported comparisons, MGAGR is described as relying on predefined groups, and supervised CompFS mostly underperformed compared to GroupFS when applied in purely unsupervised settings. The paper also states that GroupFS consistently outperforms state-of-the-art unsupervised FS in clustering across nine benchmarks spanning images, tabular data, and biological datasets (Lifshitz et al., 12 Nov 2025).

These results position GroupFS as a method whose gains are tied specifically to latent group structure. The evidence presented in the paper is strongest in settings with noise, redundancy, and interpretable but non-obvious feature organization.

6. Interpretability, limitations, and naming

A prominent claim of GroupFS is interpretability through discovered groups rather than through isolated feature scores. On the MNIST digits dataset, the method is reported to find spatially contiguous, functionally meaningful pixel groups, including upper-left pixels forming the “loop” separating 8 from 3. On the Student Performance tabular dataset, the discovered groups reportedly include alcohol-related features, indicators of academic performance or motivation, and parent-related features. These examples are used to argue that the learned groups align with meaningful patterns in the underlying data (Lifshitz et al., 12 Nov 2025).

The limitations stated in the source are specific. GroupFS currently relies on Euclidean graphs for feature and sample neighborhoods, making it less suitable if the data manifold is highly non-Euclidean. It also does not model condition- or time-dependence of group relevance (Lifshitz et al., 12 Nov 2025). These limitations delimit the current scope of the method: it is a structured unsupervised FS framework, not a general mechanism for dynamic or non-Euclidean relational data.

The term “GroupFS” also requires terminological care. In distributed systems literature, “GFS, or GroupFS in some contexts,” refers to the Google File System, a distributed file system with a master-slave, chunk-based architecture (Pan et al., 2024). This suggests that the same string may denote unrelated concepts across fields. In machine learning, however, GroupFS denotes the unsupervised feature-selection framework described above (Lifshitz et al., 12 Nov 2025).