Feature Stability Index (FSI) Overview

• Feature Stability Index (FSI) is a metric that quantifies the reproducibility of feature selection outcomes under data perturbations in both predefined and learned-feature contexts.
• FSI is computed by evaluating recurrence-based selection frequencies or aligned subspace discrepancies, often using bootstrap resampling and Procrustes alignment.
• Empirical findings show that FSI enhances ensemble ranking accuracy and robustness while highlighting challenges like threshold sensitivity and increased computational cost.

Feature Stability Index (FSI) is best understood, in the arXiv literature represented here, not as a single canonical formula but as a generic label for quantitative summaries of how reproducibly features are recovered under perturbations of data or training. In classical feature selection, stability is defined as “the ability of a feature selection method to give a consistent set of features when the training data changes,” and is often computed on selected subsets induced by repeated resampling (Zhang et al., 2018). In learned-representation settings, where features are latent, distributed, and non-identifiable across retrainings, stability shifts from raw coordinate matching to the reproducibility of aligned low-dimensional subspaces and downstream selections (Sankaran, 2021). This suggests that any rigorous account of FSI must specify the object whose reproducibility is being measured: a selected subset, an aligned latent subspace, a feature-wise selection frequency, or an entire stability path.

1. Conceptual scope of feature stability

Feature stability concerns invariance of feature-selection outcomes under perturbation. In the subset-selection setting, the perturbation is typically a change in training samples, and the stable object is the selected feature subset. In the learned-feature setting, the perturbation is again a change in data or resampling, but the stable object may be a representation subspace rather than any individual coordinate, because latent coordinates from separate training runs need not be directly comparable (Zhang et al., 2018).

This distinction is fundamental. In EFSIS, stability is explicitly subset-based: ranked lists are truncated at a threshold $t$, and stability is computed over the resulting top-$t$ subsets rather than over the full rankings. By contrast, for non-rectangular or nontabular data, the learned coordinates themselves are not identified across runs, so direct feature-wise comparison is inadequate; the relevant targets become aligned representations, aligned coordinates after Procrustes rotation, and the reproducibility of downstream selections on those aligned coordinates (Sankaran, 2021).

A plausible implication is that “FSI” is not a property of features in the abstract. It is a property of a feature-generation-and-comparison protocol: perturb, relearn or reselect, align if necessary, and then summarize reproducibility.

2. Quantitative forms of FSI-like measures

The two most relevant arXiv sources define different stability objects and different numerical summaries. One is a recurrence-based subset score for predefined features; the other separates global subspace instability from feature-wise selection reproducibility for learned features.

Setting	Quantity	Meaning
Subset-based feature selection	$S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$	Average selection frequency over the union of selected features
Learned feature subspaces	$$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$	Mean squared discrepancy from a consensus representation after alignment
Learned aligned features	$$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$	Selection frequency of aligned feature $k$

In EFSIS, the stability quantity is

$S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$

where $F$ is the set of features appearing in at least one of the $M$ selected subsets, and $freq(f)$ is the number of those subsets containing feature $t$0. This is the average selection frequency over the union of all features ever selected. If the same subset is chosen in all runs, then $t$1; if many features appear only intermittently, $t$2 decreases (Zhang et al., 2018).

In the learned-feature framework, the global quantity is not a frequency score but an aligned subspace discrepancy: $t$3 Smaller $t$4 means greater stability after alignment. Feature-level reproducibility is then measured by

$t$5

and supplemented by stability-selection paths $t$6 over the lasso penalty $t$7 (Sankaran, 2021).

These quantities are not interchangeable. The EFSIS score is a subset-based recurrence index on predefined variables; $t$8 is a global instability measure on aligned representations; $t$9 is a feature-wise reproducibility frequency defined only after alignment.

3. Subset-based FSI in ensemble feature selection

EFSIS combines data perturbation and function perturbation, and its distinctive feature is that stability is used both as an evaluation criterion and as an internal aggregation weight (Zhang et al., 2018). Starting from dataset $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$0, the method generates $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$1 bootstrap datasets

$S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$2

each produced by drawing $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$3 samples from $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$4 with replacement, where $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$5 is the original sample size. For each ranker $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$6, feature ranking on each bootstrap dataset yields ranked lists

$S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$7

Stability is not computed on the full ranked lists. Instead, for each list $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$8, the top $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M}$9 features are taken, producing $$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$0 selected subsets for ranker $$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$1. The score $$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$2 is then computed over those subsets. What is being stabilized is therefore the consistency of selected top-$$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$3 subsets under sample perturbation, not full-rank agreement.

Aggregation proceeds in two levels. Within each ranker, ranks are aggregated across bootstrap samples by rank product: $$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$4 where $$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$5 is the rank of feature $$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$6 produced by ranker $$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$7 on bootstrap dataset $$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$8. Features are sorted by $$FSS_{\mathcal A}(\mathbf Z_1,\dots,\mathbf Z_B)=\frac{1}{B}\sum_{b=1}^B \|\bar{\mathbf Z}_b-\mathbf M\|_2^2$$9 to obtain the bootstrap-aggregated ranked list $$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$0. Across rankers, EFSIS uses the stability-weighted product

$$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$1

The paper states that $$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$2 is the weight “so that a more stable ranker is assigned a higher weight.” The exact printed exponent form is therefore part of the method and should be attributed as written (Zhang et al., 2018).

This makes EFSIS unusual among feature-stability studies. Stability is not merely reported after selection; it directly modulates the final ensemble ranking. For the basic function-perturbation baseline, the same cross-ranker rank-product is used without stability weights,

$$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$3

which the paper describes as equivalent to setting $$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$4.

The paper also distinguishes two operational contexts for stability. Inside EFSIS, $$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$5 is computed over the $$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$6 bootstrap-derived subsets for each individual ranker. In the benchmark evaluation, stability is separately computed over the 10 selected subsets obtained from 10-fold cross-validation. This distinction is important because the bootstrap-based score drives the algorithm, whereas the cross-validation-based score is used for reporting performance (Zhang et al., 2018).

4. FSI for learned features and non-identifiable representations

For non-rectangular data, the main obstacle is that the variables of interest are learned rather than given. The paper on learned-feature stability formulates a feature learner as

$$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$7

with learned feature vectors $$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$8. The study instantiates this with a VAE, a supervised CNN, and a random convolutional features model (Sankaran, 2021).

The central claim is that raw learned coordinates are not directly comparable across retrainings. The authors therefore separate subspace stability from selection stability. Data are first split into a feature-learning set $$SS_{\mathcal S}^k=\frac{1}{B}\sum_{b=1}^B \mathbf 1\{k\in \mathcal S(\bar{\mathbf Z}_b,\mathbf y)\}$$9 and an inference set $k$0. Bootstrap perturbations are applied only to $k$1: if $k$2 is a bootstrap resample of $k$3, then the $k$4-th learner is trained on $k$5, producing $k$6, and the resulting representation is evaluated on the full dataset: $k$7

Each $k$8 is then centered, scaled, and reduced from $k$9 dimensions to $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$0 dimensions using PCA or sparse components analysis, yielding $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$1. The reduced representations are jointly aligned by generalized Procrustes analysis. For two centered tables $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$2, ordinary Procrustes solves

$S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$3

For $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$4 matrices, generalized Procrustes solves

$S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$5

with alternating updates and consensus

$S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$6

This absorbs sign flips, permutations, and orthogonal rotations, and changes the target of inference from raw neurons to aligned subspaces (Sankaran, 2021).

After alignment, stability is measured at two levels. Global subspace instability is quantified by $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$7. Feature-level reproducibility is quantified by $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$8, where $S_n=\frac{1}{|F|}\sum_{f\in F}\frac{freq(f)}{M},$9 is a selection procedure on aligned features. The default selection mechanism is stability selection based on lasso, with selection paths $F$0 and thresholded selected set

$F$1

In this framework, an FSI-like scalar is not primary; the main objects are aligned subspace discrepancy, feature-wise selection frequency, and the full stability curve as a function of regularization (Sankaran, 2021).

5. Empirical findings and the stability–accuracy relation

The EFSIS experiments use six microarray cancer datasets—AML, CNS, DLBCL, Prostate, Leukemia, and ColonBreast—with four individual rankers: SAM, Information Gain, GeoDE, and ReliefF. Stability is evaluated under 10-fold cross-validation, using the same subset-consistency measure as in the algorithm, while predictive performance is measured by AUC with a linear SVM. The threshold $F$2 is varied from $F$3 to $F$4 of the original number of features, and EFSIS uses $F$5 bootstraps and $F$6 rankers (Zhang et al., 2018).

Several findings are especially relevant to any discussion of FSI. Stability is dataset-dependent for individual rankers. ReliefF is described as “a very unstable method with the lowest stability score across all datasets,” even where predictive performance is strong. Basic function perturbation is moderate in stability, whereas EFSIS “consistently improves the stability of basic function perturbation.” Using Wilcoxon signed-ranks tests, the paper reports that EFSIS has significantly higher stability than basic function perturbation in all 6 datasets. On accuracy, ensemble methods are more robust across datasets, and function perturbation and EFSIS are significantly worse than the best ranker in only 4 of 54 experiments. The paper therefore frames EFSIS as achieving both high predictive accuracy and high stability, while also noting that it is “more time-consuming and more computationally expensive” because each ranker is run on many bootstrap datasets (Zhang et al., 2018).

The learned-feature study reaches related but conceptually different conclusions. In simulation, the raw learned coordinates are distributed and do not correspond one-to-one with true latent variables, while canonical correlations decay quickly after a few aligned dimensions, supporting low-dimensional reduction before stability analysis. Subspaces can be quite stable under bootstrap perturbations when enough data are allocated to feature learning: visually, random convolutional features are most stable, VAE intermediate, and CNN least stable. Unsupervised learners are often more stable than supervised ones, although the supervised CNN can produce more frequently selected response-relevant features. The best practical compromise for the learning/inference split is an even split, $F$7, and PCA works better than sparse components analysis for stable, interpretable selection curves (Sankaran, 2021).

In the spatial proteomics case study on multiplexed ion beam imaging of triple negative breast cancer, learned-feature approaches outperform a baseline ridge regression on simple pixelwise cell-type composition features except for the RCF-SCA combination. The most important aligned dimensions are not necessarily the first principal or aligned dimensions: for example, CNN feature 4 can be more stably selected than feature 3, and analogous inversions appear for RCF and VAE. The aligned embedding plots also show heterogeneity of local stability: in the VAE, samples with higher tumor-to-immune ratio are much more stable than those with low ratio. A plausible implication is that FSI may vary not only across algorithms and perturbations but also across regions of the learned representation space (Sankaran, 2021).

6. Limitations, misconceptions, and terminological ambiguity

Several recurrent misconceptions are corrected by these papers. First, feature stability is not necessarily rank correlation. In EFSIS, the score is computed on top-$F$8 subsets rather than on full rankings, and the paper explicitly distinguishes its measure from Kendall or Spearman correlation, Jaccard similarity, the Kuncheva index, the Nogueira stability measure, and a named FSI (Zhang et al., 2018). Second, for learned representations, raw coordinate identities are not the correct stability target; the stable object is often an aligned low-dimensional subspace, and the feature-level quantity of interest is the reproducibility with which aligned dimensions are selected downstream (Sankaran, 2021).

The same sources also identify substantive limitations. The EFSIS subset score is threshold-sensitive because it depends on the chosen top-$F$9 truncation and is not adjusted for overlap expected by chance (Zhang et al., 2018). In learned-feature analysis, stability can be misleading if subspace reproducibility is conflated with predictive relevance: a representation may be stable but uninformative, or predictive yet unstable. Alignment can itself introduce artifacts, sparse components analysis can produce unstable and non-monotone selection paths, the perturbation scheme matters, and there is no universal threshold for declaring a feature “stable” (Sankaran, 2021).

A final source of ambiguity is purely terminological. In numerical PDE and computational mechanics, “FSI” almost always denotes fluid–structure interaction rather than feature stability. The arXiv papers on stable partitioned algorithms for incompressible flow with shells, elastic solids, and deforming beams analyze added-mass partitioned coupling, Robin interface conditions, normal-mode amplification factors, and stability without sub-iterations; they do not define feature-selection stability indices (Banks et al., 2013, Banks et al., 2013, Li et al., 2015). For that reason, any use of “Feature Stability Index” should be explicit about domain and object of measurement.

Taken together, the literature supports a precise but non-unified view of FSI. In predefined-feature settings, an FSI-like quantity can be a recurrence-based subset consistency score. In learned-representation settings, it is more naturally decomposed into subspace reproducibility, aligned feature-wise selection frequency, and stability paths over regularization. The common principle is reproducibility under perturbation; the principal variation lies in what counts as a feature and in how cross-run comparability is established.