---
title: Feature Stability Overview
url: https://www.emergentmind.com/topics/feature-stability
type: topic
---

# Feature Stability Overview

Searching arXiv for recent and foundational papers on feature stability across feature selection, attribution, latent spaces, radiomics, and related settings.
Feature stability denotes the degree to which a feature representation, feature subset, feature ranking, or feature attribution remains invariant under perturbations of data, training realizations, algorithmic randomness, or measurement conditions. Across the literature, the term is used in several closely related senses: robustness of extracted motifs in biological sequences [1206.4822], reproducibility of selected predictors under repeated runs or resampling [2005.12483], robustness of radiomic descriptors to segmentation or acquisition variability [1910.05693], invariance of latent spaces to training perturbations [2402.11404], and consistency of post-hoc explanations under input perturbations when model predictions are preserved [2604.02532]. The common concern is reproducibility: if small perturbations induce large changes in features or explanations, downstream interpretation, scientific inference, and deployment reliability are compromised.

## 1. Conceptual scope and problem formulations

Feature stability is not a single formalism but a family of related robustness notions. In classical feature selection, it asks whether repeated executions of a selector return similar subsets or rankings when data are perturbed or algorithmic randomness changes. One formulation considers a score matrix \(S=[s_{ij}]\) over repeated runs and a corresponding rank matrix \(R=[r_{ij}]\), then studies the variability of feature ranks across runs [2005.12483]. Another formulation represents repeated selections by a binary matrix \(Z\), where \(Z_{i,f}=1\) if feature \(f\) is selected on replicate \(i\), and defines a stability index from the dispersion of the column-wise selection frequencies [2012.00001].

A second usage concerns learned representations. In latent-space studies, stability is defined as the invariance of latent spaces to minor data, training realizations, and parameter perturbations, and is decomposed into sample stability, structural stability, and inferential stability [2402.11404]. In work on learned features for non-rectangular data, stability is separated into feature subspace stability, which measures variability of aligned learned representations, and feature selection stability, which measures how often aligned features are selected downstream [2102.10388].

A third usage concerns domain-specific descriptors. In radiomics, stability refers to the robustness of extracted quantitative imaging features with respect to segmentation variability or acquisition variation [1910.05693], [2210.02759]. In biological sequence classification, stability was introduced as a property of generated motifs to evaluate the robustness of motif extraction methods under input perturbation, emphasizing both the ability to reveal changes in input data and the ability to target interesting motifs [1206.4822].

A fourth usage concerns interpretability. For post-hoc attributions, feature-attribution stability asks whether explanations remain consistent when the input is perturbed but the model prediction is held fixed [2604.02532]. Related certification work distinguishes hard stability, which requires preservation for every perturbation in a specified set, from soft stability, which quantifies the probability that prediction consistency is maintained over random perturbations [2504.13787]. This suggests that “feature stability” functions as a cross-cutting criterion for robustness of both features and feature-based explanations.

## 2. Stability metrics and mathematical characterizations

The literature employs several metric families, each tied to a specific output type. Rank-based metrics are prominent when outputs are ordered feature lists. The instability index proposed for comparing MDA, LIME, and SHAP is defined over the top \(k\) features as
\[
I(k)=\frac{1}{k}\sum_{j=1}^{k}\mathrm{Var}\bigl(r_{1j},r_{2j},\dots,r_{nj}\bigr),
\]
where features are ordered by average rank \(\bar r_j\) and larger values indicate less stable top-\(k\) rankings [2005.12483]. The appeal of this construction is that ranks are invariant to scaling of importance scores and align with “select top \(k\)” practice [2005.12483].

Overlap-based metrics are standard for subsets. One study quantifies stability of ensemble feature selectors by the average pair-wise Jaccard similarity among selected subsets:
\[
J=\frac{2}{U(U-1)}\sum_{1\le i<j\le U}\frac{|s_i\cap s_j|}{|s_i\cup s_j|},
\]
with \(J=1\) for identical subsets and \(J=0\) for disjoint ones [2108.01485]. In microbiome feature selection, the Nogueira-Brown stability index is used:
\[
\hat\Phi(Z)=1-\frac{(1/p)\sum_{f=1}^{p}\sigma_f^2}{(\bar k/p)(1-\bar k/p)},
\]
where \(\sigma_f^2=(M/(M-1))\hat p_f(1-\hat p_f)\), \(\hat p_f\) is the empirical selection frequency, and \(\bar k\) is the average subset size [2012.00001]. By construction, \(\hat\Phi\in[0,1]\), equals \(1\) iff all subsets coincide, and has expected value zero under random feature picking [2012.00001].

Information-theoretic metrics generalize across full rankings, partial rankings, and subsets. A Jensen–Shannon-based stability score maps each output list \(L^k\) to a probability vector \(p^k\), computes
\[
D_{JS}(p^1,\dots,p^K)=H\Bigl(\frac{1}{K}\sum_{k=1}^{K}p^k\Bigr)-\frac{1}{K}\sum_{k=1}^{K}H(p^k),
\]
and normalizes it as
\[
S_{JS}=1-\frac{D_{JS}(p^1,\dots,p^K)}{D^*_{JS}},
\]
so that \(S_{JS}=1\) for identical outputs and \(S_{JS}=0\) under the designated random baseline [2402.05295]. Because the construction places more probability mass on higher-ranked features, disagreements near the top contribute more heavily [2402.05295].

Adjusted measures address exchangeable or highly similar features. For data sets with similar features, unadjusted measures can penalize replacements among nearly identical predictors. The SMA family introduces chance-corrected measures that incorporate feature similarity and enforce strict upper-bound control, with SMA-Count recommended as a default [2009.12075]. This suggests that the meaning of stability depends not only on overlap but also on whether substitutable features should be regarded as equivalent.

For attribution stability, the Feature Attribution Stability Suite uses three complementary metrics on attribution pairs \((A(x),A(x'))\): Structural Similarity Index, Spearman rank correlation, and top-\(k\) Jaccard overlap [2604.02532]. The same work emphasizes prediction-invariance filtering, so explanation fragility is not conflated with model prediction changes [2604.02532]. Certification work on attributions instead defines soft stability as
\[
\tau_r(f,x,\alpha)=P_{\alpha'\sim \mathrm{Uniform}(\Delta_r(\alpha))}[f(x\odot\alpha')\simeq f(x\odot\alpha)],
\]
with \(\Delta_r(\alpha)\) the set of admissible perturbations, thereby making stability itself a probabilistic quantity [2504.13787].

For radiomics, stability is often quantified via coefficient of variation and intraclass correlation coefficient. One study based on probabilistic segmentations computes \(CV=\sigma_F/\mu_F\) and
\[
ICC=\frac{BMS-WMS}{BMS+(\ell-1)WMS},
\]
where \(\ell=25\) segmentations per tumor [1910.05693]. Another radiomics study based on simulated CT acquisitions instead uses a Wilcoxon rank-sum criterion: a feature is deemed stable across two groups if \(W<1\), and its overall stability score is the percentage of the \(28\) group-pair tests for which that criterion holds [2210.02759].

## 3. Perturbation models and evaluation protocols

The perturbation model determines what kind of stability is being measured. In feature-selection studies, perturbations usually arise from resampling, random splits, bootstraps, or intrinsic randomness of the selector. The microbiome study estimates stability by generating \(M\) bootstrap replicates, rerunning each selector on each replicate, and computing \(\hat\Phi(Z)\) from the resulting binary selection matrix [2012.00001]. The rank-based instability study varies the random seed across repeated executions of MDA, LIME, and SHAP on random forests, with \(n_{\text{repeat}}=1,100,1000\) tested and a plateau observed by \(n=100\) [2005.12483].

Ensemble feature-selection methods treat perturbations as a design component rather than merely an evaluation device. EFSIS combines data perturbation via bootstrap rank-product aggregation with function perturbation across multiple rankers, weighting each ranker by a stability score derived from its bootstrap behavior [1811.07939]. Cluster Stability Selection modifies classical stability selection for settings with highly correlated proxy variables by aggregating at the cluster level, so that selection probability is defined for clusters rather than isolated features [2201.00494]. “2D Stability Selection” introduces a second perturbation axis by injecting additive Gaussian noise into the design matrix, producing selection frequencies \(\pi_k(\sigma_j)\) over both subsampling randomness and design noise levels [2605.02205].

For learned representations, perturbations arise from repeated model training and alignment uncertainty. The latent-space workflow trains \(K\) realizations of an autoencoder, aligns clusters using k-means and the modified Jonker–Volgenant algorithm, and computes adjusted stress, anchor-set Jaccard dissimilarity, anisotropy changes, and cluster-label instability over \(500\) realizations [2402.11404]. Work on learned features from non-rectangular data uses bootstrap perturbations for the feature learner and half-sampling for downstream stability selection, then aligns the learned features via generalized Procrustes after dimensionality reduction by PCA or Sparse Components Analysis [2102.10388].

In radiomics, perturbations may reflect segmentation uncertainty or acquisition variability. Probabilistic segmentation work replaces a single hard mask with samples from a Probabilistic U-Net posterior, generating up to \(1{,}000\) 2D slice-wise masks per CT slice and filtering them to \(25\) maximally diverse samples for each 3D tumor [1910.05693]. The CT simulation study constructs in-silico acquisition perturbations with ASTRA, varying projection count and Poisson noise, and repeats each acquisition setting ten times with different noise seeds [2210.02759].

In feature-attribution stability, perturbations are applied directly to inputs. FASS groups perturbations into geometric, photometric, and compression families and only evaluates retained image pairs whose top-1 label is unchanged, optionally within a confidence tolerance [2604.02532]. Certification approaches for attributions sample perturbation masks from \(\Delta_r(\alpha)\) to estimate \(\tau_r\) by Monte Carlo [2504.13787]. Related work on Multiplicative Smoothing constructs a smoothed classifier \(f(x)=E_{s\sim\mathcal{D}}[h(x\odot s)]\), where mask noise is used to certify incremental and decremental stability of binary explanation masks [2307.05902].

## 4. Empirical findings across application domains

Several recurring empirical patterns appear across domains. In stochastic feature ranking for random forests, LIME and SHAP are reported to be more stable than MDA, with LIME most stable for small \(k\) and SHAP slightly overtaking for larger \(k\); increasing repetitions reduces instability but does not drive it to zero, and the methods do not necessarily converge to the same selected set [2005.12483]. Predictive performance differences among the three methods are reported as negligible despite substantial gaps in instability [2005.12483]. This suggests that predictive accuracy and feature stability are partially decoupled.

Microbiome experiments reach a similar conclusion. In simulations and real microbiome applications, Stability is reported to correlate strongly and negatively with false positive and false negative rates, whereas MSE correlations are weak or inconsistent [2012.00001]. In highly collinear settings, methods with lowest MSE, such as random forests in some scenarios, can exhibit very poor Stability and high error rates in true feature recovery [2012.00001]. The authors therefore conclude that Stability is a preferred criterion over MSE when the goal is reproducible biological feature identification [2012.00001].

In radiomics, stability varies sharply by feature family. Under segmentation uncertainty sampled from a Probabilistic U-Net, nearly all first-order statistics and shape features have \(ICC>0.9\), most GLCM and NGTDM features also exceed \(0.9\), GLSZM features are least stable with some ICCs as low as \(0.2\), GLRLM features show intermediate stability, and wavelet-transformed features are generally less stable than original CT counterparts [1910.05693]. Overall, \(28.7\%\) of extracted features have \(ICC<0.9\) and would be discarded under that cutoff [1910.05693]. The same work shows that segmentation variance propagates into prognostic performance, with the c-index of a four-feature Cox signature ranging from \(0.569\) to \(0.577\) across segmentation samples [1910.05693].

The CT simulation study reports that median stability is low for most radiomics features, often below \(40\%\), whereas discriminative power is high for most features, often above \(90\%\), mirroring an empirical tandem phantom study [2210.02759]. The top \(10\%\) stable and discriminative features in the simulator overlap with empirical rankings by more than \(80\%\), which is presented as evidence that the simulator reproduces acquisition-induced variability faithfully [2210.02759]. A plausible implication is that highly discriminative features need not be stable under acquisition perturbations.

For latent spaces, the \(500\)-realization study finds that structural stability can improve with increasing inter-feature correlation, but inferential stability may remain poor [2402.11404]. On the low-correlation dataset, cluster-label instability has \(P50\approx70\%\), anchor-set change \(P50\approx83\%\), adjusted stress modes near \(0.67\), and Jaccard modes near \(0.86\); on the moderately high-correlation dataset, structural metrics improve substantially, with adjusted stress mode near \(0.20\), but anchor and label variability remain non-negligible [2402.11404].

For post-hoc attributions, FASS reports that Grad-CAM is most stable across datasets, IG and GradientSHAP closely track one another, and LIME is least stable, especially on CIFAR-10 under geometric perturbations [2604.02532]. Stability depends strongly on perturbation family: geometric perturbations expose substantially greater instability than photometric changes, and without prediction-invariance filtering up to \(99\%\) of evaluated pairs involve changed predictions [2604.02532]. The soft-certification study reports non-vacuous soft-stability rates for attribution masks on vision and language tasks, with mild smoothing empirically increasing \(\hat\tau_r\) by about \(10\%-20\%\) while reducing clean accuracy by less than \(5\%\) [2504.13787]. Multiplicative Smoothing similarly reports non-trivial certified radii for explanation stability at modest accuracy cost [2307.05902].

Beyond conventional feature selection and explanation, feature stability also appears as dynamical robustness of learned representations. In GNN-based force fields for molecular dynamics, reducing edge feature correlation is reported to improve maximum stable simulation time on OOD gold clusters from \(0.03\) ps for baseline Allegro to \(10.0\) ps with a dynamic correlation penalty, while slightly improving energy and force MAE and incurring \(2.4\%\) training-time overhead [2502.12548]. In sparse autoencoders, aligned training is reported to improve cross-seed stability measured by MMCS from approximately \(0.65\)–\(0.75\) to approximately \(0.85\)–\(0.90\), while reducing dead-feature fraction from about \(20\%\) to below \(1\%\) in a cited configuration [2605.18629]. These studies use “stability” for internal representation quality rather than subset reproducibility, but they preserve the same core notion of invariance across perturbations.

## 5. Stability improvement strategies

A large part of the literature treats stability not only as an evaluation criterion but as an optimization target. The most established strategy is perturb-and-aggregate. Classical stability selection improves a base selector by subsampling and retaining only features that are consistently selected; Integrated Path Stability Selection replaces the supremum over stability paths by an integral of transformed selection probabilities and derives substantially tighter upper bounds on expected false positives than previous methods [2403.15877]. Its key score has the form
\[
S_{IPSS,f}=\Bigl\{j:\int_{\lambda_{\min}}^{\lambda_{\max}} f(\pi_j(\lambda))\,\mu(d\lambda)\ge \tau\Bigr\},
\]
and is reported to achieve actual false positives close to target while recovering substantially more true positives than classical stability selection on simulations and cancer data [2403.15877].

When correlation structure causes vote-splitting among proxies, cluster-aware aggregation is used. Cluster Stability Selection defines cluster-level selection proportions
\[
\hat\Theta_B(C_k)=\frac{1}{2B}\sum_{b=1}^{B}\Bigl[\mathbf{1}\{C_k\cap \hat S(A_b)\neq\emptyset\}+\mathbf{1}\{C_k\cap \hat S(\overline A_b)\neq\emptyset\}\Bigr]
\]
and then forms cluster representatives by weighted or unweighted averaging [2201.00494]. In the reported proxy setting, this prevents lasso stability selection from ranking an always-selected but weaker unclustered variable above the latent-signal proxy cluster [2201.00494].

A distinct strategy is to modify the representation so that stable features become easier to recover. In GNN force fields, a correlation penalty
\[
L_{\rm corr}=\frac{2}{D(D-1)}\sum_{1\le i<j\le D} r_{ij}^2
\]
is added to the training loss with a dynamic coefficient scheduler, with the aim of decorrelating feature channels [2502.12548]. In sparse autoencoders, aligned training enforces
\[
w_i^{\rm enc}\cdot w_i^{\rm dec}=1
\]
for every feature by reparameterization, thereby removing a degree of degeneracy associated with unstable and dead features [2605.18629]. In both cases, the intervention is geometric: stability is improved by constraining the internal feature basis.

Another family of strategies improves robustness by explicitly modeling uncertainty. In radiomics, the proposed workflow is to train a probabilistic segmentation model, sample plausible segmentations, compute ICC or CV for each feature, discard low-robustness features, and average the remaining stable features across masks before downstream model fitting [1910.05693]. In CT radiomics, in-silico simulation is proposed as a way to pre-screen stable features before multi-center studies [2210.02759]. In biological sequence analysis, the introduction of motif stability as a comparison criterion suggests a similar workflow: robustness of extracted motifs becomes a property to evaluate alongside discriminative utility [1206.4822].

For explanation stability, smoothing and controlled evaluation are central. FASS imposes prediction-invariance filtering and decomposes stability into structural, rank, and overlap components [2604.02532]. Soft-certification work uses Monte Carlo sampling to estimate \(\tau_r\) and provide confidence bounds without requiring smoothing or Lipschitz estimation [2504.13787]. Multiplicative Smoothing instead constructs a smoothed classifier with provable \(\lambda\)-Lipschitz behavior in feature masks, enabling certified incremental and decremental stability radii for attribution masks [2307.05902]. This suggests two broad approaches to explanation stability: measurement under carefully controlled perturbations, and certification via smoothing-induced regularity.

## 6. Relationships to reproducibility, interpretability, and common controversies

A recurring theme is that feature stability is closely tied to interpretability but not reducible to predictive performance. The random-forest ranking study states that stability of selected features with respect to intrinsic randomness is essential to the human interpretability of a machine learning algorithm, even though predictive metrics may be similar across markedly different selectors [2005.12483]. The microbiome study argues that if tiny changes to the training data cause large changes in the chosen subset, the identified biological features are likely to be artifacts rather than real signal, and reports that MSE is inadequate for evaluating this problem [2012.00001]. These findings frame stability as a reproducibility criterion rather than merely a robustness auxiliary.

At the same time, the precise object whose stability should be measured is contested. Some work focuses on subsets, some on rankings, some on latent geometries, and some on explanations. The information-theoretic study explicitly argues that a useful stability metric should handle full ranked lists, feature subsets, and partial ranked lists in a unified way [2402.05295]. The adjusted-stability literature further argues that ordinary overlap measures behave undesirably when highly similar features are interchangeable, and therefore stability should be similarity-aware [2009.12075]. A common misconception is that identical feature identifiers are necessary for stability; the adjusted-measure literature rejects that for highly correlated or otherwise exchangeable features [2009.12075].

Another controversy concerns conditioning. In attribution studies, low similarity between explanations can arise either because the explanation is fragile or because the model prediction changed. FASS argues that evaluation without prediction-invariance filtering conflates these two effects, and reports that up to \(99\%\) of evaluated perturbed pairs can involve changed predictions if no such filter is imposed [2604.02532]. This implies that some reported instability in explanation literature may reflect model sensitivity rather than explanation instability per se.

A related issue is whether stability should be defined as a hard universal guarantee or a probabilistic rate. The soft-stability framework treats hard stability as the special case \(\tau_r=1\), but allows practically meaningful intermediate guarantees via confidence-bounded Monte Carlo estimation [2504.13787]. This differs from classical feature-selection metrics, where a single scalar summary such as Jaccard, instability index, or Nogueira stability often collapses all variability into one number [2108.01485], [2005.12483], [2012.00001]. A plausible implication is that future stability analyses may increasingly distinguish between deterministic robustness and probabilistic reliability.

Finally, stability may refer either to external reproducibility or internal feature quality. In GNN force fields and sparse autoencoders, stability is improved by directly reshaping the learned feature basis [2502.12548], [2605.18629]. In those settings, the concern is not whether the same subset is selected, but whether the representation remains well-conditioned, non-degenerate, and reproducible across seeds or OOD dynamics. This broader usage expands feature stability beyond feature selection into a general property of representational geometry.

## 7. Practical frameworks and open directions

Across the cited work, several practical patterns emerge for stability-oriented workflows. First, repeated perturbation is essential. Recommendations include running at least \(100\) repetitions or until instability plateaus when using stochastic feature scorers [2005.12483], evaluating many realizations of latent spaces rather than relying on a single autoencoder [2402.11404], and using bootstrap-based estimation when only one data set is available [2012.00001]. Second, the perturbation mechanism should match the scientific uncertainty of interest: resampling for sample variability, design jittering for measurement error [2605.02205], probabilistic segmentations for delineation uncertainty [1910.05693], acquisition simulation for scanner variability [2210.02759], and input perturbations with prediction filtering for explanation robustness [2604.02532].

Third, stability should often be combined with post-processing or filtering. Radiomics workflows explicitly propose discarding features with \(ICC<0.9\) or \(CV>0.1\) and averaging stable features across segmentation samples [1910.05693]. Stability selection and its extensions retain only high-frequency features or clusters [2403.15877], [2201.00494], [2605.02205]. EFSIS weights rankers by their bootstrap-derived stability before aggregating them [1811.07939]. These procedures treat stability as a criterion for model construction, not merely model assessment.

Open directions are stated explicitly in several studies. The microbiome work calls for analytic or asymptotic distributions for Stability under dependent-replicate settings, direct incorporation of Stability into feature-selection objectives, and extensions to other perturbation regimes and structured feature domains [2012.00001]. The simulator-based ensemble-stability paper notes that extending theoretical guarantees beyond first-pick probabilities and adapting the approach to other selectors remain open [2108.01485]. The GNN force-field paper identifies higher-order feature dependencies and adaptive scheduling of correlation penalties as future directions [2502.12548]. The attribution-certification paper positions FASS as a benchmark for future work on robust post-hoc attribution [2604.02532], while the soft-stability paper argues for model-agnostic probabilistic certificates that remain informative at larger perturbation radii [2504.13787].

Taken together, these works indicate that feature stability is best viewed as a multidimensional property of reproducibility under perturbation, with distinct formalizations for subsets, rankings, latent spaces, domain-specific descriptors, and explanations. The literature consistently shows that stability can diverge sharply from predictive accuracy, that it depends strongly on the perturbation model and output representation, and that explicit stability-aware design can materially improve the reliability of both learned features and their interpretations [1206.4822], [2005.12483], [1910.05693], [2402.11404], [2604.02532].

Source: https://www.emergentmind.com/topics/feature-stability