EMA-FS: Feature Screening Methods Overview

• EMA-FS is a family of feature screening methods that use auxiliary statistics (e.g., exponential moving averages, EM-tests) to preselect relevant features and reduce model complexity.
• Variants of EMA-FS apply diverse strategies—such as evolutionary multitasking, marginal empirical-likelihood screening, and histogram-based acceleration in LightGBM—to address tasks in classification, clustering, and boosting.
• By restricting the feature space before intensive learning phases, EMA-FS methods enhance computational efficiency and model performance in high-dimensional settings.

Searching arXiv for the cited EMA-FS-related papers to ground the article in current arXiv records. EMA-based Feature Screening (EMA-FS) denotes a family of feature-screening procedures in which an auxiliary statistic is maintained or tested per feature and then used to restrict a downstream search, estimation, or training routine. In the current arXiv record, the designation is explicit for a LightGBM optimization based on an exponential moving average of split gains (Song, 24 Jun 2026). Closely related uses arise in two other lines of work: an evolutionary multitasking method that uses multiple filters to generate screened feature-selection subproblems in high-dimensional classification (Li et al., 2022), and an EM-test-based marginal screening procedure for ultrahigh-dimensional clustering (Wang et al., 2023). A broader antecedent is marginal empirical-likelihood screening, which formalizes variance-aware marginal relevance testing for linear models, generalized linear models, and general estimating-equation settings (Chang et al., 2013). Taken together, these works do not define a single canonical algorithm; they define a recurrent design pattern in which a relatively cheap feature-wise criterion is used to cull dimensions before a more expensive learner or optimizer is applied.

1. Scope, nomenclature, and variant meanings

The acronym “EMA-FS” is not used uniformly across the literature. In one case it names a concrete algorithm; in the others it functions more naturally as a descriptive umbrella for methods that combine feature screening with either EM-type inference or evolutionary multitasking.

Usage	Core mechanism	Primary setting
Explicit “EMA-FS”	Exponential moving average of per-feature split gain, followed by top-$K$ retention	Histogram-based GBDT and LightGBM
Natural descriptive use	EM-test statistic for per-feature mixture homogeneity	Ultrahigh-dimensional clustering
Conceptual near-equivalent	Multiple filter-based task generation plus multitask evolutionary optimization	High-dimensional supervised feature selection

The LightGBM method is the only one that explicitly bears the title “EMA-FS” (Song, 24 Jun 2026). The clustering paper instead presents “feature screening based on the EM-test,” but its own technical exposition identifies “EMA-FS” as a natural name for the method because screening is driven by an EM-based homogeneity test (Wang et al., 2023). The evolutionary multitasking paper does not use the term “EMA-based Feature Screening” and does not define “EMA” as an acronym, but it systematically places filter-based screening in front of an evolutionary multitask optimizer, making it conceptually close to what one would call evolutionary-multitasking-based feature screening (Li et al., 2022).

This terminological dispersion matters because the word “screening” itself changes meaning across the three lines. In the multitask evolutionary formulation, screening produces lower-dimensional surrogate tasks that help solve an original wrapper-level feature-selection problem. In the EM-test formulation, screening is a marginal inferential device for identifying cluster-relevant variables. In the LightGBM formulation, screening is an algorithm-level optimization that decides which feature histograms are worth constructing for a tree. A common misconception is therefore to treat EMA-FS as a single standardized method; the literature instead supports an acronym with multiple, method-specific realizations.

2. Evolutionary multitasking as screened wrapper feature selection

In high-dimensional classification, the multitask evolutionary formulation treats feature selection as binary combinatorial optimization over $X=(x_1,\dots,x_N)$, $x_i\in\{0,1\}$, with objective $\min f(X)$, where $f(\cdot)$ is the classification error rate (Li et al., 2022). The multitask embedding replaces the single full-space problem by one original task plus several low-dimensional tasks obtained through filter-based elimination of irrelevant features. Task 1 uses all features; additional tasks are defined by subsets retained by Relief-F, TV, and PCC after knee-point thresholding. This yields $K+1$ related tasks, with the filtered tasks acting as simplified subproblems.

The method uses three standard univariate filters. Relief-F measures discriminative power through nearest-hit and nearest-miss comparisons; TV uses per-feature variance,

$$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$

and PCC measures linear dependence between a feature and the label,

$$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$

For each filter, features are ranked by weight, a knee point is detected on the weight-versus-rank curve, and features above that knee define a relevant subset. The paper’s rationale is that using only Relief-F, as in earlier EMT-based FS methods, does not generate sufficiently diverse tasks for knowledge transfer.

Optimization is performed by a modified Competitive Swarm Optimizer with separate subpopulations per task rather than a unified MFEA-style genotype. Individuals are real-valued vectors, discretized at threshold $\delta=0.5$ for evaluation. Fitness is

$\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$

where $X=(x_1,\dots,x_N)$0 is the balanced classification error, $X=(x_1,\dots,x_N)$1 is the number of selected features, $X=(x_1,\dots,x_N)$2 is the task dimension, and $X=(x_1,\dots,x_N)$3. The error term is

$X=(x_1,\dots,x_N)$4

Feature subsets are evaluated with a $X=(x_1,\dots,x_N)$5-NN classifier, feature selection is conducted on training data via 10-fold cross-validation, and the best solution is then evaluated on the test set; all results are averaged over 30 independent runs of 10-fold CV.

The multitask contribution lies in explicit cross-task transfer. For a loser particle in task $X=(x_1,\dots,x_N)$6, transfer occurs with probability $X=(x_1,\dots,x_N)$7, and the update may use an aggregated winner

$X=(x_1,\dots,x_N)$8

with task weights chosen as $X=(x_1,\dots,x_N)$9 for the full task and $x_i\in\{0,1\}$0 for the filtered tasks. Parameter analysis favors Case A $x_i\in\{0,1\}$1 and $x_i\in\{0,1\}$2, with best overall performance for $x_i\in\{0,1\}$3. Numerous empirical results on eighteen high-dimensional datasets with 2,420–12,600 features show that MF-CSO obtains the lowest test error on 11 of 18 datasets against EA-based baselines, achieves 84 wins, 8 losses, and 34 ties in Wilcoxon comparisons against seven EAs, and almost always selects substantially fewer features than non-EMT EAs. The paper’s own ablations further indicate that knowledge transfer, aggregated winners, and CSO rather than PSO all contribute materially to performance. A plausible implication is that, within supervised wrapper feature selection, EMA-FS can be understood as a two-stage architecture in which fast filter screening creates transferable subtasks and evolutionary multitasking exploits them to guide the full-dimensional search.

3. EM-test-based screening for clustering

For ultrahigh-dimensional clustering, feature screening is formulated feature-wise through mixture homogeneity testing rather than supervised prediction (Wang et al., 2023). Let $x_i\in\{0,1\}$4 denote feature $x_i\in\{0,1\}$5 in sample $x_i\in\{0,1\}$6, and suppose each feature’s conditional distribution in cluster $x_i\in\{0,1\}$7 is $x_i\in\{0,1\}$8 from a known identifiable parametric family. Marginally,

$x_i\in\{0,1\}$9

A feature is cluster-irrelevant if $\min f(X)$0, so the mixture collapses to the base family $\min f(X)$1; it is cluster-relevant if at least two components differ, so the marginal lies in $\min f(X)$2. Screening is therefore reduced to

$\min f(X)$3

The test statistic is built from a penalized log-likelihood

$\min f(X)$4

with penalty

$\min f(X)$5

The homogeneous null fit constrains all component parameters to equality. Under the heterogeneous alternative, a fixed number $\min f(X)$6 of EM iterations is run from one or more initializations $\min f(X)$7, with responsibilities

$\min f(X)$8

For a single initialization, the EM-test statistic is

$\min f(X)$9

and the final statistic is the maximum over multiple initializations. Features are retained if

$f(\cdot)$0

with the theoretical threshold $f(\cdot)$1, $f(\cdot)$2. Empirically, $f(\cdot)$3 gives a good balance between type I and II errors.

The theory establishes nonasymptotic tail bounds under both homogeneity and heterogeneity, a limiting null distribution, sure independent screening, and model selection consistency. In particular, with $f(\cdot)$4, sure screening holds when $f(\cdot)$5 for $f(\cdot)$6, and exact recovery holds when $f(\cdot)$7 under the stated conditions. Under the null, the limiting law is analytically complicated in general but is stochastically bounded by $f(\cdot)$8; this bound is used as a conservative null for p-values. The method is marginal and computationally efficient, with total cost $f(\cdot)$9, and is highly parallelizable across features. The reported running times are about 15 seconds at $K+1$0, about 127 seconds at $K+1$1, and about 505 seconds at $K+1$2.

Simulation and real-data results show that the EM-test has strong feature-ranking quality and clustering impact. In simulations, EM-based screening followed by $K+1$3-means yields ARIs close to oracle clustering that uses only the truly relevant features, and the method remains robust under moderate parametric misspecification. In a single-cell RNA-seq study with 11,697 cells and 33,538 genes, the procedure models counts with a negative binomial family, applies the EM-test to each gene in each batch, combines p-values by a Bonferroni-type minimum combination, and selects 2,754 genes at FDR $K+1$4. Downstream PCA, Harmony batch correction, SNN graph construction, Louvain clustering, and UMAP identify 9 cell subtypes, including three additional subtypes beyond the six known in the source study. The paper also states a clear limitation: the method is not suitable for pure Bernoulli features because mixtures of Bernoulli distributions remain Bernoulli and therefore do not create a marginal heterogeneity signal.

4. Marginal empirical likelihood as a variance-aware screening paradigm

Marginal empirical-likelihood screening does not bear the EMA-FS label, but it supplies a closely related screening logic based on feature-wise test statistics that incorporate uncertainty rather than raw effect magnitude (Chang et al., 2013). In the linear-model case, for each variable $K+1$5, the marginal contribution parameter $K+1$6 is defined through

$K+1$7

so that $K+1$8 when $K+1$9. With $$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$0, the marginal empirical likelihood is

$$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$1

and the screening statistic is the empirical-likelihood ratio at zero,

$$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$2

If $$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$3 is not in the convex hull of $$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$4, the paper sets $$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$5, treating this as strong evidence of relevance.

The central methodological claim is self-studentization. The empirical-likelihood ratio behaves like a variance-normalized quadratic form, so $$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$6 reflects signal relative to noise rather than signal magnitude alone. This is the formal basis for the paper’s claim that the procedure can incorporate the level of uncertainty of marginal estimators, unlike screening rules based only on $$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$7 or $$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$8. The same construction extends to generalized linear models with canonical link, because testing $$W_{\text{TV}(a)}=\frac{1}{|S|}\sum_{j=1}^{|S|}(A(j)-\bar A)^2;$$9 reduces again to a marginal moment condition involving $$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$0 and centered $$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$1. It extends further to general estimating-equation models through feature-specific moment vectors $$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$2, with QIF for longitudinal data given as a detailed example.

The paper establishes sure screening under minimal marginal signal and tail conditions. If

$$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$3

and $$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$4 for $$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$5, then

$$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$6

under the stated assumptions. Under normal or sub-Gaussian tails, the admissible dimensionality includes regimes such as $$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$7. The paper also gives size control and selection consistency under partial orthogonality or signal separation between active and inactive variables. Practically, it recommends ranking variables by $$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$8 and selecting a top-$$W_{\text{PCC}(a)}=\frac{\sum_{i=1}^{|S|}(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^{|S|}(x_i-\bar x)^2}\sqrt{\sum_{i=1}^{|S|}(y_i-\bar y)^2}}.$$9 subset, typically with $\delta=0.5$0, before applying a second-stage penalized estimator. Iterative variants such as EL-ISIS recover variables that are marginally weak but jointly important. This suggests a general template for EMA-FS-like procedures in which the screening score is a likelihood- or divergence-based, variance-aware marginal discrepancy rather than a raw filter score.

5. Exponential-moving-average screening for histogram GBDT

The explicit EMA-FS method is an algorithm-level acceleration for histogram-based gradient boosting, especially LightGBM (Song, 24 Jun 2026). The motivating observation is that per-feature histogram construction dominates training time. On an 80K $\delta=0.5$1 500-feature synthetic workload with 4 threads and 100 trees, histogram construction takes 2.66 seconds, or 69.0% of the 3.86-second total; split finding takes 1.00 second, or 26.0%; and other work takes 0.20 seconds, or 5.0%. Because the code is already memory-bandwidth bound and eight micro-optimizations did not produce measurable speedups, the method targets the number of histograms built rather than the speed of each histogram.

For each feature $\delta=0.5$2 and boosting iteration $\delta=0.5$3, the algorithm records the maximum split gain $\delta=0.5$4 observed anywhere in tree $\delta=0.5$5, setting $\delta=0.5$6 if the feature is unused. It then updates an exponential moving average

$\delta=0.5$7

with $\delta=0.5$8. The paper uses $\delta=0.5$9 in all experiments. After a warmup of $\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$0 full-feature trees, features are sorted by current EMA gain, and only the top

$\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$1

features are allowed to participate in the next tree, where $\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$2 is the retention ratio. Unselected features are not used for histogram construction or split search in that tree, but their EMA values still decay through the same update with $\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$3. Selection occurs at the tree level rather than the node level so that LightGBM’s histogram subtraction trick remains valid.

The deterministic variant is complemented by Stochastic EMA-FS, which replaces top-$\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$4 retention by gain-weighted sampling without replacement. It defines

$\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$5

and samples $\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$6 features according to $\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$7. Here $\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$8 controls the interpolation between random subsampling and deterministic screening: $\text{fitness}=\alpha E+(1-\alpha)\frac{S}{D},$9 yields uniform sampling, while $X=(x_1,\dots,x_N)$00 approaches deterministic top-$X=(x_1,\dots,x_N)$01 retention. Both variants are implemented in about 120 lines of C++ across all six LightGBM tree learners and are fully backward-compatible. The extra per-tree costs are $X=(x_1,\dots,x_N)$02 for the EMA update, $X=(x_1,\dots,x_N)$03 for ranking, and $X=(x_1,\dots,x_N)$04 for the selection mask; these are negligible relative to the dense histogram cost.

The reported speed-accuracy trade-offs are strongly data dependent. On the dense 500-feature synthetic benchmark, EMA-FS with $X=(x_1,\dots,x_N)$05 yields 1.34$X=(x_1,\dots,x_N)$06 speedup and improves AUC from 0.9599 to 0.9610; with $X=(x_1,\dots,x_N)$07, it yields 2.61$X=(x_1,\dots,x_N)$08 speedup with AUC 0.9419. Random $X=(x_1,\dots,x_N)$09 feature subsampling is slightly faster on that benchmark, at 2.82$X=(x_1,\dots,x_N)$10, but has worse log-loss. On IEEE-CIS Fraud with 432 features, EMA-FS at $X=(x_1,\dots,x_N)$11 yields 1.45$X=(x_1,\dots,x_N)$12 speedup, whereas LightGBM feature_fraction=0.3 is slower than baseline, 5.67 seconds versus 4.87 seconds. S-EMA-FS can improve the trade-off further: on IEEE-CIS, $X=(x_1,\dots,x_N)$13, $X=(x_1,\dots,x_N)$14 gives 1.21$X=(x_1,\dots,x_N)$15 speedup while increasing AUC from 0.9236 to 0.9252. On FraudDense and FraudMixed, S-EMA-FS likewise provides simultaneous speedup and small AUC gains in several settings. By contrast, on the Bosch dataset with 968 features and more than 90% missingness, EMA-FS produces essentially no speedup because LightGBM’s sparse bin optimization already bypasses most empty values. The paper therefore recommends EMA-FS primarily for dense or moderately dense datasets with moderate to high dimensionality and a nontrivial share of weak or noisy features.

6. Comparative structure, misconceptions, and open directions

Across these variants, the common structure of EMA-FS is feature-wise scoring followed by explicit restriction of a downstream procedure. In the multitask evolutionary method, the score is a filter weight and the restriction creates low-dimensional tasks (Li et al., 2022). In the clustering method, the score is an EM-test statistic and the restriction produces a screened feature set for later clustering (Wang et al., 2023). In the LightGBM method, the score is an EMA-smoothed split gain and the restriction is operational rather than final, because it only governs which histograms are built for a tree (Song, 24 Jun 2026). The empirical-likelihood framework contributes the idea that a screening score can be self-normalized and uncertainty-aware while still remaining marginal and scalable (Chang et al., 2013).

Several misconceptions are clarified by this comparison. First, “EMA” is not a fixed acronym across the literature: it may refer to exponential moving average, to an EM-based testing mechanism, or, more loosely, to an evolutionary multitasking algorithm used as the engine behind screening. Second, “feature screening” does not always mean permanent variable elimination. The LightGBM method screens features at the per-tree level, and a feature can re-enter because older EMA gains decay rather than vanish abruptly. Third, marginal screening and wrapper search should not be conflated. The EM-test and empirical-likelihood methods are fundamentally marginal and can miss features that are jointly but not marginally informative; the clustering paper states this limitation explicitly, and the EL paper addresses related issues through iterative screening. The evolutionary multitasking method, by contrast, performs wrapper-level subset search but relies on front-end filters that are themselves univariate or largely marginal.

Open directions are specific to each lineage. The multitask paper suggests additional filters such as symmetrical uncertainty, Fisher score, RRelief-F, and ReliefMSS, alternate base learners such as SVM or logistic regression, and more sophisticated multitasking frameworks such as MFEA-II, TWO-MFEA, and MFEA-DRA (Li et al., 2022). The clustering paper identifies nonparametric or semiparametric mixtures as a natural extension and notes the unsuitability of the present method for pure binary features (Wang et al., 2023). The LightGBM paper proposes adaptive retention ratios, sparse-aware screening that accounts for per-feature processing cost, and combination with GOSS so that importance sampling occurs in both feature and instance dimensions (Song, 24 Jun 2026). The empirical-likelihood paper suggests another axis of development: profile-based or iterative screening that can recover hidden variables with weak marginal but strong conditional effects (Chang et al., 2013).

The broader implication is not that a single EMA-FS recipe has emerged, but that the literature has converged on a shared principle: feature-wise auxiliary statistics can be used to shrink the effective dimensionality before the expensive part of learning, whether the expensive component is wrapper optimization, clustering after latent-mixture inference, or per-tree histogram construction. This suggests that EMA-FS is best understood as a methodological family whose members differ in objective, supervision regime, statistical assumptions, and computational target, but share a common commitment to early, informed dimensionality reduction.