Adaptive Early Feature Selection (AEFS)

• Adaptive Early Feature Selection (AEFS) is a feature selection approach for deep recommender systems that performs instance-wise adaptive selection before the embedding layer to reduce computational cost.
• AEFS employs a dual-model architecture where a lightweight auxiliary selector guides the main predictor, achieving a 37.5% reduction in activated embedding parameters.
• Its collaborative training objective uses binary cross-entropy, embedding alignment, and prediction alignment losses to balance efficiency and prediction quality.

Searching arXiv for the AEFS paper and closely related feature-selection work to ground the article in current literature. Searching arXiv for "Adaptive Early Feature Selection for Deep Recommender Systems" and related AEFS terminology. Adaptive Early Feature Selection (AEFS) is a feature-selection framework for deep recommender systems that performs instance-wise selection before the main embedding layer, with the explicit objective of retaining the adaptivity associated with late feature selection while reducing the activated parameters of the embedding layer (Hu et al., 15 Sep 2025). It is motivated by the observation that large-scale recommender models are dominated by embedding tables and that existing methods typically optimize only one of two goals: early feature selection reduces embedding computation but usually selects a fixed global subset of fields, whereas adaptive late feature selection selects features per instance but only after embeddings have already been fetched (Hu et al., 15 Sep 2025). AEFS addresses this tension through a dual-model architecture consisting of an auxiliary selector and a main predictor, trained jointly with binary cross-entropy, embedding alignment, and prediction alignment losses; in the reported default setting, it matches the performance of adaptive late feature selection methods while achieving a 37.5% reduction in activated embedding parameters (Hu et al., 15 Sep 2025).

1. Problem setting and taxonomic position

Within the formulation used for deep recommender systems, feature selection is divided into two categories: early feature selection and late feature selection. Early feature selection selects a fixed subset of feature fields before the embedding layer and retrains the recommender model using only those selected fields. Its primary advantage is that dropping fields early reduces embedding lookups, forward computation, backward updates, and activated embedding parameters. Its limitation is that the chosen subset is global and static rather than tailored to each instance. Late feature selection, described as adaptive feature selection, dynamically adjusts feature choices for each data instance, but it operates after embeddings are already produced, so it does not substantially reduce embedding-layer cost (Hu et al., 15 Sep 2025).

Paradigm	Selection stage	Characterization
Early feature selection	Before embedding layer	Fixed subset of feature fields; retrains model
Late feature selection	After embeddings are produced	Adaptive per instance; embedding cost remains
AEFS	Before the main embedding layer	Instance-wise adaptive selection with reduced activated embedding parameters

AEFS is positioned as an overview of these two paradigms. It keeps instance-level adaptivity, which prior late-selection methods such as AdaFS and MvFS exploit for predictive quality, while moving the decision boundary to the input side of the main embedding layer so that the main model activates fewer embeddings (Hu et al., 15 Sep 2025). This positioning is especially relevant in systems with billions of ID-type features, where the embedding layer dominates model size and parameter activation.

2. Architectural design and instance-wise selection

The framework uses a dual-model architecture. The auxiliary model estimates feature importance and selects the top-$k$ fields for each instance; the main model performs the actual CTR prediction using only those selected fields. The basic deep recommender pipeline is written as

$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$

AEFS modifies this pipeline by inserting adaptive selection before the main embedding computation. After the auxiliary model computes feature scores, the selected feature indices are used to form a reduced input for the main model:

$$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$

The main model, denoted $M_m$, then embeds only the selected fields and predicts from the reduced representation:

$$\boldsymbol{E^{s}_{m}} =\text {Scaling} ( M_{m}.E(\boldsymbol{X^{s}}), \boldsymbol{W} ),$$

$P_{m} = M_{m}.P(\boldsymbol{E^{s}_{m}}).$

The auxiliary model, denoted $M_a$, is intentionally smaller and cheaper. In the reported default configuration, the main embedding dimension is $d_1=32$ and the auxiliary embedding dimension is $d_2=4$. The auxiliary computation is

$\boldsymbol{E_{a}} = M_{a}.E(\boldsymbol{X}) = [\boldsymbol{e_{a1}, \boldsymbol{e_{a2}, \ldots, e_{aN}}],$

$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$0

$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$1

$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$2

$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$3

$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$4

The defining property is instance-level adaptivity: for each user-item-context sample, the auxiliary model outputs a score vector $\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$5 and selects the top-$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$6 fields for that specific sample. The selected fields can therefore vary across instances. This differs from fixed early feature selection, which learns one global mask, and from adaptive late feature selection, which makes the adaptive choice only after all embeddings have already been computed (Hu et al., 15 Sep 2025).

3. Collaborative training objective

AEFS trains the auxiliary selector and the main predictor jointly. The optimization objective is given as

$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$7

and the algorithmic training loss is stated as

$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$8

with

$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$9

$$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$0

$$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$1

Two collaborative constraints are central. The Embedding Alignment Loss (EAL) is the fine-grained representation-level constraint:

$$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$2

The Prediction Alignment Loss (PAL) is the coarse-grained output-level constraint:

$$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$3

The role of these losses is explicitly differentiated. EAL aligns the auxiliary model’s selected-feature representation with the representation used by the main model, while PAL aligns the auxiliary model’s final predictive behavior with the main model’s output. The ablation study reports that removing either one hurts performance, and removing both hurts most (Hu et al., 15 Sep 2025). In the framework’s own interpretation, the auxiliary model is therefore not merely a heuristic scorer; it is trained to become a selector whose representation space and decision behavior remain compatible with the main recommender.

4. Embedding-layer sparsity and empirical evaluation

A central claim of AEFS is that adaptive selection can be moved early enough to reduce embedding activation. The paper derives the reduction in activated embedding parameters as

$$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$4

under the default setting $$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$5, $$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$6, and an early selection ratio of 50% (Hu et al., 15 Sep 2025). This derivation formalizes the intended trade-off: the main model activates embeddings for only half the fields, while the auxiliary selector introduces a smaller compensating cost.

The empirical evaluation uses three benchmark CTR datasets, split 8:1:1 into train, validation, and test sets (Hu et al., 15 Sep 2025).

Dataset	Scale	Field description
Avazu	About 40 million user-click records	22 feature fields
Criteo	About 45 million user-click records	26 categorical and 13 numerical fields
KDD12	Not summarized by record count in the provided data	11 categorical fields

For Criteo, numerical features are discretized via $$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$7 when $$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$8, else $$\boldsymbol{X^{s}}= \text{Selection}(\boldsymbol{X}, \boldsymbol{I}).$$9; for KDD12, rare categories are mapped to OOV with minimum frequency threshold 10. AEFS is evaluated with three mainstream backbones—MLP, DeepFM, and DCN—and compared against AutoField, OptFS, LPFS, SHARK, AdaFS, and MvFS. The reported metrics are AUC, Logloss, and PaE, with statistical significance assessed using a two-sided $M_m$0-test at $M_m$1 (Hu et al., 15 Sep 2025).

The reported findings are consistent across Avazu, Criteo, and KDD12: AEFS variants match AdaFS and MvFS in AUC and Logloss, show no statistically significant degradation, and achieve 37.5% PaE. In contrast, AutoField often degrades AUC and Logloss; OptFS, LPFS, and SHARK are generally weaker than adaptive methods; and AdaFS and MvFS are accurate but have 0% PaE (Hu et al., 15 Sep 2025). The implementation details stated for these experiments include main embedding size $M_m$2, auxiliary embedding size $M_m$3, batch size 2048, Adam, BatchNorm, Xavier initialization, and an implementation based on a public PyTorch CTR library.

5. Operational interpretation, transferability, and limitations

The operational importance of AEFS lies in the fact that embedding-layer activation, rather than prediction-layer arithmetic, is the dominant cost in many industrial recommenders. By performing instance-wise selection before the main embedding layer, AEFS is designed to reduce not only activated embedding parameters but also embedding lookups. The reported latency results indicate that lower $M_m$4 leads to lower training and inference latency, and the paper states that AEFS approximately halves the main model’s activated embedding usage relative to AdaFS while preserving an overall activated-parameter reduction of 37.5% (Hu et al., 15 Sep 2025).

The framework also reports a transferability property: the auxiliary model’s prediction layer can differ from the main model’s prediction layer without meaningful performance loss. This matters because it suggests an operational decoupling between a lightweight, relatively stable selector and a main recommender that may evolve more rapidly. The same section of results notes that pretraining is less important in AEFS than in conventional late-selection methods, that removing top-$M_m$5 reweighting hurts performance, and that the recommended auxiliary-to-main embedding ratio is $M_m$6 (Hu et al., 15 Sep 2025).

Several limitations are stated explicitly. The theoretical parameter reduction does not automatically yield full runtime savings unless the system is optimized for sparse lookups and selective embedding access. If the auxiliary model is made too small, performance drops. AEFS still relies on an embedding-based selector, so the auxiliary model adds overhead. The net gain depends on the selection ratio and the embedding-dimension ratio (Hu et al., 15 Sep 2025). A common misconception is that any adaptive feature selector will necessarily reduce the dominant cost in a deep recommender; in AEFS’s formulation, this is not true for adaptive late feature selection, because embeddings must still be looked up before late-stage pruning occurs.

6. Related usages of the acronym and broader methodological context

The acronym “AEFS” is not unique across the feature-selection literature. In "AutoEncoder Inspired Unsupervised Feature Selection" (Han et al., 2017), AEFS denotes an AutoEncoder Feature Selector for unsupervised feature selection that combines autoencoder regression and group lasso tasks. That work is motivated by high-dimensional data in computer vision and machine learning, and it argues that the method can select important features by excavating both linear and nonlinear information among features, in contrast to conventional self-representation approaches that rely only on linear assumptions (Han et al., 2017). In this earlier usage, AEFS refers neither to recommender systems nor to adaptive early instance-wise selection.

Related ideas also appear under different names in other subfields. "Adaptive Node Feature Selection For Graph Neural Networks" (Azizpour et al., 3 Oct 2025) proposes an adaptive node feature selection approach for GNNs that identifies and removes unnecessary features during training using validation-performance changes under permutation interventions. Its selection is early in the sense that pruning occurs during training rather than after a separate fitting stage. "Forward-Backward Selection with Early Dropping" (Borboudakis et al., 2017) introduces a heuristic for temporarily discarding variables that are conditionally independent of the outcome given the selected set, with optional reconsideration in later runs. That method yields a family of algorithms, FBED$M_m$7$M_m$8, FBED$M_m$9$$\boldsymbol{E^{s}_{m}} =\text {Scaling} ( M_{m}.E(\boldsymbol{X^{s}}), \boldsymbol{W} ),$$0, and FBED%%%%4$\left\{ \begin{array}{ll} \{x_1, \ldots, x_N\} & \leftarrow \mbox{quantization}(\{f_1,\ldots,f_N\}), \ \{e_1, \ldots, e_N\} & \leftarrow M.E(\{x_1, \ldots, x_N\}),\ \hat{y} & \leftarrow M.P(\{e_1, \ldots, e_N\}). \end{array} \right.$4%%%%2, which trade off speed and theoretical recovery guarantees and, in experiments, increase computational efficiency by about two orders of magnitude in high-dimensional problems while retaining predictive performance (Borboudakis et al., 2017).

This suggests that “adaptive early feature selection” can function as a broader methodological description rather than a single canonical algorithm. In that broader sense, AEFS for deep recommender systems is distinguished by two coupled commitments: the selection decision is instance-adaptive, and it is executed before the main embedding layer, where the dominant parameter-activation cost resides (Hu et al., 15 Sep 2025).