RISE: Ranking via Iterative SElection

• The paper presents RISE as a unified framework that iteratively refines rankings by scoring, selection, and context adjustment across diverse domains.
• RISE applies iterative methods in collaborative ranking, deep tabular learning, neural network pruning, and ensemble classification to overcome sparsity and low discrimination.
• The iterative approach dynamically adjusts importance scores, mitigating issues like attention bias and context dependency to yield more robust, context-aware rankings.

“Ranking via Iterative SElection (RISE)” is best understood as an Editor’s term for a family of procedures in which a ranking is not produced in a single pass, but is progressively refined through repeated scoring and selection-like updates. In the supplied literature, that pattern appears in several distinct forms: iterative random walks over bipartite graphs for neighbor-based collaborative ranking, iterative feature exclusion for feature importance ranking in deep tabular learning, iterative sensitivity-based pruning before and during neural network training, and iterative random-subspace reweighting for high-dimensional ensemble classification. The ranked objects differ across these settings—users, pairwise preferences, item representatives, tabular features, weights, nodes, or predictor variables—but the structural motif is the same: intermediate scores alter the effective search space or propagation dynamics used to compute later scores (Shams et al., 2018, Shaninah et al., 2024, Verdenius et al., 2020, Tian et al., 2020).

1. Conceptual scope

The shared setting across these methods is an optimization or inference problem in which one-shot ranking is inadequate because the importance of an entity depends on context. In neighbor-based collaborative ranking, user similarity depends on concordant preferences, while concordant preferences depend on similar users. In deep tabular learning, the importance of a feature can change depending on which other features are present. In pruning, the importance of a parameter depends on which other parameters remain. In random-subspace classification, the usefulness of a variable depends on which other variables are sampled into the same subspace.

This suggests that RISE is a design pattern for context-dependent ranking rather than a single canonical algorithm. The common loop has three parts: compute a score, modify the active structure by selection, exclusion, pruning, or reweighting, and then recompute scores on the modified structure. In the supplied papers, these loops are realized by PageRank-style random walks, repeated feature masking, repeated sensitivity ranking with intermediate sparsity targets, and repeated subspace generation biased by previous selection frequencies (Shams et al., 2018, Shaninah et al., 2024, Verdenius et al., 2020, Tian et al., 2020).

A frequent misconception is to equate iterative ranking with greedy top-$K$ selection. That description fits neither IteRank nor the other methods cleanly. IteRank “does not simply iteratively select top items”; instead, it iterates over multiple coupled variables. The Iterative Feature Exclusion module ranks features by repeated exclusion, not by stepwise forward inclusion. SNIP-it and SNAP-it repeatedly rank parameters or nodes by elasticity-like sensitivity statistics and prune in stages. Iterative RaSE ranks variables indirectly through feature-selection frequencies $\eta_l$ derived from repeatedly selected subspaces (Shams et al., 2018, Shaninah et al., 2024, Verdenius et al., 2020, Tian et al., 2020).

2. Recurrent algorithmic structure

Across the four formulations, the iterative mechanism can be summarized as follows.

Work	Iterated quantities	Selection or update mechanism
IteRank	$S$, $c$, $h$, $p$	Bipartite random walks on UPNet and PRNet
IFE / IFENet	$a_j$, $A$, $S$	Exclude one feature per iteration and aggregate attention
SNIP-it / SNAP-it	Sensitivity or elasticity scores	Re-rank and prune to intermediate sparsity $\kappa_i$
Iterative RaSE	$\eta_l$0, $\eta_l$1, $\eta_l$2	Reweight random-subspace generation by previous frequencies

The principal distinction among these methods is the object being selected. In IteRank, the effective selection pressure is encoded by graph edges and restart vectors $\eta_l$3 and $\eta_l$4. In IFE, “selection” is implemented indirectly via systematic exclusion: at iteration $\eta_l$5, the feature $\eta_l$6 is masked, attention is recomputed, and the resulting attention vector contributes to a global ranking. In SNIP-it and SNAP-it, low-ranked components are removed at each stage to reach an intermediate sparsity target. In iterative RaSE, no feature is permanently removed during the iteration; instead, higher-frequency features receive higher probability under the next subspace distribution (Shams et al., 2018, Shaninah et al., 2024, Verdenius et al., 2020, Tian et al., 2020).

Another recurrent feature is the presence of coupled variables. IteRank alternates between user similarity and preference concordance in Phase 1, and between extended concordance and representative significance in Phase 2. IFE alternates across exclusion contexts and then compresses the resulting context-specific attention matrix into a single importance vector. SNIP-it alternates between current network structure and sensitivity ranking. RaSE alternates between subspace selection and variable-frequency estimation. A plausible implication is that RISE-style methods are especially natural when latent importance cannot be specified independently of the evolving support structure.

3. Graph-based collaborative ranking: IteRank

IteRank was proposed for neighbor-based collaborative ranking (NCR), where the objective for a target user $\eta_l$7 is to estimate unknown pairwise concordances $\eta_l$8, aggregate them into an overall ranking over items $\eta_l$9, and return top-$S$0 items. The input preference relation is encoded as

$S$1

Traditional NCR proceeds in three consecutive steps: compute user-user similarity, estimate concordance of pairwise preferences from neighbors, and infer a total item ranking. The paper identifies two sparsity-induced failures in that pipeline: the rare common preferences problem, where many user-user similarities are exactly $S$2 or are based on very few co-compared pairs, and the low discrimination flaw, where most pairwise concordances are set to $S$3. In typical NCR, more than $S$4 of pairs receive zero concordance in practice (Shams et al., 2018).

IteRank breaks that rigid three-stage pipeline by using two bipartite graphs. The User–Preference Network is

$S$5

with users $S$6, pairwise preferences

$S$7

and edges $S$8 when user $S$9 explicitly supports preference $c$0. On this graph, IteRank defines a preference-to-user transition matrix $c$1 and a user-to-preference transition matrix $c$2, both column-stochastic, and alternates

$c$3

where $c$4 is the similarity vector, $c$5 is the concordance vector, $c$6 is a restart distribution over the target user’s direct preferences, and the paper uses $c$7. This random walk is PageRank-like: similarity is increased for users attached to high-concordance preferences, while concordance is increased for preferences supported by high-similarity users.

Phase 2 introduces item representatives. For each item $c$8, the method defines a desirable representative $c$9 and an undesirable representative $h$0, forming

$h$1

The Preference–Representative Network is

$h$2

where each preference connects to the desirable representative of the winner and the undesirable representative of the loser. With transition matrices $h$3 and $h$4, converged Phase-1 concordance $h$5, and personalization vector

$h$6

Phase 2 alternates

$h$7

with $h$8 typically. The final item score is then

$h$9

The significance of this construction lies in how it addresses both NCR sparsity problems. Phase 1 propagates similarity indirectly through the whole user-preference graph, so almost all user pairs receive non-zero similarity values and similarity levels become highly discriminative. Phase 2 spreads mass through item representatives and assigns non-zero extended concordance even to unobserved but structurally related preferences, thereby resolving low discrimination. Empirically, Table 8 reports that EigenRank assigns non-zero concordance to less than $p$0 of possible pairs and only $p$1–$p$2 distinct levels, whereas IteRank Phase 2 yields non-zero concordance for approximately $p$3 of pairs and millions of distinct levels, including $p$4 different level values in Epinions. On recommendation quality, the reported NDCG@10 values include $p$5 on Epinions at UPL$p$6, $p$7 on ML-1M at UPL$p$8, and $p$9 on MovieTweetings at UPL$a_j$0, each exceeding the listed baselines at those operating points (Shams et al., 2018).

4. Iterative feature exclusion in deep tabular learning

The Iterative Feature Exclusion (IFE) module addresses deep tabular learning, where each instance is $a_j$1 and the tasks include classification and regression, although the experiments focus on classification. The motivating claim is that deep tabular models with internal feature selection often use unidimensional feature importance and therefore ignore contextual dependence, under-express feature interactions, and may suffer from attention bias and attention generalization limitations. IFE replaces one-shot feature scoring with a multi-pass exclusion process in which each feature is excluded once and attention is recomputed (Shaninah et al., 2024).

For a single input $a_j$2, the method runs $a_j$3 iterations. At iteration $a_j$4, it constructs a binary mask $a_j$5 with $a_j$6 and $a_j$7 for $a_j$8, and applies

$a_j$9

Each iteration has its own fully connected weight matrix $A$0. The masked input produces

$A$1

where $A$2 is Softmax over classes. Attention scores are then formed from an amplified version of $A$3: with amplification coefficient $A$4, the weights are multiplied by $A$5 and exponentiated, and the attention vector is computed as

$A$6

Across all exclusions, the attention vectors are concatenated,

$A$7

and the global feature-importance vector is obtained by mean aggregation followed by Softmax,

$A$8

The interpretive point of the module is explicit. Because feature $A$9 is removed at iteration $S$0, each other feature is evaluated under multiple altered contexts. The paper characterizes this as capturing both local and global interactions: local, because a feature is scored when a particular other feature is absent; global, because the final $S$1 aggregates across all exclusion contexts. The article also frames IFE as a “complementary-space RISE”: rather than adding features step-by-step, it starts from the full feature set and removes one feature at a time to observe how the attention landscape changes.

IFE is integrated into IFENet, where the learned importance scores reweight the input: $S$2 The weighted input then passes through a simple fully connected predictor with a first hidden layer of size $S$3 and ReLU activation, followed by an output layer with $S$4 units for classification. Training uses the standard cross-entropy loss, and gradients flow through the weighting step into the IFE attention units. The main computational trade-off is the iterative cost: with $S$5 masking iterations and an FC forward of cost $S$6 per iteration, the module costs $S$7, an extra factor of $S$8 over standard single-pass attention.

The reported empirical evidence has two parts. For feature ranking, IFE is evaluated by NDCG@$S$9 against a GradientSHAP-derived “ground truth” ranking; across TELCO, HELOC, Titanic, and UCI-STP, the NDCG@$\kappa_i$0 curves are consistently closer to $\kappa_i$1 than those of XGBoost. For predictive performance, IFENet improves over a plain FNN on all four datasets, including TELCO accuracy $\kappa_i$2, HELOC $\kappa_i$3, Titanic $\kappa_i$4, and UCI-STP $\kappa_i$5, with corresponding F1 improvements. Table 5 further reports that IFENet attains the highest accuracy and F1 on TELCO and HELOC, is close to DANet on Titanic, and clearly dominates on UCI-STP with accuracy $\kappa_i$6 and F1 $\kappa_i$7 (Shaninah et al., 2024).

5. Iterative ranking of sensitivity statistics for pruning

The pruning formulation begins from SNIP’s sensitivity criterion. For a network $\kappa_i$8, a loss $\kappa_i$9, and multiplicative gates $\eta_l$00, the gated network is $\eta_l$01, and the sensitivity of parameter $\eta_l$02 is

$\eta_l$03

By the chain rule,

$\eta_l$04

Hence the saliency score is essentially the gradient-weight product, usually taken in magnitude. The paper reinterprets this criterion through functional elasticity,

$\eta_l$05

so that

$\eta_l$06

which differs from the SNIP score only by division by the common scalar $\eta_l$07 and therefore preserves the ranking (Verdenius et al., 2020).

SNIP-it replaces one-shot pruning by iterative ranking of these sensitivity statistics. At each pruning step $\eta_l$08, the current network is optionally trained for $\eta_l$09 epochs, an intermediate target sparsity $\eta_l$10 is set according to the rule of thumb

$\eta_l$11

sensitivities are recomputed on the partially pruned network using one mini-batch, and the lowest-ranked components are pruned so that total sparsity reaches $\eta_l$12. The defaults reported are $\eta_l$13 pruning steps and accumulated $\eta_l$14 samples for each sensitivity computation. When $\eta_l$15, pruning is performed entirely before training; when $\eta_l$16, pruning is interleaved with optimization.

The same principle extends to structured pruning through SNAP-it. Instead of gating individual weights, the method introduces gates $\eta_l$17 on nodes or channels in layer $\eta_l$18 and defines node sensitivity

$\eta_l$19

with the corresponding elasticity

$\eta_l$20

Structured SNIP-it then re-ranks channels or neurons at each stage and prunes the lowest-ranked ones, subject to the practical constraint that input and output nodes are not pruned and entire layers are not disconnected.

The significance of the iteration is that parameter importance is context-dependent. The paper argues that parameters that appear only moderately important in the dense network may become crucial after other parameters are removed, so re-ranking after each pruning step gives these components “another chance.” This iterative mechanism is presented as a remedy to criticisms of SNIP—namely that its sensitivity criterion may not propagate training signal properly or may even disconnect layers—without requiring GraSP’s second-order machinery.

The reported results emphasize extreme sparsity. On CIFAR-10 Conv6, SNIP-it during training achieves $\eta_l$21 accuracy at $\eta_l$22 sparsity and harmonic mean $\eta_l$23, compared with IMP-global at $\eta_l$24 and $\eta_l$25 sparsity, and HoyerSquare at $\eta_l$26 and $\eta_l$27 sparsity. On CIFAR-10 ResNet18, SNIP-it during training reports $\eta_l$28 accuracy at $\eta_l$29 sparsity and harmonic mean $\eta_l$30. On Imagenette ResNet18, SNIP-it during training reports $\eta_l$31 accuracy at $\eta_l$32 sparsity and harmonic mean $\eta_l$33. For structured pruning on Imagenette VGG16, SNAP-it achieves $\eta_l$34 accuracy, $\eta_l$35 weight sparsity, $\eta_l$36 node sparsity, inference FLOPs $\eta_l$37 of baseline, training FLOPs $\eta_l$38, and training time $\eta_l$39–$\eta_l$40 lower than the dense baseline. The paper also reports more balanced layer-wise sparsity profiles and improved robustness relative to plain SNIP under small-$\eta_l$41 Carlini–Wagner attacks (Verdenius et al., 2020).

6. Random subspace ensembles and iterative variable ranking: RaSE

RaSE addresses sparse classification with i.i.d. data $\eta_l$42, $\eta_l$43, $\eta_l$44. A base classifier trained on subspace $\eta_l$45 using learning algorithm $\eta_l$46 is written $\eta_l$47. In one-shot RaSE, for each weak learner $\eta_l$48, the algorithm generates $\eta_l$49 random subspaces $\eta_l$50, evaluates each using a criterion $\eta_l$51, selects

$\eta_l$52

trains the base classifier on $\eta_l$53, and aggregates by

$\eta_l$54

The threshold $\eta_l$55 is chosen by empirical risk minimization, and the feature frequency

$\eta_l$56

serves as a feature-importance score (Tian et al., 2020).

The selection criterion emphasized in the paper is the ratio information criterion (RIC), based on a weighted symmetrized Kullback–Leibler divergence. Its empirical form is

$\eta_l$57

where $\eta_l$58 is the effective number of free parameters in the model restricted to $\eta_l$59. For LDA, Proposition 6 gives

$\eta_l$60

and for QDA, Proposition 7 provides the corresponding mean-and-covariance form with quadratic terms and penalty $\eta_l$61.

The theoretical foundation is built around a minimal discriminative set $\eta_l$62, defined by

$\eta_l$63

with minimal cardinality. Proposition 3 states that any discriminative set containing $\eta_l$64 attains Bayes risk. Proposition 5 shows that the KL term in RIC is maximized by $\eta_l$65, because adding irrelevant variables does not change the KL divergence, while omitting any signal strictly decreases it. Theorem 4 then establishes screening consistency and weak consistency of RIC under high-dimensional assumptions. Separate LDA and QDA theorems specialize these results to explicit signal-strength conditions. On the ensemble side, Theorem 1 shows conditional risk convergence to the infinite-ensemble classifier at rate $\eta_l$66 at jump points of the aggregation distribution and exponentially fast otherwise, while Theorem 2 provides an analogous Monte Carlo variance bound.

The difficulty in high dimensions is coverage probability. Under hierarchical uniform subspace sampling,

$\eta_l$67

which can be extremely small when $\eta_l$68 is large. This is the reason for iterative RaSE. After one RaSE pass, the algorithm uses the empirical frequencies $\eta_l$69 to define reweighted probabilities

$\eta_l$70

and samples the next generation of random subspaces from a restrictive multinomial distribution based on $\eta_l$71. Theorem 10 shows that, under a stepwise detectable condition and suitable growth conditions, after

$\eta_l$72

iterations the probability $\eta_l$73 converges to $\eta_l$74, while the required $\eta_l$75 is much smaller than in one-shot RaSE. The practical interpretation is direct: iterative selection frequencies form a ranking, and that ranking guides later selection, which sharply reduces the number of random subspaces needed to find a desirable subspace (Tian et al., 2020).

7. Interpretive themes, limitations, and relation among formulations

Taken together, these methods suggest that RISE-style procedures are most useful when importance is inherently relational. In IteRank, the relation is between users, preferences, and item representatives. In IFE, it is among features under different masked contexts. In SNIP-it and SNAP-it, it is among parameters or nodes in a partially pruned network. In iterative RaSE, it is among variables that co-occur in candidate subspaces. A plausible implication is that iterative ranking is particularly effective when sparsity, interaction effects, or support dependence make one-shot scores unstable or uninformative (Shams et al., 2018, Shaninah et al., 2024, Verdenius et al., 2020, Tian et al., 2020).

The supplied papers also identify method-specific limitations. In IteRank, Phase 2 has worst-case complexity $\eta_l$76 because PRNet conceptually spans the full set of pairwise preferences, and the authors note that very large item catalogs may require pruning, sampling, or approximate PRNet constructions. In IFE, the cost of iterative exclusion is $\eta_l$77, so large $\eta_l$78 is a direct computational bottleneck, and zero-masking may be semantically unnatural for some domains. In SNIP-it and SNAP-it, pruning schedules such as the sequence $\eta_l$79, number of steps $\eta_l$80, and interval $\eta_l$81 remain heuristic, and the paper explicitly leaves automatic sparsity selection open. In RaSE, one-shot random subspace search may require very large $\eta_l$82 in high-dimensional settings, which is precisely why the iterative variant is introduced.

A second misconception is that iterative ranking necessarily improves results by adding complexity alone. The supplied evidence points instead to a more specific mechanism: iteration changes the effective context in which scores are computed. IteRank propagates concordance through graph structure to overcome rare common preferences and low discrimination. IFE forces attention reallocation when dominant features are excluded, thereby mitigating attention bias. SNIP-it re-estimates sensitivity after each pruning event, which reduces the risk of pruning parameters that become important only in a sparser network. Iterative RaSE uses selection frequencies to focus later subspace generation on likely signals.

This suggests a broad but technically coherent view of RISE. It is not tied to a particular modality, objective, or optimization primitive. Rather, it denotes a recurrent architecture in which ranking emerges from repeated interaction between scores and support structure. In the supplied literature, that architecture appears in recommendation, tabular feature ranking, neural network pruning, and high-dimensional classification, with each domain instantiating the same basic principle through a different mathematical apparatus.