--- title: Comprehensive Relative Importance (CRI) url: https://www.emergentmind.com/topics/comprehensive-relative-importance-cri type: topic --- # Comprehensive Relative Importance (CRI) Searching arXiv for the specified paper and closely related work on relative importance and variable importance. Search query: arXiv 2509.10853 Variable Selection Using Relative Importance Rankings Comprehensive Relative Importance (CRI) is a relative-importance measure for quantifying the unique contribution of each predictor to the explanatory power of a linear model, especially in the presence of multicollinearity or high-dimensional data with \(p>n\). In the variable-selection framework developed in "Variable Selection Using Relative Importance Rankings" [2509.10853], CRI generalizes the earlier Relative Weight (RW) approach to arbitrary predictor matrices, serves as a computationally efficient approximation to General Dominance (GD), and is used not only for post-hoc explanation but also as a filter-based ranking device before model construction. Its central methodological premise is that variable ranking should account for both direct and combined effects of predictors rather than relying only on marginal correlation. ## 1. Definition and statistical role In the usage of [2509.10853], CRI belongs to the family of relative importance (RI) measures. RI measures had traditionally been employed as post-hoc explanatory tools, whereas the paper proposes using CRI as a filter-based variable ranking tool for variable selection prior to model building. This repositioning is consequential because the ranking criterion is intended to reflect explanatory contribution under predictor dependence, including settings with suppressors, weak predictors, and clusters of highly correlated variables. The defining feature of CRI is that it attempts to quantify the unique contribution of each predictor while preserving information about predictor dependence. The paper contrasts this with marginal correlation, which ignores dependencies among predictors. The resulting interpretation is that CRI is not merely a descriptive attribution device but a ranking functional designed for pre-model selection in correlated designs. A common misconception is that RI measures are intrinsically secondary to model fitting. The framework in [2509.10853] directly contests that view by treating RI scores as the first stage of a filter paradigm, with estimation deferred to least squares or ridge regression after ranking has been established. ## 2. Mathematical construction CRI is defined for a predictor matrix \(X\) of rank \(r\) through a singular value decomposition \[ X = U_r S_r V_r^\top . \] The construction then uses a generalized orthogonal predictor matrix for transforming and allocating variance. The CRI importance vector \(D(X)\) is \[ D(X) = \left[(V_r S_r V_r^\top) \odot (V_r S_r V_r^\top)\right] \left[(V_r U_r^\top y) \odot (V_r U_r^\top y)\right], \] where \(\odot\) denotes the Hadamard product. The first term is described as a reallocation of explained variance from the orthogonal predictors back to original variables, while the second term projects the response onto the orthogonal predictor space and quantifies the proportion of variance in \(y\) explained by the orthogonal predictors [2509.10853]. This formulation is explicitly designed to work regardless of whether \(X\) is of full rank. That generality is a key distinction from the classical RW construction. In the special case where \(X\) is full column rank and \(pp\), CRI.Z reduces to the Correlation-Adjusted marginal coRrelation (CAR) score. The simplification is motivated computationally, but it also changes how explained variance is reassigned to the original predictors. ## 3. Relation to GD, RW, and CRI.Z The statistical position of CRI is most easily understood by comparing it with GD, RW, and CRI.Z. GD is presented as the theoretical ideal for variable importance because it directly measures average incremental contribution to \(R^2\) over all possible models, but it is computationally prohibitive except for very small \(p\). Specifically, GD requires fitting \(2^p-1\) models, which makes exhaustive evaluation infeasible for moderate or large predictor sets [2509.10853]. CRI is therefore framed as an efficient approximation to GD that retains the main advantage of GD—considering direct and joint effects—while remaining applicable for any \(X\), including \(p \gg n\) and singular designs. CRI.Z is a further simplification oriented toward computational efficiency and, according to the paper, often toward improved selection performance. | Measure | Core characterization | Computational implication | |---|---|---| | GD | Theoretical ideal; average incremental contribution to \(R^2\) over all possible models | Requires fitting \(2^p-1\) models | | CRI | Efficient approximation of GD; generalizes RW; works for any \(X\) | Scalable under \(p \gg n\) or singularity | | CRI.Z | Identity-reallocation simplification of CRI | Faster to compute; reduces to CAR when \(n>p\) | RW appears here as the full-rank special case recovered by CRI. This suggests that CRI should be viewed less as a replacement for RW than as its extension to arbitrary predictor matrices. It also suggests that CRI.Z occupies a distinct position: it is computationally simpler and, in the paper’s empirical analysis, often more competitive for selection, even if full CRI may remain preferable for post-hoc explanatory fidelity. ## 4. Use in variable ranking and filter-based selection The variable-selection use of CRI in [2509.10853] follows a filter paradigm. First, CRI, CRI.Z, GD, CAR, or another RI measure is computed for all predictors. Second, predictors are ranked by their scores. Third, a sequence of models is constructed by incrementally including the top \(k\) ranked variables. Fourth, the resulting models are fitted using least squares or ridge regression. The paper states the workflow as follows: compute the scores \(d=f(X,y)\), construct a sequence of models by including top \(k\) ranked variables, and fit with least squares or ridge regression. It further notes that no lasso is used in this fitting step so as not to introduce secondary variable selection. In this design, ranking and coefficient estimation are separated: RI performs screening and ordering, while least squares or ridge performs model fitting. This separation is methodologically important because it clarifies what CRI is intended to optimize. CRI is not a penalized estimator and is not itself a sparse regression fit. Its role is to rank predictors in a way that captures direct and combined effects, after which standard estimators can be used for the chosen model size. The paper recommends tuning the number of variables included using a validation set, and it notes that Ridge-CRI.Z can be used if prediction stability or variance is a concern [2509.10853]. ## 5. Empirical behavior and comparative evidence The empirical results in [2509.10853] are organized around two tasks: variable ranking and predictive modeling. For ranking, CRI, GD, and CRI.Z are reported to offer superior rankings over marginal correlation, denoted SIS, especially in the presence of suppressors and weak predictors. In simulations with clusters of highly correlated predictors and suppressor effects, CRI and CRI.Z are described as consistently identifying true variables earlier, requiring smaller model sizes to capture all true signals. For downstream modeling and support recovery, the study reports that RI-based methods consistently match or outperform the lasso and relaxed lasso, particularly in high-correlation settings, when predictors are grouped or strongly correlated, and in high-dimensional settings with \(p \gg n\). Ridge-regularized RI models, especially Ridge-CRI.Z, are reported to improve prediction error and often exceed even the relaxed lasso in both selection accuracy, measured by F1-score, and prediction error, measured by Relative Test Error (RTE). The paper also differentiates performance by signal-to-noise regime. At low to moderate SNR, lasso is described as excelling because of its bias-variance tradeoff. At high SNR and high correlation, RI-based methods, especially CRI.Z and Ridge-CRI.Z, are reported to outperform lasso and best subset by providing robust variable ranking and stable model fitting. In the clustered-predictor setting identified as Example 5, LS-RI variants, especially LS-CRI.Z, achieve the highest support recovery rates, while forward stepwise and best subset degrade sharply with increasing correlation. SIS performs well only when marginal correlations align with support and is described as fragile in masked or joint-effect cases [2509.10853]. A second misconception addressed by these results is that lasso-like methods exhaust the relevant design space for variable selection. The empirical study instead presents RI-based ranking as a competitive alternative, especially when predictor correlation is itself the source of benchmark-method failure. ## 6. Related meanings in adjacent literatures The phrase “comprehensive relative importance” is not confined to the CRI measure of [2509.10853]. In "All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously" [1801.01489], the comprehensive object is Model Class Reliance (MCR), defined as the range \[ [ MCR_{-}(\epsilon),\ MCR_{+}(\epsilon) ]\ :=\ \left[ \min_{f \in \mathcal{R}(\epsilon)} MR(f),\ \max_{f \in \mathcal{R}(\epsilon)} MR(f) \right]. \] There, comprehensive relative importance refers to the full spectrum of reliance a variable could have among all well-performing models in a Rashomon set, rather than to a single-variable ranking score for a linear predictor matrix. A related but distinct line is Relative Feature Importance (RFI), which generalizes Permutation Feature Importance and Conditional Feature Importance by evaluating the importance of a feature relative to an arbitrary conditioning set \(G\) through \[ \operatorname{RFI}_j^G = \tilde{R}(f) - R(f), \] with \(\tilde{X}_j \sim P(X_j \mid X_G)\) and \(\tilde{X}_j \perp (Y,\overline{G}) \mid X_G\) [2007.08283]. RFI is therefore a perturbation-based importance framework, not a dominance-style variance reallocation method. In multicriteria decision analysis, the same descriptive phrase appears in yet other forms. "Qualitative Decision Methods for Multi-Attribute Decision Making" [1508.00879] treats relative importance as a strict partial order over criteria and defines qualitative dominance through a witness criterion \(X_w\). "As simple as possible but not simpler in Multiple Criteria Decision Aiding: the ROR-SMAA level dependent Choquet integral approach" [1905.07941] uses a comprehensive importance notion based on the Shapley index averaged across evaluation levels, \[ \phi(\mu^L, i) = \frac{\int_\alpha^\beta \phi(\mu^L, i, t)\,dt}{\beta-\alpha}. \] These usages concern trade-offs among criteria and interactions in aggregation models rather than variable selection in regression. This suggests that “Comprehensive Relative Importance” functions across subfields as a descriptive label for importance measures that integrate dependencies, alternatives, or interactions, but in the specific terminology of [2509.10853] it denotes a particular SVD-based RI measure for ranking predictors prior to model building. The implementation accompanying that paper is available at the repository identified there: `https://github.com/tien-endotchang/RI-variable-selection` [2509.10853]. Source: https://www.emergentmind.com/topics/comprehensive-relative-importance-cri