Robust variance estimation under near-collinearity

Develop a formally robust variance estimator for conditional SAGE importance inference when feature correlations are sufficiently high that the conditioning distribution is nearly singular and the plug-in influence-function standard error underestimates sampling variability.

Background

The paper studies conditional SAGE feature importance with a one-step, cross-fitted estimator and Wald confidence intervals. Its theoretical guarantees rely on nuisance-estimation rates that can fail under near-collinearity: as correlations approach one, conditional distributions become nearly degenerate, the relevant conditioning covariance matrices become ill-conditioned, and the second-order remainder may no longer be negligible at practical sample sizes.

Simulations with correlation 0.9 show that the point estimator remains consistent, but the empirical sampling variability does not contract at the expected square-root rate while the plug-in influence-function standard error does. Consequently, Wald intervals increasingly undercover. The paper recommends grouped importance or bootstrap variance estimates in this regime, but leaves the construction of a formally robust variance estimator unresolved.