When Does Heteroskedasticity Matter? A Contrast-Specific Theory of Robust Inference

Abstract: Conventional heteroskedasticity diagnostics ask whether the conditional variance of the regression disturbance varies with covariates. This paper asks a different question: when does that variation matter for inference on the estimand of interest? The paper develops a contrast-specific theory characterizing when covariance perturbations are inferentially relevant. We show that, for any linear contrast $a'β$ in a linear regression, the difference between the heteroskedasticity-robust variance and the pooled fixed-design variance is governed by the empirical covariance between conditional error variance and a contrast-specific leverage score. Thus, heteroskedasticity may be present in the model yet first-order irrelevant for a particular coefficient or linear combination. Conversely, modest heteroskedasticity may have a large inferential effect if it is concentrated on observations that are highly informative for the contrast of interest. We characterize the effect exactly through a heteroskedasticity relevance ratio and a standard-error inflation factor, relate the result to pairs and residual bootstrap procedures, and extend the decomposition to general covariance structures, where off-diagonal dependence contributes a separate contrast-specific term. The results provide a unified way to understand why robust, clustered, and bootstrap standard errors can differ across coefficients in the same regression.