Variance Estimation for Saturated Fixed-Effect Specifications

Abstract: We characterize the asymptotic behavior of conventional variance estimators in linear regression with high-dimensional fixed effects under a drift in which both the proportional fixed-effect dimension $ρn = d{K_n}/n \to ρ\in [0,1)$ and the residual treatment variance $τn^2 = nQ{K_n} \to τ^2 \in (0, \infty]$ are non-degenerate. Three findings emerge. First, under strict exogeneity and conditional homoskedasticity, the Cattaneo--Jansson--Newey-corrected $t$-statistic is asymptotically exact for any $τ^2 > 0$: there is no Stock--Yogo-style threshold in $τ^2$. Second, the Eicker--White HC0 estimator is biased downward by a fixed factor $(1-ρ)$, producing over-rejection that grows with saturation. Third, HC3 over-corrects in the opposite direction by a factor $1/(1-ρ)$. The leave-one-out estimator (HC2) removes the first-order leverage distortion and is asymptotically exact under homoskedasticity or design-balanced heteroskedasticity; under general heteroskedasticity with non-uniform leverage, HC2 retains an additional bias of order $ρ|μ- ω^2|$ that we characterize. An empirical application to Piotroski F-Score returns in CEE markets illustrates the predicted variance hierarchy in real data.

• The paper demonstrates that the HC0 estimator is downward biased by a factor of (1-ρ), inflating t-statistic variance.
• It shows that the leave-one-out (HC2) variance estimator achieves asymptotically exact inference even with unbalanced leverage.
• Simulation and empirical evidence validate theoretical predictions, emphasizing robust variance methods for high-dimensional fixed-effect models.

Variance Estimation in Saturated Fixed-Effect Linear Models: Theoretical Results and Implications

Introduction

The paper "Variance Estimation for Saturated Fixed-Effect Specifications" (2607.05215) systematically analyzes variance estimation in linear regression models with high-dimensional or "saturated" fixed effects (FE). It introduces a unified asymptotic framework where both the dimension of the fixed effects relative to sample size ($\rho_n = d_{K_n}/n$) and the residual treatment variance ($\tau_n^2 = n Q_{K_n}$) are allowed to drift jointly, reflecting the properties of modern applied econometric specifications with a high degree of FE absorption. The work focuses on the behavior of classical and robust variance estimators—including the Eicker–White (HC0), HC1, HC2 (leave-one-out), and HC3 estimators—in these settings, and provides precise characterizations of their bias and asymptotic properties under general conditions.

Asymptotic Framework and Fixed-Effect Saturation

The core asymptotic regime considers both the proportional FE dimension and the score (residual treatment) variance drifting to non-degenerate limits:

• FE dimension: $\rho_n = d_{K_n}/n \to \rho \in [0,1)$, with $d_{K_n}$ the number of FE parameters and $n$ the sample size.
• Residual treatment variance: $\tau_n^2 = n Q_{K_n} \to \tau^2 \in (0,\infty]$, with $Q_{K_n}$ the variance of the regressor $X$ after projecting out the FE.

This framework is directly relevant for multi-way FE models (worker–firm, panel, event-study, TWFE), triple-differences, and high-dimensional cell-based designs dominating modern empirical practice.

Unlike the instrumental variables (IV) context, where weak identification produces a critical value distortion governed by the first-stage F-statistic (Stock–Yogo-type threshold), the saturated FE linear model with strict exogeneity and conditional homoskedasticity maintains the unbiasedness of the within estimator regardless of $Q_{K_n}$, precluding a weak-identification or first-stage–type critical value distortion.

Theoretical Results: Variance Estimators and t-statistics

The paper provides a systematic theoretical characterization of the standard and robust variance estimators:

• HC0 (Eicker–White) estimator is downward biased by a factor $(1-\rho)$; this induces t-statistics with variance inflated by $\tau_n^2 = n Q_{K_n}$0 and significant size distortion that increases with FE saturation.
• HC3 estimator exhibits over-correction, consistently producing an upward bias by a factor $\tau_n^2 = n Q_{K_n}$1, leading to t-statistics with reduced variance and substantial under-rejection, especially for large $\tau_n^2 = n Q_{K_n}$2.
• HC2 (leave-one-out, LO) and HC1 (DOF-corrected HC0) estimators are shown to deliver asymptotically exact inference when leverage is uniform or under "design-balanced" heteroskedasticity. However, HC2 remains asymptotically exact even with unbalanced leverage, while HC1 can deviate.
• Cattaneo–Jansson–Newey (CJN)-corrected t-statistics (using the residual-based variance estimator with the Frisch–Waugh–Lovell orthogonalization) yield exact finite-sample $\tau_n^2 = n Q_{K_n}$3 distributions with no dependency on $\tau_n^2 = n Q_{K_n}$4, confirming that there is no Stock–Yogo-type threshold for identification in these models.

The explicit limiting distributions of t-statistics using these estimators under the stated drift are as follows:

Estimator	Limit of estimated variance (under homoskedasticity)	Asymptotic t-stat variance
HC0	$\tau_n^2 = n Q_{K_n}$5	$\tau_n^2 = n Q_{K_n}$6
HC1, HC2	$\tau_n^2 = n Q_{K_n}$7	$\tau_n^2 = n Q_{K_n}$8
HC3	$\tau_n^2 = n Q_{K_n}$9	$\rho_n = d_{K_n}/n \to \rho \in [0,1)$0

Under heteroskedasticity, the estimators’ behavior is characterized by the limiting effective design variance $\rho_n = d_{K_n}/n \to \rho \in [0,1)$1, where $\rho_n = d_{K_n}/n \to \rho \in [0,1)$2 is the score-variance target and $\rho_n = d_{K_n}/n \to \rho \in [0,1)$3 measures cross-leverage variance. The leave-one-out (HC2) standard errors are shown to be consistent provided the design is balanced or approximately so; otherwise, a bias proportional to $\rho_n = d_{K_n}/n \to \rho \in [0,1)$4 appears, but simulations suggest this bias is small in practice unless the FE structure is highly unbalanced.

Simulation Evidence

Extensive Monte Carlo simulations corroborate all theoretical results. Notably:

• HC0 substantially over-rejects at moderate or high FE saturation (e.g. empirical size ~16% at $\rho_n = d_{K_n}/n \to \rho \in [0,1)$5 for a nominal 5% test), even with heteroskedastic errors.
• HC3 is overly conservative, with rejection rates falling far below nominal levels.
• HC2/LO and HC1 have size close to nominal in balanced designs; in unbalanced settings, HC2/LO remains correct while HC1 can deviate downward.
• No dependence on $\rho_n = d_{K_n}/n \to \rho \in [0,1)$6 (residual treatment variance) is observed for the inference properties, confirming the absence of a weak-identification issue as found in IV designs.

Empirical Application

An empirical illustration examines Piotroski F-Score as a predictor of forward equity returns in CEE markets using a Visegrad firm-year panel with moderate FE saturation ($\rho_n = d_{K_n}/n \to \rho \in [0,1)$7–$\rho_n = d_{K_n}/n \to \rho \in [0,1)$8). Findings demonstrate:

• Substantial spread between variance estimators: Standard errors using HC0 are up to 15% smaller, and using HC3 up to 35% larger, compared to LO in moderately saturated designs.
• Consistent agreement with theoretical ratios: The observed standard error ratios across HC0/LO, HC1/LO, and HC3/LO match the predictions from the analytical results to within 1–2 percentage points, even in real data.
• Implications for empirical practice: Application to actual data confirms that misuse of conventional robust standard errors grossly misstates inferential accuracy when FE saturation is non-negligible.

Implications and Recommendations

The theoretical and empirical results collectively establish several important implications:

• Applied inference with saturated FEs should not rely on standard HC0 or HC3 estimators. Both produce severe distortion and either overstate or understate precision as sample saturation increases.
• Leave-one-out (HC2) standard errors, or HC1 when leverage is uniform, should be default choices. In most balanced FE panels, HC2 and HC1 will yield indistinguishable results, but HC2 is robust to leverage imbalance.
• Reporting the residual treatment variance $\rho_n = d_{K_n}/n \to \rho \in [0,1)$9 is crucial. It quantifies the identifying variation that survives FE absorption and communicates the effective sample size for inference.
• No Stock–Yogo threshold is required or meaningful in saturated FE OLS designs under the standard model, but analogous issues may arise if exogeneity fails (e.g., measurement error in treatment), emphasizing the need for model-specific diagnostics in those cases.

Future Directions

The analytical framework admits natural generalizations, including:

• Extension to vector-valued treatments and multivariate FE projections.
• Accommodation of dependent errors through clustered or network-robust analogues of leave-one-out estimation.
• Integration with modern difference-in-differences and event-study designs, particularly with complex cohort interactions or staggered adoption where FE saturation and leverage imbalance are frequent and consequential.

Conclusion

This work provides a definitive asymptotic characterization of variance estimation for saturated fixed-effect specifications, establishing leave-one-out standard errors (HC2) as uniquely robust to FE saturation and highlighting the quantitative unreliability of HC0/HC3 in such settings. The results have immediate practical impact on empirical practices in applied econometrics, offering new diagnostic tools and robust inferential procedures for high-dimensional FE designs. The paper’s recommendations are relevant across a variety of domains—including labor, trade, and asset pricing—where multi-way fixed effects and high-dimensional projections are standard.

Reference:

"Variance Estimation for Saturated Fixed-Effect Specifications" (2607.05215)