Overstuffed sandwiches and separation anxiety: finite-sample variance estimation for penalized GEE with near-separated binary data

Abstract: Penalized generalized estimating equations (PGEE) stabilize point estimation for longitudinal binary data under near-separation, but inference still depends on how the sandwich variance is corrected. Existing corrections for PGEE can overadjust in high-leverage directions, require restrictive pooling assumptions, or add global regularization without explaining the bias. We establish first-order asymptotics for PGEE along convergent interior-root sequences and derive a matrix characterization of the parameter-specific overcorrection induced by full leverage adjustment. Finite-sample calibration is limited by both mean bias and the variability of leverage-corrected variance estimates. We propose $\hat{V}<em>{AR}$, which keeps the score-level leverage correction and adds a finite-sample upward translation dominated at first order by the finite-population factor, with a smaller centering term. In simulations, $\hat{V}</em>{AR}$ gives conservative or near-nominal type I error in low-event, small-$N$ settings, including $N = 10$, where several standard corrections remain anti-conservative and pooling estimators are unavailable for unbalanced designs.

• The paper introduces a novel estimator that corrects leverage-induced bias in penalized GEE, ensuring accurate variance estimation in small, near-separated samples.
• It rigorously compares traditional variance corrections, highlighting the overcorrection of MD and undercorrection of KC through comprehensive simulation and theoretical results.
• The estimator V incorporates finite-population adjustments and maintains robust type I error control, even in imbalanced designs and rare-event settings.

Finite-Sample Variance Estimation for Penalized GEE with Near-Separated Binary Data

Introduction

The paper "Overstuffed sandwiches and separation anxiety: finite-sample variance estimation for penalized GEE with near-separated binary data" (2604.18863) systematically addresses the problem of inference instability in small-sample, longitudinal binary data analysis with generalized estimating equations (GEE), especially when event rates are low and covariates approach or achieve separation. The main focus is on penalized GEE (PGEE) with Firth-type bias reduction, clarifying both large-sample properties and mechanisms for variance estimation under near-separation, where conventional sandwich estimators and their corrections can fail.

PGEE: Motivation and Asymptotic Theory

Penalized GEE leverages a Firth-type penalty to mitigate bias and instability in estimating regression parameters for sparse, cluster-correlated binary data. In small samples, classic GEE’s sandwich variance estimator is prone to downward bias, skewing type I error rates and producing unreliable confidence intervals. When separation or near-separation occurs, estimation instability is particularly acute, often leading to non-convergent fits or parameter explosions.

The paper establishes first-order asymptotics for PGEE: in the large-sample limit, PGEE is consistent and asymptotically normal, with limiting distribution matching that of GEE when roots converge interiorly. The penalty behaves as an $O(1)$ term and does not alter the asymptotic distribution, justifying the use of standard GEE variance estimation approaches under PGEE in the stable regime.

Small-Sample Variance Estimation: Taxonomy and Bias Analysis

Four main classes of variance corrections are reviewed and analyzed:

• Leverage-correction estimators: These use adjustments based on cluster-specific hat matrices $(I-H_{ii})^{-1}$, intended to counteract the shrinkage effect of fitted residuals. The canonical cases are Kauermann-Carroll (KC) with $c=1/2$ and Mancl-DeRouen (MD) with $c=1$. However, the MD correction completely inverts self-shrinkage but ignores cross-subject contamination, leading to parameter-specific overcorrection, as formally quantified by the paper’s matrix results.
• Additive-stabilizer estimators: E.g., the Morel-Bokossa-Neerchal (MBN) estimator applies a ridge term and finite-population correction, producing conservative inference but without a direct mechanism for tuning bias across parameters.
• Pooling estimators: These pool residuals and correlation estimates across subjects, requiring design constraints such as equal numbers of repeated measurements and homogeneous within-cluster correlation structures, limiting applicability.
• Hybrid estimators: These combine mechanisms, such as leverage corrections with pooling, or apply mean-centering to mitigate bias (e.g., Ford-Westgate, Rogers-Stoner, Fan-Zhang-Zhang).

The paper delivers strong theoretical results:

• A matrix characterization of parameter-specific leverage overcorrection via $B_{\mathrm{lev}} = \sum_i A_i (I_0 - A_i)^{-1}A_i$, directly relating overcorrection to the information structure per parameter and demonstrating that treatment effects suffer maximal overcorrection in imbalanced designs.
• Finite-sample bias comparison: MD overcorrects the sandwich bias by a positive semidefinite matrix of order $N_{\min}/(N_{\min}-1)$; KC undercorrects. Only pooling estimators avoid leverage-driven biases but at the cost of design flexibility.

Proposed Estimator: Score-Level Leverage Correction with Conservative Calibration

Motivated by the instability in existing corrections—especially at the level of individual parameters—the paper proposes a new estimator, denoted $V$, which:

• Retains score-level leverage correction $(I-H_{ii})^{-1}$ to neutralize self-shrinkage bias.
• Incorporates a finite-sample upward translation via multiplicative finite-population and Bessel factors, plus mean-centering, producing conservative standard errors without the anti-conservatism seen in MD or underestimation typical in KC.
• Is robust to unbalanced designs and handles datasets where pooling-based estimators are not computable.

Theoretical analysis guarantees that $V$ is consistent asymptotically and quantifies its finite-sample excess bias as strictly positive definite, thereby offering explicit protection against anti-conservative inference.

Simulation Study: Type I Error, SE Calibration, and Power

A comprehensive simulation study spans 192 scenarios, manipulating number of subjects, event rates, correlation structures, and design imbalance. Fourteen variance estimators—including thirteen from literature and the new $V$—are compared.

Strong numerical results include:

• Type I error for treatment-effect inference remains near or below nominal levels at $(I-H_{ii})^{-1}$0 and 10% event rates for $(I-H_{ii})^{-1}$1 ($(I-H_{ii})^{-1}$2), while KC ($(I-H_{ii})^{-1}$3) and LZ ($(I-H_{ii})^{-1}$4) are substantially anti-conservative.
• Median SE/SimSE for $(I-H_{ii})^{-1}$5 is slightly above the target (overestimates), whereas KC and LZ understate variability. The upward calibration from $(I-H_{ii})^{-1}$6 matches theoretical predictions.
• In unbalanced or misspecified correlation scenarios, $(I-H_{ii})^{-1}$7 maintains robust performance while many estimators fail to compute or remain anti-conservative.

Figure 1: Type I error for $(I-H_{ii})^{-1}$8 in simulation scenarios by N and event rate, demonstrating conservative calibration of $(I-H_{ii})^{-1}$9 relative to leverage- and pooling-based corrections.

Figure 2: Median SE/SimSE ratio across estimators, showing $c=1/2$0 consistently above or near target values especially in small N, low event regimes.

Applied Data: Leverage Diagnostics and Practical Implications

The paper provides analyses on two real datasets: a toenail onychomycosis clinical trial (quasi-complete separation in small subsamples) and contagious bovine pleuropneumonia (high cluster imbalance and engineered separation).

Applied findings:

• In $c=1/2$1 toenail subsamples, standard errors for treatment effect vary by a factor of 3 across estimators; $c=1/2$2 and additive-stabilizer estimators yield the most conservative p-values, often crossing significance thresholds that leverage corrections do not.
• Overcorrection diagnostic $c=1/2$3 is highly parameter-specific, saturating for treatment effect in highly imbalanced settings.
• In the CBPP dataset, $c=1/2$4 produces the greatest SE inflation and avoids anti-conservative inference, despite clusters being highly imbalanced and pooling corrections being inapplicable.

Figure 3: Type I error under unbalanced repeated-measures design illustrating the robust performance of $c=1/2$5 even when pooling estimators are not feasible.

Figure 4: Overcorrection ratios for $c=1/2$6 (treatment vs within-subject parameters) across applied datasets, highlighting substantial parameter-specific leverage effects in small N.

Discussion and Implications

The paper’s results highlight that existing leverage corrections, when used in penalized GEE near separation, can overinflate variance in specific parameter directions, primarily the treatment effect with imbalanced arms. No single scalar correction achieves uniform calibration. The new estimator $c=1/2$7 is robust, conservative, and explicitly accounts for finite-sample effects through a deliberate upward translation.

Practical recommendations:

• For small N, low-event, non-pooling PGEE, $c=1/2$8 offers the best type I error control with standard Wald inference.
• Overcorrection diagnostics ($c=1/2$9) should always be reported, as substantial differences signal instability and risk of misleading inference.
• Pooling corrections can be safe but are often infeasible in unbalanced designs; additive stabilizers are overly conservative but computationally robust.
• Simulation and real-data evidence reveals that leverage corrections alone cannot guarantee safety, and finite-sample conservative calibration is essential.

Conclusion

This work establishes rigorous theoretical foundations for PGEE variance estimation in finite samples, precisely quantifies leverage-driven bias per parameter, and introduces a conservative estimator $c=1$0 that addresses the anti-conservative failures of existing corrections without sacrificing practical applicability. The methodology is especially relevant for small-sample, rare-event, and unbalanced longitudinal studies typical in biomedical research, impacting both the design of trials and the conduct of inference. Future directions include parameter-specific adaptive corrections and joint integration of variance and degrees-of-freedom calibration for improved finite-sample inference in high-leverage designs.