Appropriate SMD threshold for balance diagnostics in digital advertising experiments

Establish an appropriate standardized mean difference threshold for diagnosing covariate imbalance in impressed user characteristics when evaluating Meta advertising experiments, rather than relying on the conventional 0.20 threshold originating from behavioral sciences contexts.

Background

The authors assess audience balance using both t-test p-value distributions and standardized mean differences (SMDs), noting common heuristic thresholds (e.g., 0.20) used in other fields. They caution that these thresholds come from different research contexts and may not fit digital advertising experiments.

This explicit uncertainty highlights the methodological open question of calibrating SMD thresholds specifically for Meta ad delivery settings, where audience composition and platform algorithms differ from the contexts that motivated conventional SMD cutoffs.