Testing Sign Congruence Between Two Parameters (2405.11759v3)
Abstract: We test the null hypothesis that two parameters $(\mu_1,\mu_2)$ have the same sign, assuming that (asymptotically) normal estimators $(\hat{\mu}_1,\hat{\mu}_2)$ are available. Examples of this problem include the analysis of heterogeneous treatment effects, causal interpretation of reduced-form estimands, meta-studies, and mediation analysis. A number of tests were recently proposed. We recommend a test that is simple and rejects more often than many of these recent proposals. Like all other tests in the literature, it is conservative if the truth is near $(0,0)$ and therefore also biased. To clarify whether these features are avoidable, we also provide a test that is unbiased and has exact size control on the boundary of the null hypothesis, but which has counterintuitive properties and hence we do not recommend. We use the test to improve p-values in Kowlaksi (2022) from information contained in that paper's main text and to establish statistical significance of some key estimates in Dippel et al. (2021).
- Berger, Roger L. (1989): “Uniformly More Powerful Tests for Hypotheses Concerning Linear Inequalities and Normal Means,” Journal of the American Statistical Association, 84, 192–199.
- Brinch, Christian N., Magne Mogstad, and Matthew Wiswall (2017): “Beyond LATE with a Discrete Instrument,” Journal of Political Economy, 125, 985–1039.
- Craig, Cecil C. (1936): “On the Frequency Function of xy𝑥𝑦xyitalic_x italic_y,” The Annals of Mathematical Statistics, 7, 1 – 15.
- Datta, Somnath (1995): “On a modified bootstrap for certain asymptotically nonnormal statistics,” Statistics & Probability Letters, 24, 91–98.
- Dippel, Christian, Robert Gold, Stephan Heblich, and Rodrigo Pinto (2021): “The Effect of Trade on Workers and Voters,” The Economic Journal, 132, 199–217.
- Efron, B. (1979): “Bootstrap Methods: Another Look at the Jackknife,” The Annals of Statistics, 7, 1–26.
- Gail, M. and R. Simon (1985): “Testing for Qualitative Interactions between Treatment Effects and Patient Subsets,” Biometrics, 41, 361–372.
- Kim, Deborah (2023): “Testing Sign Agreement,” available at https://www.dropbox.com/scl/fi/e8b9o1bt9ful8s21zh5yi/JMP_DKIM.pdf?rlkey=8rsfnc2572e4gnopflj1y1mlc&dl=0.
- Kowalski, Amanda E (2022): “Behaviour within a Clinical Trial and Implications for Mammography Guidelines,” The Review of Economic Studies, 90, 432–462.
- Liu, Yiqi and Francesca Molinari (2024): “Inference for an Algorithmic Fairness-Accuracy Frontier,” available at https://arxiv.org/abs/2402.08879.
- MacKinnon, David, Chondra Lockwood, Jeanne Hoffman, Stephen West, and Virgil Sheets (2002): “MacKinnon DP, Lockwood CM, Hoffman JM, West SG, Sheets V. A comparison of methods to test mediation and other intervening variable effects,” Psychological methods, 7, 83–104.
- Montoya, Amanda Kay and Andrew F. Hayes (2017): “Two-condition within-participant statistical mediation analysis: A path-analytic framework.” Psychological methods, 22 1, 6–27.
- Russek-Cohen, Estelle and Richard M. Simon (1993): “Qualitative Interactions in Multifactor Studies,” Biometrics, 49, 467–477.
- Slough, Tara and Scott A. Tyson (2022): “Sign-Congruence, External Validity, and Replication,” Discussion paper, Emory University.
- van Garderen, Kees Jan and Noud van Giersbergen (2022): “A Nearly Similar Powerful Test for Mediation,” ArXiv 2012.11342.