On new tests for the Poisson distribution based on empirical weight functions

Abstract: We propose new goodness-of-fit tests for the Poisson distribution. The testing procedure entails fitting a weighted Poisson distribution, which has the Poisson as a special case, to observed data. Based on sample data, we calculate an empirical weight function which is compared to its theoretical counterpart under the Poisson assumption. Weighted Lp distances between these empirical and theoretical functions are proposed as test statistics and closed form expressions are derived for L1, L2 and L1 distances. A Monte Carlo study is included in which the newly proposed tests are shown to be powerful when compared to existing tests, especially in the case of overdispersed alternatives. We demonstrate the use of the tests with two practical examples.

• The paper introduces new goodness-of-fit tests for the Poisson distribution employing empirical weight functions to compare observed and theoretical PMFs.
• It develops nine test statistics using varying weight functions and Lp metrics, demonstrating superior power against overdispersed alternatives in Monte Carlo studies.
• The tests are validated with empirical data, such as sparrow nests and horse kicks, offering robust diagnostic tools compared to traditional methods.

New Tests for the Poisson Distribution Using Empirical Weight Functions

Introduction

The Poisson distribution is extensively used for modeling count data in various applications, including biostatistics and epidemiology. Given its widespread utilization, verifying whether data follows a Poisson distribution is crucial. The paper "On new tests for the Poisson distribution based on empirical weight functions" (2402.12866) introduces new goodness-of-fit tests for the Poisson distribution, addressing gaps in existing methodologies and enhancing the power of tests, particularly when confronting overdispersed alternatives.

Conceptual Framework

Traditional methods for assessing goodness-of-fit for the Poisson distribution face limitations, especially in the context of nuanced alternative distributions that exhibit dispersion properties. The authors propose leveraging the weighted Poisson distribution, a generalization that reweights the standard Poisson distribution through an ascertainment method originally described by Fisher. By using an empirical weight function derived from observed data, the methodology aligns the empirical and theoretical probability mass functions (PMFs).

Definitions and Formulation

The weighted Poisson distribution involves a discrete random variable with PMF adjusted by weights, making it versatile for diverse count distributions. The empirical weight function, $w^*$, which equates the fitted PMF to the empirical PMF, forms the base for testing. The weighted $L_p$ distances between $w^*$ and $\mathbf{1}$ (indicating a true Poisson distribution) are computed, focusing on $L_1$, $L_2$, and $L_{\infty}$ metrics.

Proposed Tests

The paper develops nine distinct test statistics by computing weighted distances using various weight functions:

• Empirical PMF: Generates test statistics $T_{n,f_n}^{(p)}$.
• Fitted Poisson PMF: Generates test statistics $T_{n,f_{\widehat{\lambda}}}^{(p)}$.
• Exponential Weight Function: Adopts $\textrm{e}^{-x}$ as a weight, resulting in $T_{n,L}^{(p)}$.

The new tests are particularly structured to address limitations of finite sample sizes and to provide meaningful test statistics by emphasizing weight on smaller count values, thereby maintaining finite distances.

Numerical Evaluation

The authors conduct a comprehensive Monte Carlo study, using a warp-speed bootstrap methodology to assess test performance against a spectrum of alternative distributions, both equidispersed and over/under-dispersed. The tests reveal that while traditional goodness-of-fit tests like Kolmogorov-Smirnov and Anderson-Darling retain utility, the newly proposed tests often surpass them in power when faced with overdispersed alternatives.

Results Overview

For sample sizes of 30, 50, and 100, the tests maintain nominal significance levels while demonstrating significant power against overdispersed alternatives. For underdispersed alternatives, some of the new tests underperform relative to traditional methods, notably $ID_n$ in the presented scenarios. The $T_{n,f_n}^{(1)}$ and $T_{n,L}^{(1)}$ emerge as consistently competitive across multiple alternative distributions.

Practical Applications

Two empirical examples demonstrate the applicability of the new tests:

1. Sparrow Nests Frequency: The data departs significantly from a Poisson model, validated by p-values below 0.1 for most tests.
2. Prussian Army Horse Kicks: Here, most tests do not reject the Poisson assumption at the same significance level, supporting the model's appropriateness for this dataset.

Conclusion

The introduction of tests based on empirical weight functions offers a robust framework for assessing Poisson distribution compatibility. These tests, particularly with their sensitivity to overdispersion, provide valuable diagnostic tools beyond traditional methods. Future research may further refine the insensitivity to parameter choice in weight functions and explore extensions to related distributions, enhancing robustness across diverse applications.