Poisson Regression: Methods & Extensions

• Poisson regression is a statistical method for modeling count data using a log link function, ensuring interpretable multiplicative effects.
• Extensions like negative binomial and COM-Poisson models address overdispersion and underdispersion, enhancing model robustness.
• Regularization techniques, Bayesian approaches, and advanced diagnostics enable effective application in high-dimensional and sparse data settings.

Poisson regression (PR) is a principal statistical method for modeling count data where the outcome variable represents the number of events occurring within a fixed interval or space. The classical Poisson regression model, grounded in the exponential family of distributions, assumes the mean and variance of the response are equal, and models the log of the expected count as a linear function of predictors. Despite its widespread applicability across fields—social sciences, biostatistics, epidemiology, engineering, and beyond—classical PR is often limited by challenges such as overdispersion, underdispersion, multicollinearity, sparsity, and the need for interpretability in high-dimensional and structured data. Recent research has thus produced substantial innovations, including extensions and alternatives to classical PR that enhance its flexibility and robustness.

1. Classical Poisson Regression: Foundations, Estimation, and Use Cases

The fundamental PR model assumes independent observations $Y_i$ such that $Y_i \sim \mathrm{Poisson}(\mu_i)$, with the log link: $\log(\mu_i) = X_i^\top \beta,$ where $\mu_i$ is the expected count, $X_i$ is the covariate vector, and $\beta$ is the coefficient vector. The log-likelihood for a sample is

$$\ell(\beta) = \sum_i \left[ y_i X_i^\top \beta - \exp(X_i^\top \beta) - \log(y_i!) \right].$$

Maximum likelihood estimation (MLE) is typically used, leveraging iteratively reweighted least squares in the GLM framework. The model’s key assumptions are equidispersion ($\text{Var}(Y_i) = \mathbb{E}(Y_i)$), no unmeasured confounding, and correct functional form of $\mu_i$.

In classical settings, PR provides interpretable coefficients (log increments), robust estimation, and readily computable diagnostics (Pearson and deviance residuals, leverage, influence). However, its equidispersion assumption is frequently violated in practice.

2. Generalizations and Flexible Count Regression Models

To address overdispersion and underdispersion (i.e., $\text{Var}(Y_i) >$ or $<\ \mathbb{E}(Y_i)$), a spectrum of models has emerged:

• Negative Binomial Regression: Incorporates gamma heterogeneity, capturing overdispersion.
• COM-Poisson Regression (Sellers et al., 2010): Extends the Poisson by introducing a dispersion parameter $\nu$, yielding the pmf:

$$P(Y = y) = \frac{\lambda^y}{(y!)^\nu Z(\lambda, \nu)},$$

with $$Z(\lambda, \nu) = \sum_{s=0}^\infty \lambda^s / (s!)^\nu$$. For $\nu=1$ it reduces to the Poisson; as $\nu \to \infty$, to the Bernoulli. Estimation proceeds via MLE, and the model can be cast as a GLM with log-link for $\lambda$. A likelihood ratio test of $H_0: \nu = 1$ versus $H_1: \nu \ne 1$ assesses whether PR is appropriate or a more flexible model like COM-Poisson is needed.

• Discrete Weibull Regression (Klakattawi et al., 2015): Allows direct modeling of over- and under-dispersion via a distinct parameterization; estimation via MLE.

Key advantages of these models are robust performance across a broad range of dispersion levels and, when reparametrization is used (e.g., modeling the mean directly in COM-Poisson (Jr et al., 2018)), retention of coefficient interpretability.

3. High-Dimensional, Sparse, and Regularized Poisson Regression

Modern applications increasingly require PR in high-dimensional contexts (genomics, text, finance). Regularization and sparsity are crucial:

• Lasso and Group-Lasso for Poisson Regression (Ivanoff et al., 2014): The log-intensity $g(x)$ is modeled as a linear combination over a dictionary $\{\phi_j\}$, and coefficients are selected via $\ell_1$ (Lasso) or group penalties:

$$\hat\beta^L \in \arg\min_{\beta}\{-\ell(\beta) + \sum_{j} \lambda_j |\beta_j|\},$$

with data-driven penalty weights calibrated using Poisson-specific concentration inequalities.

• Square-root Lasso-inspired Methods (Jia et al., 2017): Penalized weighted score function approaches adjust for natural heteroscedasticity without requiring data-dependent tuning of penalty parameters, mirroring variance invariance in square-root Lasso. The estimation problem is formulated as

$$\min_\beta \frac{1}{n} \sum_{i=1}^n \left[ y_i e^{-\frac{1}{2} X_i^\top \beta} + e^{\frac{1}{2} X_i^\top \beta} \right] + \lambda \|\beta\|_1,$$

with provable $\ell_1$-consistency and scale-free tuning.

• Variational Bayes for Sparse Poisson Regression (Kharabati et al., 2023): Non-conjugate mean-field VB methods with Laplace, spike-and-slab, or Bernoulli-product priors are efficient, provide full posterior approximations, and can outperform frequentist Lasso or SCAD in estimation and uncertainty quantification in simulations and real-world data.

Cardinality-constrained model selection is effected via mixed-integer conic programming and safe screening (Kurihara et al., 17 Apr 2025), enabling optimal sparse solutions and computational scalability.

4. Extensions: Nonlinear, Mixture, and Robust Poisson Regression

• Mixture Models with Liu-Type Shrinkage (Ghanem et al., 2023): Combats multicollinearity and population heterogeneity (e.g., heart disease severity staging) by combining mixture of experts with Liu-type shrinkage estimators, enhancing robustness of coefficient estimates while preserving classification accuracy.
• Robust Regression for Sparse and Heterogeneous Data (Capponi et al., 1 Sep 2025): Introduces a general family of moment estimators unifying Poisson, gamma, and NLS estimation, flexibly adapting to heteroskedasticity and sparsity via a tunable parameter $\kappa$, with cross-validation used for optimal estimator selection. PR (i.e., $\kappa=0$) may not be optimal in finance/economics when excess zeros or complex mean-variance scaling are present.
• Stochastic and Nonconvex Programming Approaches (Anh et al., 16 Jan 2024): Satisfy chance constraints under predictor uncertainty via nonconvex optimization. Constraints on probability distributions of the predicted means are transformed into deterministic constraints, solved by advanced solvers, slightly improving robust estimation under predictor uncertainty.

5. Bayesian, Survey, and Experimental Design Extensions

• Efficient Bayesian Posterior Sampling (D'Angelo et al., 2021): Negative binomial and Pólya-gamma data augmentation facilitate rapid, accurate MH and adaptive importance sampling for high-dimensional PR, outperforming HMC in certain regimes.
• Poisson Regression with Survey Data (Kazemitabar, 2014): For partially observed data under random sampling without replacement, the mean is adjusted by the sampling ratio $\gamma$. Under specific asymptotic regimes (large number of cases), unbiased estimation is restored for the observed counts, correcting bias induced by incomplete sampling.
• Algebraic and Geometric Design Theory (Kahle et al., 2015): The optimality region of a particular experimental design for PR (e.g., in the Rasch model) is characterized by polynomial inequalities (semi-algebraic sets). The design problem can be approached algebraically and geometrically via spectrahedral relaxations, enabling partitioning of parameter space by local optimality.

6. Applications: Signal Processing, Forecasting, and Sequence Modeling

• Signal Denoising and Imaging (Zhu et al., 2013): Bootstrap Poisson regression is applied for denoising Poisson photon count data, followed by robust local nonparametric regression for baseline removal in X-ray spectral images, outperforming competing methods under low-signal and outlier-rich conditions.
• Neural Embeddings with Poisson Regression Loss (Wei et al., 2023): Approximating Levenshtein distance in DNA storage via neural embeddings trained with Poisson negative log-likelihood loss leverages the count nature of sequence edits, reducing approximation error and skewness compared to alternatives; the optimal embedding dimension is determined using an eigenvalue spectrum criterion.
• Hybrid Deep Learning Frameworks (Das et al., 20 Sep 2025): Poisson regression loss is seamlessly integrated with dimensionality reduction (PCA) and deep learning (Seq2Seq LSTM) to predict power outages, ensuring predictions are non-negative and count-valued. Empirical results show improved robustness and accuracy, particularly in high-event-count regimes.

7. Diagnostics, Inference, and Interpretation

Standard inferential procedures involve confidence intervals derived from the observed or Fisher information, parametric or nonparametric bootstrapping for small samples, and robust residual analysis (Pearson, deviance). Model adequacy is assessed via diagnostic plots and quantile residuals. Dispersion testing (e.g., likelihood ratio test for $\nu=1$ in COM-Poisson) guides model choice. Coefficient interpretation for PR is multiplicative on the mean scale; more flexible count models require nuanced interpretation, e.g., effects on the mean or higher moments, or via distributional shifts.

In summary, Poisson regression remains a pivotal tool for modeling count data and sparse non-negative phenomena, but optimal performance and interpretability in modern data-rich, heterogeneous, and high-dimensional environments rely on sophisticated methodological extensions, regularization, robust estimation, and model diagnostics. Recent research unifies and extends classical PR, providing powerful, scalable, and interpretable frameworks for both statistical inference and machine learning applications across diverse scientific domains.