CGLMM: Analytic Marginal Likelihoods

• CGLMM is a two-level mixed model that integrates exponential-family regression with random effects through analytic marginal likelihoods.
• It systematically identifies algebraic conditions for simultaneous conjugacy across distributions like Gaussian, Poisson, gamma, and binomial.
• The framework enables efficient maximum likelihood inference and scalability by using closed-form expressions for marginal means, variances, and correlations.

The conjugate generalized linear mixed model (CGLMM) is a class of two-level mixed models that unifies exponential-family regression with random effects and achieves computational tractability through analytic (closed-form) marginal likelihoods. Here, "conjugacy" refers to the capacity for explicit integration over the random effects due to an exponential family structure, not Bayesian updating. The CGLMM framework provides a systematic account of when such closed-form tractability is possible, the necessary algebraic conditions, representative model classes (Gaussian, Poisson, binomial, gamma), and the statistical and computational properties that result (Lee et al., 2017, Molenberghs et al., 2011).

1. Exponential-Family Structure and Classical Conjugacy

A fundamental component of the CGLMM is the one-parameter exponential family:

$$f_{Y|\theta}(y|\theta, \phi) = \exp\left\{ \frac{y \theta - b(\theta)}{\phi} + c(y, \phi) \right\}$$

with canonical parameter $\theta$, cumulant function $b(\theta)$, and known dispersion parameter $\phi$. The classical conjugate prior for $\theta$ is

$$\pi(\theta|\chi, \nu) \propto \exp\{ \chi \theta - \nu b(\theta) \}$$

where $\chi$, $\nu$ are hyperparameters. When this prior is combined with the exponential-family likelihood, the posterior remains in the same family (Bayesian conjugacy). The CGLMM transfers this analytic property to the two-level clustered data setting, permitting closed-form marginal likelihoods (Lee et al., 2017).

2. CGLMM Hierarchical Formulation and Analytic Marginalization

Consider clustered data $y_{ij}$ ($j=1,..., n_i$ in group $\theta$0). Each group has a random effect $\theta$1 drawn from the conjugate prior; conditional on $\theta$2, $\theta$3 are independent exponential-family variables. The marginal likelihood for all groups is:

$\theta$4

which collapses into a closed form via the ratio of normalizing constants (i.e., marginalization is analytic) (Lee et al., 2017). The factorization over groups enables incorporation of group-level covariates by allowing $\theta$5 and $\theta$6 to depend on group-level predictors, e.g., $\theta$7. This structure is referred to as "group-level conjugacy."

3. Simultaneous Conjugacy and Covariate Incorporation

To admit unit-level covariates while preserving analytic tractability, the framework introduces "simultaneous conjugacy." Here, the canonical parameter for each observation is expressed as:

$\theta$8

$\theta$9

with boundary conditions ensuring reduction to the group-level case when $b(\theta)$0. The necessary and sufficient condition for analytic marginal likelihood is that both $b(\theta)$1 and $b(\theta)$2 are affine in $b(\theta)$3 for all $b(\theta)$4, so that after substitution, the group-level integral remains in the conjugate family. This notion is termed "simultaneous conjugacy," and its satisfaction determines which exponential-family models permit unit-level covariate effects in analytic form (Lee et al., 2017).

4. Canonical Examples

The principal one-parameter exponential-family distributions supporting simultaneous conjugacy include:

Distribution	Random Effect Prior	Unit-level Covariates	Marginal Likelihood Type
Gaussian	Normal	Linear in mean	Closed-form, full covariate support
Poisson	Gamma	Log-linear in mean	Closed-form
Gamma	Inverse-Gamma	Scale-multiplicative	Closed-form
Binomial	Beta	Only group-level	Closed-form for group-level only

For the binomial, no nontrivial solution exists for simultaneous conjugacy—unit-level covariates cannot be incorporated while preserving analytic marginalization. For the other cases, the integral remains analytic due to the maintained affine structure. Explicit algebraic forms for the marginal log-likelihood are given in terms of sums over group- and unit-level sufficient statistics and the normalizing constant for the conjugate prior (Lee et al., 2017).

5. Mean-Variance Relationships and Correlation Structure

CGLMMs yield explicit formulas for marginal means, variances, and (in many cases) covariances:

$b(\theta)$5

$b(\theta)$6

In the Poisson–gamma case, the marginal distribution is negative binomial, showing extra-Poisson variation due to overdispersion from the gamma mixing. Marginal correlations can, in the strong conjugacy cases (e.g., Poisson–gamma–normal), be written in closed form, reflecting both overdispersion and cluster-induced dependence (Molenberghs et al., 2011).

6. Maximum Likelihood Inference and Computational Advantages

The explicit marginal likelihood enables standard maximum likelihood inference, including direct maximization via quasi-Newton or Fisher scoring. The analytic form makes the likelihood efficient to evaluate ($b(\theta)$7 per group; sums and special functions only), unlike quadrature-based or simulation-based approaches. Second derivatives (Hessian) yield exact standard errors. Large-scale, streaming, or parallelizable inference is feasible since group-sufficient statistics suffice for updating and memory usage can be greatly reduced. These computational advantages distinguish CGLMMs from more general GLMMs requiring intractable integrals (Lee et al., 2017, Molenberghs et al., 2011).

7. Extension, Limitations, and Related Methodologies

The scope of analytic tractability under simultaneous conjugacy is precisely characterized by the form of the link and prior as per the affine constraints. Binary responses with unit-level covariates (binomial–beta) do not admit analytic marginal likelihoods, requiring either approximation (e.g., logit–probit link relations) or full numeric integration (Molenberghs et al., 2011). The CGLMM also serves as a foundation for models that integrate both forms of overdispersion (conjugate random effects) and subject-specific effects (normal random effects), as elaborated in the family of generalized linear models for repeated measures, which further address estimation methods, identifiability, and correlation structures within clusters.

The primary implication is that CGLMMs formally unify the classical approach to overdispersion (conjugate random effect mixtures: beta–binomial, negative–binomial, gamma–frailty) and modern mixed-effects regression, providing a rigorous, frequentist, computationally efficient framework for hierarchical exponential-family data, provided the conditions for closed-form analytic marginalization are met (Lee et al., 2017, Molenberghs et al., 2011).