Conjugate GLMMs (CGLMMs)

• Conjugate GLMMs are a class of mixed models that enable closed-form marginal likelihood computation through conjugacy between exponential family likelihoods and random effects.
• They integrate conjugate random effects to capture overdispersion and Gaussian random effects to model intra-cluster correlations, ensuring efficient and exact inference.
• CGLMMs unify common models like Gaussian, Poisson, beta-binomial, and gamma, reducing computational burden while accurately addressing hierarchical data structures.

A conjugate generalized linear mixed model (CGLMM) is a class of mixed models for clustered and longitudinal data, distinguished by the feature that, for particular choices of random effects and link functions, the marginal likelihood can be computed in closed form without recourse to numerical integration. CGLMMs unify several well-known models—including Gaussian, Poisson, negative-binomial, beta-binomial, and gamma models—under a single analytic framework. Central to the construction is the conjugacy between the exponential family conditional likelihood and the distribution of (latent) random effects, facilitating analytic marginalization. This enables efficient and exact maximum-likelihood inference in settings that exhibit both overdispersion and hierarchical multilevel structure (Molenberghs et al., 2011, Lee et al., 2017).

1. Model Formulation: Hierarchical and Conjugate Structure

In the general CGLMM framework, the observed outcome $Y_{ij}$ for subject (or cluster) $i=1,\ldots,N$ and measurement $j=1,\ldots,n_i$ is modeled conditionally on two types of latent random effects:

• Conjugate random effects $\theta_i$ capture overdispersion—extra variability not explained by the canonical exponential-family model. The form of $G(\alpha,\beta)$, the distribution of $\theta_i$, is chosen to be conjugate to the conditional likelihood (e.g., Gamma for Poisson, Beta for Bernoulli) (Molenberghs et al., 2011).
• Normal (Gaussian) random effects $b_i \sim N(0,D)$ of dimension $q$ induce correlation within clusters or repeated measures, capturing subject-level heterogeneity.

The CGLMM is typically specified as follows:

$$f(y_{ij} \mid \theta_i, b_i) = \exp\left\{ \phi^{-1} \left[ y_{ij} \lambda_{ij} - \psi(\lambda_{ij}) \right] + c(y_{ij}, \phi) \right\},$$

where $\lambda_{ij}$ is the canonical parameter, and the conditional mean satisfies: $i=1,\ldots,N$0 with link function $i=1,\ldots,N$1, fixed effects $i=1,\ldots,N$2, and design matrices $i=1,\ldots,N$3.

The conjugate random effect modifies the mean at the observation level: $i=1,\ldots,N$4 where $i=1,\ldots,N$5 is a function of the fixed and normal random effects.

2. Exponential Family Structure and Conjugacy Conditions

Marginalization of the random effects relies on expressing both the outcome distribution and the random-effects prior in compatible (conjugate) exponential-family form (Lee et al., 2017). For a one-parameter exponential family: $i=1,\ldots,N$6 with canonical parameter $i=1,\ldots,N$7 and cumulant $i=1,\ldots,N$8, conjugate priors take: $i=1,\ldots,N$9

Closed-form marginalization is possible if the mapping $j=1,\ldots,n_i$0 and $j=1,\ldots,n_i$1 induced by covariates is affine in $j=1,\ldots,n_i$2 and $j=1,\ldots,n_i$3: $j=1,\ldots,n_i$4 Those conditions are satisfied for Gaussian, Poisson, and gamma models with nontrivial choices of unit- and group-level covariates, and for the binomial under group-level predictors alone.

3. Marginal Likelihood and Analytic Marginalization

The likelihood contribution for subject $j=1,\ldots,n_i$5 is expressed as: $j=1,\ldots,n_i$6 Analytic marginalization is performed over the conjugate random effect $j=1,\ldots,n_i$7, leveraging conjugacy to obtain, e.g., negative-binomial or beta-binomial-type densities. The remaining integral over the normal random effects $j=1,\ldots,n_i$8 is handled via classical approaches such as Gauss–Hermite quadrature or Laplace approximation (Molenberghs et al., 2011).

A key property of CGLMMs is that for a wide range of settings—including those with both group- and unit-level covariates in Gaussian, Poisson, and gamma families—the entire marginal likelihood is available in closed form, yielding substantial computational advantages (Lee et al., 2017).

4. Key Special Cases and Model Families

Table 1 summarizes the principal CGLMM special cases for clustered data (Lee et al., 2017):

Family	Conditional Model	Conjugate Prior	Marginal
Gaussian	$j=1,\ldots,n_i$9	$\theta_i$0	Gaussian-integrated
Poisson	$\theta_i$1	$\theta_i$2	Negative-binomial
Binomial	$\theta_i$3	$\theta_i$4	Beta-binomial
Gamma	$\theta_i$5	$\theta_i$6	Inverse-gamma

Context and significance: For Poisson outcomes, the CGLMM recovers the negative-binomial–normal model. For binary outcomes, it yields the beta–binomial–normal model, and for continuous measures, the hierarchical normal model. In each case, overdispersion and cluster-level correlation are jointly modeled.

5. Marginal Covariance, Correlation, and Overdispersion

CGLMMs allow explicit derivation of the marginal covariance and correlation structure for repeated measures or clustered observations. In the Poisson–Gamma–Normal case, for independent $\theta_i$7,

$\theta_i$8

Marginal variances thus combine overdispersion (additional variation from conjugate effects) and correlation from normal random effects (Molenberghs et al., 2011).

An immediate implication is that the closed-form correlation between observations within a cluster can be obtained analytically, facilitating inference on dependence structures and the impact of modeling choices on variance decomposition.

6. Inference, Computation, and Practical Implementation

With explicit marginal likelihoods, parameter estimation proceeds via direct maximum likelihood. After analytic marginalization over conjugate random effects, gradient and Hessian computations are facilitated, supporting classical Wald and likelihood-ratio test theory. Remaining numerical integration (over $\theta_i$9) is typically one-dimensional or low-dimensional, depending on the correlation structure (Molenberghs et al., 2011). In big-data and federated learning environments, groupwise sufficient statistics can be calculated and communicated, simplifying computation and supporting privacy (Lee et al., 2017).

Standard estimation techniques include adaptive quadrature, Laplace approximation, or expectation-maximization (when treating random effects as missing data). In applied practice, platforms such as SAS NLMIXED can accommodate CGLMMs by analytic specification of the closest closed-form (Molenberghs et al., 2011).

7. Illustrative Data Applications and Model Diagnostics

CGLMMs have been applied to epileptic seizure counts, clinical trials on onychomycosis, and childhood asthma survival. For the Poisson–Gamma–Normal CGLMM fit to epileptic seizure data with 45 placebo and 44 active subjects and weekly repeated counts, maximum likelihood estimates yield both overdispersion and random effect variance: $G(\alpha,\beta)$0 (s.e. 0.2113) and $G(\alpha,\beta)$1 (s.e. 0.1850). The overdispersion index, $G(\alpha,\beta)$2, quantifies the departure from Poisson variance (Molenberghs et al., 2011).

In treatment comparison, inference may differ from simpler mixed models; in this example, slope differences found to be significant under the Poisson–Normal disappear under the fuller CGLMM that accommodates overdispersion.

Closed-form marginal correlations allow model comparison and diagnostic assessment, with combined model correlations systematically below those from purely normal-effects models, reflecting the greater absorption of variance by the independent conjugate effects.

---

Primary references: (Molenberghs et al., 2011, Lee et al., 2017).