---
title: Conjugate GLMMs (CGLMMs)
url: https://www.emergentmind.com/topics/conjugate-glmms-cglmms
type: topic
---

# Conjugate GLMMs (CGLMMs)

A conjugate generalized linear mixed model (CGLMM) is a class of mixed models for clustered and longitudinal data, distinguished by the feature that, for particular choices of random effects and link functions, the marginal likelihood can be computed in closed form without recourse to numerical integration. CGLMMs unify several well-known models—including Gaussian, Poisson, negative-binomial, beta-binomial, and gamma models—under a single analytic framework. Central to the construction is the conjugacy between the exponential family conditional likelihood and the distribution of (latent) random effects, facilitating analytic marginalization. This enables efficient and exact maximum-likelihood inference in settings that exhibit both overdispersion and hierarchical multilevel structure [1101.0990], [1709.06288].

## 1. Model Formulation: Hierarchical and Conjugate Structure

In the general CGLMM framework, the observed outcome $Y_{ij}$ for subject (or cluster) $i=1,\ldots,N$ and measurement $j=1,\ldots,n_i$ is modeled conditionally on two types of latent random effects:

- **Conjugate random effects** $\theta_i$ capture overdispersion—extra variability not explained by the canonical exponential-family model. The form of $G(\alpha,\beta)$, the distribution of $\theta_i$, is chosen to be conjugate to the conditional likelihood (e.g., Gamma for Poisson, Beta for Bernoulli) [1101.0990].
- **Normal (Gaussian) random effects** $b_i \sim N(0,D)$ of dimension $q$ induce correlation within clusters or repeated measures, capturing subject-level heterogeneity.

The CGLMM is typically specified as follows:

\[
f(y_{ij} \mid \theta_i, b_i) = \exp\left\{ \phi^{-1} \left[ y_{ij} \lambda_{ij} - \psi(\lambda_{ij}) \right] + c(y_{ij}, \phi) \right\},
\]
where $\lambda_{ij}$ is the canonical parameter, and the conditional mean satisfies:
\[
g(E[Y_{ij} \mid \theta_i, b_i]) = X_{ij}^T \beta + Z_{ij}^T b_i,
\]
with link function $g(\cdot)$, fixed effects $\beta$, and design matrices $X_{ij}, Z_{ij}$.

The conjugate random effect modifies the mean at the observation level:
\[
E[Y_{ij} \mid \theta_i, b_i] = \theta_{ij} \, \kappa_{ij},
\]
where $\kappa_{ij}$ is a function of the fixed and normal random effects.

## 2. Exponential Family Structure and Conjugacy Conditions

Marginalization of the random effects relies on expressing both the outcome distribution and the random-effects prior in compatible (conjugate) exponential-family form [1709.06288]. For a one-parameter exponential family:
\[
f(y \mid \theta, \phi) = \exp\left\{ \frac{y\,\theta - b(\theta)}{\phi} + c(y,\phi) \right\}
\]
with canonical parameter $\theta$ and cumulant $b(\theta)$, conjugate priors take:
\[
g(\theta \mid \chi, \nu) = g_0(\chi, \nu) \exp\{ \chi \theta - \nu b(\theta) \}.
\]

Closed-form marginalization is possible if the mapping $x \mapsto \theta(x)$ and $b(\theta(x))$ induced by covariates is affine in $\theta_0$ and $b(\theta_0)$:
\[
\theta(x) = p(x) \theta_0 + q(x) b(\theta_0) + r(x), \qquad b(\theta(x)) = s(x) \theta_0 + t(x) b(\theta_0) + u(x).
\]
Those conditions are satisfied for Gaussian, Poisson, and gamma models with nontrivial choices of unit- and group-level covariates, and for the binomial under group-level predictors alone.

## 3. Marginal Likelihood and Analytic Marginalization

The likelihood contribution for subject $i$ is expressed as:
\[
L_i(\beta, D, \alpha) = \int_{\mathbb{R}^q} \int \left[ \prod_{j=1}^{n_i} f(y_{ij} \mid b_i, \theta_i ; \beta)\right] f(\theta_i; \alpha) f(b_i; D) d\theta_i db_i.
\]
Analytic marginalization is performed over the conjugate random effect $\theta_i$, leveraging conjugacy to obtain, e.g., negative-binomial or beta-binomial-type densities. The remaining integral over the normal random effects $b_i$ is handled via classical approaches such as Gauss–Hermite quadrature or Laplace approximation [1101.0990].

A key property of CGLMMs is that for a wide range of settings—including those with both group- and unit-level covariates in Gaussian, Poisson, and gamma families—the entire marginal likelihood is available in closed form, yielding substantial computational advantages [1709.06288].

## 4. Key Special Cases and Model Families

Table 1 summarizes the principal CGLMM special cases for clustered data [1709.06288]:

| Family        | Conditional Model           | Conjugate Prior         | Marginal                |
|---------------|----------------------------|-------------------------|-------------------------|
| Gaussian      | $y_{ij} \sim N(\mu_i, \sigma^2)$    | $\mu_i \sim N(\lambda, \kappa^2)$         | Gaussian-integrated     |
| Poisson       | $y_{ij} \sim \mathrm{Poi}(\mu_i)$   | $\mu_i \sim \text{Gamma}(A,B)$            | Negative-binomial       |
| Binomial      | $y_{ij} \sim \mathrm{Bin}(n_{ij}, p_i)$ | $p_i \sim \mathrm{Beta}(A,B)$             | Beta-binomial           |
| Gamma         | $y_{ij} \sim \Gamma(A, \mu_i/A)$    | $\mu_i \sim \text{Inv-Gamma}(C,D)$        | Inverse-gamma           |

**Context and significance:** For Poisson outcomes, the CGLMM recovers the negative-binomial–normal model. For binary outcomes, it yields the beta–binomial–normal model, and for continuous measures, the hierarchical normal model. In each case, overdispersion and cluster-level correlation are jointly modeled.

## 5. Marginal Covariance, Correlation, and Overdispersion

CGLMMs allow explicit derivation of the marginal covariance and correlation structure for repeated measures or clustered observations. In the Poisson–Gamma–Normal case, for independent $\theta_{ij}$,
\[
\operatorname{Cov}(Y_{ij}, Y_{ik}) = \alpha \beta^2 E[\kappa_{ij} \kappa_{ik}] = \alpha \beta^2 \exp\{ X_{ij}^T\beta + X_{ik}^T\beta \} \exp \left(\frac12 z_{ij}^T D z_{ij} + \frac12 z_{ik}^T D z_{ik} + z_{ij}^T D z_{ik} \right).
\]
Marginal variances thus combine overdispersion (additional variation from conjugate effects) and correlation from normal random effects [1101.0990].

An immediate implication is that the closed-form correlation between observations within a cluster can be obtained analytically, facilitating inference on dependence structures and the impact of modeling choices on variance decomposition.

## 6. Inference, Computation, and Practical Implementation

With explicit marginal likelihoods, parameter estimation proceeds via direct maximum likelihood. After analytic marginalization over conjugate random effects, gradient and Hessian computations are facilitated, supporting classical Wald and likelihood-ratio test theory. Remaining numerical integration (over $b_i$) is typically one-dimensional or low-dimensional, depending on the correlation structure [1101.0990]. In big-data and federated learning environments, groupwise sufficient statistics can be calculated and communicated, simplifying computation and supporting privacy [1709.06288].

Standard estimation techniques include adaptive quadrature, Laplace approximation, or expectation-maximization (when treating random effects as missing data). In applied practice, platforms such as SAS NLMIXED can accommodate CGLMMs by analytic specification of the closest closed-form [1101.0990].

## 7. Illustrative Data Applications and Model Diagnostics

CGLMMs have been applied to epileptic seizure counts, clinical trials on onychomycosis, and childhood asthma survival. For the Poisson–Gamma–Normal CGLMM fit to epileptic seizure data with 45 placebo and 44 active subjects and weekly repeated counts, maximum likelihood estimates yield both overdispersion and random effect variance: $\alpha=2.4640$ (s.e. 0.2113) and $d=1.1289$ (s.e. 0.1850). The overdispersion index, $\operatorname{Var}(Y)/E[Y]=1+(\alpha \beta)\kappa_{ij}$, quantifies the departure from Poisson variance [1101.0990].

In treatment comparison, inference may differ from simpler mixed models; in this example, slope differences found to be significant under the Poisson–Normal disappear under the fuller CGLMM that accommodates overdispersion.

Closed-form marginal correlations allow model comparison and diagnostic assessment, with combined model correlations systematically below those from purely normal-effects models, reflecting the greater absorption of variance by the independent conjugate effects.

---

*Primary references:* [1101.0990], [1709.06288].

Source: https://www.emergentmind.com/topics/conjugate-glmms-cglmms