Mixed-Effects Linear Regression

• Mixed-effect linear regression models are statistical tools that integrate fixed effects to capture overall trends and random effects to account for cluster-specific deviations.
• They utilize ML and REML estimation methods to provide unbiased variance component estimates and address complex data structures like random intercepts and slopes.
• Applications span biostatistics, econometrics, and social sciences, enabling robust analysis of longitudinal, multilevel, and clustered studies.

A mixed-effect linear regression model extends classical linear regression to account for dependence between observations due to hierarchical, clustered, or repeated-measures study designs. It incorporates both fixed effects—parameters associated with the entire population or certain repeatable levels of experimental factors—and random effects, which model subject- or cluster-specific deviations. Mixed-effect linear models are foundational in the analysis of longitudinal, multilevel, or clustered data across domains such as biostatistics, econometrics, and social sciences.

1. Model Specification and Structure

The canonical mixed-effect linear regression model is represented as

$$y_{ij} = x_{ij}^\top\beta + z_{ij}^\top b_i + \varepsilon_{ij},\quad b_i\sim N(0,\Sigma),\, \varepsilon_{ij}\sim N(0,\sigma^2)$$

where:

• $y_{ij}$: response for observation $j$ in cluster (or subject) $i$,
• $x_{ij}$: vector of fixed-effect covariates (common to all clusters),
• $\beta$: fixed-effect coefficients,
• $z_{ij}$: vector of random-effect covariates (vary by cluster),
• $b_i$: cluster-specific random effect, typically multivariate normal with covariance $\Sigma$,
• $\varepsilon_{ij}$: residual error, usually assumed i.i.d. normal.

This model framework accommodates random intercepts, random slopes, and general random-effects structures. Marginally, the distribution of $y_{ij}$ is Gaussian with mean $x_{ij}^\top\beta$ and covariance that reflects both between- and within-cluster variation.

Extensions include functional data models that allow both response and covariates to be functions over a domain and admit random processes as random effects (Liu et al., 2016), as well as partially linear models where fixed-effects coefficients are complemented by nonparametric or high-dimensional nuisance functions (Emmenegger et al., 2021).

2. Estimation Methodologies

Parameter estimation in mixed-effect linear regression proceeds via maximum likelihood (ML) or restricted maximum likelihood (REML), both of which require integrating over the distribution of random effects due to their “latent” character. The marginal log-likelihood is:

$$\ell(\beta, \Sigma, \sigma^2) = -\frac{1}{2} \Bigl\{ \log|V| + (y - X\beta)^\top V^{-1}(y - X\beta) \Bigr\}$$

with total covariance $V = Z \Sigma Z^\top + \sigma^2 I$.

REML adjusts for fixed-effect estimation and is preferred for unbiased estimation of variance components (Leckie, 2019). Algorithms for maximizing the likelihood include EM, Newton-Raphson, and variants using sparse Cholesky factorization for high-dimensional or multivariate models (Adjakossa et al., 2017). Closed-form solutions for the fixed effects emerge from the generalized least squares estimator given variance components.

When random effects or errors deviate from normality, variance-mean mixture models (e.g., normal inverse Gaussian, generalized hyperbolic) and stochastic-gradient-based ML estimation generalize the framework (Asar et al., 2018).

3. Model Selection and Penalization

Model selection in mixed-effect regression addresses both fixed and random effects. Four principal strategies are recognized (Müller et al., 2013):

• Information Criteria (IC): AIC, BIC, and conditional AIC adapt the likelihood with parameter penalties. For BIC, the presence of dependence reduces the effective sample size, and criteria such as the effective-sample-size BIC ($\log n_{\text{eff}}$) provide improved theoretical justification and model selection consistency (Shen et al., 2021).
• Penalized Likelihood: $\ell_1$ (lasso), SCAD, and group penalties regularize high-dimensional mean or covariance components. Group LASSO is suitable for random-effect structures, ensuring positive semidefiniteness of the random-effects covariance (Hultman et al., 18 Mar 2025). Shrinkage methods provide oracle properties under fixed dimensionality and appropriate tuning.
• Fence Procedures: Methods such as adaptive Fence use simulation-based selection to identify optimal submodels under a lack-of-fit plus variability criterion.
• Bayesian Techniques: Spike-and-slab priors for variance components, Bayes factors, and DIC support model averaging and selection, at substantially increased computational cost.

For models with massive numbers of categorical predictors and interactions, approaches recasting them as group-specific random effects yield scalable and theoretically consistent prediction, as in the PMMP framework (Sun et al., 2024).

4. Algorithmic Innovations and Extensions

Contemporary developments extend model flexibility and computational tractability:

• Gradient Boosting: Gradient boosting tailored for mixed models employs alternating updates for fixed and random components, BLUP-based baselearners, and AIC/cross-validation for early stopping. Proper separation of fixed/random effect updates rectifies bias, selection imbalance, and convergence issues in high-dimensional settings (Griesbach et al., 2020).
• Double Machine Learning (DML): In models with high-dimensional or nonparametric nuisance covariates, DML applies machine learning (e.g., random forests) to orthogonalize out nuisance effects from both predictors and response, yielding root-$N$ consistent, semiparametrically efficient estimation of fixed effects (Emmenegger et al., 2021).
• Multivariate & Functional Data: For multivariate responses, models explicitly parameterize cross-equation random-effects correlation (block Cholesky factorization, profiled deviance minimization) (Adjakossa et al., 2017). In the functional setting, spline/EM approaches estimate both mean trajectories and random deviations (Liu et al., 2016).
• Mixture-of-Experts with Mixed Effects: MEMoE generalizes LMMs to capture subgroup-specific fixed effects alongside subject-level random effects, using EM with Laplace integration and robust (sandwich) standard errors (Yue et al., 8 Mar 2026).

5. Applications and Interpretation

Mixed-effect linear regression is the de facto standard for longitudinal, multilevel, and cluster-randomized designs. Applications include:

• Longitudinal Biomedical Data: Quantifying individual- and group-level trajectories of disease markers, cognitive decline, or biometrics (Leckie, 2019).
• Functional/High-Dimensional Predictors: Modeling environmental or imaging data with underlying time- or space-varying structure (Liu et al., 2016).
• Sparse High-Dimensional Problems: Matrix-valued covariates in imaging or genomics exploit Kronecker-structured random effects to capture both mean and covariance sparsity (Hultman et al., 18 Mar 2025).
• Predictive Oncology: Patient-specific tumor growth forecasting leverages individual random effects for improved prediction over fixed effects alone (Nasiri et al., 2018).
• Categorical or Interaction-Dense Settings: Substantial categorical feature spaces, addressed via group random effects and scalable likelihoods (Sun et al., 2024).

Interpretation centers on fixed effects as population-average relationships, variance components as partitioners of variation (intra- vs inter-group), and BLUPs as cluster- or subject-specific predicted deviations. Intraclass correlation (ICC) quantifies the degree of clustering in the data.

6. Model Diagnostics, Robustness, and Practical Considerations

Standard diagnostics include graphical assessment of residual and random-effect normality, inspection of variance component plausibility, and evaluation of model-implied vs empirical correlations. Likelihood ratio tests (with appropriate mixture null distributions for variance components) and information criteria support structure comparison. Robust standard errors and sandwich variance estimates are mandated in complex or misspecified settings (Yue et al., 8 Mar 2026).

Choice of estimation and selection method is determined by data dimensionality, goal (inference vs prediction), and computational resources. High-dimensional or structured data require penalized likelihood or boosting (Griesbach et al., 2020, Hultman et al., 18 Mar 2025), whereas moderate-sized, well-structured problems remain amenable to standard REML/ML or Bayesian approaches. For non-Gaussian data or heavy-tailed distributions, variance-mean mixture methods are available (Asar et al., 2018).

Widespread software support exists (e.g., lme4, nlme, grbLMM, dmlalg) allowing routine application in standard statistical environments.

7. Theoretical Properties and Future Directions

Mixed-effect linear regression models enjoy extensive theoretical support: under standard regularity, ML and REML estimators are consistent and asymptotically normal; double machine learning yields semiparametric efficiency in the presence of high-dimensional nuisance (Emmenegger et al., 2021). Information criteria, when corrected for effective sample size, retain model selection consistency under dependence (Shen et al., 2021). Penalized and Bayesian procedures provide variable selection and post-selection inference under various sparsity and prior structures.

Active research areas include high-dimensional regularization of covariance structures, robust modeling under non-Gaussian and misspecified settings, computation for large-scale and complex random-effects structures, and integration with machine learning workflows for automated model fitting and selection.

---

References:

• "Double Machine Learning for Partially Linear Mixed-Effects Models with Repeated Measurements" (Emmenegger et al., 2021)
• "Bayesian Information Criterion for Linear Mixed-effects Models" (Shen et al., 2021)
• "Estimating Functional Linear Mixed-Effects Regression Models" (Liu et al., 2016)
• "Gradient Boosting for Linear Mixed Models" (Griesbach et al., 2020)
• "Profiled deviance for the multivariate linear mixed-effects model fitting" (Adjakossa et al., 2017)
• "Model Selection in Linear Mixed Models" (Müller et al., 2013)
• "Multilevel models for continuous outcomes" (Leckie, 2019)
• "Linear Mixed-Effects Models for Non-Gaussian Repeated Measurement Data" (Asar et al., 2018)
• "Regularized Parameter Estimation in Mixed Model Trace Regression" (Hultman et al., 18 Mar 2025)
• "A Random-effects Approach to Regression Involving Many Categorical Predictors and Their Interactions" (Sun et al., 2024)
• "Mixed Effects Mixture of Experts: Modeling Double Heterogeneous Trajectories" (Yue et al., 8 Mar 2026)
• "Mixed-Effect Modeling for Longitudinal Prediction of Cancer Tumor" (Nasiri et al., 2018)
• "Extended multivariate generalised linear and non-linear mixed effects models" (Crowther, 2017)