Mixed-Effects Scaling Models

• Mixed-effects scaling models are advanced statistical frameworks that explicitly parameterize heteroscedastic variance as a function of covariates and random effects in hierarchical or longitudinal data.
• They employ dual submodels for the location (mean) and scale (variance) components, enabling joint estimation and improved model-based diagnosis.
• Estimation techniques such as adaptive Gaussian quadrature, Laplace approximations, and EM-type algorithms ensure efficient computation and robust inference in complex data settings.

Mixed-effects scaling models, including the mixed-effects location–scale model (MELSM), generalize classical linear mixed models by explicitly parameterizing within-cluster (or within-subject) residual variance as a function of covariates and random effects. This framework enables joint modeling of both mean and heteroscedastic variance structures across complex, hierarchical, or longitudinal data settings, with substantial implications for robust inference, variance decomposition, and model-based diagnosis.

1. Classical Mixed-Effects Models and the Need for Scale Modeling

A conventional linear mixed model (LMM) for repeated measures or clustered data employs the form

$$y_{ij} = x_{ij}^T\beta + z_{ij}^T b_i + \varepsilon_{ij}, \qquad b_i \sim N(0, D),\quad \varepsilon_{ij}\sim N(0, \omega^2)$$

where $b_i$ are random effects (e.g., subject- or cluster-specific), and the residual variance $\omega^2$ is assumed homoscedastic. However, in many applications, this assumption is violated: units may vary not only in mean levels but also in dispersion, due to biological, institutional, or process heterogeneity. Neglecting this variance heterogeneity can result in underestimated (or overestimated) standard errors and lead to biased inference for both fixed and random effects (Jeanselme et al., 23 May 2025, Leckie et al., 2021).

2. Mathematical Specification of Mixed-Effects Scaling Models

The mixed-effects location–scale model extends the LMM by introducing a parametric model for the log-residual variance:

• Location (mean) submodel:

$$y_{ij} = x_{ij}^T\beta + z_{ij}^T b_i + \varepsilon_{ij},\qquad b_i \sim N(0,D)$$

• Scale (variance) submodel:

$$\log(\sigma_{ij}^2) = w_{ij}^T\alpha + u_i,\qquad u_i \sim N(0, \tau^2)$$

• Conditional errors:

$$\varepsilon_{ij} | b_i, u_i \sim N(0, \sigma_{ij}^2),\quad \sigma_{ij}^2 = \exp(w_{ij}^T\alpha + u_i)$$

Covariates $x_{ij}$ and $w_{ij}$ may overlap; $z_{ij}$ and $u_i$ encode random effects for mean and log-variance, respectively (Jeanselme et al., 23 May 2025). Correlation between $b_i$ and $u_i$ can be parameterized for joint modeling of mean–variance interplay.

3. Estimation Procedures

Estimation in MELSMs requires integration over latent random effects in both mean and variance components, complicating the likelihood:

$$\ell(\beta, \alpha, D, \tau^2) = \sum_{i=1}^N \log \iint \Bigg[ \prod_{j=1}^{n_i} (2\pi\sigma_{ij}^2)^{-1/2} \exp\left\{ -\frac{(y_{ij}-x_{ij}^T\beta - z_{ij}^T b_i)^2}{2\sigma_{ij}^2} \right\} \Bigg] \phi(b_i;0, D)\phi(u_i;0,\tau^2) db_i du_i$$

This is maximized by adaptive Gaussian quadrature or Laplace approximations. Maximum likelihood (ML) or restricted maximum likelihood (REML) criteria are used in frequentist settings, while full Bayesian inference relies on MCMC, as in brms (Jeanselme et al., 23 May 2025, Leckie et al., 2021).

For generalized response distributions (e.g., generalized hyperbolic or mixtures for skew/extreme observations), the corresponding likelihoods require further integration or specialized moment-based initialization and Newton–Raphson algorithms for efficiency and robustness (Fujinaga et al., 2022, Schumacher et al., 2020). Approximate EM-type algorithms are deployed for models with complex error structures.

4. Robustness and Misspecification: Simulation Evidence

Extensive simulation studies have demonstrated that:

• Fitting standard LMMs to heteroscedastic data results in downward-biased estimates of random effect variances and poor (sub-nominal) coverage probabilities for fixed effects (as low as 60–80%), due to underestimated standard errors.
• Correctly specified MELSMs maintain negligible bias and proper coverage for fixed and random effects, even under moderate sample sizes.
• Scale misspecification (e.g., omitting relevant predictors from the variance model) inflates standard errors for mean-parameters but does not induce mean-bias. Location misspecification (mis-specified mean model) biases scale estimates because variance components absorb unmeasured trends (Jeanselme et al., 23 May 2025).
• These results generalize to multilevel contexts, e.g., schools or clusters, where joint modeling of mean and variance components separates “high-consistency” and “boom-bust” units (Leckie et al., 2021).

Model	Mean Estimate Bias	Std. Error Coverage	Variance Estimate Bias
Standard LMM	~0	60–80%	Downward
Correct MELSM	~0	~95%	Negligible

Under misspecification, coverage deteriorates primarily for the neglected submodel.

5. Practical Recommendations and Extensions

Best practice entails:

• Systematically assessing variance heterogeneity using diagnostic residual plots and formal tests (e.g., Levene’s).
• Introducing a variance (scale) submodel when data suggest heteroscedasticity (biological variability, learning, etc.).
• Using likelihood-ratio tests, Bayesian credible intervals, or Bayes factors to justify inclusion of variance effects (Jeanselme et al., 23 May 2025).
• In small samples, parsimony is crucial: begin with random intercepts, add slope components only as needed to avoid overfitting.
• MELSM naturally generalizes to generalized linear mixed-effects models (GLMMs) by linking both mean and variance (dispersion) parameters to covariates, and extends to models with non-normal outcomes or joint models incorporating time-to-event data.
• Joint modeling of mean and variance allows institutional or group-level variance heterogeneity to be separated from individual-level variability, yielding interpretable metrics of “consistency” vs. “volatility” (Leckie et al., 2021).

6. Model Extensions and Robustification

Advanced variants extend MELSM to accommodate non-normal data or increase robustness:

• Mixed-effects scaling models based on the generalized hyperbolic distribution introduce skewness and heavy tails at both mean and dispersion levels, requiring a combination of method-of-moments initialization and Newton–Raphson updating for tractable and stable estimation (Fujinaga et al., 2022).
• Scale mixture extensions, such as the use of skew-normal mixtures for random effects and errors, increase resilience to outliers and provide flexible modeling of asymmetric and heavy-tailed processes. Efficient estimation leverages EM-type algorithms with linearization and block-wise maximization (Schumacher et al., 2020).
• Single-index mixed-effects scaling models for high-dimensional non-Gaussian survey data allow for monotonic, nonparametric link functions and incorporate skewed random effects, heavy-tailed residuals, and survey-weight adjustment in a Bayesian MCMC framework. Applications include the assessment of disease progression in complex medical survey data (Liu et al., 25 Sep 2025).

7. Computational and Software Considerations

Empirical scalability is critical for MELSM in high-dimensional or large-scale grouped data. Krylov subspace methods (conjugate gradient and stochastic Lanczos quadrature) enable inference for generalized mixed-effects models with crossed random effects at dramatically reduced runtimes compared to Cholesky-based computations. These methods, implemented in C++/R/Python libraries (e.g., GPBoost), support tractable estimation and uncertainty quantification when the number of random effects is large (Kündig et al., 14 May 2025).

Expectation propagation (EP) algorithms with block-sparse reparameterization facilitate scalable, distributed approximate Bayesian inference for mixed-effects regression, achieving linear scaling with the number of groups and maintaining statistical accuracy (Zhou et al., 23 Sep 2024).

• (Jeanselme et al., 23 May 2025) Assessing the impact of variance heterogeneity and misspecification in mixed-effects location-scale models
• (Leckie et al., 2021) Mixed-effects location scale models for joint modelling school value-added effects on the mean and variance of student achievement
• (Fujinaga et al., 2022) Mixed-effects location-scale model based on generalized hyperbolic distribution
• (Schumacher et al., 2020) Approximate inferences for nonlinear mixed effects models with scale mixtures of skew-normal distributions
• (Liu et al., 25 Sep 2025) An Interpretable Single-Index Mixed-Effects Model for Non-Gaussian National Survey Data
• (Kündig et al., 14 May 2025) Scalable Computations for Generalized Mixed Effects Models with Crossed Random Effects Using Krylov Subspace Methods
• (Zhou et al., 23 Sep 2024) Scalable Expectation Propagation for Mixed-Effects Regression