Stochastic EM (SEM)

• Stochastic EM (SEM) is a class of estimation methods that replaces the exact E-step with stochastic simulation techniques to overcome intractable latent variable integrations.
• It employs diverse strategies such as single imputation, Monte Carlo averaging, and recursive Robbins–Monro updates, tailored to the model’s latent structure.
• SEM enhances robustness and efficiency by enabling tractable likelihood maximization, particularly in state-space, mixture models, and high-dimensional settings.

Searching arXiv for recent and foundational papers on stochastic EM variants to ground the article in the literature. arXiv search query: "all:stochastic EM algorithm latent variables SAEM MCEM" Stochastic expectation-maximization (SEM) denotes a class of incomplete-data estimation procedures obtained by relaxing the exact E-step of EM. In the standard formulation, with observed data $x$, latent data $z$, complete data $(x,z)$, and incomplete-data likelihood $L(\theta;x)=\int f(x,z;\theta)\,dz$, EM iterates the auxiliary function $$Q(\theta,\theta^{(i-1)})=\mathbb{E}[\log f(x,z;\theta)\mid x,\theta^{(i-1)}]$$ and the maximization step $$\theta^{(i)}=\arg\max_\theta Q(\theta,\theta^{(i-1)})$$ (Tadayon, 2018). SEM-type methods replace that exact conditional expectation by stochastic simulation, Monte Carlo averaging, stochastic approximation, or subset-based estimates when the conditional law of the missing data is unavailable or too costly to compute exactly. The literature does not use the label uniformly: some papers reserve “SEM” for single random completion, some use it for Monte Carlo E-steps, and others distinguish SAEM, online EM, or variance-reduced stochastic EM as more specific descendants (Gruffaz et al., 2024).

1. Incomplete-data structure and the EM baseline

The common substrate of SEM is the incomplete-data model. The latent component may be a discrete assignment matrix $Z$ in mixture models, a permutation matrix $\Pi$ in shuffled regression, a full state trajectory $x_{0:T}$ in state-space models, hidden units and missing records in probabilistic neural networks, or dropout-completed responses in longitudinal data (Blömer et al., 2013). In nonlinear state-space form, one representative factorization is

$$p_{\theta}(x_{0:T}, y_{1:T}) = p_{\theta} ( x_0 ) ~ \prod_{t =1}^T p_{\theta} (x_{t}\mid x_{t-1} ) ~ \prod_{t =1}^T p_{\theta} ( y_{t}\mid x_t ),$$

with observed-data likelihood

$z$0

and EM auxiliary function

$z$1

(Chau et al., 2018).

Across applications, the same obstruction recurs: direct maximization of the observed-data likelihood is difficult because the missing-data conditional law is high-dimensional, combinatorial, or analytically awkward. In shuffled linear regression, exact EM is intractable because $z$2 requires summing over $z$3 permutations (Abid et al., 2018). In nonlinear mixed-effects models, the conditional law $z$4 is not analytically tractable (Karimi et al., 2019). In robust state-space imaging with compound-Gaussian noise, the posterior $z$5 couples states and latent scales, so the exact E-step is not available in closed form (Arab et al., 22 Jun 2026).

This suggests that SEM is less a single algorithm than a strategy for substituting tractable stochastic operations for an intractable conditional expectation.

2. Main meanings of “SEM” in the literature

The term “stochastic EM” is used in several non-equivalent ways. Some papers treat them as members of one family; others insist on sharper distinctions.

Variant	Stochastic replacement	Representative paper
Classical SEM	One simulated completion $z$6, then maximize $z$7	(Tadayon, 2018)
MCEM-like SEM	Monte Carlo average of sampled latent completions or trajectories in the E-step	(Chau et al., 2018)
SAEM	Robbins–Monro recursion on sufficient statistics, score, or information, often with MCMC	(Gruffaz et al., 2024)
Online or variance-reduced stochastic EM	Random subset or minibatch sufficient-statistic updates, sometimes with control variates	(Kereta et al., 2021)

A classical statement of the single-imputation form is

$z$8

which the SAEM review explicitly separates from MCEM and SAEM (Tadayon, 2018). By contrast, particle-based state-space SEM approximates the E-step with

$z$9

using sampled smoothing trajectories; that paper notes that this is “generally named Stochastic EM (SEM) algorithm in the literature,” while also remarking that it is close to Monte Carlo EM (Chau et al., 2018).

A further refinement appears in SAEM. There the stochastic object is not a fresh Monte Carlo approximation at each iteration, but a recursion such as

$(x,z)$0

or, in sufficient-statistic form,

$(x,z)$1

with decreasing step sizes satisfying Robbins–Monro conditions (Gruffaz et al., 2024). A plausible implication is that “SEM” functions as an umbrella label, while MCEM, SAEM, online EM, and variance-reduced SEM denote structurally distinct stochastic relaxations of EM.

3. Simulation mechanisms for the latent variables

The stochastic step in SEM is determined by the geometry of the latent space. In mixture models, SEM samples hard assignments from the posterior responsibilities $(x,z)$2, then updates weights, means, and covariances from the sampled allocation (Blömer et al., 2013). In shuffled linear regression, the latent variable is a permutation matrix, and the E-step is approximated by Metropolis–Hastings over the permutation group with local swap proposals; the M-step then uses the posterior average of sampled permutation matrices (Abid et al., 2018).

In nonlinear state-space models, the latent variable is often a full trajectory. A prominent construction combines a Conditional Particle Filter with Backward Simulation (CPF-BS). One forward conditional particle pass produces filtering particles, and repeated backward simulation then generates $(x,z)$3 smoothing trajectories from an approximation of

$(x,z)$4

with backward kernel

$(x,z)$5

(Chau et al., 2018). The same CPF-BS idea is reused in nonparametric state-space estimation, where SEM alternates between latent-state smoothing and a local-linear-regression update of the unknown dynamics (Chau et al., 2020).

In SAEM for nonlinear mixed-effects models, latent random effects are simulated by MCMC kernels targeting $(x,z)$6. A notable acceleration uses an independent Metropolis–Hastings proposal

$(x,z)$7

where $(x,z)$8 is the conditional mode and $(x,z)$9 comes from a Laplace approximation or, for continuous data, from linearization of the structural model (Karimi et al., 2019). In robust state-space radio interferometric imaging, the stochastic E-step is a block Gibbs sampler: FFBS samples the state trajectory conditional on latent textures, and each texture variable has a Gamma full conditional

$L(\theta;x)=\int f(x,z;\theta)\,dz$0

(Arab et al., 22 Jun 2026).

Other domains use different kernels. Hybrid probabilistic neural networks perform stochastic completion by Gibbs-style single-site updates using local conditional probabilities derived from a Gibbs distribution (Paass, 2013). Longitudinal selection models with non-ignorable dropout generate missing responses from the Gaussian conditional outcome model and accept proposals according to the current logistic dropout model (Gad et al., 2022). These examples make clear that SEM is defined less by a specific sampler than by the decision to replace an exact E-step integral with sampled latent completions.

4. Objective functions, sufficient statistics, and M-steps

The M-step in SEM retains the complete-data perspective of EM, but its algebra depends strongly on model structure. In Gaussian state-space models with sampled latent trajectories $L(\theta;x)=\int f(x,z;\theta)\,dz$1, the Monte Carlo complete-data objective yields explicit covariance updates: $L(\theta;x)=\int f(x,z;\theta)\,dz$2

$L(\theta;x)=\int f(x,z;\theta)\,dz$3

(Chau et al., 2018).

In shuffled regression, the exact M-step has closed form: $L(\theta;x)=\int f(x,z;\theta)\,dz$4 and the stochastic version replaces $L(\theta;x)=\int f(x,z;\theta)\,dz$5 by the empirical average of sampled permutations (Abid et al., 2018). In exploratory item factor analysis, SAEM maintains recursively averaged sufficient statistics of augmented Gaussian responses and updates loadings through eigenanalysis of

$L(\theta;x)=\int f(x,z;\theta)\,dz$6

so the M-step becomes a factor-extraction problem rather than a generic nonlinear optimization (Geis, 2019).

Not every descendant of SEM is pure likelihood maximization. In discriminative sdEM, the model remains generative and exponential-family, but the optimized criterion is a discriminative loss such as negative conditional log-likelihood or Hinge loss. The update is written in expectation-parameter space as a stochastic natural-gradient step,

$L(\theta;x)=\int f(x,z;\theta)\,dz$7

with the natural-gradient identity supplied by exponential-family duality (Masegosa, 2014). In nonparametric state-space estimation, the authors explicitly call the method “SEM-like” rather than genuine SEM because the update of the transition map $L(\theta;x)=\int f(x,z;\theta)\,dz$8 is a catalog-based local linear regression step, not the maximization of a single expected complete-data log-likelihood (Chau et al., 2020).

A recurrent pattern nevertheless remains: once the latent variables or sufficient statistics have been stochastically updated, the maximization step is typically much simpler than the original observed-data optimization.

5. Convergence theory, approximation error, and common misconceptions

A central misconception is that all SEM variants inherit the deterministic monotonicity of exact EM. The literature does not support that view. Exact EM increases the likelihood each iteration and converges to a local maximum under standard conditions, but particle-based SEM, EnKS-EM approximations, and other stochastic variants generally do not come with a formal monotonicity theorem (Chau et al., 2018). In mixture cure models, the SEM iterates are described not as pointwise convergent but as converging in distribution to a stationary Gaussian regime, so final estimates are taken from post-burn-in summaries rather than a fixed point (Barui et al., 2021).

Theory is strongest in several specific directions. For Gaussian mixture models, one-step proximity results show that, with high probability, SEM updates for weights, means, and covariances are close to the corresponding EM updates when the effective component mass is large enough; the same work reports that the stochastic variant runs nearly twice as fast (Blömer et al., 2013). For SAEM, the standard Robbins–Monro conditions

$L(\theta;x)=\int f(x,z;\theta)\,dz$9

govern convergence of the stochastic approximation recursion (Tadayon, 2018). In biased-MCMC SAEM, asymptotically biased kernels do not recover the exact stationary set, but the iterates converge to bias-controlled neighborhoods of it; the asymptotic error depends on the induced MCMC bias rather than vanishing automatically (Gruffaz et al., 2024).

Another misconception is that “SEM,” “MCEM,” and “SAEM” are interchangeable. The review and methodological papers repeatedly separate them: classical SEM uses one stochastic completion, MCEM uses Monte Carlo averages in the E-step, and SAEM uses recursive stochastic approximation, often with MCMC (Tadayon, 2018). This suggests that precise terminology matters, particularly when discussing convergence, memory, and computational cost.

6. Empirical behavior across application domains

SEM and its descendants appear in mixture estimation, state-space inference, missing-data regression, survival analysis, psychometrics, imaging, and probabilistic neural modeling. In shuffled linear regression, treating the unknown permutation probabilistically and using stochastic EM yields lower parameter error, less sensitivity to initialization, and significantly better performance on partially shuffled data than the hard-EM formulation that chooses a single best permutation (Abid et al., 2018). In cure-rate survival modeling with exponentiated Weibull lifetimes, SEM and EM are similar when initialization is good, but SEM is markedly more robust to poor starting values; the paper highlights divergence-rate differences under badly chosen starts (Barui et al., 2021). In longitudinal dropout models with missing responses and covariates, SEM is combined with multiple imputation for covariates and a selection model for non-ignorable dropout, with Monte Carlo standard errors obtained from a Louis-type information approximation (Gad et al., 2022).

State-space models provide some of the clearest large-scale SEM case studies. CPF-BS-SEM for nonlinear state-space models estimates parameters accurately with few particles, often outperforming EnKS-based EM when dynamics are strongly nonlinear (Chau et al., 2018). Its nonparametric extension, npSEM, alternates between stochastic smoothing and local-linear-regression updates of the transition operator, and the paper reports that iteratively updating the dynamics catalog is essential for good reconstruction (Chau et al., 2020). In robust radio interferometric imaging, a heavy-tailed SAEM with Gibbs updates for latent states and textures improves reconstruction fidelity under radio-frequency interference and outperforms both a Gaussian EM algorithm and an oracle RTS smoother (Arab et al., 22 Jun 2026).

Large-scale optimization has motivated further refinements. SPIDER-EM imports SPIDER/SARAH variance reduction into the E-step and establishes

$$Q(\theta,\theta^{(i-1)})=\mathbb{E}[\log f(x,z;\theta)\mid x,\theta^{(i-1)}]$$0

for smooth nonconvex latent-variable estimation (Fort et al., 2020). In penalised PET reconstruction, stochastic variance-reduced EM updates sufficient statistics rather than gradients, and the resulting SVREM substantially outperforms OSEM-type baselines while retaining EM-style surrogate structure (Kereta et al., 2021). A plausible implication is that modern SEM research increasingly treats stochasticity not merely as a workaround for intractable E-steps, but as an algorithmic design space encompassing variance reduction, control variates, biased and unbiased MCMC, and discriminative or nonparametric extensions of the EM paradigm.