Incremental Seeded EM

• Incremental seeded EM is a method that integrates latent-variable inference with incremental updates and structured seeding to refine the initialization and update process.
• It employs stochastic approximation, hard-EM relabeling, and restart schemes across settings like mixture models, semantic segmentation, and point-set alignment.
• Convergence analyses reveal stationarity guarantees and improved iteration complexities, although nonconvexity and stability challenges remain.

Incremental Seeded Expectation Maximization denotes a family of EM-based procedures in which latent-variable inference is combined with incremental parameter or sufficient-statistics updates and with some form of seeding. Across the cited literature, the term is used for several closely related patterns: stochastic-approximation EM for streaming data, finite-sum incremental EM with memory and variance reduction, hard-EM relabeling driven by partial annotations, seeded initialization for mixture models, and domain-specific restart or alignment schemes that preserve the EM backbone while altering initialization and update order (Tran et al., 27 Jan 2026, Fort et al., 2020, Yan et al., 2021, Kuang et al., 7 Jul 2025, Evangelidis et al., 2016, Blömer et al., 2013, Karimi et al., 2019). The unifying principle is that latent responsibilities or expected sufficient statistics are not recomputed from scratch at every pass; instead, they are updated incrementally, while seeds determine the initial state, constrain the E-step, or provide structured restarts.

1. Formal structure and EM recursions

In its classical latent-variable form, EM optimizes an incomplete-data objective by alternating an E-step and an M-step. For models with tractable complete-data likelihood,

$$Q_{\mathrm{EM}}(\theta \mid \theta_t) = \mathbb{E}_{z \mid x,y,\theta_t}\!\left[\log p(y,z \mid x;\theta)\right].$$

Incremental variants replace full-dataset recomputation with samplewise or blockwise updates of sufficient statistics. In the online formulation emphasized for streaming settings, data arrive as an i.i.d. stream $X=(x,y)$ from a stationary distribution $\pi$, and one studies

$L(\theta)=\mathbb{E}[f(\theta;X)].$

For mixture-of-experts regression, a key instance is

$$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$

When EM applies, sufficient statistics are updated by stochastic approximation,

$$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$

with responsibilities

$r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$

followed by

$$\theta_{t+1}=\arg\max_\theta Q_{\mathrm{EM}}(\theta;S_{t+1}).$$

The same expectation-space viewpoint appears in finite-sum SAEM and FIEM. There, with

$$F(\theta)=\frac{1}{n}\sum_{i=1}^n f_i(\theta)+R(\theta),$$

the M-step map is

$$T(s)\in \arg\min_{\theta\in\Theta}\{-s^\top \phi(\theta)+R(\theta)\},$$

and EM is analyzed through the reparameterized objective $X=(x,y)$0 and mean field $X=(x,y)$1 (Tran et al., 27 Jan 2026, Fort et al., 2020).

This structure makes explicit that incremental seeded EM is not defined by one particular update rule. Rather, the common core is an EM-compatible latent-variable surrogate together with an incremental recursion for either sufficient statistics, per-index memories, or pseudo-labels.

2. The meaning of “seeded”

The meaning of seeding varies by application, but in every case it specifies the initial latent structure more informatively than an unstructured random start.

In finite mixtures and clusterwise linear regression, seeds are parameter or responsibility initializations. Recommended strategies include K-means on an initial batch, K-means on $X=(x,y)$2, random initialization with multiple restarts, perturbed ground truth in simulations, and logistic warm starts for gating parameters. In the FIEM framework, seeding simply fixes the initial sufficient statistics $X=(x,y)$3 and stored memories $X=(x,y)$4; the subsequent control-variate dynamics are unchanged, and under the stated assumptions seeding does not alter the nonasymptotic rates, though it may reduce the initial gap $X=(x,y)$5 (Fort et al., 2020, Kuang et al., 7 Jul 2025).

In incremental semantic segmentation, seeds are not merely initial values but observed labels. Incoming samples are $X=(x,y)$6, where $X=(x,y)$7 is the set of annotated pixels. For $X=(x,y)$8, the seed label is fixed; for $X=(x,y)$9, the label is latent and is imputed in a hard-EM E-step subject to a seed-consistency constraint that excludes the current task’s novel classes from unlabeled regions (Yan et al., 2021).

In point-set alignment, seeds are the current central GMM parameters together with initial rigid poses for a newly arriving set. The incremental EM step then aligns the new set to the existing model without reprocessing all past sets (Evangelidis et al., 2016).

In Gaussian-mixture initialization, seeding is sequential in the number of components: components are added one-by-one using a criterion based on the minimum Mahalanobis distance to the current partial GMM. The paper terms these procedures SphericalGonzalez and Adaptive, and both build partial models $\pi$0 before a final EM phase (Blömer et al., 2013).

Setting	Seed definition	Incremental mechanism
Finite-sum latent-variable EM	Initial $\pi$1 and memory $\pi$2	SAEM, iEM, FIEM updates
Semantic segmentation	Annotated pixels $\pi$3	Hard-EM relabeling plus SGD M-step
Clusterwise linear regression	K-means or random parameter seeds; elite restarts	EM with Cluster Revival and Elite Recombination
Point-set alignment	Current GMM and initial pose $\pi$4	New-set responsibilities and sufficient-statistics fusion

A common misconception is that seeding is only an initialization heuristic. The segmentation formulation shows that seeds can also act as hard constraints in the E-step, while clusterwise regression uses seeded restarts as an integral part of the optimization procedure.

3. Incremental EM within the stochastic MM framework

A central theoretical development is the embedding of incremental stochastic EM into incremental stochastic Majorization-Minimization. The MM construction introduces a majorizer for the instantaneous loss,

$\pi$5

with equality at $\pi$6, and parameterizes the surrogate in exponential-family-like form

$\pi$7

The stochastic update becomes

$\pi$8

When a latent-variable representation exists and the EM $\pi$9-function can be written in the same exponential-family form, online EM is recovered as a special case: the MM sufficient-statistic recursion coincides with the stochastic EM sufficient-statistic recursion, and the surrogate minimization coincides with the EM M-step (Tran et al., 27 Jan 2026).

This generalization relaxes explicit EM requirements. It removes the need for an explicit latent-variable model and for closed-form E/M steps, provided one can construct a valid majorizer that upper-bounds the per-sample loss and is tangent at the current iterate. The paper uses this to treat softmax-gated mixture-of-experts models, for which “no stochastic EM algorithm is available.” In that setting, the negative log-likelihood is shown to be non-Lipschitz, non-smooth, and nonconvex, so generic incremental EM/MM methods depending on global Lipschitz smoothness or convexity are not directly applicable. The tailored surrogate combines a corrected quadratic upper bound for the gating softmax log-sum-exp with linearized expert terms, and the resulting updates solve linear systems for gating and expert blocks while maintaining online sufficient statistics (Tran et al., 27 Jan 2026).

A plausible implication is that “incremental seeded EM” is best understood not as a single closed-form algorithm, but as the EM-specialized region of a broader incremental surrogate-optimization design space.

4. Convergence theory and computational regimes

For stochastic incremental MM, almost sure convergence is established under assumptions A1–A11, including differentiability, a unique surrogate minimizer map $L(\theta)=\mathbb{E}[f(\theta;X)].$0, bounded second moments, stability, and Robbins–Monro step sizes. The main stationarity guarantee is

$L(\theta)=\mathbb{E}[f(\theta;X)].$1

provided the Lyapunov, stability, and step-size conditions hold. The Lyapunov analysis uses

$L(\theta)=\mathbb{E}[f(\theta;X)].$2

together with martingale-difference noise control and the mean-field condition

$L(\theta)=\mathbb{E}[f(\theta;X)].$3

(Tran et al., 27 Jan 2026).

Finite-sum incremental EM admits sharper nonasymptotic statements. In the global-convergence analysis of incremental EM, if $L(\theta)=\mathbb{E}[f(\theta;X)].$4 is drawn uniformly from $L(\theta)=\mathbb{E}[f(\theta;X)].$5, then iEM satisfies

$L(\theta)=\mathbb{E}[f(\theta;X)].$6

which yields an $L(\theta)=\mathbb{E}[f(\theta;X)].$7 iteration complexity for reaching an $L(\theta)=\mathbb{E}[f(\theta;X)].$8-stationary point in expectation. Variance-reduced methods improve this to $L(\theta)=\mathbb{E}[f(\theta;X)].$9 through SEMVR and FIEM (Karimi et al., 2019).

The FIEM analysis sharpens this picture. With constant step size and random termination, two strategies are given for achieving an $$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$0-approximate stationary point: one with

$$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$1

and one with

$$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$2

The first favors higher-accuracy regimes; the second improves scaling in $$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$3 for small-to-medium accuracy. The stationarity measures

$$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$4

satisfy $$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$5, so the EM-specific mean-field residual controls gradient stationarity in expectation (Fort et al., 2020).

These guarantees do not imply global optimality. The recurrent result is convergence to stationary points, not to the global maximum of a nonconvex likelihood.

5. Major application domains

In softmax-gated mixture-of-experts regression, the conditional model is

$$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$6

For softmax-gated Gaussian and multinomial logistic MoE, incremental stochastic MM updates online surrogate statistics, gating parameters, expert parameters, and variances. Empirically, the method “consistently outperforms widely used stochastic optimizers, including stochastic gradient descent, root mean square propagation, adaptive moment estimation, and second-order clipped stochastic optimization.” On the dent maize genotypes dataset, Incremental MM and gated logistic warm start (Incremental MM*) achieve “the best or second-best metrics (MSE≈0.12; NRMSE≈0.14)” with stable 5-fold CV behavior (Tran et al., 27 Jan 2026).

In online incremental semantic segmentation, the E-step imputes missing pixel labels by

$$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$7

retained only if the confidence exceeds $$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$8, while seed pixels remain fixed. The M-step optimizes

$$L(\theta)=\mathbb{E}_{(x,y)}[-\log p(y\mid x;\theta)].$$9

with rehearsal, adaptive class-level sampling, and cosine normalization. On PASCAL VOC 2012, the 15-1 disjoint setting gives old $$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$0–$$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$1, new $$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$2–$$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$3, and all $$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$4, which is reported as a gain over MiB* of $$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$5 vs $$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$6 $$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$7. On ADE20K 50-50, the method reaches all $$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$8 and is reported to beat LwF.MC* by $$S_{t+1}=(1-\alpha_{t+1})S_t+\alpha_{t+1}s(x_{t+1},y_{t+1},r_{t+1}),$$9 mIoU $r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$0 vs $r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$1 (Yan et al., 2021).

In clusterwise linear regression, ISEM or EMis augments the standard EM recursion for finite mixtures of linear regressions with two restart mechanisms: Elite Recombination and Cluster Revival. Elite Recombination recombines cluster proposals from low-error solutions; Cluster Revival regenerates collapsed clusters from data absorbed by a “supercluster” using Edge-Point K-flat or Center-Point Splitting. The same work introduces Resolvability,

$r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$2

and X-predictability,

$r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$3

as diagnostics for model quality and predictive ambiguity. Over “14,000 independent problems,” runtime is reported empirically as

$r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$4

and for $r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$5 the learned mapping between pairwise resolvability and recovery accuracy is summarized as $r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$6 (Kuang et al., 7 Jul 2025).

In joint alignment of multiple point sets, incremental EM treats transformed observations

$r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$7

as samples from a central GMM with an optional uniform outlier component. A newly arriving point set is incorporated by computing responsibilities against the current GMM, updating its rigid transform by weighted Procrustes, and then updating global sufficient statistics $r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$8 without revisiting all previous data. This yields an online EM formulation for registration rather than clustering or regression (Evangelidis et al., 2016).

6. Limitations, misconceptions, and extensions

Several limitations recur across the literature. Nonconvexity is ubiquitous: mixture-of-experts objectives are nonconvex, clusterwise regression suffers from collapse and local minima, and EM-type guarantees are stationary-point guarantees rather than guarantees of global recovery. Stability assumptions are also substantial. In the stochastic MM analysis, stability of the SA iterates is taken as a standing assumption; the text notes that such stability “may be ensured via truncation or projection schemes in general SA but is taken as a standing assumption here.” In FIEM and related variance-reduced schemes, memory overhead is intrinsic because per-index sufficient statistics must be stored (Tran et al., 27 Jan 2026, Fort et al., 2020).

A second misconception is that EM monotonicity survives unchanged under all incremental seeded variants. It does not. The semantic-segmentation formulation is explicitly a stochastic, hard-EM variant for which “classical EM monotonicity of $r_{t+1}(k)=p(z=k\mid x_{t+1},y_{t+1};\theta_t),$9 is not guaranteed under hard assignments and stochastic updates” (Yan et al., 2021).

A third misconception is that seeding changes the formal rates. In the SAEM/FIEM analysis, seeding changes the initial state and can reduce $$\theta_{t+1}=\arg\max_\theta Q_{\mathrm{EM}}(\theta;S_{t+1}).$$0, but “under the paper’s assumptions (A1–A5), seeding does not alter the nonasymptotic rates” (Fort et al., 2020).

The principal extensions proposed in the cited work are informative priors or proximal warm-up terms, adaptive seeding, mini-batching, Polyak averaging, and hybrid MM–EM updates in which tractable blocks use closed-form EM while intractable blocks use MM surrogates. This suggests a practical decision rule already stated explicitly for mixture-of-experts: use incremental seeded EM “whenever the latent-variable model admits closed-form expected sufficient statistics and easy M-steps,” and prefer incremental stochastic MM “when EM is unavailable, unstable, or impractical—particularly for softmax-gated MoE with polynomial gating/expert functions” (Tran et al., 27 Jan 2026).