Multi-step Newton Iteration

• Multi-step Newton Iteration is an iterative parameter estimation procedure that uses staged Newton updates to adapt to shifting data distributions in online continual learning.
• It integrates a random effects model to account for task-specific deviations and reduces computational cost by minimizing frequent matrix inversions through effective batch grouping.
• Empirical and theoretical validations show its superior accuracy, lower mean squared error, and asymptotic normality, enabling reliable statistical inference in non-stationary environments.

A Multi-step Newton Iteration Algorithm, as developed for online continual learning in non-stationary environments, is an iterative parameter estimation procedure that incrementally updates model parameters as data arrives sequentially from heterogeneous tasks. The framework, operating under statistically principled assumptions, is designed to mitigate catastrophic forgetting by leveraging multi-stage Newton updates and a random effects model for task heterogeneity. Importantly, it achieves both computational efficiency—especially in reducing repeated matrix inversions—and provable asymptotic normality of estimators, facilitating statistical inference after continual adaptation (Lu et al., 10 Aug 2025).

1. Algorithmic Structure and Staging

The Multi-step Newton Iteration (MSNI) algorithm is organized into sequential stages, each corresponding to a different segment of streaming data (potentially different tasks with task-specific parameter drift). The process begins by computing an initial estimate using a fixed number of early batches via the One-stage Newton Iteration (OSNI):

$$\hat{\theta}_{\text{stage},0} = \operatorname{argmin}_{\theta \in \Theta} \frac{1}{\lfloor K^{\alpha} \rfloor}\sum_{k=1}^{\lfloor K^{\alpha} \rfloor} \frac{1}{n_k}\sum_{i=1}^{n_k} l(X_{(k,i)},Y_{(k,i)}, \theta)$$

where $K$ is the number of mini-batches, $n_k$ is the batch size, and $l$ is the loss function.

Subsequent stages $t$ aggregate new data and perform a Newton-type correction:

$$\hat{\theta}_{\text{stage},t} = \hat{\theta}_{\text{stage},t-1} - \left[\frac{1}{\lfloor K^{\alpha_t} \rfloor}(H_{\text{agg}})\right]^{-1} \left[\frac{1}{\lfloor K^{\alpha_t} \rfloor}(g_{\text{agg}})\right]$$

where the aggregated Hessians and gradients incorporate all previously used data with the latest available parameter iterate for their evaluation. The exponents $\alpha_t$ (with $0 < \alpha_1 < \dots < \alpha_T = 1$) control the stage split and batch grouping.

This design results in each stage using grouped data for its update, interpolating between per-batch online updating and full-memory batch estimation, but crucially avoids excessive memory usage and expensive repeated matrix inversions.

2. Statistical Framework for Task Heterogeneity

Catastrophic forgetting in continual learning is attributed to distributional (task) drift, which the MSNI addresses by modeling each batch/task's parameter as a random effect:

$\theta_k = \theta^* + \eta_k$

Here, $\theta^*$ captures the task-invariant “global” parameter, while $\eta_k$ is a random fluctuation specific to batch/task $k$. The global objective becomes estimating $\theta^*$ under these random effects, typically by minimizing an expectation over both observed data and this latent variation.

Aggregating batch-wise losses and curvatures, the estimation process reflects a weighted average—each batch's contribution is modulated by local curvature (Hessian), statistically aligning updates even as tasks differ substantially.

3. Computational Efficiency and Matrix Inversion Reduction

A central computational benefit of the MSNI algorithm is the substantial reduction in the number of required large matrix inversions. Early-stage OSNI involves one inversion, and as the procedure advances, later stages operate over increasing batch aggregates, allowing further reuse of Hessian estimates and less frequent inversion due to batch grouping:

• High-quality parameter initialization in the first stage reduces variance and sets an accurate trajectory.
• Subsequent Newton steps use an ever-larger sample size for Hessian/gradient aggregation, further stabilizing the computation.
• Frequent and expensive inversions (as required in per-batch Newton methods) are thus replaced by a few strategically staged inversions, achieving both computational and statistical efficiency.
• In settings with high heterogeneity, early accurate correction mitigates the effects of drift without repeatedly recalculating large inverses.

This approach is particularly advantageous in high-dimensional settings or with resource constraints, as encountered in practical online continual learning scenarios.

4. Asymptotic Normality and Statistical Inference

The MSNI framework provides non-asymptotic error bounds and achieves asymptotic normality of the estimator. Under prescribed conditions on the growth of the model's dimensionality $p$ and the number of processed batches $K$, the estimation obeys:

$$\|\hat{\theta}_{\text{stage,T}} - \theta^*\| = O_p\left(\sqrt{\frac{p}{K}}\right)$$

Moreover, normal approximation holds for any fixed direction $v$:

$$\frac{\sqrt{K}\, v^{\top}(\hat{\theta}_{\text{stage,T}} - \theta^*)}{\left\{v^{\top}\Sigma^{-1}\mathbb{E}(Z_k^{\otimes 2})\Sigma^{-1}v\right\}^{1/2}} \stackrel{d}{\to} \mathcal{N}(0,1)$$

where $\Sigma$ is the expected Hessian, and $Z_k$ the batch gradient. This enables hypothesis testing and confidence interval construction in online continual learning—capabilities generally absent from standard continual learning heuristics.

5. Handling Catastrophic Forgetting in Online Continual Learning

The staged MSNI estimator is specifically constructed to mitigate catastrophic forgetting under storage constraints. By combining:

• A random effects model that captures per-batch/task deviations from the global parameter,
• Partial loss aggregation with staged parameter corrections,
• Use of recent gradient and Hessian information relevant to current and past distributions,

the approach “calibrates” the global parameter after each task or batch acclimatization. This limits the abrupt parameter shifts responsible for catastrophic forgetting and maintains performance across sequentially presented, non-stationary tasks.

Empirical results confirm that this approach achieves significant accuracy improvements and lower mean squared error compared to batch estimators (WLSE) and continual learning baselines (GEM), especially when task non-stationarity is pronounced.

6. Empirical Validation: Synthetic and Real Data

Validation is demonstrated across both synthetic and canonical benchmarks:

• Synthetic experiments: Results on simulated linear and logistic regression under two settings—fully random batch-wise parameters and block-task sequences—show that MSNI achieves uniformly lower MSE than both WLSE and episodic memory-based continual learning methods.
• MNIST/CIFAR-10: Using domain-incremental learning protocols, the algorithm consistently attains superior Average Incremental Accuracy (AIA) and exhibits favorable Forward Transfer (FWT) and Backward Transfer (BWT), especially in settings with large inter-task variation.

Performance remains robust regardless of training regime, and both accuracy and efficiency are preserved as the parameter dimension diverges.

7. Theoretical and Practical Implications

The statistical formulation underlying MSNI establishes a basis for analyzing and diagnosing continual learning systems rigorously. By blending Newton’s rapid convergence properties with a random effects model and staged computation, the algorithm simultaneously achieves near-optimal convergence rates, asymptotic normality, resilience to catastrophic forgetting, and computational scalability in streaming, resource-constrained environments.

The framework provides a foundation for subsequent developments in statistically principled, resource-efficient online continual adaptation in high-dimensional, non-stationary data streams (Lu et al., 10 Aug 2025).