Extreme Learning Machines (ELM)

Updated 30 January 2026

Extreme Learning Machines (ELM) are a type of single-hidden layer feedforward network that uses random weights and non-iterative learning techniques.
ELM offers rapid training speeds and effective generalization, making it suitable for large-scale and real-time applications.
Practical use cases of ELM include image classification, regression, and pattern recognition, providing a computationally efficient alternative to traditional networks.

Operator Splitting Method (OSM) refers to a class of numerical methods for solving differential, variational, and monotone-inclusion problems by decomposing the original operator (or system) into a sum of simpler components, each of which is integrated or solved separately. This methodology is foundational across PDEs, DAEs, control, optimization, computational physics, and high-dimensional simulation, providing a rigorous structure for leveraging sparsity, structure preservation, and multirate effects.

1. Mathematical Foundations and Abstract Formulation

The prototypical operator splitting scenario involves a Cauchy problem or inclusion of the form

$\frac{du}{dt} = (A_1 + A_2 + \cdots + A_N)u,$

or, in monotone operator theory, find $x$ such that $0 \in A(x) + B(x)$ (or $A(x) + B(x) + C(x)$ in the three-operator case). Each $A_i$ is typically chosen to expose structure (e.g., linear, dissipative, Hamiltonian, constraint-enforcing) enabling specialized sub-solvers or analytic flows.

Splitting replaces the original flow $u(t) = e^{t(A_1+...+A_N)}u_0$ by compositions of flows $e^{\alpha_{i} t A_i}$ (with step weights $\alpha_i$ ), or by more general sub-integrators where exact exponentials are unavailable. The method’s order, stability, and structure preservation properties depend on the algebraic conditions on these coefficients, the sub-operator properties, and the chosen composition.

2. Classical Techniques: Lie–Trotter, Strang, and High-Order Schemes

The two-operator Lie–Trotter splitting advances the solution with sequential flows:

$u_{n+1} = e^{\Delta t A_2} e^{\Delta t A_1} u_n,$

yielding first-order global accuracy.

Strang (symmetric) splitting,

$u_{n+1} = e^{\frac{\Delta t}{2}A_1} e^{\Delta t A_2} e^{\frac{\Delta t}{2}A_1} u_n,$

achieves second-order accuracy. The local error is controlled by the leading commutators in the Baker–Campbell–Hausdorff expansion.

For $N$ -splitting ( $N > 2$ ), Strang-type symmetric compositions generalize as

$S_h(A_1,\ldots,A_N) = e^{\frac{h}{2}A_1} \cdots e^{hA_N} \cdots e^{\frac{h}{2}A_1},$

with order conditions determined by commutator calculus. For arbitrary $N$ and higher order, recent complex-valued methods (e.g., CLT-2 for $N$ -split) have been introduced, ensuring positive real weights in all fractional steps and favorable A-stability (Spiteri et al., 2024).

High-order splitting schemes are constructed by multi-stage compositions. Efficient third-order methods for two operators cannot avoid negative weights, and for higher order, optimizing stability and error constants (e.g., via the Local Error Measure—LEM) is crucial (Wei et al., 4 Jan 2025).

3. Structure Preservation: Energy, Dissipativity, and Symplecticity

In systems like linear port-Hamiltonian flows,

$\dot{x} = (J - R)\nabla H(x) + Bu,$

where $J$ is skew-symmetric, $R$ positive semi-definite, and $H$ quadratic, the splitting must preserve dissipativity:

$\frac{dH}{dt} = -\nabla H(x)^T R \nabla H(x) + y^T u \leq y^T u,$

and maintain the passivity structure (Lorenz et al., 2024).

A symmetric splitting (Strang or impulse methods), with energy-preserving integrators (e.g., implicit midpoint), ensures the discrete analog of this inequality, so the schemes do not artificially excite or dissipate energy inconsistent with the physical model.

For stochastic PDEs such as Maxwell's equations with additive noise, splitting into one-dimensional subsystems simultaneously preserves multi-symplectic laws and the correct growth rate of expected energy, under suitable regularity assumptions and careful numerical implementation (Chen et al., 2021).

4. Multirate, Scalar, and Distributed Splitting

Operator splitting methods leverage multirate potential by decoupling fast and slow subsystems, enabling macro-stepping for slow variables and micro-stepping for fast dynamics. This is particularly effective when coupled subsystems evolve on disparate timescales, such as mass-spring-damper chains with stiff and soft blocks (Lorenz et al., 2024).

Distributed operator splitting generalizes to large-scale monotone inclusion problems on networked graphs, where coefficient matrices encode local variables and inter-node communications, and fixed-point convergence is achieved via averaged or conically quasi-averaged operators. This yields highly scalable, decentralized algorithms applicable to consensus optimization, dual decomposition, and networked control (Dao et al., 21 Apr 2025).

5. Optimization, Monotone Inclusions, and Accelerated Splitting

Three-operator (Davis–Yin) splitting solves inclusions of the form $0 \in A x + B x + C x$ where $A, B$ are set-valued maximally monotone, and $C$ is single-valued and cocoercive:

1
2
3

x_B = J_{γB}(z)
x_A = J_{γA}(2x_B - z - γC x_B)
z⁺  = z + λ(x_A - x_B)

Unification of classical Forward–Backward and Douglas–Rachford schemes arises as special cases via appropriate operator choices. In convex optimization, these splittings deliver near-optimal rates and permit acceleration (Nesterov-type) for strongly monotone scenarios (Davis et al., 2015), with recent extensions to inertial, line-search, and composite settings.

Variable and operator splitting (VOS) introduces independent auxiliary variables, transforming classic gradient flow into a coupled dynamical system. This enables strong Lyapunov functions, yielding explicit accelerated algorithms (AOR, EPC), and optimal complexity, including for minimax and saddle-point problems (Chen et al., 7 May 2025).

6. Applications in PDEs, DAEs, Control, and Simulation

OSM is essential in the efficient numerical solution of PDEs arising in finance (option pricing) via ADI, IMEX, and complementarity-projection splitting schemes (Hout et al., 2015), phase-field models (e.g., Cahn–Hilliard equations) with unconditional discrete energy stability (Li et al., 2021), and nonlinear ODEs via separation of variables and Koopman–Lie semigroup techniques (Banjara et al., 21 Jun 2025).

In DAEs, care must be taken to preserve algebraic constraints; splitting must respect the index-1 structure and energy/power conservation (e.g., ε-regularization in port-Hamiltonian DAEs) (Bartel et al., 2023).

Efficient simulation of multi-physics systems such as dynamic gas flows in pipeline networks is achieved via explicit split-step methods leveraging the analytic flows of linear and nonlinear sub-operators, ensuring causality, mass-conservation, and robustness to network topology (Dyachenko et al., 2016).

Operator splitting with capillary relaxation (OSCAR) separates viscous and capillary scales in two-phase flows, achieving dramatic speed-ups and controllable splitting error at low capillary number (Maes et al., 2021).

7. Practical Considerations: Cost, Stability, and Efficiency

The computational advantage of operator splitting methods emerges from four factors:

Decomposition yields smaller linear systems or enables closed-form sub-flows.
Multirate and scalar splitting reduces the dimensionality of computational blocks.
Stability is optimized by proper ordering of operators and tailored sub-integrators, including replacement of unstable implicit steps by explicit schemes at minimal loss of formal accuracy (Wei et al., 4 Jan 2025).
Block-iterative, asynchronous, or distributed implementations allow parallelization and scalability, critical for network control, large-scale optimization, and gray-box modeling (Combettes, 2023, Tang et al., 26 May 2025).

In multi-operator scenarios ( $N > 2$ ), efficient compositions—e.g., complex-valued CLT-2 methods—permit favorable stability with positive real steps and serve as bases for constructing high-order, $N$ -split schemes (Spiteri et al., 2024, Spiteri et al., 2023).

Convergence rates are controlled by the sub-integrator order and the algebraic satisfaction of commutator-based constraints. Structure preservation and guaranteed monotonicity (e.g., dissipativity, passivity) in the discrete scheme are essential in physical models and optimization.

References:

Operator splitting for coupled linear port-Hamiltonian systems (Lorenz et al., 2024)
A pair of Second-order complex-valued, N-split operator-splitting methods (Spiteri et al., 2024)
A new efficient operator splitting method for stochastic Maxwell equations (Chen et al., 2021)
A Three-Operator Splitting Scheme and its Optimization Applications (Davis et al., 2015)
Improving the stability and efficiency of high-order operator-splitting methods (Wei et al., 4 Jan 2025)
A general approach to distributed operator splitting (Dao et al., 21 Apr 2025)
Fast operator splitting methods for obstacle problems (Liu et al., 2022)
Operator-Splitting Methods for Neuromorphic Circuit Simulation (Shahhosseini et al., 28 May 2025)
Split-as-a-Pro: behavioral control via operator splitting and alternating projections (Tang et al., 26 May 2025)
The Geometry of Monotone Operator Splitting Methods (Combettes, 2023)
Application of Operator Splitting Methods in Finance (Hout et al., 2015)
The operator-splitting method for Cahn-Hilliard is stable (Li et al., 2021)
Operator Splitting Methods: Numerical Solutions of Ordinary Differential Equations via Separation of Variables (Banjara et al., 21 Jun 2025)
Operator splitting for semi-explicit differential-algebraic equations and port-Hamiltonian DAEs (Bartel et al., 2023)
GeoChemFoam: Operator Splitting based time-stepping for efficient Volume-Of-Fluid simulation of capillary-dominated two-phase flow (Maes et al., 2021)
Accelerated Gradient Methods Through Variable and Operator Splitting (Chen et al., 7 May 2025)
Operator Splitting Method for Simulation of Dynamic Flows in Natural Gas Pipeline Networks (Dyachenko et al., 2016)
Operator splitting for abstract Cauchy problems with dynamical boundary condition (Csomós et al., 2020)
Beyond Strang: A practical assessment of some second-order 3-splitting methods (Spiteri et al., 2023)