---
title: Hybrid Multilayer Extreme Learning Machine
url: https://www.emergentmind.com/topics/hybrid-multilayer-extreme-learning-machine-hml-elm
type: topic
---

# Hybrid Multilayer Extreme Learning Machine

Searching arXiv for the cited HML-ELM and CIW-ELM papers to ground the article in current records.
Hybrid Multilayer Extreme Learning Machine (HML‑ELM) is a hierarchical Extreme Learning Machine architecture for image classification that combines a multilayer stack of ELM‑based autoencoders (ELM‑AEs) for self‑taught feature extraction with a Simplified Interval Type‑2 Fuzzy ELM (SIT2‑FELM) for supervised classification [2507.08047]. In the reported formulation, the model is an $(L+1)$‑layer network in which layers $1,\dots,L$ perform unsupervised multilayer feature encoding of an input data matrix $X$, while layer $L+1$ receives the final representation $X^L$ and produces class outputs $Y$ through an Interval Type‑2 fuzzy classifier [2507.08047]. A closely related line of work, although not itself a multilayer HML‑ELM implementation, provides a closed‑form mechanism for constructing data‑driven input weights in Extreme Learning Machines by expressing them as random linear combinations of training samples; this method has been identified as directly beneficial to HML‑ELM because it preserves the single‑shot training style while replacing purely random input weights with sample‑dependent ones [1406.2889].

## 1. Historical placement and motivating context

HML‑ELM was introduced against the background of several ELM‑based model families with different trade‑offs in depth, uncertainty modelling, and computational cost [2507.08047]. Standard single‑layer ELM uses random input weights and biases together with closed‑form output weights obtained via pseudoinverse, yielding very fast training but a shallow representational structure that is limited for complex natural signals such as video, audio, and images [2507.08047]. ML‑ELM extends ELM through multiple ELM‑AEs, thereby improving representation learning on high‑dimensional data, but remains crisp and therefore does not explicitly model uncertainty in features or outputs [2507.08047].

A further development, ML‑IT2FELM, employs Interval Type‑2 fuzzy autoencoders and an IT2‑FELM classifier with Karnik–Mendel type‑reduction, and is described as achieving strong robustness to uncertainty [2507.08047]. However, the Karnik–Mendel procedure is iterative and involves sorting operations and repeated switching‑point searches, which impose high computational cost and create a bottleneck for real‑time, cost‑sensitive applications such as UAV deployment [2507.08047]. ML‑FELM reduces some of that burden by using Type‑1 fuzzy sets in both autoencoders and classifier, but it is reported as less robust than Interval Type‑2 fuzzy logic systems and more computationally expensive than crisp ML‑ELM [2507.08047].

The design objective of HML‑ELM is to combine the fast, non‑iterative learning of ELM and ML‑ELM with the robustness to uncertainty associated with Interval Type‑2 fuzzy logic, while avoiding the training overhead of fuzzy autoencoders and the sorting and iteration cost of Karnik–Mendel type‑reduction [2507.08047]. The resulting architecture therefore keeps ELM‑AEs in the unsupervised stack and restricts Interval Type‑2 fuzziness to the final classifier, where a simplified SC type‑reduction procedure replaces Karnik–Mendel iteration [2507.08047].

The 2014 work on explicit computation of input weights in ELMs provides an adjacent theoretical and algorithmic motive for hybridization in multilayer settings [1406.2889]. That work shows that, in backpropagation‑trained multilayer perceptrons and in linear SVMs, trained separating hyperplanes are linear combinations of training samples; it then transfers that observation to ELM by constructing input weights from random sign combinations of training examples rather than from i.i.d. uniform randomness [1406.2889]. A plausible implication is that this closed‑form, data‑driven weighting scheme can be inserted into supervised components of HML‑ELM without sacrificing the one‑shot character of ELM training.

## 2. Architectural composition and data flow

The reported HML‑ELM architecture consists of two main phases: self‑taught feature extraction and supervised feature classification [2507.08047]. In the first phase, the input $X^0$ is processed by a stack of $L$ ELM‑AEs, producing a hierarchy of representations
$$
X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.
$$
Each layer transforms its input through a random hidden mapping followed by an analytical least‑squares reconstruction solve, and the output features are propagated forward to the next layer [2507.08047].

Using the notation of the model, the training set is written as $(X^s,T)$ with $X^s=[x_1^s,\dots,x_P^s]$ and $T=[t_1,\dots,t_P]$, where $P$ denotes the number of samples [2507.08047]. For hidden layer $s=1,\dots,L$, the hidden mapping is
$$
h^{s}(x_p^s) = g(x_p^s a + b) = [h_1(x_p^s),\dots,h_{M_s}(x_p^s)],
$$
where $a$ and $b$ are orthogonal random weights and biases, $M_s$ is the number of hidden nodes, and $g(\cdot)$ is the activation function, linear or nonlinear [2507.08047]. The hidden output matrix is
$$
H^s = [h^{s}(x_1^s), \dots, h^{s}(x_P^s)]^T \in \mathbb{R}^{P \times M_s}.
$$

The ELM‑AE layer then solves for output weights analytically. In the sparse or compressed case,
$$
\pmb{\beta}^{s} =
\left( \frac{I}{C} + (H^{s})^T H^{s} \right)^{-1} (H^{s})^T X^{s},
$$
where $C$ is the regularization parameter [2507.08047]. In the equal‑dimension case,
$$
\pmb{\beta}^{s} = (H^{s})^{-1} X^s,\quad (\pmb{\beta}^s)^T \pmb{\beta}^s = I.
$$
Feature propagation is then expressed as
$$
H^s = g(H^{s-1} \cdot \pmb{\beta}^s),\quad
X^{s+1} = g(X^s \cdot \pmb{\beta}^s),
$$
with the final high‑level representation $X^L$ delivered to the SIT2‑FELM classifier [2507.08047].

This architecture is explicitly decoupled: the unsupervised stack performs self‑taught encoding without labels, and only the final classifier uses supervised targets [2507.08047]. The reported emphasis on an “ELM random feature space” prior to unsupervised learning suggests that random orthogonal projections remain foundational even within the hybrid design [2507.08047].

## 3. ELM‑AE stack and self‑taught representation learning

At each unsupervised layer, the ELM‑AE computes
$$
h^{s}(x_p^s) = g(x_p^s a + b),
$$
subject to
$$
a^T a = I,\quad b^T b = 1,
$$
where $a=[a_1,\dots,a_{M_s}]$ is the matrix of randomly generated orthogonal input weights and $b=[b_1,\dots,b_{M_s}]$ is the vector of orthogonal hidden biases [2507.08047]. The reconstruction of the input is
$$
\hat{X}^s = H^s \pmb{\beta}^s,
$$
with targets set equal to the inputs $X^s$ [2507.08047].

The associated optimization problem is formulated as
$$
\min_{\pmb{\beta}^s} \| H^s \pmb{\beta}^s - X^s\|_2^2 + \frac{1}{C} \|\pmb{\beta}^s\|_2^2,
$$
which admits the same closed‑form solution given above for the sparse case [2507.08047]. This is central to the ELM‑AE role inside HML‑ELM: each layer is trained independently through a single least‑squares solve rather than through gradient descent or joint backpropagation across layers [2507.08047].

The term “self‑taught” is used because labels are not involved in this stage and because each autoencoder independently reconstructs its own input [2507.08047]. The forward propagation rule
$$
X^{s+1} = g(X^{s} \cdot \pmb{\beta}^s)
$$
yields progressively higher‑level representations with no iterative fine‑tuning across layers [2507.08047]. The reported interpretation is that this combination of randomness and orthogonality provides good generalization while reducing training cost relative to backpropagation‑based deep models [2507.08047].

Within the broader HML‑ELM discussion, explicit computation of input weights supplies a distinct but compatible mechanism for replacing purely random input projections by random linear combinations of training samples [1406.2889]. In that formulation,
$$
W^{(1)} = R\,X,
$$
where $X \in \mathbb{R}^{h \times k}$ is the training data matrix and $R \in \mathbb{R}^{p \times h}$ is a random sign matrix with entries in $\{-1,1\}$ [1406.2889]. The resulting vectors are normalized,
$$
\mathbf{w}_a^{(1)} = \frac{\mathbf{r}_a}{\lVert \mathbf{r}_a \rVert},
$$
so that each hidden neuron weight becomes a random sum or difference of training examples rather than an arbitrary random direction [1406.2889]. The original discussion identifies this as useful for HML‑ELM because it provides a principled, closed‑form, data‑driven alternative to random input weights while preserving single‑pass training.

## 4. SIT2‑FELM classifier and simplified Interval Type‑2 type‑reduction

The final classification stage in HML‑ELM is a Simplified Interval Type‑2 Fuzzy ELM, which uses Interval Type‑2 fuzzy sets in the hidden layer and an SC type‑reduction‑based output layer trained under the ELM paradigm [2507.08047]. An Interval Type‑2 fuzzy set is characterized by a Footprint of Uncertainty bounded by a lower membership function $\underline{\mu}_{\tilde{A}_{jk}}$ and an upper membership function $\overline{\mu}_{\tilde{A}_{jk}}$ [2507.08047]. For rule $j$ and input dimension $k$, the rule antecedent is interval‑valued, and the rule base takes the form
$$
R^j:\ \text{IF } x_1 \text{ is } \tilde{A}_{j1} \text{ AND } \dots \text{ AND } x_N \text{ is } \tilde{A}_{jN}\ \text{THEN } y_i\ \text{is } w_{ij},\ j=1,\dots,M.
$$

For each rule $j$, the interval firing strengths are
$$
\tilde{F}^j = [\underline{f}_j(\vec{x}_p),\overline{f}_j(\vec{x}_p)],
$$
and with product t‑norm,
$$
\underline{f}_j(\vec{x}_p) = \prod_{k=1}^N \underline{\mu}_{jk}, \quad
\overline{f}_j(\vec{x}_p) = \prod_{k=1}^N \overline{\mu}_{jk}.
$$
Standard center‑of‑sets type‑reduction is
$$
Y_{COS} = \frac{\sum_{j=1}^{M} Y^j \tilde{F}^j}{\sum_{j=1}^{M} \tilde{F}^j} = [y_l, y_r],
$$
with left and right endpoints $y_l$ and $y_r$ usually obtained through Karnik–Mendel iteration [2507.08047]. Because Karnik–Mendel is iterative and sorting‑based, the HML‑ELM classifier replaces it with a simplified SC procedure [2507.08047].

In the SIT2‑FELM formulation, the uncertainty width is defined as
$$
\Delta u_{ij} = \overline{f}_j - \underline{f}_j,\quad j=1,\dots,M_f,
$$
and binary variables $z_{lj,i},z_{rj,i}\in\{0,1\}$ are introduced to determine how much of $\Delta u_{ij}$ contributes to the left and right endpoints [2507.08047]. The resulting closed‑form expressions are
$$
y_l^i = \frac{ \sum_{j = 1}^{M_f} \overline{f}_j w_{ij} - \sum_{j=1}^{M_f} (1 - z_{lj,i})\Delta u_{ij} w_{ij} }{ \sum_{j = 1}^{M_f} \overline{f}_j - \sum_{j=1}^{M_f} (1 - z_{lj,i})\Delta u_{ij} }
$$
and
$$
y_r^i = \frac{ \sum_{j = 1}^{M_f} \overline{f}_j w_{ij} - \sum_{j=1}^{M_f} (1 - z_{rj,i})\Delta u_{ij} w_{ij} }{ \sum_{j = 1}^{M_f} \overline{f}_j - \sum_{j=1}^{M_f} (1 - z_{rj,i})\Delta u_{ij} }.
$$
The reported role of the SC algorithm is to choose the binary variables appropriately, using derivative‑based criteria, so that $y_l^i$ and $y_r^i$ are obtained without Karnik–Mendel iteration or sorting [2507.08047]. Final crisp outputs are then computed as
$$
y_i = \frac{y_l + y_r}{2}.
$$

Initial consequent estimation is performed by the Nie–Tan approximation,
$$
y_p^i  =
\frac{\sum_{j=1}^{M_f} (\underline{f}_j + \overline{f}_j) w_{ij}}{\sum_{j=1}^{M_f} \underline{f}_j + \sum_{j=1}^{M_f} \overline{f}_j},
$$
with
$$
\varphi_{pj} =
\frac{\underline{f}_j + \overline{f}_j}{\sum_{j=1}^{M_f} \underline{f}_j + \sum_{j=1}^{M_f} \overline{f}_j}.
$$
The hidden row vector for sample $p$ is then
$$
h_p = [\varphi_{p1} x_{p1}, \ldots ,  \varphi_{p1} x_{pN}, \ldots,
\varphi_{pM_f} x_{p1}, \ldots ,\varphi_{pM_f} x_{pN}],
$$
from which the hidden matrix $H_0$ is assembled and used to solve
$$
Q_A = H_0^\dagger T
$$
for initial consequents [2507.08047]. A refined hidden matrix $H_B$ is then formed after SC‑based type‑reduction, and
$$
Q_B = H_B^\dagger T
$$
produces the refined consequent parameters [2507.08047].

The intended effect of this construction is to retain the robustness of Interval Type‑2 fuzzy logic while reducing the computational burden normally associated with IT2 type‑reduction [2507.08047].

## 5. Training pipeline, parameterization, and computational profile

The end‑to‑end HML‑ELM training pipeline for image classification begins with preprocessing that transforms raw images into a fixed vector representation and constructs a training matrix $X^0 \in \mathbb{R}^{P \times N}$ together with a one‑hot target matrix $T \in \mathbb{R}^{P \times \tilde{N}}$ [2507.08047]. The unsupervised feature extraction phase then trains $L$ ELM‑AE layers. For each layer $s$, orthogonal input weights $a^s$ and biases $b^s$ are generated randomly, hidden outputs are computed as
$$
H^s = g(X^s a^s + b^s),
$$
and analytic output weights are obtained either by
$$
\pmb{\beta}^{s} =  \left( \frac{I}{C} + (H^{s})^T H^{s} \right)^{-1} (H^{s})^T X^{s}
$$
or, in the equal‑dimension case, by
$$
\pmb{\beta}^s = (H^s)^{-1} X^s.
$$
The next representation is then
$$
X^{s+1} = g(X^s \cdot \pmb{\beta}^s).
$$
After the final ELM‑AE, the model passes $X^L$ to the SIT2‑FELM classifier [2507.08047].

Supervised training of the SIT2‑FELM begins by randomly setting fuzzy membership parameters $m_{jk}$ and spreads $[\sigma_{jk}^1,\sigma_{jk}^2]$ for the Interval Type‑2 fuzzy sets associated with each rule and input dimension [2507.08047]. Initial consequents are estimated with the Nie–Tan approximation through the pseudoinverse solve $Q_A = H_0^\dagger T$, after which the SC algorithm determines the auxiliary binary variables used in the refined endpoint computations and the refined solve $Q_B = H_B^\dagger T$ [2507.08047].

The reported computational structure is notable because each ELM‑AE layer requires only a single matrix inversion and matrix multiplications, while SIT2‑FELM replaces Karnik–Mendel iteration and sorting with closed‑form SC calculations and linear solves [2507.08047]. The paper states that overall training time is significantly lower than CNN and KM‑based IT2‑FELM, with complexity roughly scaling linearly with the number of samples and quadratically with the number of hidden nodes or fuzzy rules for the matrix operations [2507.08047].

The explicit input‑weight method in ELMs has a related computational argument. It preserves the standard ELM training step $\boldsymbol{\beta}=H^+T$ and changes only the construction of $W^{(1)}$, replacing i.i.d. uniform random weights with class‑conditioned random linear combinations of training examples [1406.2889]. The original description states that all weights for this method can be computed in a single pass, and that the additional cost of weight construction is negligible compared with the pseudoinverse step for typical ELM sizes [1406.2889]. This suggests a natural compatibility with HML‑ELM components whose dominant cost already lies in matrix solves rather than iterative gradient updates.

## 6. Benchmarks, UAV deployment, and relation to adjacent ELM variants

The reported empirical evaluation of HML‑ELM covers benchmark image classification and an active UAV classification and transport task [2507.08047]. The benchmark datasets are MNIST, Fashion‑MNIST, CIFAR‑10, and a custom object dataset [2507.08047]. MNIST contains 70,000 grayscale images of size $28\times 28$ with 60,000 training and 10,000 test images across 10 classes; Fashion‑MNIST uses the same size and splits; CIFAR‑10 contains 60,000 color images of size $32\times 32\times 3$ with 50,000 training and 10,000 test images across 10 classes; and the object dataset contains 36,386 images of four geometric objects, preprocessed to $52\times 52$ pixels, or 2704 input features, with approximately 9,000 images per class [2507.08047]. Accuracy is measured as
$$
\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN},
$$
and the protocol uses ten independent runs per model with mean accuracy and training time reported [2507.08047].

For these benchmark problems, ELM is described as having the fastest training but the lowest accuracy, whereas CNN attains the highest accuracy at the cost of order‑of‑magnitude higher training time and many more parameters [2507.08047]. Among ELM‑family multilayer models, ML‑IT2FELM tends to achieve the best accuracy but at the highest computational cost; ML‑FELM improves on ML‑ELM under noisy conditions; and HML‑ELM is reported to achieve accuracy close to ML‑IT2FELM and CNN while using far fewer parameters than CNN and less training time than both CNN and ML‑IT2FELM [2507.08047]. On the object dataset specifically, CNN yields the highest accuracy, while HML‑ELM is second best and requires roughly one‑sixth of the CNN training time [2507.08047].

The UAV application uses a Parrot Bebop 2 quadcopter with an onboard 1080p RGB camera, a foldable robotic arm and gripper actuated through Arduino and Bluetooth, and ROS middleware for high‑level control on a remote computer running Ubuntu 16.04 with MATLAB, Python, and C++ [2507.08047]. The mission is to take off in an indoor arena of approximately $2.3\,\text{m} \times 3.2\,\text{m}$, navigate to a station where one of four objects is presented, classify it actively using HML‑ELM, grasp it, fly to a second station with four labeled containers, classify the label, and deliver the object to the corresponding container [2507.08047].

The image processing pipeline in this deployment converts RGB to HSV, applies blurring and thresholding, performs segmentation and morphological cleaning, extracts a centered $52\times 52$ patch, flattens it to a $1\times 2704$ vector, and sends it to HML‑ELM [2507.08047]. In the active classification loop, a batch of images is collected, per‑image class predictions are obtained, an empirical posterior is computed,
$$
P(c|o_j) = \frac{\text{number of times } Y_{\text{HML‑ELM}} = o_j}{\text{number of collected images}},
$$
and the class is selected by
$$
\hat{c} = \argmax_{c} P(c|o_i).
$$
If $P(\hat{c}|o_i) \ge t_c$, the classification is accepted; the text gives $t_c=0.82$ as an example and also states that HML‑ELM achieves high active classification accuracy with threshold $t_c(\hat{c}) \geq 0.8$ [2507.08047].

Within this active scenario, CNN is reported to have the highest static classification accuracy but excessive computational load for efficient real‑time deployment, while ELM and ML‑ELM are fast but less robust under noisy conditions, varying viewpoints, and illumination [2507.08047]. HML‑ELM is described as providing a reliable real‑time classifier in the control loop, with classification errors decreasing as the threshold increases and with particular robustness in noisy, motion‑blurred, and partially occluded views [2507.08047].

The earlier explicit input‑weight ELM results are narrower in scope but relevant to hybrid multilayer design [1406.2889]. On MNIST, CIW‑ELM is reported to achieve lower error than standard ELM for a given hidden layer size; the example given is that ELM requires a 784‑3000‑10 network to reach approximately 96% accuracy, whereas CIW‑ELM reaches approximately 96% with 784‑700‑10 [1406.2889]. On iris and wine, CIW‑ELM outperforms ELM for small hidden layers, whereas on abalone both methods perform similarly and the benefit appears weaker because the task is essentially regression recast as classification [1406.2889]. This suggests that, where HML‑ELM is used for discriminative image classification rather than regression‑like prediction, data‑driven input‑weight construction may be especially compatible with compact architectures.

## 7. Strengths, limitations, and interpretive issues

The strengths explicitly attributed to HML‑ELM are fast training and inference, robustness to noise and uncertainty, effective self‑taught representation learning, computational efficiency through SC type‑reduction, fewer parameters than deep CNNs, and successful deployment on a UAV performing a perception‑driven manipulation task [2507.08047]. These properties follow from the combination of random hidden parameters, closed‑form output weights, multilayer ELM‑AE feature extraction, and Interval Type‑2 fuzzy classification without Karnik–Mendel iteration [2507.08047].

Several limitations and assumptions are also stated [2507.08047]. The feature extractor is based solely on ELM‑AEs rather than fuzzy autoencoders, so uncertainty is not modelled at the feature level. Performance, while close, does not surpass top CNN performance on all benchmarks. The fuzzy classifier is only in the final layer, so deeper fuzzy architectures are not explored. Regularization and the number of layers or nodes must be tuned per dataset [2507.08047]. In addition, because HML‑ELM relies on a final supervised classifier after an unsupervised feature stack, its advantages are most directly substantiated for classification tasks rather than for regression settings.

The explicit input‑weight approach raises further design considerations when transferred into multilayer or hybrid systems [1406.2889]. The original method is intrinsically class‑dependent, since it allocates hidden units by class and constructs class‑wise random sign combinations of samples. It is therefore naturally suited to supervised classification layers and not directly applicable to unlabeled unsupervised layers without modification [1406.2889]. The original discussion also notes weaker benefit on regression‑type tasks, emphasizing that the method is “intrinsically based on class information” [1406.2889]. This suggests that, inside HML‑ELM, sample‑combination weight construction is most coherent at supervised stages or at layers whose purpose is explicitly discriminative.

A recurring misconception in the surrounding ELM literature is that hybridization necessarily implies iterative deep learning in the backpropagation sense. The reported HML‑ELM formulation does not adopt back‑propagation fine‑tuning across layers; each ELM‑AE is trained independently through a one‑shot least‑squares solve, and the final SIT2‑FELM is likewise trained through linear systems after fuzzy feature construction [2507.08047]. Another possible misconception is that Interval Type‑2 robustness requires full Karnik–Mendel computation at all layers. The defining claim of HML‑ELM is instead that a single SIT2‑FELM output stage with SC type‑reduction can preserve much of the robustness advantage while avoiding the principal computational bottleneck [2507.08047].

Potential extensions mentioned for HML‑ELM include deeper architectures with more ELM‑AE layers or multiple fuzzy layers, alternative fuzzy logic variants such as zSlice IT2 FLSs or general Type‑2 fuzzy logic, applications to time series, audio, video, and other robotic platforms, and hybridization with convolutional components [2507.08047]. These are presented as directions rather than validated results. A plausible implication, when combined with the explicit input‑weight work, is that future hybrid multilayer ELMs may vary not only in their choice of fuzzy or crisp classifier but also in how data‑dependent the hidden projections are at different depths [1406.2889].

Source: https://www.emergentmind.com/topics/hybrid-multilayer-extreme-learning-machine-hml-elm