Hybrid Multilayer Extreme Learning Machine

• Hybrid Multilayer Extreme Learning Machine (HML‑ELM) is a hierarchical model that integrates multiple ELM‑AE layers for unsupervised feature extraction with a robust Interval Type‑2 fuzzy classifier for supervised tasks.
• The unsupervised feature extraction phase uses one-shot least‑squares solutions in each ELM‑AE layer, ensuring fast training and scalable representation learning.
• The final classifier employs a simplified SC type‑reduction method, significantly reducing computational cost while preserving the uncertainty modeling benefits of Interval Type‑2 fuzzy logic.

Searching arXiv for the cited HML-ELM and CIW-ELM papers to ground the article in current records. Hybrid Multilayer Extreme Learning Machine (HML‑ELM) is a hierarchical Extreme Learning Machine architecture for image classification that combines a multilayer stack of ELM‑based autoencoders (ELM‑AEs) for self‑taught feature extraction with a Simplified Interval Type‑2 Fuzzy ELM (SIT2‑FELM) for supervised classification (Hernandez-Hernandez et al., 10 Jul 2025). In the reported formulation, the model is an $(L+1)$‑layer network in which layers $1,\dots,L$ perform unsupervised multilayer feature encoding of an input data matrix $X$, while layer $L+1$ receives the final representation $X^L$ and produces class outputs $Y$ through an Interval Type‑2 fuzzy classifier (Hernandez-Hernandez et al., 10 Jul 2025). A closely related line of work, although not itself a multilayer HML‑ELM implementation, provides a closed‑form mechanism for constructing data‑driven input weights in Extreme Learning Machines by expressing them as random linear combinations of training samples; this method has been identified as directly beneficial to HML‑ELM because it preserves the single‑shot training style while replacing purely random input weights with sample‑dependent ones (Tapson et al., 2014).

1. Historical placement and motivating context

HML‑ELM was introduced against the background of several ELM‑based model families with different trade‑offs in depth, uncertainty modelling, and computational cost (Hernandez-Hernandez et al., 10 Jul 2025). Standard single‑layer ELM uses random input weights and biases together with closed‑form output weights obtained via pseudoinverse, yielding very fast training but a shallow representational structure that is limited for complex natural signals such as video, audio, and images (Hernandez-Hernandez et al., 10 Jul 2025). ML‑ELM extends ELM through multiple ELM‑AEs, thereby improving representation learning on high‑dimensional data, but remains crisp and therefore does not explicitly model uncertainty in features or outputs (Hernandez-Hernandez et al., 10 Jul 2025).

A further development, ML‑IT2FELM, employs Interval Type‑2 fuzzy autoencoders and an IT2‑FELM classifier with Karnik–Mendel type‑reduction, and is described as achieving strong robustness to uncertainty (Hernandez-Hernandez et al., 10 Jul 2025). However, the Karnik–Mendel procedure is iterative and involves sorting operations and repeated switching‑point searches, which impose high computational cost and create a bottleneck for real‑time, cost‑sensitive applications such as UAV deployment (Hernandez-Hernandez et al., 10 Jul 2025). ML‑FELM reduces some of that burden by using Type‑1 fuzzy sets in both autoencoders and classifier, but it is reported as less robust than Interval Type‑2 fuzzy logic systems and more computationally expensive than crisp ML‑ELM (Hernandez-Hernandez et al., 10 Jul 2025).

The design objective of HML‑ELM is to combine the fast, non‑iterative learning of ELM and ML‑ELM with the robustness to uncertainty associated with Interval Type‑2 fuzzy logic, while avoiding the training overhead of fuzzy autoencoders and the sorting and iteration cost of Karnik–Mendel type‑reduction (Hernandez-Hernandez et al., 10 Jul 2025). The resulting architecture therefore keeps ELM‑AEs in the unsupervised stack and restricts Interval Type‑2 fuzziness to the final classifier, where a simplified SC type‑reduction procedure replaces Karnik–Mendel iteration (Hernandez-Hernandez et al., 10 Jul 2025).

The 2014 work on explicit computation of input weights in ELMs provides an adjacent theoretical and algorithmic motive for hybridization in multilayer settings (Tapson et al., 2014). That work shows that, in backpropagation‑trained multilayer perceptrons and in linear SVMs, trained separating hyperplanes are linear combinations of training samples; it then transfers that observation to ELM by constructing input weights from random sign combinations of training examples rather than from i.i.d. uniform randomness (Tapson et al., 2014). A plausible implication is that this closed‑form, data‑driven weighting scheme can be inserted into supervised components of HML‑ELM without sacrificing the one‑shot character of ELM training.

2. Architectural composition and data flow

The reported HML‑ELM architecture consists of two main phases: self‑taught feature extraction and supervised feature classification (Hernandez-Hernandez et al., 10 Jul 2025). In the first phase, the input $X^0$ is processed by a stack of $L$ ELM‑AEs, producing a hierarchy of representations

$$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$

Each layer transforms its input through a random hidden mapping followed by an analytical least‑squares reconstruction solve, and the output features are propagated forward to the next layer (Hernandez-Hernandez et al., 10 Jul 2025).

Using the notation of the model, the training set is written as $(X^s,T)$ with $1,\dots,L$0 and $1,\dots,L$1, where $1,\dots,L$2 denotes the number of samples (Hernandez-Hernandez et al., 10 Jul 2025). For hidden layer $1,\dots,L$3, the hidden mapping is

$1,\dots,L$4

where $1,\dots,L$5 and $1,\dots,L$6 are orthogonal random weights and biases, $1,\dots,L$7 is the number of hidden nodes, and $1,\dots,L$8 is the activation function, linear or nonlinear (Hernandez-Hernandez et al., 10 Jul 2025). The hidden output matrix is

$1,\dots,L$9

The ELM‑AE layer then solves for output weights analytically. In the sparse or compressed case,

$X$0

where $X$1 is the regularization parameter (Hernandez-Hernandez et al., 10 Jul 2025). In the equal‑dimension case,

$X$2

Feature propagation is then expressed as

$X$3

with the final high‑level representation $X$4 delivered to the SIT2‑FELM classifier (Hernandez-Hernandez et al., 10 Jul 2025).

This architecture is explicitly decoupled: the unsupervised stack performs self‑taught encoding without labels, and only the final classifier uses supervised targets (Hernandez-Hernandez et al., 10 Jul 2025). The reported emphasis on an “ELM random feature space” prior to unsupervised learning suggests that random orthogonal projections remain foundational even within the hybrid design (Hernandez-Hernandez et al., 10 Jul 2025).

3. ELM‑AE stack and self‑taught representation learning

At each unsupervised layer, the ELM‑AE computes

$X$5

subject to

$X$6

where $X$7 is the matrix of randomly generated orthogonal input weights and $X$8 is the vector of orthogonal hidden biases (Hernandez-Hernandez et al., 10 Jul 2025). The reconstruction of the input is

$X$9

with targets set equal to the inputs $L+1$0 (Hernandez-Hernandez et al., 10 Jul 2025).

The associated optimization problem is formulated as

$L+1$1

which admits the same closed‑form solution given above for the sparse case (Hernandez-Hernandez et al., 10 Jul 2025). This is central to the ELM‑AE role inside HML‑ELM: each layer is trained independently through a single least‑squares solve rather than through gradient descent or joint backpropagation across layers (Hernandez-Hernandez et al., 10 Jul 2025).

The term “self‑taught” is used because labels are not involved in this stage and because each autoencoder independently reconstructs its own input (Hernandez-Hernandez et al., 10 Jul 2025). The forward propagation rule

$L+1$2

yields progressively higher‑level representations with no iterative fine‑tuning across layers (Hernandez-Hernandez et al., 10 Jul 2025). The reported interpretation is that this combination of randomness and orthogonality provides good generalization while reducing training cost relative to backpropagation‑based deep models (Hernandez-Hernandez et al., 10 Jul 2025).

Within the broader HML‑ELM discussion, explicit computation of input weights supplies a distinct but compatible mechanism for replacing purely random input projections by random linear combinations of training samples (Tapson et al., 2014). In that formulation,

$L+1$3

where $L+1$4 is the training data matrix and $L+1$5 is a random sign matrix with entries in $L+1$6 (Tapson et al., 2014). The resulting vectors are normalized,

$L+1$7

so that each hidden neuron weight becomes a random sum or difference of training examples rather than an arbitrary random direction (Tapson et al., 2014). The original discussion identifies this as useful for HML‑ELM because it provides a principled, closed‑form, data‑driven alternative to random input weights while preserving single‑pass training.

4. SIT2‑FELM classifier and simplified Interval Type‑2 type‑reduction

The final classification stage in HML‑ELM is a Simplified Interval Type‑2 Fuzzy ELM, which uses Interval Type‑2 fuzzy sets in the hidden layer and an SC type‑reduction‑based output layer trained under the ELM paradigm (Hernandez-Hernandez et al., 10 Jul 2025). An Interval Type‑2 fuzzy set is characterized by a Footprint of Uncertainty bounded by a lower membership function $L+1$8 and an upper membership function $L+1$9 (Hernandez-Hernandez et al., 10 Jul 2025). For rule $X^L$0 and input dimension $X^L$1, the rule antecedent is interval‑valued, and the rule base takes the form

$X^L$2

For each rule $X^L$3, the interval firing strengths are

$X^L$4

and with product t‑norm,

$X^L$5

Standard center‑of‑sets type‑reduction is

$X^L$6

with left and right endpoints $X^L$7 and $X^L$8 usually obtained through Karnik–Mendel iteration (Hernandez-Hernandez et al., 10 Jul 2025). Because Karnik–Mendel is iterative and sorting‑based, the HML‑ELM classifier replaces it with a simplified SC procedure (Hernandez-Hernandez et al., 10 Jul 2025).

In the SIT2‑FELM formulation, the uncertainty width is defined as

$X^L$9

and binary variables $Y$0 are introduced to determine how much of $Y$1 contributes to the left and right endpoints (Hernandez-Hernandez et al., 10 Jul 2025). The resulting closed‑form expressions are

$Y$2

and

$Y$3

The reported role of the SC algorithm is to choose the binary variables appropriately, using derivative‑based criteria, so that $Y$4 and $Y$5 are obtained without Karnik–Mendel iteration or sorting (Hernandez-Hernandez et al., 10 Jul 2025). Final crisp outputs are then computed as

$Y$6

Initial consequent estimation is performed by the Nie–Tan approximation,

$Y$7

with

$Y$8

The hidden row vector for sample $Y$9 is then

$X^0$0

from which the hidden matrix $X^0$1 is assembled and used to solve

$X^0$2

for initial consequents (Hernandez-Hernandez et al., 10 Jul 2025). A refined hidden matrix $X^0$3 is then formed after SC‑based type‑reduction, and

$X^0$4

produces the refined consequent parameters (Hernandez-Hernandez et al., 10 Jul 2025).

The intended effect of this construction is to retain the robustness of Interval Type‑2 fuzzy logic while reducing the computational burden normally associated with IT2 type‑reduction (Hernandez-Hernandez et al., 10 Jul 2025).

5. Training pipeline, parameterization, and computational profile

The end‑to‑end HML‑ELM training pipeline for image classification begins with preprocessing that transforms raw images into a fixed vector representation and constructs a training matrix $X^0$5 together with a one‑hot target matrix $X^0$6 (Hernandez-Hernandez et al., 10 Jul 2025). The unsupervised feature extraction phase then trains $X^0$7 ELM‑AE layers. For each layer $X^0$8, orthogonal input weights $X^0$9 and biases $L$0 are generated randomly, hidden outputs are computed as

$L$1

and analytic output weights are obtained either by

$L$2

or, in the equal‑dimension case, by

$L$3

The next representation is then

$L$4

After the final ELM‑AE, the model passes $L$5 to the SIT2‑FELM classifier (Hernandez-Hernandez et al., 10 Jul 2025).

Supervised training of the SIT2‑FELM begins by randomly setting fuzzy membership parameters $L$6 and spreads $L$7 for the Interval Type‑2 fuzzy sets associated with each rule and input dimension (Hernandez-Hernandez et al., 10 Jul 2025). Initial consequents are estimated with the Nie–Tan approximation through the pseudoinverse solve $L$8, after which the SC algorithm determines the auxiliary binary variables used in the refined endpoint computations and the refined solve $L$9 (Hernandez-Hernandez et al., 10 Jul 2025).

The reported computational structure is notable because each ELM‑AE layer requires only a single matrix inversion and matrix multiplications, while SIT2‑FELM replaces Karnik–Mendel iteration and sorting with closed‑form SC calculations and linear solves (Hernandez-Hernandez et al., 10 Jul 2025). The paper states that overall training time is significantly lower than CNN and KM‑based IT2‑FELM, with complexity roughly scaling linearly with the number of samples and quadratically with the number of hidden nodes or fuzzy rules for the matrix operations (Hernandez-Hernandez et al., 10 Jul 2025).

The explicit input‑weight method in ELMs has a related computational argument. It preserves the standard ELM training step $$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$0 and changes only the construction of $$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$1, replacing i.i.d. uniform random weights with class‑conditioned random linear combinations of training examples (Tapson et al., 2014). The original description states that all weights for this method can be computed in a single pass, and that the additional cost of weight construction is negligible compared with the pseudoinverse step for typical ELM sizes (Tapson et al., 2014). This suggests a natural compatibility with HML‑ELM components whose dominant cost already lies in matrix solves rather than iterative gradient updates.

6. Benchmarks, UAV deployment, and relation to adjacent ELM variants

The reported empirical evaluation of HML‑ELM covers benchmark image classification and an active UAV classification and transport task (Hernandez-Hernandez et al., 10 Jul 2025). The benchmark datasets are MNIST, Fashion‑MNIST, CIFAR‑10, and a custom object dataset (Hernandez-Hernandez et al., 10 Jul 2025). MNIST contains 70,000 grayscale images of size $$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$2 with 60,000 training and 10,000 test images across 10 classes; Fashion‑MNIST uses the same size and splits; CIFAR‑10 contains 60,000 color images of size $$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$3 with 50,000 training and 10,000 test images across 10 classes; and the object dataset contains 36,386 images of four geometric objects, preprocessed to $$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$4 pixels, or 2704 input features, with approximately 9,000 images per class (Hernandez-Hernandez et al., 10 Jul 2025). Accuracy is measured as

$$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$5

and the protocol uses ten independent runs per model with mean accuracy and training time reported (Hernandez-Hernandez et al., 10 Jul 2025).

For these benchmark problems, ELM is described as having the fastest training but the lowest accuracy, whereas CNN attains the highest accuracy at the cost of order‑of‑magnitude higher training time and many more parameters (Hernandez-Hernandez et al., 10 Jul 2025). Among ELM‑family multilayer models, ML‑IT2FELM tends to achieve the best accuracy but at the highest computational cost; ML‑FELM improves on ML‑ELM under noisy conditions; and HML‑ELM is reported to achieve accuracy close to ML‑IT2FELM and CNN while using far fewer parameters than CNN and less training time than both CNN and ML‑IT2FELM (Hernandez-Hernandez et al., 10 Jul 2025). On the object dataset specifically, CNN yields the highest accuracy, while HML‑ELM is second best and requires roughly one‑sixth of the CNN training time (Hernandez-Hernandez et al., 10 Jul 2025).

The UAV application uses a Parrot Bebop 2 quadcopter with an onboard 1080p RGB camera, a foldable robotic arm and gripper actuated through Arduino and Bluetooth, and ROS middleware for high‑level control on a remote computer running Ubuntu 16.04 with MATLAB, Python, and C++ (Hernandez-Hernandez et al., 10 Jul 2025). The mission is to take off in an indoor arena of approximately $$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$6, navigate to a station where one of four objects is presented, classify it actively using HML‑ELM, grasp it, fly to a second station with four labeled containers, classify the label, and deliver the object to the corresponding container (Hernandez-Hernandez et al., 10 Jul 2025).

The image processing pipeline in this deployment converts RGB to HSV, applies blurring and thresholding, performs segmentation and morphological cleaning, extracts a centered $$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$7 patch, flattens it to a $$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$8 vector, and sends it to HML‑ELM (Hernandez-Hernandez et al., 10 Jul 2025). In the active classification loop, a batch of images is collected, per‑image class predictions are obtained, an empirical posterior is computed,

$$X^0 \rightarrow X^1 \rightarrow X^2 \rightarrow \dots \rightarrow X^L.$$9

and the class is selected by

$(X^s,T)$0

If $(X^s,T)$1, the classification is accepted; the text gives $(X^s,T)$2 as an example and also states that HML‑ELM achieves high active classification accuracy with threshold $(X^s,T)$3 (Hernandez-Hernandez et al., 10 Jul 2025).

Within this active scenario, CNN is reported to have the highest static classification accuracy but excessive computational load for efficient real‑time deployment, while ELM and ML‑ELM are fast but less robust under noisy conditions, varying viewpoints, and illumination (Hernandez-Hernandez et al., 10 Jul 2025). HML‑ELM is described as providing a reliable real‑time classifier in the control loop, with classification errors decreasing as the threshold increases and with particular robustness in noisy, motion‑blurred, and partially occluded views (Hernandez-Hernandez et al., 10 Jul 2025).

The earlier explicit input‑weight ELM results are narrower in scope but relevant to hybrid multilayer design (Tapson et al., 2014). On MNIST, CIW‑ELM is reported to achieve lower error than standard ELM for a given hidden layer size; the example given is that ELM requires a 784‑3000‑10 network to reach approximately 96% accuracy, whereas CIW‑ELM reaches approximately 96% with 784‑700‑10 (Tapson et al., 2014). On iris and wine, CIW‑ELM outperforms ELM for small hidden layers, whereas on abalone both methods perform similarly and the benefit appears weaker because the task is essentially regression recast as classification (Tapson et al., 2014). This suggests that, where HML‑ELM is used for discriminative image classification rather than regression‑like prediction, data‑driven input‑weight construction may be especially compatible with compact architectures.

7. Strengths, limitations, and interpretive issues

The strengths explicitly attributed to HML‑ELM are fast training and inference, robustness to noise and uncertainty, effective self‑taught representation learning, computational efficiency through SC type‑reduction, fewer parameters than deep CNNs, and successful deployment on a UAV performing a perception‑driven manipulation task (Hernandez-Hernandez et al., 10 Jul 2025). These properties follow from the combination of random hidden parameters, closed‑form output weights, multilayer ELM‑AE feature extraction, and Interval Type‑2 fuzzy classification without Karnik–Mendel iteration (Hernandez-Hernandez et al., 10 Jul 2025).

Several limitations and assumptions are also stated (Hernandez-Hernandez et al., 10 Jul 2025). The feature extractor is based solely on ELM‑AEs rather than fuzzy autoencoders, so uncertainty is not modelled at the feature level. Performance, while close, does not surpass top CNN performance on all benchmarks. The fuzzy classifier is only in the final layer, so deeper fuzzy architectures are not explored. Regularization and the number of layers or nodes must be tuned per dataset (Hernandez-Hernandez et al., 10 Jul 2025). In addition, because HML‑ELM relies on a final supervised classifier after an unsupervised feature stack, its advantages are most directly substantiated for classification tasks rather than for regression settings.

The explicit input‑weight approach raises further design considerations when transferred into multilayer or hybrid systems (Tapson et al., 2014). The original method is intrinsically class‑dependent, since it allocates hidden units by class and constructs class‑wise random sign combinations of samples. It is therefore naturally suited to supervised classification layers and not directly applicable to unlabeled unsupervised layers without modification (Tapson et al., 2014). The original discussion also notes weaker benefit on regression‑type tasks, emphasizing that the method is “intrinsically based on class information” (Tapson et al., 2014). This suggests that, inside HML‑ELM, sample‑combination weight construction is most coherent at supervised stages or at layers whose purpose is explicitly discriminative.

A recurring misconception in the surrounding ELM literature is that hybridization necessarily implies iterative deep learning in the backpropagation sense. The reported HML‑ELM formulation does not adopt back‑propagation fine‑tuning across layers; each ELM‑AE is trained independently through a one‑shot least‑squares solve, and the final SIT2‑FELM is likewise trained through linear systems after fuzzy feature construction (Hernandez-Hernandez et al., 10 Jul 2025). Another possible misconception is that Interval Type‑2 robustness requires full Karnik–Mendel computation at all layers. The defining claim of HML‑ELM is instead that a single SIT2‑FELM output stage with SC type‑reduction can preserve much of the robustness advantage while avoiding the principal computational bottleneck (Hernandez-Hernandez et al., 10 Jul 2025).

Potential extensions mentioned for HML‑ELM include deeper architectures with more ELM‑AE layers or multiple fuzzy layers, alternative fuzzy logic variants such as zSlice IT2 FLSs or general Type‑2 fuzzy logic, applications to time series, audio, video, and other robotic platforms, and hybridization with convolutional components (Hernandez-Hernandez et al., 10 Jul 2025). These are presented as directions rather than validated results. A plausible implication, when combined with the explicit input‑weight work, is that future hybrid multilayer ELMs may vary not only in their choice of fuzzy or crisp classifier but also in how data‑dependent the hidden projections are at different depths (Tapson et al., 2014).