Hierarchical Extreme Learning Machine (HELM)

• Hierarchical Extreme Learning Machine (HELM) is a multilayer feedforward framework that hierarchically stacks ELM modules using random projections for rapid feature extraction and classification.
• It leverages closed-form, layerwise training to eliminate iterative backpropagation, significantly accelerating training while maintaining robust performance.
• HELM supports various extensions—including kernelized, fuzzy, and online sequential variants—proving effective in image processing, anomaly detection, and medical diagnosis.

A Hierarchical Extreme Learning Machine (HELM) is a multilayer feedforward computational framework that hierarchically stacks Extreme Learning Machine (ELM) modules for feature extraction and classification. HELM systematically exploits unsupervised or supervised stacking, typically employing random projection layers and closed-form least-squares solvers at each stage. The approach achieves high efficiency and performance in single-pass or online learning scenarios, with demonstrable success across image classification, time series anomaly detection, and structured medical diagnosis applications (Addanki, 2020, Salman et al., 2022, Dong et al., 2023, Yang et al., 2019, Hernandez-Hernandez et al., 10 Jul 2025).

1. Conceptual and Architectural Foundations

HELM architectures generalize shallow ELMs by compositional layerwise feature learning. The canonical structure comprises multiple “encoding” layers—each an ELM-based autoencoder or single-layer unsupervised learner—followed by a final supervised ELM or advanced variant (e.g., kernel ELM, interval type-2 fuzzy ELM) for classification or regression.

Generalized Layering Paradigms

• Unsupervised layers: Stacked sparse or ridge-regularized ELM autoencoders extract increasingly abstract representations from high-dimensional input. These layers are trained independently, typically without backpropagation (Salman et al., 2022, Yang et al., 2019, Hernandez-Hernandez et al., 10 Jul 2025).
• Supervised layer: A final ELM or differentiated classifier (e.g., kernelized, fuzzy, or one-class ELM) operates on the final-stage features for supervised learning or one-class anomaly detection (Salman et al., 2022, Dong et al., 2023, Hernandez-Hernandez et al., 10 Jul 2025).
• Hybrid variants: Extensions such as hierarchical interval type-2 fuzzy ELM leverage fuzzy inference as the classifier, robustifying prediction in adverse/noisy settings (Hernandez-Hernandez et al., 10 Jul 2025).

The architectural design is driven by the principle of closed-form, non-iterative optimization at every layer, enabling substantial computational acceleration relative to backpropagation-based deep learning.

2. Mathematical Formulation and Training Procedures

Basic ELM Layer

Given inputs $X \in \mathbb{R}^{d \times N}$, the hidden layer output is $H = g(W X + b)$, where $W$ and $b$ are randomly sampled and fixed. The output weight matrix $\beta$ is obtained via

$\beta = H^\dagger T,$

with $T$ the target matrix and $H^\dagger$ the Moore-Penrose pseudoinverse. In $\ell_2$-regularized variants,

$\beta = (H^T H + \lambda I)^{-1} H^T T.$

Hierarchical/Stacked Structure

Let $H = g(W X + b)$0; then each encoding layer $H = g(W X + b)$1 computes

$H = g(W X + b)$2

with $H = g(W X + b)$3 minimizing (possibly sparsity-regularized) reconstruction loss such as

$H = g(W X + b)$4

Output/Classification Layer

• Canonical ELM classifier: as above.
• Kernel ELM (KELM): Utilizes a kernel matrix $H = g(W X + b)$5 (or user-defined kernel $H = g(W X + b)$6), with decision function

$H = g(W X + b)$7

• Interval Type-2 Fuzzy Layer: Applies rule-based fuzzy inference with closed-form consequent computations (e.g., SC algorithm for type-reduction, cost-efficient over Karnik-Mendel) (Hernandez-Hernandez et al., 10 Jul 2025).
• Online Sequential ELM (OS-ELM): For streaming, HELM can utilize recursive least squares updates,

$H = g(W X + b)$8

with correlation matrix $H = g(W X + b)$9 similarly updated (Addanki, 2020).

3. Training Algorithms and Implementation Schemes

Across applications, HELM instantiates a consistent layerwise pipeline:

1. Feature Extraction (Unsupervised Stacking):
   • For $W$0 to $W$1:
     • Randomize $W$2, $W$3.
     • Encode input or previous-layer features via $W$4.
     • Solve for $W$5 to minimize layer-specific loss.
     • Output $W$6 or $W$7.
2. Supervised Classification:
   • Form intermediary representation $W$8.
   • Train a supervised ELM variant on $W$9.
3. (Optional) Online/Sequential Learning:
   • Update $b$0 and $b$1 recursively as new data streams in (Addanki, 2020).

Pseudo-code for standard semi-supervised anomaly detection (Dong et al., 2023):

$b$7

4. Empirical Performance and Comparative Analysis

HELM consistently demonstrates competitive or state-of-the-art results on diverse benchmarks:

Application Domain	Performance (Summary)	Reference
Scene Recognition (Scene-15)	97.8% (HELM) vs 87.9% (flat OS-ELM)	(Addanki, 2020)
Medical: ADHD r-fMRI	Sensitivity 97.17%, Specificity 99.76%, AUC ~0.993	(Salman et al., 2022)
Industrial: Hydraulic Anomaly	Accuracy 99.5%, F1-score 0.985, FPR 0.015	(Dong et al., 2023)
Remote Sensing (Cassini ISS contour)	F1 = 0.58 vs. 0.12–0.30 (CNN/ELM), train <10 s	(Yang et al., 2019)
UAV Image Classification (HML-ELM variant)	~94% real-time accuracy, near-CNN with ≪5× time	(Hernandez-Hernandez et al., 10 Jul 2025)

HELM typically outperforms shallow ELMs and traditional machine learning models; in resource-constrained or online scenarios, it surpasses conventional deep nets in efficiency by large margins.

5. Variants and Extensions

Several HELM variants target domain-specific requirements:

• Kernelized stacking: KELM for nonlinearly separable tasks.
• Interval type-2 fuzzy output: Enhances robustness to uncertainty, e.g., in UAV vision (Hernandez-Hernandez et al., 10 Jul 2025).
• Semi-supervised/one-class models: Hierarchical ELM stacking with final one-class detector for anomaly detection (Dong et al., 2023).
• Online-sequential learning: OS-ELM update for streaming, concept drift, and nonstationary settings (Addanki, 2020).
• Hierarchy depth and width design: Most empirical analyses found optimal depth $b$2–$b$3, with performance often plateauing or degrading with greater depth due to overfitting in absence of backpropagation (Salman et al., 2022).
• Custom feature extractors and domain fusion: Multi-modal streaming, subspace extraction (e.g., dynamic brain connectivity, image channel separation) (Addanki, 2020, Salman et al., 2022).

6. Computational Complexity and Practical Tradeoffs

The hierarchical design of HELM leverages shallow but wide networks, maximizing diversity via random projections rather than deep nonlinear transformations (Addanki, 2020). Key advantages:

• Closed-form per-layer training: Avoids iterative optimization; computational time is dominated by pseudoinverse calculation, $b$4 per layer.
• No backpropagation or end-to-end fine-tuning: Each stage is independently optimized in a single shot.
• Scalability: Efficient both for batch and streaming modes due to the non-iterative, memory-bounded updates.
• Regularization and tuning: Ridge or sparsity terms ($b$5 or $b$6) stabilize the pseudoinverse, control overfitting, and facilitate generalization—parameter selection achieved via grid search or cross-validation (Salman et al., 2022, Yang et al., 2019).

7. Applications and Future Directions

HELM has been employed in vision (image/video), structured signal processing (hydraulic, brain signals), and control (UAVs), delivering rapid inference with strong accuracy. Notable extensions under consideration include:

• Deepened hierarchies or convolutional ELM autoencoders for more complex data structures (Hernandez-Hernandez et al., 10 Jul 2025).
• Advanced output modules: Bayesian ELMs, probabilistic/fuzzy inference, or additional kernelization to further exploit data structure.
• Multi-modal learning: Multiple subnetworks for heterogeneous streams fused at later stages (Addanki, 2020).
• AutoML/parameter learning: Automated selection of layer width, number, and regularization hyperparameters.
• Application to dynamic, high-dimensional, and streaming data settings where non-iterative online updates are critical (Addanki, 2020, Dong et al., 2023).

HELM thus represents a modular paradigm for hierarchical learning—uniting high-throughput, scalable architectures with closed-form optimality at each layer, and offering a rapid deployment path for classification, anomaly detection, and real-time control in scientific, engineering, and biomedical domains (Addanki, 2020, Salman et al., 2022, Yang et al., 2019, Dong et al., 2023, Hernandez-Hernandez et al., 10 Jul 2025).