MACNN-BiLSTM Architecture Overview

• The paper introduces MACNN-BiLSTM, a composite framework combining CNN, BiLSTM, and multi-head attention to enhance both accuracy and interpretability in motor intention recognition.
• The CNN module hierarchically extracts local spatial features from structured data, transforming input sequences through layered convolutions and effective normalization.
• The integration of bidirectional LSTM and multi-head attention mechanisms enables the model to capture long-range dependencies, achieving up to 94.67% accuracy in AR-SSVEP BCI tasks.

The MACNN-BiLSTM (Multi-Head Attention CNN-BiLSTM) architecture is a composite neural network framework that combines convolutional neural networks (CNN), bidirectional long short-term memory (BiLSTM) layers, and a multi-head attention mechanism. It is designed to jointly exploit local spatial patterns, bidirectional temporal dependencies, and context-selective weighting in structured sequence data such as EEG or network traffic time series. The MACNN-BiLSTM was originally introduced in the context of AR-SSVEP-based motor intention recognition with the aim of achieving high real-time accuracy and interpretable decision-making in brain-computer interface (BCI) applications (Yang et al., 7 Dec 2025). It has also been adopted for high-precision tasks in IoT intrusion detection (Naeem et al., 25 Mar 2025), demonstrating broad applicability in time series classification domains.

1. Input Feature Representation and Preprocessing

The first stage in MACNN-BiLSTM is domain-specific feature extraction and sample normalization. In AR-SSVEP motor intention detection, eight occipital and parieto-occipital EEG channels ($C=8$) are used, with ten temporal-spectral features computed per channel—such as peak frequency, band-limited power spectral densities (theta, alpha, beta), and statistical moments (mean, standard deviation, skewness, max, min)—resulting in a feature matrix $\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$. Each feature is z-scored across the training set ($\mu_f = 0,\, \sigma_f = 1$) to enforce standardization and facilitate stable optimization (Yang et al., 7 Dec 2025). Analogous normalization is performed for time windowed IoT intrusion detection features, where up to 115 normalized scalars per time step are provided as CNN input (Naeem et al., 25 Mar 2025).

2. CNN Module for Local and Hierarchical Feature Extraction

A stack of convolutional layers serves to hierarchically extract and abstract locally-correlated features. In the AR-SSVEP MACNN-BiLSTM, the input $8 \times 10$ "image" is fed into three 2D convolutional layers:

• Conv2D(1→16, kernel $3 \times 3$), stride $1 \times 1$, padding $1 \times 1$, activation: ReLU.
• Conv2D(16→32, kernel $3 \times 3$), stride $2 \times 2$, padding $1 \times 1$, activation: ReLU.
• Conv2D(32→64, kernel $\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$0), stride $\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$1, padding $\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$2, activation: ReLU.

After the third convolution, the output of size $\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$3 is flattened to create a sequence of length 20 (from $\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$4), each with 64 features: $\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$5. The convolution operation per layer follows

$\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$6

For sequential event detection in IoT traffic, the CNN module is composed of three 1D convolutional layers, each followed by batch normalization, ReLU, pooling, and dropout for noise reduction and regularization. This structure efficiently captures local spatiotemporal correlations essential for subsequent sequential modeling (Naeem et al., 25 Mar 2025).

3. BiLSTM for Bidirectional Sequence Modeling

After convolutional feature extraction, the sequence is passed to a BiLSTM layer. Each time step’s input in AR-SSVEP MACNN-BiLSTM ($\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$7) is processed both in the forward and backward directions, each with a hidden size $\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$8. The LSTM equations for each cell (forward direction) are:

$\mathbf{X}^{(0)} \in \mathbb{R}^{8 \times 10}$9

The hidden states from both directions are concatenated at each step: $\mu_f = 0,\, \sigma_f = 1$0, yielding BiLSTM outputs $\mu_f = 0,\, \sigma_f = 1$1 (Yang et al., 7 Dec 2025). For the IoT model the BiLSTM also uses 128 units per direction and outputs a comparable sequence of hidden representations (Naeem et al., 25 Mar 2025).

4. Multi-Head Attention Mechanism

The MACNN-BiLSTM distinguishes itself from previous attention CNN-BiLSTM models by using a transformer-style multi-head attention layer. In the AR-SSVEP architecture:

• Four attention heads ($\mu_f = 0,\, \sigma_f = 1$2), with projection dim $\mu_f = 0,\, \sigma_f = 1$3, model dim $\mu_f = 0,\, \sigma_f = 1$4.
• For each head:

$\mu_f = 0,\, \sigma_f = 1$5

with $\mu_f = 0,\, \sigma_f = 1$6.

• Per-head scaled dot-product attention:

$\mu_f = 0,\, \sigma_f = 1$7

• Outputs from all heads are concatenated and projected, followed by a residual connection and layer normalization:

$\mu_f = 0,\, \sigma_f = 1$8

This design enables the model to focus on multiple, potentially distinct, aspects of temporal patterns relevant for complex tasks such as motor intention discrimination.

5. Integration, Pooling, and Classification

After attention, temporal pooling (mean or max) is applied along the sequence axis to yield a fixed-length vector $\mu_f = 0,\, \sigma_f = 1$9:

$8 \times 10$0

This representation is used as the input to a fully connected classification head:

• FC1: $8 \times 10$1, activation: ReLU, dropout: 0.5
• FC2: $8 \times 10$2 (for AR-SSVEP four-class recognition)
• Output: Softmax activation for class probabilities

The training loss is categorical cross-entropy, and Adam optimizer is employed (learning rate $8 \times 10$3, weight decay $8 \times 10$4, StepLR schedule with $8 \times 10$5 per 100 epochs, up to 300 epochs, batch size 64, regularization via dropout and early stopping) (Yang et al., 7 Dec 2025).

6. Empirical Performance and Interpretability

On AR-SSVEP-based BCI datasets, the MACNN-BiLSTM achieves a reported best accuracy of 94.67% at a 1.5-second window, surpassing CNN-BiLSTM and other baseline architectures (Yang et al., 7 Dec 2025). For IoT intrusion detection, its general attention CNN-BiLSTM variant achieves 99.0% overall accuracy on the N-BaIoT dataset, with high F1 scores, MCC, and Cohen’s Kappa (Naeem et al., 25 Mar 2025).

Post hoc SHAP (SHapley Additive exPlanations) analysis enables quantification and visualization of feature importance for the network's classification decisions. For AR-SSVEP, SHAP values highlight the strong relevance of specific EEG channel-feature combinations (notably, PO6 alpha_psd, PO5 std, and PO4 beta_psd) to model predictions, underscoring the value of the MACNN-BiLSTM for both high accuracy and interpretable neurotechnology (Yang et al., 7 Dec 2025).

7. Broader Applicability and Comparative Models

While the MACNN-BiLSTM in (Yang et al., 7 Dec 2025) targets BCI with multi-head attention, previous attention-augmented CNN-BiLSTM architectures have demonstrated efficacy across diverse time series domains:

• Intrusion detection (contextual feature extraction and context selection (Naeem et al., 25 Mar 2025))
• Earthquake prediction networks (spatial CNN encoding, BiLSTM sequence learning, additive attention, regression output (Kavianpour et al., 2021))

This suggests that the MACNN-BiLSTM's attention-augmented spatiotemporal modeling paradigm generalizes effectively to other pattern-rich sequence domains where both local structure and long-range dependencies modulate class-discriminative signals. The distinctive contribution of multi-head attention, as opposed to single-head or additive attention mechanisms, lies in its ability to attend to multiple, complementary positions or features simultaneously, which plausibly increases robustness in scenarios with high-dimensional, multimodal, or noisy input data.

---

References:

• Enhancing Interpretability of AR-SSVEP-Based Motor Intention Recognition via CNN-BiLSTM and SHAP Analysis on EEG Data (Yang et al., 7 Dec 2025)
• Efficient IoT Intrusion Detection with an Improved Attention-Based CNN-BiLSTM Architecture (Naeem et al., 25 Mar 2025)
• A CNN-BiLSTM Model with Attention Mechanism for Earthquake Prediction (Kavianpour et al., 2021)