Multiclass ML Intrusion Detection Framework

• Multiclass ML-based intrusion detection frameworks are integrated systems that classify network traffic into benign and multiple attack types using advanced machine learning techniques.
• They leverage multi-stage pipelines with privacy-preserving encoding, transformer-based feature extraction, and softmax layers for both binary and multiclass predictions.
• Incremental learning and Gaussian Mixture Modeling enable continuous discovery and integration of new attack types, ensuring high detection accuracy.

A multiclass machine learning based intrusion detection framework refers to an integrated system that identifies, classifies, and continuously adapts to multiple attack types in network traffic using advanced ML architectures, especially those supporting multiclass output spaces. These frameworks are designed to address evolving intrusion scenarios by leveraging high-dimensional feature extraction, adaptive learning, and robust unknown-attack discovery, thereby extending beyond simple benign/malicious discrimination to fine-grained attack attribution and incremental adaptation.

1. System Architecture and Data Flow

A typical multiclass ML-based intrusion detection framework consists of a multi-stage pipeline for traffic ingestion, feature transformation, hierarchical decision making, and incremental model updates (Adjewa et al., 4 Nov 2024).

• Data Acquisition & Privacy-Preserving Encoding: Raw network flows (e.g., from the CSE-CIC-IDS2018 dataset) are passed through modules such as Privacy-Preserving Fixed-Length Encoding (PPFLE) to obscure sensitive values, followed by sub-word tokenization via ByteLevelBPETokenizer.
• Feature Representation: Tokens are embedded using the first four layers of a pretrained BERT encoder, with the [CLS] token serving as a fixed-length, high-dimensional embedding capturing protocol-agnostic traffic structure.
• Binary Detection: An initial binary classification head, implemented as softmax over {benign, malicious}, achieves perfect recall (100%), ensuring no false negatives or false positives in practice at this stage.
• Multiclass Attack Identification: Detected malicious flows are processed by a multiclass head—softmax output over all known attack types plus an “other” node—enabling both fine-grained attribution and explicit detection of unknown attacks.
• Unknown Attack Discovery and Model Update: Samples classified as “other” are buffered for periodic clustering using Gaussian Mixture Models (GMM) in the BERT embedding space. When a cluster threshold is reached, the number of emergent attack classes is determined by silhouette analysis, the head is dynamically expanded, and the model is fine-tuned on the union of legacy and newly discovered attacks using replay/exemplar-based incremental learning.

This continuous data flow allows the system to ingest new flows, update its attack taxonomy over time, and maintain high operational accuracy with minimal manual intervention (Adjewa et al., 4 Nov 2024).

2. Transformer-Based Feature Extraction Mechanisms

The transformation of network flows to effective, semantically rich embeddings exploits Transformer architectures:

• Attention Mechanism: Input tokens embeddings $X\in\mathbb{R}^{L\times d_\text{model}}$ are mapped via linear projections to $Q$, $K$, and $V$, where scaled dot-product attention is performed:

$$\mathrm{Attention}(Q,K,V) = \mathrm{softmax}\left( \frac{QK^T}{\sqrt{d_k}} \right) V$$

• Multi-Head Attention: Input is split across $h$ heads, each with independent projections, and concatenated:

$$\mathrm{MultiHead}(X) = \mathrm{Concat}( \mathrm{head}_1, \ldots, \mathrm{head}_h ) W^O$$

• Positional Encoding: Trigonometric functions encode sequence order for permutation-invariant architectures:

$$PE_{pos,2i} = \sin\left( \frac{pos}{10000^{2i/d_\text{model}}} \right), \quad PE_{pos,2i+1} = \cos\left( \frac{pos}{10000^{2i/d_\text{model}}} \right)$$

• Deep Encoder Blocks: Residual connections, layer normalization, and feed-forward layers are composed, providing stable, high-capacity feature transformation:

$$Z^\ell = \mathrm{LayerNorm}( X^{\ell-1} + \mathrm{MultiHead}(X^{\ell-1}) )$$

$$X^\ell = \mathrm{LayerNorm}( Z^\ell + \mathrm{FeedForward}(Z^\ell) )$$

The [CLS] token from the BERT embedding serves as the holistic flow-level representation input to downstream detection modules (Adjewa et al., 4 Nov 2024).

3. Unknown Attack Discovery via Gaussian Mixture Modeling

To support the detection and incorporation of previously unseen attack types, multiclass ML-based frameworks implement unsupervised clustering directly in the induced deep embedding space:

• GMM Model: For $N$ candidate embeddings $x_n$, the probability is modeled as a weighted sum of Gaussian distributions:

$$p(x_n|\theta) = \sum_{k=1}^{K} \pi_k \mathcal{N}(x_n|\mu_k, \Sigma_k), \quad \sum_k \pi_k = 1$$

• Parameter Updates via EM:
  • E-step:

  $$\gamma_{nk} = \frac{\pi_k\mathcal{N}(x_n|\mu_k,\Sigma_k)}{\sum_j \pi_j \mathcal{N}(x_n|\mu_j, \Sigma_j)}$$ - M-step:

  $$N_k = \sum_n \gamma_{nk}, \quad \pi_k \leftarrow N_k/N, \quad \mu_k \leftarrow (1/N_k) \sum_n \gamma_{nk} x_n, \quad \Sigma_k \leftarrow (1/N_k) \sum_n \gamma_{nk} (x_n-\mu_k)(x_n-\mu_k)^T$$

• Model Expansion: Upon convergence, each cluster $k$ is mapped to a new attack label $X_k^\text{new}$. The system dynamically augments its classification head and proceeds with incremental fine-tuning (Adjewa et al., 4 Nov 2024).

This capacity for unsupervised attack taxonomy growth is critical for real-world deployment in evolving threat landscapes.

4. Incremental and Continuous Learning Strategies

Rather than requiring full model retraining with each new attack type, the architecture supports continual expansion:

• Detection of “Other” Samples: Samples not assigned to any known class accumulate until a preset threshold triggers the GMM clustering.

• Head Expansion and Focused Fine-Tuning: Only softmax weights for new and previous classes (with replayed exemplars) are updated; the backbone encoder remains frozen to maintain embedding consistency.

• No Catastrophic Forgetting: This targeted adjustment avoids the destructive forgetting effect often observed in non-incremental learning regimes, preserving high detection rates across both legacy and nascent classes (Adjewa et al., 4 Nov 2024).

This mechanism yields high detection accuracy throughout system evolution, with each update maintaining ≥90% accuracy, precision, recall, and F1 over the entire attack taxonomy, as measured in real-time deployment scenarios.

5. Evaluation Metrics and Empirical Results

Comprehensive evaluation is performed at each stage using standard metrics:

• Binary Stage: 100% recall and 0% false positives/negatives are achieved in attack/benign discrimination.

• Multiclass Stage: Initial class accuracy and recall are both ~95.6%. Incorporation of new clusters (e.g., DoS-Hulk, DDoS-HOIC, SSH-BF, FTP-BF, Infiltration) shows minor degradation (to ~94%, then ~92%), but no class falls below 90% recall, even as taxonomy complexity grows.

• Robustness Demonstration: Plotted evolution of accuracy, precision, recall, and F1 across successive model updates confirms resilience to incremental updates and dynamic threat profiles (Adjewa et al., 4 Nov 2024).

Metric Definition	Formula
Accuracy	$(TP+TN)/(TP+TN+FP+FN)$
Precision	$TP/(TP+FP)$
Recall	$TP/(TP+FN)$
F1-score	$$2 \cdot \mathrm{Precision} \cdot \mathrm{Recall} / (\mathrm{Precision}+\mathrm{Recall})$$

Deployments using these frameworks achieve near real-time throughput suitable for volatile network environments and maintain state-of-the-art detection fidelity.

6. Training and Inference Algorithms

The approach specifies distinct procedures for initial training and continuous operation:

Initial Training (Algorithm 1)

1. Preprocess all flows (PPFLE + BPE tokenizer).

2. Compute BERT embeddings.

3. Train binary classifier (benign vs. malicious).

4. Assign known attack labels, “other” for unknowns.

5. Fine-tune multiclass classifier (softmax over known + “other”).

Continuous Inference & Update (Algorithm 2)

• For each new batch:
  • Encode, embed, classify (binary → multiclass).
  • Buffer “other” outputs.
  • When buffer exceeds a threshold: perform silhouette analysis to set $K$, GMM clustering, re-label, expand classifier head, fine-tune only heads with old and new labels, clear buffer.
  • Output current attack predictions, including new attack classes (Adjewa et al., 4 Nov 2024).

This schema operationalizes a closed-loop, self-updating, data-driven defense pipeline.

7. Applications and Significance

The outlined multiclass ML-based intrusion detection framework is engineered for:

• Adaptive Defense: Providing realtime, adaptive monitoring for enterprise networks, IoT mesh environments, and cloud-edge systems.
• Emerging Threat Discovery: Rapid surfacing and explicit integration of novel attack vectors via unsupervised embedding-space clustering.
• Scalability: Supporting tens of thousands of flows per second with minimal retraining, suitable for edge deployment and central SIEM orchestration.
• Future-Proofing: Seamless expansion of the attack taxonomy makes the solution robust to attack evolution and adversarial adaptation.

The approach demonstrates superior performance in evolving environments, offering both granular attribution and resilience to previously unseen intrusion types through a rigorously engineered machine learning backbone (Adjewa et al., 4 Nov 2024).