---
title: 'CICIoMT2024: IoMT Intrusion Detection Benchmark'
url: https://www.emergentmind.com/topics/ciciomt2024
type: topic
---

# CICIoMT2024: IoMT Intrusion Detection Benchmark

CICIoMT2024 is a benchmark dataset for Internet of Medical Things intrusion detection and anomaly detection, used to model cyberattacks against heterogeneous healthcare-connected devices operating over Wi-Fi, MQTT, and Bluetooth/BLE. In the literature, it is described as a realistic IoMT traffic corpus collected from laboratory testbeds containing real and simulated devices, labeled for benign traffic and multiple attack families, and employed in supervised, unsupervised, explainable, federated, hierarchical, and cross-domain IDS studies [2502.11854].

## 1. Origin, scope, and reported corpus views

A detailed source description identifies CICIoMT2024 as originating from the Canadian Institute for Cybersecurity, with one study explicitly attributing it to Dadkhah et al. (2024) and describing traffic capture in a lab IoMT network using CIC FlowMeter, followed by ground-truth labeling scripts [2508.10346]. Multiple studies agree that the benchmark is intended for realistic medical-IoT intrusion detection, but they do not report a single uniform corpus size or a single fixed feature representation.

| Study | Reported corpus view | Reported representation |
|---|---|---|
| [2502.11854] | `≃ 300 K records, split roughly 80% benign vs. 20% attack` | `25–40 features per record` |
| [2509.10563] | `approximately 1 000 000 network-flow records` | `46 total` features |
| [2410.23306] | `on the order of 2.6 million flow samples` | `31 raw features` |
| [2508.10346] | `9 378 297 records` | `multi-protocol flow features` |
| [2603.10776] | traffic from `40 IoMT devices (25 real, 15 simulated)` | `45 numeric features` |
| [2508.12470] | traffic from `40 IoMT devices (25 physical, 15 simulated)` | `83 numerical flow-level features` |

These discrepancies are directly present in the published record. This suggests that CICIoMT2024 is being used through multiple exported forms, preprocessing regimes, or task-specific subsets rather than as a single universally fixed tensorization. A related implication is that reported metrics are not directly comparable unless the exact corpus view, feature extraction pipeline, and split protocol are matched.

One paper characterizes CICIoMT2024 as “the first multi-protocol, multi-device, multi-scenario benchmark for IoMT anomaly detection” [2502.11854]. That characterization is consistent with the breadth of downstream uses reported across anomaly detection, multiclass intrusion classification, zero-day detection, and federated continual learning.

## 2. Devices, protocols, and attack taxonomy

The benchmark is consistently described as covering heterogeneous medical endpoints. Reported device types include wearable sensors, health monitors, diagnostic instruments, and Bluetooth-enabled medical peripherals [2502.11854]; patient monitors, infusion pumps, wearable sensors, routers, gateways, and auxiliary hardware used to emulate a realistic hospital network [2509.10563]; and common clinical endpoints such as wearable monitors, infusion pumps, imaging scanners, and remote sensors [2410.23306]. One federated-learning study states that traffic was collected from `40 IoMT devices (25 real, 15 simulated)` over Wi-Fi, MQTT, and Bluetooth [2603.10776].

Protocol coverage is likewise multi-modal. The dataset has been described as integrating MQTT publish/subscribe telemetry, Wi-Fi/TCP–IP traffic flows, and Bluetooth Low Energy power traces [2502.11854]. Other studies focus on flow-level representations derived from Wi-Fi, MQTT, and BLE traffic [2508.12470]. This protocol diversity is central to the benchmark’s role in evaluating IDS models that must generalize across device classes and network stacks rather than within a single transport regime.

The attack taxonomy is broad but not presented identically in every paper. Several studies describe `18 attack types` grouped into `five` major families: MQTT, DoS, DDoS, Reconnaissance, and Spoofing [2603.10776]. A hierarchical IDS study lists DoS floods over TCP, UDP, ICMP, and SYN; DDoS floods over TCP, UDP, SYN, and ICMP; Reconnaissance attacks including Ping Sweep, OS Scan, Port Scan, and VulScan; Spoofing via ARP spoofing; and MQTT attacks such as Malformed Data, Connect Flood, and Publish Flood [2508.10346]. A cross-domain IDS paper instead reports `six labeled traffic classes`, consisting of Normal plus five attack categories, and isolates “Man-in-the-Middle (MITM): ARP and DNS spoofing” as a category [2508.12470]. This suggests that downstream studies sometimes regroup the same low-level attack scripts into different label ontologies for specific classification tasks.

A CNN-based study provides an explicit class-level count for a `19 classes (18 attack types plus benign)` setting, where `DDOS-UDP` dominates with `1 998 026` samples and `Recon-OS_Scan` has only `926` flows [2410.23306]. That extreme imbalance is repeatedly identified as one of the benchmark’s core research challenges.

## 3. Feature space and preprocessing conventions

Feature definitions vary across corpus views, but the reported feature families are consistent. Commonly used attributes include packet and flow headers, byte counts, packet counts, inter-arrival-time statistics, TCP flag counts, protocol identifiers, flow duration, port numbers, and packet-length statistics [2410.23306]. In richer versions of the dataset, device-centric metrics such as CPU utilization, memory load, queue depth, auxiliary sensor readings, and Bluetooth power-consumption features are also present [2509.10563]. A cross-domain IDS paper using an `83`-feature representation lists source and destination IP and port, protocol, forward and backward packet counts, byte counts, flow bytes per second, inter-arrival times, ACK/SYN/RST counts, initial window bytes, and payload statistics such as packet length mean and standard deviation [2508.12470].

Preprocessing pipelines differ, but several recurring patterns appear. Standardization via z-score normalization is reported in multiple studies, using forms such as $\hat{x} = (x-\mu)/\sigma$ or $x_i' = (x_i-\mu)/\sigma$ [2509.10563]. Other work applies Min–Max normalization to $[0,1]$, written as
$$
X_{\mathrm{norm}} = \frac{X-X_{\min}}{X_{\max}-X_{\min}},
$$
either globally or “on each client” in a federated setting [2508.10346]. Missing values are variously imputed with the feature-wise mean, dropped, or removed together with rows containing NA values [2502.11854]. Categorical protocol fields are one-hot encoded in some pipelines, label-encoded in others, and passed directly to CatBoost with its native “ordered target statistics” encoding in one explainability study [2509.10563].

Several papers perform explicit imbalance mitigation. Reported methods include random under/oversampling [2502.11854], RandomOverSampler by simple duplication on CICIoMT2024 [2508.12470], and uniform oversampling of all minority classes until each class matches the majority-class cardinality [2604.06254]. One federated-learning study additionally caps each attack class to `at most 10 000 samples per attack class in training, 2 000 per class in testing, per time period` [2603.10776].

Task-specific feature engineering is also common. For sequential models in anomaly detection, sliding windows of length $T=4$ are used to construct tensors $\{x_{t-T+1},\dots,x_t\}$, while non-sequential models receive appended window summaries such as mean, variance, and min/max [2502.11854]. An explainable-ensemble study computes `FlowEntropy = –∑_i (p_i log₂ p_i)` over packet lengths and `FlagRatio_RST = rst_count/(syn_count+1)` as derived metrics [2509.10563].

## 4. Evaluation regimes and benchmark tasks

CICIoMT2024 is not tied to a single evaluation task. Published work uses it for binary `Benign` versus `Malicious` detection, `6-way` category classification, `19-class` subtype classification, attack-specific detection for `DoS` and `DDoS`, active-versus-idle time-series modeling, device-specific Bluetooth anomaly detection, known-versus-unknown zero-day filtering, and continual learning under synthetic concept drift [2410.23306].

The benchmark therefore supports both flat and layered problem formulations. In one hierarchical IDS, the “Near Edge” root layer performs binary “Normal vs. Anomaly” detection using meta-learning or one-class classification; the second layer classifies flagged flows as “Known” or “Unknown”; the “Cloud” layers then perform five-category and `18`-subcategory Random Forest classification [2508.10346]. In a federated continual-learning formulation, data arrive over periods $t_0 \dots t_6$, and the task is to update a global LSTM model under drift while controlling catastrophic forgetting [2603.10776].

Train/test protocols also vary considerably. Reported configurations include `80% training / 20% testing, stratified by attack vs. benign` with `5-fold cross-validation on training data for hyperparameter tuning (GridSearch)` [2502.11854]; a `70/30 train/test split` [2509.10563]; `70% train, 15% validation, 15% test` with `5-fold cross-validation` inside training [2410.23306]; and `stratified 10-fold cross-validation` with nested hyperparameter tuning [2508.10346]. This heterogeneity matters for interpretation: very high accuracy under one split protocol does not establish identical performance under another, especially when label grouping, class balance, or temporal drift differ.

## 5. Representative results on the benchmark

Reported results on CICIoMT2024 range from moderate difficulty in heterogeneous anomaly-detection settings to near-perfect scores in some supervised classification settings.

| Study | Setting | Reported result |
|---|---|---|
| [2502.11854] | Multi-Protocol Data (MQTT + Wi-Fi) | `Ensemble (XGBoost): balanced precision ≃0.83, recall ≃0.83, F1 ≃0.83; false positives ↓ by ~50% vs. AE` |
| [2509.10563] | Random Forest / CatBoost | `RF: Acc=99.92 %, F1=99.96 %, AUC=100 %, train time=140 s`; `CatBoost: Acc=99.90 %, F1=99.95 %, AUC=100 %, train time=23 s` |
| [2410.23306] | Proposed 1D-CNN | `0.99` accuracy on `2-class`, `6-class`, and `19-class` tasks |
| [2508.12470] | BiGAT-ID | `Overall accuracy: 99.13%`, `False positive rate: 0.0013%`, `Inference time per instance: 0.0002 seconds` |
| [2508.10346] | Root-level usfAD | `Accuracy 99.77`, `Macro F1 97.81`, `Weighted F1 99.77` |
| [2604.06254] | SE ViT–BiLSTM after balancing | `Accuracy 98.16`, `FPR 0.0036`, `Inference latency 0.00014 s/inst` |

The anomaly-detection study that foregrounds CICIoMT2024 as a multi-protocol benchmark reports more differentiated outcomes than the near-saturated multiclass studies. On `Multi-Protocol Data (MQTT + Wi-Fi)`, `Isolation Forest` achieves anomaly precision `0.90`, anomaly recall `0.76`, and `F1 ≃0.76`, whereas the `Autoencoder` shows `high FPs`; the stacked `XGBoost` ensemble improves to `F1 ≃0.83` and cuts false positives by about `50%` versus the autoencoder [2502.11854]. On `Attack-Specific Data (DoS/DDoS)`, however, `LSTM, CNN-LSTM, GRU all achieve precision=recall=F1=1.0 on DDoS detection`, and the stacked ensemble reaches `test accuracy = 1.0, ROC AUC ≃1.0` [2502.11854].

The CNN study reports `0.99` accuracy, precision, recall, and F1 on `2-class` and `6-class` tasks, and `0.99/0.98/0.99/0.98` on the `19-class` task, outperforming Logistic Regression, AdaBoost, a Deep NN, and Random Forest in that setup [2410.23306]. A separate explainable-ensemble study on a `1 000 000`-flow corpus view reports `AUC=100 %` for both Random Forest and CatBoost, with CatBoost training about six times faster (`23 s` versus `140 s`) [2509.10563]. Cross-domain work using CICIoMT2024 and EdgeIIoTset reports `99.13 percent` accuracy and `0.0002 seconds per instance` for BiGAT-ID on the medical dataset [2508.12470].

These results collectively show that CICIoMT2024 can appear nearly solved under some static supervised protocols while remaining materially challenging in settings emphasizing false positives, rare classes, temporal structure, or cross-period generalization.

## 6. Explainability, zero-day detection, continual learning, and limitations

Beyond top-line accuracy, CICIoMT2024 has become a testbed for interpretability. In explainable ensemble learning, SHAP and LIME are used to analyze Random Forest and CatBoost predictions, with `IAT` and `rst_count` dominating Random Forest explanations, and `PacketRate` and `srate` dominating CatBoost explanations; `SHAP global explanations on 1 000 instances took 3.5 s for CatBoost vs. ~80 s for RF` [2509.10563]. LIME examples further show that a flow with `rst_count=543` and `PacketRate=1.72 pkt/s` can be flagged as `attack` because high `rst_count` and low `PacketRate` push the surrogate model over threshold [2509.10563]. This positions the benchmark as useful not only for detection accuracy but also for analyst-facing rationale generation.

Zero-day evaluation is another major line of work. A hierarchical IDS reports that the first layer “detects zero-day attacks with high accuracy without needing new datasets,” and compares `usfAD`, `LOF`, and `IF` for unknown-versus-known attack detection [2508.10346]. In that study, `usfAD` attains `91.03 %` F1 for unknown `DoS`, `88.48 %` for unknown `MQTT`, `60.96 %` for unknown `DDoS`, `81.51 %` for unknown `Recon`, and `26.29 %` for unknown `Spoof.` [2508.10346]. These class-dependent outcomes are a useful corrective to the misconception that near-perfect closed-set multiclass accuracy implies robust zero-day performance.

Continual learning under drift exposes a different failure mode. In an incremental federated-learning study, `Static` binary accuracy falls from `97.85` at `t₁` to about `63–64` by `t₄–t₆`, while `Simple Incremental` collapses to `45.93` at `t₆`; by contrast, `Cumulative Incremental` averages `93.30`, `Representative Incremental` averages `95.73`, and `Retain (100)` averages `91.92` in binary classification [2603.10776]. In `6-way` classification, `Cumulative Incremental` reaches average accuracy `66.7`, `Representative Incremental` `64.5`, and `Retain (1000)` `64.6`, whereas `Static` averages `48.5` and `Simple Incremental` `45.4` [2603.10776]. The same study reports that cumulative replay gives the best accuracy but substantially higher training time, while representative and retention methods reduce training time by `30–60 %` with only a `2–4 %` accuracy drop [2603.10776]. This suggests that CICIoMT2024 is useful for studying catastrophic forgetting and resource-constrained retraining, not only conventional IID benchmarking.

The benchmark’s limitations are also explicitly discussed. One anomaly-detection paper states that CICIoMT2024 “does not yet cover every real-world IoMT attack (e.g. spoofed BLE pairing)” and proposes future extensions to `CoAP` and `LoRaWAN`, plus `federated learning for privacy` and `edge deployment and real-time inference` [2502.11854]. The federated study notes `No automatic drift-detection`, the use of `IID splits`, and the absence of more realistic `federated class-incremental learning` with local, sparse first appearances [2603.10776]. A hybrid SE ViT–BiLSTM study additionally states that `no formal hypothesis tests` are provided, although performance gains after balancing are consistently observed in confusion matrices and ROC curves [2604.06254].

Taken together, the literature presents CICIoMT2024 as a high-variance but increasingly central IoMT security benchmark: broad in protocol coverage, heterogeneous in published preprocessing and task formulations, often highly separable in static supervised settings, yet still informative for false-positive control, explainability, zero-day detection, class imbalance, and continual adaptation under evolving threat landscapes.

Source: https://www.emergentmind.com/topics/ciciomt2024