Enhancing Anomaly-Based Intrusion Detection Systems with Process Mining

Abstract: Anomaly-based Intrusion Detection Systems (IDSs) ensure protection against malicious attacks on networked systems. While deep learning-based IDSs achieve effective performance, their limited trustworthiness due to black-box architectures remains a critical constraint. Despite existing explainable techniques offering insight into the alarms raised by IDSs, they lack process-based explanations grounded in packet-level sequencing analysis. In this paper, we propose a method that employs process mining techniques to enhance anomaly-based IDSs by providing process-based alarm severity ratings and explanations for alerts. Our method prioritizes critical alerts and maintains visibility into network behavior, while minimizing disruption by allowing misclassified benign traffic to pass. We apply the method to the publicly available USB-IDS-TC dataset, which includes anomalous traffic affected by different variants of the Slowloris DoS attack. Results show that our method is able to discriminate between low- to very-high-severity alarms while preserving up to 99.94% recall and 99.99% precision, effectively discarding false positives while providing different degrees of severity for the true positives.

• The paper introduces a process mining approach to assign interpretable, process-based alarm severity ratings within anomaly-based intrusion detection systems.
• It employs unsupervised clustering and Petri net-based process discovery to distinguish true network threats from misclassified benign flows.
• Empirical evaluation demonstrates high precision (up to 99.99%) and recall (up to 99.94%) by filtering alerts via defined severity bands.

Process Mining for Enhanced Anomaly-Based Intrusion Detection

Motivation and Problem Statement

Anomaly-based intrusion detection systems (IDS) are central to network defense, particularly due to their ability to detect abnormal activity without relying solely on predefined signatures. Despite widespread adoption of deep learning architectures for IDS, notable performance comes with an enduring deficit: the lack of actionable, trustworthy explanations that connect detection decisions to the domain semantics, especially at packet sequence granularity. Most explainable AI (XAI) techniques applied to IDS, including SHAP and LIME, deliver only post-hoc feature attribution, failing to articulate attack dynamics and sequential packet behavior.

This paper proposes a methodology leveraging process mining to address these deficiencies in XAI for IDS. Specifically, it introduces process-based alarm severity rating and explanations for alerts, offering a mechanism to prioritize and filter critical alarms while maintaining network visibility and minimizing disruptions caused by false positives. Unlike feature-based XAI interventions, this approach grounds alarm rating in the control-flow of packets, utilizing explicit process models extracted from packet-level event logs.

Figure 1: The proposed method integrates process mining into anomaly-based IDS for process-based explanation and alarm ranking.

Methodological Framework

Anomaly-Based IDS Pipeline

The approach involves preprocessing raw network traffic (PCAPs) into aggregate flows using tools such as CICFlowMeter and NTLFlowLyzer, then training conventional anomaly-based detectors—One-Class SVM, Autoencoders (AE), and Variational Autoencoders (VAE)—on benign flows. A restrictive threshold is chosen to maximize false positives in validation, providing robust inputs for subsequent process mining.

At inference, the trained IDS flags incoming flows as anomalous or benign, but only positively flagged flows undergo process-based severity partitioning.

Process Mining Layer

Process mining begins with unsupervised clustering of packet-level traces from false positive flows to extract stateful event logs. Each log is subjected to process discovery—primarily via the inductive miner—to yield Petri nets capturing legitimate TCP event patterns. The process models encode control-flow relations between observed packet transitions and serve as the ground truth for benign behaviors misclassified as anomalies.

Figure 2: Petri net visualization encapsulates the control-flow structure of TCP event patterns.

During inference, event logs from new anomalous flows are aligned against the Petri nets using alignment-based conformance checking. Alignments quantify the similarity between flow sequences and process-based false positive characterizations. Cosine similarity over alignment vectors is used to rank alarm severity: low similarity implies high anomaly severity, while high similarity signals probable false positives.

Severity bands are defined as: Very High ([0, 0.01)), High ([0.01, 0.25)), Medium ([0.25, 0.75)), Low ([0.75, 0.99)), Very Low ([0.99, 1.0]), providing a discretized, tunable scale for selective alert filtering.

Empirical Evaluation

Dataset, Techniques, and Metrics

Experiments target the USB-IDS-TC dataset, which encompasses normal flows and variants of the Slowloris DoS attack (slow loris, high-intensity slow loris, slow post), each engineered to subvert network behavior while mimicking legitimate TCP packet streams.

IDS configuration follows best practices: OCSVM with RBF kernel, AE/VAE with deep multilayer architecture and appropriate regularization. The process mining component uses k-means clustering (k=2, window=3) and inductive mining for process model extraction.

Alarm severity tuning and filtering are evaluated using adjusted recall and precision metrics that consider only flows at or above a given severity band. This allows quantification of the trade-off between retaining true positives and discarding false positives through severity-based triage.

Numerical Results

The methodology achieves up to 99.94% recall and 99.99% precision on the AE when including alerts through the "Low" severity band, discarding most false positives while maintaining detection of high-severity attacks. True positives are concentrated in very-high, high, and medium severity bands, while false positives disproportionately occupy low and very low bands.

Figure 3: AE classifier recall/precision and proportion of TP/FP across severity bands, highlighting effective false positive rejection in low-severity intervals.

Process-based alignment metrics confirm that flows flagged as true positives are much less similar to the process mines of false positive flows than FPs, especially in the very-low severity region.

Figure 4: Distribution of TP/FP across severity bands and mean alignment counts per band, demonstrating clear separation between malicious and benign-like flows.

Theoretical and Practical Implications

The integration of process mining with anomaly-based IDS offers a model-agnostic XAI paradigm that connects detection outcomes to interpretable packet-level control-flow, addressing key gaps in trustworthiness and actionable intelligence. Severity rating enables selective alert triage, facilitating mitigation of analyst fatigue and prioritization of actionable alarms. Quantitative evidence supports the assertion that process-based alarm ranking can sharply reduce false positives while retaining legitimate detection rates, even under challenging "low-and-slow" adversarial scenarios.

Theoretically, process mining brings process science rigor to network security analytics, enabling conformance checking and dynamic process reconstruction. Practically, this opens avenues for real-time, model-agnostic integration with evolving IDS infrastructures and diverse network protocols.

Future Directions

Anticipated developments include automatic tuning of state-space characterizations, incorporation of multiple process discovery algorithms, personalized alarm cost metrics for specific alignment deviations, expanding coverage to broader attack datasets and protocols, and benchmarking against alternative alarm ranking schemes.

Conclusion

This paper demonstrates that process mining enriched anomaly-based IDS achieves interpretable, discriminative alarm severity rating, enabling selective filtering of alerts with minimal impact on detection performance. The approach is validated on Slowloris DoS attacks, with strong recall and precision figures, and achieves clear separation between genuine and misclassified benign flows. The framework is extensible, model-agnostic, and well-suited for integration into advanced network security operations, providing a substantial advance in the intersection of process mining and explainable IDS.