Knowledge-Driven Modulation of Neural Networks with Attention Mechanism for Next Activity Prediction (2312.08847v1)

Abstract: Predictive Process Monitoring (PPM) aims at leveraging historic process execution data to predict how ongoing executions will continue up to their completion. In recent years, PPM techniques for the prediction of the next activities have matured significantly, mainly thanks to the use of Neural Networks (NNs) as a predictor. While their performance is difficult to beat in the general case, there are specific situations where background process knowledge can be helpful. Such knowledge can be leveraged for improving the quality of predictions for exceptional process executions or when the process changes due to a concept drift. In this paper, we present a Symbolic[Neuro] system that leverages background knowledge expressed in terms of a procedural process model to offset the under-sampling in the training data. More specifically, we make predictions using NNs with attention mechanism, an emerging technology in the NN field. The system has been tested on several real-life logs showing an improvement in the performance of the prediction task.

Overview of the Paper

The paper discusses an innovative approach in the field of Predictive Process Monitoring (PPM), which is a subset of Process Mining. PPM focuses on using historical execution data to forecast future activities within ongoing processes. The objective of the paper is to improve the accuracy of predictions, especially for rare or changing process patterns, by incorporating background knowledge in the form of procedural process models such as Petri nets.

Neural Networks in Next Activity Prediction

In PPM, Next Activity Prediction (NAP) tasks are typically performed using Neural Networks (NNs). While NNs excel in general cases, they falter with variants not well-represented in the training data—an issue that can surface in exceptional circumstances or when the process undergoes a concept drift. To overcome this limitation, the authors propose a method enhancing NN predictions by infusing them with background knowledge using an attention mechanism.

Symbolic[Neuro] System

The proposed Symbolic[Neuro] system integrates NNs with an attention mechanism to predict future activities, guided by an underlying procedural process model. This approach is unique as it modulates the NN's output during the prediction phase itself. By utilizing this combined score–one derived from the NN predictions and another from model compliance during suffix construction — it systematically favors predictions that align more with the established background knowledge.

Experimental Evaluation

The system's effectiveness was evaluated on several real-life logs and a synthetic log tailored to mimic exceptional process executions. Across these evaluations, the proposed method illustrates a marked improvement in the predictive performance, particularly in cases where the training data is skewed or when the process execution deviates from regular patterns. The authors thoroughly explain the method's efficacy, underscoring its aptitude for adapting to concept drifts within processes.

Conclusion

In summary, this paper puts forth a robust approach to augmenting the capability of PPM systems by leveraging background process knowledge. This is accomplished through the modulation of NN outputs, specifically by applying an attention-based architecture and procedural process models during prediction tasks. The key takeaway is a significant enhancement in predictive quality for processes with rare or evolving patterns, implying the potential for better resource planning and management in various domains where complex process execution is pivotal.