Chameleons do not Forget: Prompt-Based Online Continual Learning for Next Activity Prediction

Abstract: Predictive process monitoring (PPM) focuses on predicting future process trajectories, including next activity predictions. This is crucial in dynamic environments where processes change or face uncertainty. However, current frameworks often assume a static environment, overlooking dynamic characteristics and concept drifts. This results in catastrophic forgetting, where training while focusing merely on new data distribution negatively impacts the performance on previously learned data distributions. Continual learning addresses, among others, the challenges related to mitigating catastrophic forgetting. This paper proposes a novel approach called Continual Next Activity Prediction with Prompts (CNAPwP), which adapts the DualPrompt algorithm for next activity prediction to improve accuracy and adaptability while mitigating catastrophic forgetting. We introduce new datasets with recurring concept drifts, alongside a task-specific forgetting metric that measures the prediction accuracy gap between initial occurrence and subsequent task occurrences. Extensive testing on three synthetic and two real-world datasets representing several setups of recurrent drifts shows that CNAPwP achieves SOTA or competitive results compared to five baselines, demonstrating its potential applicability in real-world scenarios. An open-source implementation of our method, together with the datasets and results, is available at: https://github.com/SvStraten/CNAPwP.

• The paper introduces CNAPwP, a framework integrating general and expert prompts for online continual learning to mitigate catastrophic forgetting in next activity prediction.
• It employs a multi-head self-attention backbone with dynamic event windowing and prefix bucketing to handle streaming data and concept drifts.
• Empirical results on synthetic and real-world logs show superior accuracy and near-zero forgetting compared to state-of-the-art baselines.

Prompt-Based Online Continual Learning for Next Activity Prediction

Problem Context and Motivation

The paper advances the field of Predictive Process Monitoring (PPM), specifically the online Next Activity Prediction (NAP) problem under concept drift and recurrency. In dynamic process environments, changes in underlying process distributions (concept drifts) and periodic recurrency of process tasks result in catastrophic forgetting when most existing NAP frameworks are updated on recent data. While image classification in continual learning (CL) has seen effective mitigation of forgetting via memory-, regularization-, architecture-, or prompt-based paradigms, the temporal and sequential dependencies in NAP make straightforward transfer of these methods insufficient.

CNAPwP: Architecture and Methodology

The paper introduces Continual Next Activity Prediction with Prompts (CNAPwP), which adapts the DualPrompt [26] prompt-based continual learning algorithm for the NAP domain. CNAPwP integrates both general prompts (G-Prompt) for task-invariant knowledge and expert prompts (E-Prompt) for task-specific adaptation into a multi-head self-attention (MHSA) backbone. The architecture modularly addresses major continual learning requirements as follows:

• Input Streaming and Preprocessing: Event logs are transformed into streams of one-hot encoded prefixes, with dynamic windowing for real-time processing. Prefixes are bucketed according to length to enable efficient batch updates and prefix-dependent parameter selection during both training and inference.
• Prompt Detection and Drift Handling: Upon detecting a concept drift, a prefix tree–based comparison automatically determines task recurrence or novelty. Existing E-Prompts are re-used for recurring tasks; for novel drifts, new prompt parameters are randomly initialized.
• Parameter Updates: Both G- and E-prompts, as well as model weights, are updated via cross-entropy loss using bucketed event streams, allowing the architecture to achieve plasticity for new concepts while preserving knowledge relevant to previous or parallel tasks.

Distinct from rehearsal- or memory-based CL, the system operates in a strictly online regime, per-event, without requiring storage of past event logs (beyond short buffer sizes for drift detection).

Evaluation Protocol and Datasets

CNAPwP is evaluated against five baselines: DynaTrainCDD [11], TFCLPM [25], a GAN-based approach [23], Landmark, and Incremental Update (w = Last Drift) [16]. Datasets include three carefully constructed synthetic logs with controlled recurrent drifts (RandomTasks, ImbalancedTasks, RecurrentTasks) and two highly variable real-world logs (Recurrent BPIC2015, BPIC2017). Evaluation emphasizes not only average accuracy, but also introduces the task-specific forgetting metric: the accuracy delta on re-encountering tasks versus the initial exposure, capturing the practical effect of catastrophic forgetting in realistic process environments.

Experimental Results

Strong Numerical Results and Empirical Claims

• On RandomTasks and RecurrentTasks (multi-task and recurrent drift scenarios), CNAPwP achieves highest average accuracy (.813 and .789 respectively), outperforming all baselines, including TFCLPM and DynaTrainCDD.
• On ImbalancedTasks and BPIC2017 (fewer drifts/frequent tasks), CNAPwP is narrowly outperformed by TFCLPM, but analysis shows higher long-term retention and less instability on task recurrences.
• Task-specific forgetting analysis confirms near-zero or negative forgetting for CNAPwP across most tasks and datasets—contrasting observed severe forgetting in DynaTrainCDD and updating baselines especially during task recurrence.
• Processing overhead, while higher for CNAPwP (typically 3–25 ms per event), remains within realistic business process response times. An ablation study demonstrates that both G- and E-prompts are necessary for optimal performance; using only one or neither significantly increases forgetting and reduces average accuracy (\$.789 \$ for both, vs. .770/.773 with one, .709 without prompts).

Contradictory/Unexpected Claims

• Despite its architectural complexity and per-event prompt detection, CNAPwP achieves practical runtime, significantly outpacing the Landmark baseline (which is update-inefficient despite being accurate).
• The GAN-based approach, despite its design for sequence generation, consistently underperforms (accuracy as low as .604 on Recurrent BPIC2015), empirically supporting the claim that adversarial learning without explicit memory or prompt structuring is ill-suited for fast drift adaptation in online NAP scenarios.

Theoretical and Practical Implications

The formalism of NAP as a continual learning task under recurring concept drift creates rigorous requirements: real-time adaptation, resistance to catastrophic forgetting, task-incrementality, and no reliance on full event log history. The paper establishes that prompt-based continual learning—if equipped with event stream segmentation, task recognition, and modular prompt assignment—can bridge the gap between image-sequence continual learning and process sequence prediction. These findings suggest that:

• Prompt-based approaches generalize beyond NLP/image domains to structured temporal prediction tasks.
• Explicit modeling of both task-invariant and task-specific knowledge via G/E prompts is empirically necessary for high retention and rapid adaptation—which may inform future design of sequence-based CL algorithms beyond NAP.

Practically, CNAPwP enables deployment of predictive process monitors in environments with privacy or storage restrictions, recurring seasonal or behavioral patterns, and rapidly evolving operational policies, addressing major barriers in industrial PPM.

Future Directions

Open challenges remain in automated and adaptive concept drift detection, efficient task-recognition mechanics in highly complex logs, and further reducing prompt overhead for deployment in ultra-low-latency environments. The current matching on prefix trees, while effective, induces additional computation in environments with high trace variance. Improving batch parallelization and optimizing the bucketing strategy are possible future avenues. Integrating robust drift detectors could move the framework closer to fully automated, model-free continual NAP.

Conclusion

This paper advances online continual learning for Next Activity Prediction by proposing a prompt-based framework, CNAPwP, that mitigates catastrophic forgetting and achieves superior knowledge retention under recurrent drift. Through rigorous experimentation across synthetic and real-world datasets, the approach demonstrates state-of-the-art accuracy and forgetting resistance, highlighting the significance of prompt composition for temporal sequence CL tasks. These findings constitute a reference point for further research on practical, adaptable, and memory-efficient process prediction architectures under dynamic environments.