Pseudo Replay-based Class Continual Learning for Online New Category Anomaly Detection in Advanced Manufacturing (2312.02491v3)

Abstract: The incorporation of advanced sensors and machine learning techniques has enabled modern manufacturing enterprises to perform data-driven classification-based anomaly detection based on the sensor data collected in manufacturing processes. However, one critical challenge is that newly presented defect category may manifest as the manufacturing process continues, resulting in monitoring performance deterioration of previously trained machine learning models. Hence, there is an increasing need for empowering machine learning models to learn continually. Among all continual learning methods, memory-based continual learning has the best performance but faces the constraints of data storage capacity. To address this issue, this paper develops a novel pseudo replay-based continual learning framework by integrating class incremental learning and oversampling-based data generation. Without storing all the data, the developed framework could generate high-quality data representing previous classes to train machine learning model incrementally when new category anomaly occurs. In addition, it could even enhance the monitoring performance since it also effectively improves the data quality. The effectiveness of the proposed framework is validated in three cases studies, which leverages supervised classification problem for anomaly detection. The experimental results show that the developed method is very promising in detecting novel anomaly while maintaining a good performance on the previous task and brings up more flexibility in model architecture.

• The paper introduces a pseudo replay-based continual learning framework that addresses catastrophic forgetting by synthesizing data with SMOTE.
• It utilizes an oversampling method to simulate prior class distributions without extensive data storage, enhancing anomaly detection.
• The framework was validated in additive manufacturing, flexibly adapting classifier architectures to achieve high accuracy for both known and new anomalies.

In the field of advanced manufacturing, particularly in additive manufacturing, ensuring quality while adapting to new and unforeseen anomalies is crucial. Conventional machine learning models often face a challenge termed "catastrophic forgetting," where a model, when trained on new data, loses the ability to identify scenarios it was previously trained on. A paper has sought to address this issue by proposing a novel framework using pseudo replay-based continual learning to enable online new category anomaly detection.

This framework introduces a technique for class incremental learning that does not require storing extensive amounts of data—something that has traditionally impeded memory-based continual learning due to storage constraints. The approach synthesizes high-quality data that mimic previous class characteristics, a procedure which circumvents the issue of data storage limits and augments the data's quality.

The core innovation of this research lies in its use of an oversampling-based data generation method, specifically the Synthetic Minority Oversampling Technique (SMOTE). SMOTE is traditionally used to address class imbalances by generating synthetic data for minority classes. Here, it's repurposed as a pseudo-data generator to simulate previous class distributions, allowing the continual learning model to train on these pseudo data when exposed to new anomalies.

The effectiveness of the proposed framework was validated using a real-world additive manufacturing scenario, where two distinct types of process anomalies were introduced in a controlled environment. The continual learning model demonstrated superior performance in detecting both known and new anomalies compared to traditional methods. It not only maintained a high detection accuracy for previously learned tasks but also proved flexible when incorporating new data, thus reducing the impact of catastrophic forgetting.

Another significant outcome of this paper is the framework's flexibility in terms of the model structure. Unlike other continual learning approaches, where a model's architecture remains fixed, the proposed framework allows for the deployment of different classifiers based on varied data patterns. As a practical illustration, the paper showed that a classifier architecture that performed well on initial tasks could be swapped for a more suitable architecture without compromise when addressing subsequent tasks.

In conclusion, this research presents a promising development in the field of smart manufacturing systems. By employing a pseudo replay-based continual learning framework that leverages data generation techniques like SMOTE, this method offers a solution for effectively incorporating incremental learning in the face of limited data storage capacity. It shows potential not just in additive manufacturing but in any scenario where the seamless integration of new information into existing processes is crucial. Future explorations intend to investigate various generators and assess the applicability across different manufacturing processes, solidifying the practical impact of this innovative approach.