FaultFormer: Pretraining Transformers for Adaptable Bearing Fault Classification (2312.02380v3)

Abstract: The growth of global consumption has motivated important applications of deep learning to smart manufacturing and machine health monitoring. In particular, analyzing vibration data offers great potential to extract meaningful insights into predictive maintenance by the detection of bearing faults. Deep learning can be a powerful method to predict these mechanical failures; however, they lack generalizability to new tasks or datasets and require expensive, labeled mechanical data. We address this by presenting a novel self-supervised pretraining and fine-tuning framework based on transformer models. In particular, we investigate different tokenization and data augmentation strategies to reach state-of-the-art accuracies using transformer models. Furthermore, we demonstrate self-supervised masked pretraining for vibration signals and its application to low-data regimes, task adaptation, and dataset adaptation. Pretraining is able to improve performance on scarce, unseen training samples, as well as when fine-tuning on fault classes outside of the pretraining distribution. Furthermore, pretrained transformers are shown to be able to generalize to a different dataset in a few-shot manner. This introduces a new paradigm where models can be pretrained on unlabeled data from different bearings, faults, and machinery and quickly deployed to new, data-scarce applications to suit specific manufacturing needs.

• The paper introduces FaultFormer, a transformer-based framework that classifies bearing faults using vibration signal data.
• It employs data augmentation and spectral domain techniques to enhance model accuracy and handle noisy, limited datasets.
• The model's adaptable pretraining strategies enable robust performance across various industrial applications, achieving 99% accuracy.

In an innovative leap forward in machine health monitoring, researchers have unveiled a powerful new AI framework aptly named FaultFormer, which utilizes a transformer-based encoder for the precise classification of bearing faults using vibration signals. This breakthrough offers invaluable insights for industries reliant on machinery and could dramatically enhance predictive maintenance strategies.

Transformers, primarily known for their impact in the fields of natural language processing and computer vision, are now making waves in the engineering field thanks to their capacity to understand complex sequences in data. The FaultFormer framework is no exception, as it harnesses the Transformer's strength in processing sequential data and its adept attention mechanism that uniquely identifies relevant data patterns for robust classification.

The brilliance of FaultFormer lies in its methodology, where vibration signal data undergoes specific augmentations, including noise introduction, cropping, and shifting. The data is then represented in the spectral domain – a technique that allows the AI to focus on the most crucial Fourier modes representing the signal most effectively. This not only enhances the model's accuracy but also enables it to grapple with limited and noisy data typical in real-world scenarios.

FaultFormer's ability to grasp both local and global trends in vibration data enables it to classify faults with stunning accuracy. In testing, the model achieved a remarkable 99% accuracy rate, showing considerable promise for its application in smart manufacturing where machine health monitoring and predictive maintenance are critical for efficiency and cost savings.

Moreover, FaultFormer is designed with adaptability in mind. Two distinct pretraining strategies have been devised to ensure the model can quickly adjust to new data, scenarios, or types of machinery. This flexible learning backbone means FaultFormer could be a universal solution adaptable across various industries and equipment without the need for extensive reprogramming.

In essence, FaultFormer signals a major advancement in smart manufacturing and industrial AI, shifting away from traditional reactive maintenance approaches to more sophisticated, data-driven strategies. This innovative fusion of AI with machine health diagnostics could not only prevent costly downtime but pave the way for more efficient, self-regulating manufacturing systems, heralding a new era of industrial automation.