A Tiny Transformer for Low-Power Arrhythmia Classification on Microcontrollers (2402.10748v2)

Abstract: Wearable systems for the continuous and real-time monitoring of cardiovascular diseases are becoming widespread and valuable assets in diagnosis and therapy. A promising approach for real-time analysis of the electrocardiographic (ECG) signal and the detection of heart conditions, such as arrhythmia, is represented by the transformer machine learning model. Transformers are powerful models for the classification of time series, although efficient implementation in the wearable domain raises significant design challenges, to combine adequate accuracy and a suitable complexity. In this work, we present a tiny transformer model for the analysis of the ECG signal, requiring only 6k parameters and reaching 98.97% accuracy in the recognition of the 5 most common arrhythmia classes from the MIT-BIH Arrhythmia database, assessed considering 8-bit integer inference as required for efficient execution on low-power microcontroller-based devices. We explored an augmentation-based training approach for improving the robustness against electrode motion artifacts noise, resulting in a worst-case post-deployment performance assessment of 98.36% accuracy. Suitability for wearable monitoring solutions is finally demonstrated through efficient deployment on the parallel ultra-low-power GAP9 processor, where inference execution requires 4.28ms and 0.09mJ.

• The paper introduces a tiny transformer achieving 98.97% accuracy for classifying five arrhythmia types.
• It employs an augmentation-based training approach to enhance noise robustness on a compact 49kB model.
• The model delivers practical performance with 4.28ms inference time and 0.09mJ energy on the GAP9 processor.

Tiny Transformer Model for Efficient Arrhythmia Classification on Wearables

Introduction

The necessity for real-time, low-power, and accurate health monitoring systems has skyrocketed due to the increasing prevalence of cardiovascular diseases globally. Wearable devices offer a promising solution by enabling continuous monitoring of patients' cardiac health. However, deploying sophisticated machine learning models like transformers on such devices poses challenges due to their computational and memory requirements. In contrast, this paper by Busia et al. introduces a compact transformer-based model designed for arrhythmia classification directly on microcontrollers. Remarkably, it achieves a remarkable 98.97% accuracy in identifying five common arrhythmia classes using the MIT-BIH Arrhythmia Database, with additional robustness against noise and efficient deployment on the GAP9 processor.

Model Design and Implementation

The authors propose a "tiny" transformer architecture, tailored to meet the constraints of low-power wearable devices without compromising on accuracy. Key aspects of the design include:

• Compact Architecture: A simplified transformer model that utilizes an augmentation-based training approach, maintaining high accuracy while significantly reducing computational and storage demands.
• Efficiency on Microcontrollers: The model's lightweight framework, requiring a mere 49kB footprint for deployment, aligns with the limited resources available on typical wearable microcontrollers.
• Augmentation for Noise Robustness: To address real-world challenges like signal noise from electrode movement, the training leveraged a dataset augmented with noise, enhancing the model's robustness.

Experimental Results

The researchers conducted exhaustive experiments to validate the model's performance, including:

• High Classification Accuracy: The tiny transformer model demonstrated an exceptional ability to classify arrhythmias, with a worst-case post-deployment accuracy of 98.36%, accounting for signal noise.
• Resource Efficiency: Deployed on the GAP9 processor, the model required only 4.28ms for inference and 0.09mJ of energy, showcasing its suitability for real-time analysis in wearable devices.
• Noise Augmentation Benefit: Incorporating noise-augmented data during training proved essential for enhancing the model's noise robustness without significant loss in classification accuracy.

Practical Implications and Future Directions

This work opens new avenues for the deployment of advanced machine learning models on resource-constrained wearable devices. The tiny transformer model balances accuracy, computational demand, and energy efficiency, making it viable for long-term, real-time cardiac monitoring. Future research could explore further optimization techniques for model compression and efficiency, as well as extending the approach to other types of biomedical signals and health monitoring applications.

Conclusion

The tiny transformer model presented by Busia et al. is a significant step forward in embedding sophisticated AI models into wearable healthcare devices. Its ability to perform accurate arrhythmia classification in real-time on low-power platforms addresses a critical need in continuous cardiac health monitoring. This work not only demonstrates the potential of transformers in the medical domain but also highlights the importance of model efficiency for practical deployment in wearables.