Train-On-Request: An On-Device Continual Learning Workflow for Adaptive Real-World Brain Machine Interfaces (2409.09161v2)

Abstract: Brain-machine interfaces (BMIs) are expanding beyond clinical settings thanks to advances in hardware and algorithms. However, they still face challenges in user-friendliness and signal variability. Classification models need periodic adaptation for real-life use, making an optimal re-training strategy essential to maximize user acceptance and maintain high performance. We propose TOR, a train-on-request workflow that enables user-specific model adaptation to novel conditions, addressing signal variability over time. Using continual learning, TOR preserves knowledge across sessions and mitigates inter-session variability. With TOR, users can refine, on demand, the model through on-device learning (ODL) to enhance accuracy adapting to changing conditions. We evaluate the proposed methodology on a motor-movement dataset recorded with a non-stigmatizing wearable BMI headband, achieving up to 92% accuracy and a re-calibration time as low as 1.6 minutes, a 46% reduction compared to a naive transfer learning workflow. We additionally demonstrate that TOR is suitable for ODL in extreme edge settings by deploying the training procedure on a RISC-V ultra-low-power SoC (GAP9), resulting in 21.6 ms of latency and 1 mJ of energy consumption per training step. To the best of our knowledge, this work is the first demonstration of an online, energy-efficient, dynamic adaptation of a BMI model to the intrinsic variability of EEG signals in real-time settings.

• The paper introduces a Train-On-Request workflow that enables on-demand retraining, achieving up to 92% classification accuracy in BMIs.
• It integrates continual learning techniques like Experience Replay and Learning without Forgetting to prevent catastrophic forgetting.
• The method demonstrates rapid calibration (1.6 minutes) and energy efficiency (1 mJ per step) on a RISC-V ultra-low-power SoC.

Train-On-Request: An On-Device Continual Learning Workflow for Adaptive Real-World Brain Machine Interfaces

The paper titled "Train-On-Request: An On-Device Continual Learning Workflow for Adaptive Real-World Brain Machine Interfaces" presents a significant advancement in the optimization of Brain-Machine Interfaces (BMIs) for real-world applications. The primary contribution lies in the development and evaluation of a Train-On-Request (TOR) workflow, which facilitates user-specific model adaptation on demand, thereby addressing the variability and inconsistencies inherent in electroencephalography (EEG) signals.

Overview and Methodology

BMIs aim to bridge the communication gap between users and devices by leveraging brain signals, particularly EEG. Traditional BMI systems often rely on pre-trained models; however, these models tend to degrade over time when exposed to novel conditions. This phenomenon necessitates periodic re-training to maintain accuracy. The TOR workflow proposed in this paper enables near real-time re-training directly on the edge device, enhancing the practical usability of BMIs.

The paper introduces TOR as a flexible, on-device continual learning (CL) strategy. The key components of the methodology include:

1. On-Demand Training: Users can initiate model retraining during usage whenever a decrease in classification performance is detected.
2. Continual Learning Techniques: Integration with Experience Replay (ER) and Learning without Forgetting (LwF) strategies to mitigate the effects of catastrophic forgetting.
3. Edge Implementation: Demonstration of the workflow on a RISC-V ultra-low-power SoC (GAP9), ensuring the feasibility of on-device learning.

Empirical Evaluation

The evaluation utilized a dataset recorded from a wearable BMI headband. The key performance metrics were accuracy of the classification, re-calibration time, and energy consumption per training step. Results highlighted several critical advancements:

• Accuracy: The TOR workflow achieved up to 92% accuracy, outperforming traditional transfer learning methods in maintaining high performance across sessions.
• Calibration Time: The re-calibration time was as low as 1.6 minutes, representing a 46% reduction compared to naive transfer learning workflows.
• Energy Efficiency: Deployment on the GAP9 SoC demonstrated high energy efficiency with only 21.6 ms of latency and 1 mJ of energy consumption per training step.

Practical and Theoretical Implications

The proposed TOR workflow has several practical implications. Firstly, the reduction in re-calibration time means that users can adapt the BMI model in real-world settings without significant interruptions. This enhancement significantly boosts the user experience and satisfaction. Secondly, the ability to perform on-device learning ensures that BMI systems can be more autonomous and less reliant on external computational resources, which is crucial for applications in remote or resource-constrained environments.

From a theoretical perspective, integrating CL techniques such as ER and LwF within the TOR workflow addresses the issue of catastrophic forgetting, ensuring sustained performance over multiple sessions. This paper provides a robust framework for future developments in adaptive BMIs, emphasizing the importance of dynamic model updates tailored to individual users.

Future Developments

Several avenues for future development are evident. The synergy between TOR workflow and more sophisticated CL algorithms can be further explored to enhance model robustness. Additionally, expanding the paper to include diverse and larger datasets can validate the generalizability of the proposed methodology. Finally, integrating advanced signal processing techniques for EEG data can further improve the signal-to-noise ratio, leading to even higher classification accuracies.

Conclusion

This paper contributes a comprehensive and efficient on-device continual learning framework for BMIs, addressing the challenges of inter-session variability and user adaptation in real-world conditions. The empirical results underscore the potential of the TOR workflow in achieving high accuracies with reduced re-calibration times and minimal energy consumption. These advancements pave the way for more reliable, user-centric BMI systems capable of real-time adaptation, significantly impacting the usability of BMIs in non-clinical settings. The proposed methodology and findings set a solid foundation for future research and development in adaptive BMI technologies.