Real-Time Monitoring of User Stress, Heart Rate and Heart Rate Variability on Mobile Devices (2210.01791v1)

Abstract: Stress is considered to be the epidemic of the 21st-century. Yet, mobile apps cannot directly evaluate the impact of their content and services on user stress. We introduce the Beam AI SDK to address this issue. Using our SDK, apps can monitor user stress through the selfie camera in real-time. Our technology extracts the user's pulse wave by analyzing subtle color variations across the skin regions of the user's face. The user's pulse wave is then used to determine stress (according to the Baevsky Stress Index), heart rate, and heart rate variability. We evaluate our technology on the UBFC dataset, the MMSE-HR dataset, and Beam AI's internal data. Our technology achieves 99.2%, 97.8% and 98.5% accuracy for heart rate estimation on each benchmark respectively, a nearly twice lower error rate than competing methods. We further demonstrate an average Pearson correlation of 0.801 in determining stress and heart rate variability, thus producing commercially useful readings to derive content decisions in apps. Our SDK is available for use at www.beamhealth.ai.

• The paper introduces the Beam AI SDK, a novel method for real-time stress, heart rate, and HRV monitoring using a smartphone camera to analyze facial pulse waves.
• Empirical evaluation shows the SDK achieves high heart rate estimation accuracy (e.g., 0.65 MAE on UBFC) and strong correlation with gold-standard measurements.
• This technology enables mental health applications to integrate stress monitoring without needing external wearables, enhancing accessibility, but requires formal medical validation.

Real-Time Monitoring of User Stress, Heart Rate, and Heart Rate Variability on Mobile Devices

The paper introduces the Beam AI SDK, a novel software development kit designed to monitor user stress, heart rate, and heart rate variability in real-time via smartphone applications. This technology addresses the growing need for stress monitoring capabilities within mobile apps by leveraging the user's selfie camera to track subtle color variations in skin regions, thereby extracting a pulse wave. These measurements allow the determination of stress levels using the Baevsky Stress Index, along with heart rate and heart rate variability.

Technological Contributions

The Beam AI SDK is composed of three core modules: Pulse Extractor, Inter-Beat Interval Processor, and Biometric Estimator. Each module plays a vital role in processing data derived from the user's facial video feed to deliver real-time physiological insights. The Pulse Extractor utilizes novel photoplethysmography techniques to capture pulse waves by examining minute color changes on the user's facial skin. Subsequently, the Inter-Beat Interval Processor identifies intervals between detected pulse peaks, and the Biometric Estimator computes heart rate, heart rate variability, and stress from these intervals, ensuring that the SDK operates entirely on-device to safeguard user privacy.

Empirical Evaluation

The empirical evaluation underscores the effectiveness of the Beam AI SDK across several benchmarks, including the UBFC and MMSE-HR datasets. The technology showcased remarkable accuracy in estimating heart rates, with minimal mean average errors (MAE) of 0.65 beats per minute on UBFC and 1.72 beats per minute on MMSE-HR datasets, outpacing competing methodologies. Moreover, the SDK exhibited strong Pearson correlation scores indicative of its high fidelity in following physiological signal trends.

The internal evaluation with Beam AI's dataset, involving a 20-minute recording using an iPhone 13, aligned closely with gold-standard measurements from a Polar H10 chest strap. This further validates the SDK's reliability under typical smartphone use conditions.

Implications and Future Directions

Practically, this technology enables mental health applications to incorporate stress monitoring features without necessitating additional hardware, such as wearable devices, expanding accessibility. Theoretically, it elucidates novel applications of video-based bio-signal processing algorithms.

Future endeavors include wider platform support, adaptations to varied environmental conditions, and enhancements in pulse wave extraction robustness. An ongoing empirical paper in Vancouver is expected to broaden the SDK's applicability across diverse smartphone usage patterns, lighting conditions, and user demographics.

Conclusion

The Beam AI SDK signifies a significant advancement in real-time physiological monitoring via mobile devices by achieving superior accuracy in heart rate and stress measurement while operating with remarkable processing efficiency. Despite these promising results, the paper duly acknowledges the necessity of formal medical validations before its application in clinical diagnostics.

Thus, the Beam AI SDK sets a new precedent in mobile-based health monitoring, potentially influencing future developments in non-contact biometry and the broader AI community.