HeartStream: Continuous Cardiac Monitoring

• HeartStream is a comprehensive system integrating wearable, camera-based, and multimodal sensors to extract high-fidelity cardiovascular metrics like HR and HRV.
• It utilizes advanced signal processing and scalable streaming analytics, including techniques such as rPPG and event-based vision, for real-time risk and stress detection.
• The framework supports resource-efficient, privacy-preserving, on-device computation, making it suitable for clinical, telehealth, and consumer applications.

HeartStream refers to a broad spectrum of technologies, systems, and methodologies for continuous, non-invasive cardiac monitoring and physiological signal extraction, leveraging advances across wearable sensors, remote photoplethysmography (rPPG), event-based vision, streaming analytics, on-device computation, and multimodal data fusion. The unifying goal of HeartStream systems is to extract high-fidelity cardiovascular metrics—such as heart rate (HR), heart rate variability (HRV), and, in more advanced variants, clinical event or risk prediction—via unobtrusive or contactless mechanisms for real-time analysis and actionable feedback. Research under the HeartStream umbrella spans fundamental biophysical mapping, scalable distributed health analytics, robust and energy-efficient embedded neural architectures, and new modalities including event cameras and ultrasound, with demonstrated applications from telehealth to online streaming platforms.

1. Biophysical Mapping and Foundational Signal Dynamics

Early work foundational to HeartStream (Solberg et al., 2014) employed MRI to obtain qualitative and quantitative maps of thoracic tissue and cardiovascular movements, identifying key displacements that serve as ground truth for radar or contactless measurement system design. By analyzing MRI sequences during suspended respiration, major insights emerged:

• The largest heart-induced thoracic displacements occur in the anterior and left regions of the heart, with in-plane motion on the order of 1 cm; adjacent lung vessel movements were measured at 2–3 mm.
• Aortic segmentation provided both positional displacements and dynamic dilatation profiles, with the effective radius variation formula $\hat{r} = \sqrt{ab}$ (major and minor axes).
• Mechanical coupling between heart and aorta was documented, with aorta dilatation reflecting internal blood pressure variations superimposed upon mechanically-induced lateral shifts (up to 1.5 mm).
• Edge-detection using the Canny method accumulated boundary movements across slices and cardiac phases, mapping both static and dynamic tissue behavior.

This physiological mapping underpins later HeartStream technologies by establishing the amplitude, spatial localization, and mechanical interplay of cardiac-induced motions to inform sensor placement, resolution requirements, and diagnostic interpretation, especially for radar/microwave and imaging-based approaches.

2. Architectures for Continuous Cardiac Analytics and Stress Detection

Distributed and real-time HeartStream systems leverage streaming analytics infrastructure to ingest, preprocess, and interpret multivariate bio-signals at population scale (Sandha et al., 2017). A reference platform integrates Apache Kafka (data ingestion—fault tolerant, multi-user) with Apache Spark Streaming (real-time, modular, scalable analytics), supporting simultaneous ECG and blood pressure streams from multiple devices/users.

Key workflows include:

• ECG/HRV feature extraction via peak detection (R-peak identification, QRS, QT and waveform morphology), diastolic/systolic BP calculation, window-based signal normalization, and artifact handling.
• Risk prediction for heart failure employs a naive Bayes classifier parameterized by established ECG and BP irregularity prevalence, yielding percentage risk (e.g., 1.2% for normal, 12.5% for S-point anomalies, 20.6% for ST depression).
• Stress detection relies on RMSSD computation: $$\mathrm{RMSSD} = \sqrt{\frac{\sum_{n=1}^{N}(I[n] - I[n-1])^2}{N-1}}$$, with windowed HRV drops or LF/HF ratio shifts flagging elevated sympathetic tone.
• The architecture ensures real-time, interactive feedback at individualized or aggregate levels, and is validated on public datasets (e.g., Physionet) with multi-user simulation.

The HeartStream analytics paradigm enables scalable, robust cardiac event detection and wellness monitoring with dynamic feature selection and continuous adaptation.

3. Wearable, Non-Contact, and Multimodal HeartStream Systems

HeartStream systems span a range of sensing modalities and deployment approaches optimized for accuracy, user compliance, and privacy:

a) Wearable and On-Device Monitoring (Tailor et al., 2020, Burrello et al., 2022, Giordano et al., 21 Oct 2024, Giordano et al., 21 Oct 2024)

• Chest-mounted and wrist-worn devices utilize microphones or ultrasound transducers for body/heart sound acquisition and Doppler-based blood flow detection.
• Embedded algorithms include STFT for spectral feature extraction (log-magnitude coefficients), Random Forest or TCN inference, and Hidden Markov (semi-Markov) models for precise cardiac phase segmentation (S₁, S₂, systole, diastole).
• Quantized TCNs deploy on microcontrollers (e.g., STM32WB55) with real-time, integer arithmetic; best models achieve 4.41 BPM MAE with 412 kB memory and 1.79–47.65 mJ inference energy.
• Envelope detectors and 2D FFTs are applied in ultrasound-based systems, with performance near gold-standard ECG (r=0.99, 0.69±1.99 bpm mean error at the radial artery), and run-time of ~71 ms per window (~5.8 mW power).

b) Camera-Based and Contactless rPPG (Gudi et al., 2020, Zhuang et al., 2022, Napolean et al., 2022, Wang et al., 11 Apr 2024, Bian et al., 25 Oct 2024, Lyu et al., 2 Feb 2025, Moustafa et al., 14 May 2025)

• Algorithms combine facial/skin region detection with pixelwise temporal analysis; key innovations include Plane-Orthogonal-to-Skin projection, dynamic band-pass filtering, head-motion frequency subtraction, and superpixel/spatiotemporal representation.
• Lightweight DCT-based frequency decomposition and attention-enhanced signal reconstruction (e.g., Multi-Frequency Mode Signal Fusion; spectrum self-attention) improve robustness to real-world lighting and motion variability.
• Event camera approaches transform high-frequency, asynchronous light intensity events into frame-like data for CNN-based cardiac pulse estimation, with superior RMSE at 60/120 FPS (2.54/2.13 bpm).
• New architectures (e.g., zero-parameter temporal shift, parallel short/long-range branches, soft-attention) permit on-device, energy-aware recognition with up to 74.2% accuracy improvement and 51.2% latency reduction on mobile platforms.

c) Multimodal and Audio-Visual Fusion (Lyu et al., 2 Feb 2025)

• CardioLive introduces audio-visual deep fusion (CardioNet), synchronizing video (camera/DOI) and audio (raw, SincNet-inspired filters) via self-attention and frequency-domain convolution, achieving robust HR estimation (MAE 1.79 BPM), outperforming pure video or audio solutions.
• System design addresses variable frame rates (FPS), unsynchronized streams (0.3s drift threshold with realignment), and delivers >98 FPS throughput on major streaming platforms.

4. Resource Efficiency, Privacy, and Accessibility

Recent research emphasizes resource-efficient and privacy-preserving deployment:

• Quantized and architecture-searched models (e.g., Q-PPG) enable on-body inference below 2 seconds, on battery-operated wearables suitable for population-scale monitoring.
• Brain-inspired frameworks (CCNN in HR-RST (Wang et al., 11 Apr 2024)) enable robust skin segmentation independent of conventional facial detection—allowing measurements on non-facial body parts (e.g., palm, forearm, sole), essential for special populations (infants, burn victims) and privacy concerns.
• On-device and edge inference eliminates the need for cloud streaming of raw physiological data, addressing major privacy barriers for telehealth and public applications.
• Adaptive duty-cycling, efficient pre-processing, and attention-based spatiotemporal models (UbiHR (Bian et al., 25 Oct 2024)) further reduce computational and memory footprints, expanding HeartStream applicability to commodity hardware and ubiquitous contexts.

5. Clinical Applications, Diagnostics, and Advanced Phenotyping

HeartStream platforms support a range of clinical, wellness, and research applications:

• Cardiovascular risk stratification from continuous wearable data (LSTM/DeepHeart (Ballinger et al., 2018)) achieves c-statistics of 0.8451 (diabetes), 0.8086 (hypertension), etc., by leveraging multi-task semi-supervised pretraining and massively multi-week biometric time series.
• Automated heart sound abnormality detection (dual-stream CNN+GRU with MFCC features and attention (Rashid et al., 2022)) on PhysioNet data achieves 87.11% accuracy, repurposing cost-effective PCG recorders for large-scale screening.
• Generative modeling (CHeart (Qiao et al., 2023)) enables conditionally-aware simulation and completion of 4D cardiac anatomy and motion, integrating clinical factors (age, gender, SBP) into spatiotemporal β-VAE-LSTM architectures—supporting data augmentation, anomaly detection, and personalized risk profiling.
• HeartStream extension to affective computing, entertainment, lie detection, and personalized streaming experiences is realized in CardioLive, Dišimo (Mladenovic et al., 2018), and EvoK (Shandilya et al., 2021), featuring non-disruptive feedback and social/interpersonal physiological sharing for enhanced engagement and well-being.

6. Technical Challenges and Ongoing Directions

Despite rapid advances, several technical and translational challenges remain:

• Algorithmic: Handling extreme lighting, non-stationary backgrounds, and complex motion patterns; achieving reliable segmentation and tracking in occluded or group settings; extending to new sensor modalities and cross-domain data fusion.
• Systemic: Standardized benchmarking (VicarPPG 2, CleanerPPG (Gudi et al., 2020)), open-source/collaborative data deployment, and ensuring clinical validity and bio-equivalence.
• Practical: User engagement, privacy management, social acceptability (e.g., form factor, ambient feedback), and regulatory integration into broader healthcare workflows.
• Research directions include interpretability (attention weights, explainable AI), confounder-aware neural architectures (for medication or co-morbidity effects), further compression and deployment on edge/IoT platforms, and real-world validation in high-variance and resource-constrained populations.

---

HeartStream thus denotes the convergence of physiological modeling, scalable analytics, robust hardware-aware signal processing, and multimodal, privacy-aware design to enable continuous, high-fidelity cardiac monitoring across clinical, consumer, and ubiquitous computing domains. The evolving landscape incorporates foundational biophysics, scalable event detection, energy-efficient neural architectures, and seamless integration into everyday workflows and social contexts.