HeAR -- Health Acoustic Representations (2403.02522v1)

Abstract: Health acoustic sounds such as coughs and breaths are known to contain useful health signals with significant potential for monitoring health and disease, yet are underexplored in the medical machine learning community. The existing deep learning systems for health acoustics are often narrowly trained and evaluated on a single task, which is limited by data and may hinder generalization to other tasks. To mitigate these gaps, we develop HeAR, a scalable self-supervised learning-based deep learning system using masked autoencoders trained on a large dataset of 313 million two-second long audio clips. Through linear probes, we establish HeAR as a state-of-the-art health audio embedding model on a benchmark of 33 health acoustic tasks across 6 datasets. By introducing this work, we hope to enable and accelerate further health acoustics research.

• The paper introduces a self-supervised masked autoencoder framework that robustly analyzes respiratory sounds for health monitoring.
• It leverages 313 million audio clips to generalize across nine health tasks, outperforming benchmarks in cough and spirometry analysis.
• The study demonstrates scalable data efficiency and sets the stage for advanced non-invasive diagnostic tools in respiratory care.

An Evaluation of HeAR: Health Acoustic Representations for Machine Learning Applications in Health Monitoring

The paper introduces HeAR, a novel self-supervised learning-based deep learning framework aimed at advancing the field of health acoustics by analyzing non-semantic respiratory sounds for health monitoring and disease detection. HeAR is developed to address limitations in current machine learning systems that are often narrowly trained and hence exhibit poor generalization across various health tasks. This approach leverages a scalable self-supervised learning method using masked autoencoders trained on a substantial dataset of acoustic health signals—specifically, 313 million two-second-long audio clips, highlighting its potential to enrich the domain of health acoustics.

Methodology and Objectives

The framework is composed of multiple components: a health acoustic event detector, an audio encoder based on masked autoencoders, and a task-specific evaluation module for various health acoustic tasks. The training of the audio encoder is supported by a large, unlabeled dataset harvested from non-copyrighted content on YouTube. This scale of data is expected to foster better generalization across nine health-related acoustic tasks, which include health acoustic event detection, cough-based disease/condition inference, and spirometry measurements.

The chosen self-supervised learning architecture, inspired by MAEs, aims to learn acoustics-based representations that are both robust and transferable across tasks. These representations are benchmarked against established systems such as TRILL, FRILL, and BigSSL-CAP12 among others. A diverse benchmark of 33 tasks across multiple datasets, including FSD50K, FluSense, and proprietary datasets from CIDRZ in Zambia, is employed to establish the efficacy and superiority of HeAR.

Results and Key Findings

HeAR's performance is notably superior in several tasks, achieving the best results in 17 of the 33 tasks evaluated, specifically shining in the domains of cough inference and spirometry estimation. HeAR demonstrates robust performance in generalized classification of respiratory diseases, such as tuberculosis, using cough audio data, and accurate spirometry estimations in COPD patient monitoring scenarios. In scenarios requiring cross-device evaluation, HeAR maintains consistent and high performance, underscoring its potential utility in real-world applications where varied audio recording equipment would be used.

The significance of training data size is also evident in this work; as the training data pool increases, the performance and robustness of the audio encoder improve, demonstrating data scalability positively influences results. Moreover, through high mean reciprocal rank scores, HeAR exhibits exceptional data efficiency, maintaining high accuracy even when trained on significantly reduced datasets.

Implications and Future Directions

This paper demonstrates the potential of large-scale self-supervised learning architectures in the field of health acoustics, especially in underserved domains such as respiratory health monitoring and diagnostics. Although the research focuses on using linear probes, future work could investigate fine-tuning the entire model to optimize performance further. Additionally, issues such as performance generalization, demographic biases, and clinically relevant thresholds require further exploration through exhaustive clinical trials and validations before HeAR can be fully integrated into healthcare systems.

Looking ahead, the research paves the way for comprehensive studies in health acoustics, improving upon foundational technologies like HeAR. Innovations such as model distillation or quantization may further optimize these encoders for real-time processing on mobile platforms, which is crucial for their deployment in under-resourced settings. With continued advancements, such systems could potentially aid healthcare practitioners worldwide, particularly in regions where respiratory diseases pose significant public health challenges, providing an efficient, cost-effective, and non-invasive diagnostic tool.