A Large-Scale Database and Predictive Model of Listener-Rated Ease of Speech Understanding in Commercial Hearing Aids

Abstract: HearAdvisor aims to provide hearing-aid consumers with audio-performance metrics and recordings that reflect real listening experience. For speech-related metrics, HearAdvisor has historically used HASPIv2, a metric designed to predict objective intelligibility and validated primarily under simulated distortions. Its relationship to consumer-rated ease of understanding for commercial hearing aids is uncertain. Here we introduce a large-scale perceptual dataset and learned metric for listener-rated perceived benefit for speech understanding. Website visitors with self-reported hearing loss completed a blind, MUSHRA-inspired listening test in which they rated recordings of commercial hearing aids on a five-point "Ease of Understanding" scale. The dataset contains 151,608 ratings, 104,298 after quality screening, spanning 10,394 binaural acoustic-manikin recordings from 83 commercial products across 72 realistic acoustic scenes. To predict these ratings, we pass aided audio and a matched clean-speech reference through a frozen Whisper encoder, subtract their internal representations, and train a small MLP head on the resulting difference embedding. On devices held out of training, the learned metric substantially outperforms HASPIv2 at the scene level (overall r = 0.92 vs. 0.83; loud = 0.89 vs. 0.75; quiet = 0.79 vs. 0.58). In loud scenes, performance reaches the split-half reliability of the listener ratings; in quiet scenes, it approaches that ceiling. The model also responds sensibly to controlled gain and SNR manipulations. Together, the dataset and model provide a new way to predict listener-rated ease of speech understanding for real commercial hearing-aid recordings.

• The paper introduces a large-scale dataset capturing listener ratings for ease of speech understanding in commercial hearing aids using realistic acoustic scenes.
• It employs a frozen Whisper encoder with a domain-specific MLP head to generate 768-dimension embeddings and predict scene-level speech ease ratings using weighted MSE loss.
• Achieving high correlations (up to r=0.92) with listener ratings, the model outperforms existing intelligibility metrics, offering a scalable tool for consumer-centric hearing aid evaluation.

Listener-Rated Speech Understanding in Commercial Hearing Aids: Dataset and Predictive Model

Dataset Construction and Listener-Rated Metrics

The study introduces a large-scale perceptual dataset capturing listener ratings of speech understanding ease for recordings of commercial hearing aids. Device recordings were conducted using a KEMAR acoustic manikin fitted according to both manufacturers' initial protocols and NAL-NL2 targets for the N3 standard audiogram. These recordings spanned 72 spatially- and acoustically-realistic scenes reproduced in an anechoic chamber, representing everyday environments and configurations of speech and noise. Each device is evaluated in both quiet ($<70$ dB SPL) and loud ($>70$ dB SPL) backgrounds, grouped for analysis due to distinct behavioral profiles across these conditions.

Listener ratings were crowdsourced via HearAdvisor’s Blind Listening Challenge, wherein participants with mild, moderate, or severe hearing loss were instructed to audition the recordings through calibrated consumer hardware (headphones or speakers). Using a MUSHRA-like paradigm, listeners rated ease of speech understanding on a five-point scale. To ensure validity, sessions were filtered by device audition conditions, self-reported hearing loss, and quality-control anchors, resulting in 104,298 ratings across 10,394 audio recordings from 83 commercial devices.

Model Architecture and Training

The predictive model leverages a frozen Whisper-small encoder as a speech foundation model. Both aided (device) audio and matched clean reference speech are mean-pooled and differenced from selected encoder layers to form a 768-dimension embedding. This representation is processed by a domain-specific MLP head tailored for either loud (layer 5) or quiet (layer 2) backgrounds.

Figure 1: Model overview illustrating difference embedding construction via Whisper encoder and MLP head for scene-wise prediction.

Training is conducted at the scene level, aggregating ratings across six talker configurations within each acoustic background to improve sample stability. The MLP head is trained with weighted MSE loss, where targets are weighted by root rating count, using AdamW for optimization. To counter initialization sensitivity, ensemble averaging over five random seeds is performed for each head.

Experimental Evaluation and Comparative Analysis

The model’s predictions on devices held out from training exhibit high correlation with aggregated listener ratings, achieving $r=0.92$ overall ($0.89$ in loud scenes, $0.79$ in quiet scenes—device-level $r=0.91$ and $0.85$, respectively). These values reach the empirical split-half reliability ceiling of the mean listener responses in loud scenes, suggesting minimal prediction noise relative to inter-rater consensus.

Figure 2: Comparison of HASPIv2 and learned metric correlation with listener ratings across scene types (loud and quiet).

By contrast, HASPIv2, a prior reference-based intelligibility metric, correlates less strongly to listener-rated ease ($r=0.83$ overall, $0.75$ loud, $0.58$ quiet). The learned metric, grounded in real consumer device recordings and subjective judgments, more accurately models perceived benefit across commercial product variations and acoustic environments.

Sensitivity and Interpretability

To assess sensitivity, the model’s outputs were evaluated under controlled manipulations: degree of gain undershoot relative to NAL-NL2 and SNR augmentation via noise attenuation. Results show that gain undershoot consistently degrades predicted ease across all scene types, while SNR improvements yield greater benefits in loud scenes—consistent with expected audibility and noise-limitation interactions for hearing-impaired listeners.

Figure 3: Model responses to gain undershoot and SNR enhancement manipulations, demonstrating heightened sensitivity in loud acoustic scenes.

Implications, Limitations, and Future Directions

The approach addresses the gap between objective intelligibility prediction and subjective ease of understanding judgments, validating that foundation models paired with expert aggregation and task-specific adaptation provide robust consumer-centric metrics. The breadth of devices evaluated, spanning prescription to OTC, ensures broad relevance for clinical, consumer, and commercial hearing technology stakeholders.

Practically, this enables rapid and scalable prediction of user experience for new devices under realistic conditions, informing device tuning, comparative studies, and consumer advisory. Theoretically, it strengthens evidence that task adaptation of large speech models enhances their utility for subjective perceptual prediction in normative and non-normative populations.

Limitations include focus on the N3 audiogram, lack of device setting granularity, uncontrolled listener calibration, and limited environmental diversity. Future work should generalize across audiograms, refine calibration protocols, and expand the acoustic scene matrix to guarantee ecological validity. Integration of richer metadata and multi-modal listener profiles may further heighten predictive accuracy and clinical applicability.

Conclusion

The reported database and model provide a rigorous, scalable metric for predicting listener-rated ease of speech understanding in commercial hearing aids, outperforming established intelligibility baselines and approaching human reliability limits in noisy environments. The methodology demonstrates the efficacy of foundation model adaptation for subjective perceptual tasks, holds promise for broader consumer device evaluation, and defines clear priorities for future expansion and validation.