YAMNet+ Hearing-Aid Scene Recognition

• YAMNet+ is a lightweight sound recognition model adapted from pretrained YAMNet, designed for classifying 14 audiologically relevant scenes using the AHEAD-DS dataset.
• It employs full end-to-end fine-tuning with transfer learning, achieving robust performance (mAP=0.83, accuracy=0.93) while ensuring efficient deployment on edge devices.
• The architecture retains the MobileNet-style backbone of YAMNet but replaces the 521-class output with a specialized 14-class layer to support hearing-aid applications.

Searching arXiv for the named model and closely related YAMNet-based work to ground the article in the current literature. YAMNet+ is a lightweight sound recognition model for audiologically relevant environment classification, introduced together with the AHEAD-DS dataset for hearing-assistive applications. In its strictest usage, the term denotes the transfer-learned YAMNet-based baseline proposed for 14 hearing-aid-relevant scene classes, with explicit support for edge deployment on smartphones. More broadly, the literature also uses “YAMNet+” informally for systems that retain pretrained YAMNet as an audio backbone and add task-specific heads, decision logic, or deployment wrappers; however, several such papers explicitly state that they do not introduce a model formally named YAMNet+ (Zhong et al., 14 Aug 2025).

1. Naming, scope, and conceptual boundaries

The named model “YAMNet+” appears in "A dataset and model for recognition of audiologically relevant environments for hearing aids: AHEAD-DS and YAMNet+" (Zhong et al., 14 Aug 2025). In that work, YAMNet+ is presented as a small, open-source baseline for sound-based scene recognition in hearing-assistive settings, with AHEAD-DS serving as the associated public benchmark. The model is designed for deployment on edge devices such as smartphones connected to hearing devices, including hearing aids and wireless earphones with hearing-aid functionality (Zhong et al., 14 Aug 2025).

A recurring point in adjacent work is that many YAMNet-derived systems are not new YAMNet backbones. Rather, they are transfer-learning adaptations built around pretrained YAMNet features. This distinction matters because “YAMNet+” can otherwise be misread as a fundamentally redesigned architecture. Several papers explicitly say that they do not introduce a model called YAMNet+ even when they add a new classifier head, a deployment framework, or application-level logic on top of YAMNet (Bhat et al., 2020, Ding et al., 8 Sep 2025, Paruchuri et al., 3 Aug 2025). In that narrower architectural sense, the named YAMNet+ of AHEAD-DS is best understood as a task-retargeted YAMNet workflow rather than a replacement for the original MobileNet-style backbone (Zhong et al., 14 Aug 2025).

A related misconception is that YAMNet+ denotes a single canonical extension across the literature. The available work instead suggests two usages. One is formal and specific to the AHEAD-DS hearing-aid scene-recognition model (Zhong et al., 14 Aug 2025). The other is descriptive: YAMNet plus a task-specific head, temporal model, or deployment wrapper in a downstream system (Bhat et al., 2020, Altwlkany et al., 2024).

2. AHEAD-DS and the hearing-aid scene-recognition task

AHEAD-DS was created to provide a standardized, publicly available dataset with consistent labels relevant to hearing aids, because prior datasets were described as proprietary, incomplete, inconsistently labeled, or not released in a reproducible ready-to-use form (Zhong et al., 14 Aug 2025). The dataset is built mainly from HEAR-DS and CHiME 6 Dev. HEAR-DS supplies environmental sounds and music, while CHiME 6 Dev supplies speech source material; 44 of 52 CHiME 6 Dev files were used, yielding 4840 speech clips after segmentation (Zhong et al., 14 Aug 2025).

Each AHEAD-DS recording is 10 s, single-channel, 16-bit integer, and 16 kHz. The full dataset contains 9968 recordings, split into 6980 training, 1001 validation, and 1987 testing examples, corresponding approximately to a 70% / 10% / 20% split performed per label after mixing and standardization (Zhong et al., 14 Aug 2025).

The target label space contains 14 audiologically relevant classes:

• Pure environments: cocktail_party, interfering_speakers, in_traffic, in_vehicle, music, quiet_indoors, reverberant_environment, wind_turbulence
• Speech-in-environment classes: speech_in_traffic, speech_in_vehicle, speech_in_music, speech_in_quiet_indoors, speech_in_reverberant_env, speech_in_wind_turbulence

The paper motivates this label design by emphasizing that speech embedded in particular environments is more useful for hearing-aid behavior than detecting the environment alone; for example, detecting speech_in_traffic is more informative than detecting traffic without speech (Zhong et al., 14 Aug 2025).

AHEAD-DS is imbalanced. Reported class counts include 1496 for music, 1496 for speech_in_music, 1334 for cocktail_party, 1047 for interfering_speakers, 222 for reverberant_environment, and 221 for speech_in_reverberant_env. This imbalance is one reason the model is trained with focal loss (Zhong et al., 14 Aug 2025).

The mixing protocol is also explicit. Half of the environmental clips were mixed with speech and half remained unmixed. Of the 4840 speech clips, 3793 were mixed with environmental sounds and 1047 were assigned to interfering_speakers without further mixing. Speech/environment mixtures were created at signal-to-noise ratios of $\{-10,-5,0,5,10\}\,\mathrm{dB}$ after level standardization and RMS equalization (Zhong et al., 14 Aug 2025).

3. Architecture and transfer-learning formulation

YAMNet+ takes raw waveform audio as input. The waveform is transformed into a log mel spectrogram, and the model uses the same basic YAMNet / MobileNet-style convolutional backbone and the same log-mel-spectrogram frontend as original YAMNet (Zhong et al., 14 Aug 2025). Input clips are single-channel, sampled at 16 kHz, and stored as 16-bit integers in WAV format in the dataset (Zhong et al., 14 Aug 2025).

The model inherits YAMNet’s fixed windowing configuration: waveform audio is split into 960 ms windows with 480 ms overlap, with zero-padding applied if needed. For a 10 s clip, this yields 20 windows. Sound recognition is performed on every 960 ms window, and scores are not combined for a waveform divided across multiple windows; each window is treated separately. Reported accuracy is therefore window-level rather than clip-level (Zhong et al., 14 Aug 2025).

The backbone is described as a convolutional block followed by 13 blocks, each containing depthwise separable convolution, batch normalization, ReLU, convolution, batch normalization, and ReLU, and then a global average pooling penultimate layer (Zhong et al., 14 Aug 2025). Relative to original YAMNet, the principal architectural change is task adaptation at the output layer: the original 521-class AudioSet task is replaced with a 14-class audiologically relevant scene-recognition task (Zhong et al., 14 Aug 2025).

The final layer is a fully connected layer with sigmoid activation and output length equal to the number of class labels. The paper reports 3,751,369 parameters when initialized with the original 521-class output and 3,743,831 parameters when initialized for 14 classes (Zhong et al., 14 Aug 2025).

Transfer learning is central to the model definition. YAMNet+ is initialized from YAMNet weights pretrained on AudioSet. Two regimes are compared: freezing all layers except the last layer, and making all layers trainable. The best overall configuration is the fully trainable one, so the final model is not merely a frozen feature extractor with a shallow head; it is fine-tuned end-to-end from AudioSet initialization (Zhong et al., 14 Aug 2025).

One architectural constraint follows from that choice: the paper states that the 960 ms window is effectively fixed by transfer learning, because changing it would break compatibility with the pretrained YAMNet setup. This suggests that the “plus” in YAMNet+ lies primarily in retargeting, training, and deployment rather than in altering the frontend geometry (Zhong et al., 14 Aug 2025).

4. Optimization, augmentation, and empirical behavior

The default training configuration uses Adam with learning rate $10^{-5}$, a maximum of 100 epochs, and a decay-on-plateau schedule: after 3 epochs without validation improvement, the learning rate is decayed by a factor of 0.5; after 6 epochs without improvement, early stopping is applied. Training data are shuffled each epoch. The loss is focal loss with $\alpha = 0.25$ and $\gamma = 2$, together with label smoothing at factor $0.1$ (Zhong et al., 14 Aug 2025).

Data augmentation is applied on the fly during training only. Each augmentation is applied independently with probability 0.5. The reported augmentations are gain in the range $-6\,\mathrm{dB}$ to $+6\,\mathrm{dB}$, additive noise sampled from $\mathcal{U}(-0.003,0.003)$, and time stretch or shrink in the range 0.9 to 1.1 (Zhong et al., 14 Aug 2025).

The test-set headline performance on AHEAD-DS is $\mathrm{mAP}=0.83$ and accuracy $=0.93$ (Zhong et al., 14 Aug 2025). The default validation configuration, with AudioSet initialization, augmentation, label smoothing, and all layers trainable, reports validation loss 0.11, validation mAP 0.85, and validation accuracy 0.95 (Zhong et al., 14 Aug 2025).

Setting	Val. mAP	Val. accuracy
Default configuration	0.85	0.95
No label smoothing	0.82	0.95
No augmentation	0.84	0.94
Only last layer trainable	0.87	0.80
Random initialization	0.61	0.15

These ablations are methodologically important. Random initialization yields validation loss 0.53, validation mAP 0.61, and validation accuracy 0.15, while the paper states that both training and validation loss remained high at 0.53. The authors interpret this as underfitting rather than overfitting, and conclude that transfer learning from pretrained YAMNet is essential for this dataset size (Zhong et al., 14 Aug 2025).

The “only last layer trainable” regime produces validation mAP 0.87 but validation accuracy 0.80. The paper explicitly argues that this is not the better model overall, because mAP can remain high even with many false negatives when false positives are limited. On that reading, full fine-tuning offers the more balanced solution (Zhong et al., 14 Aug 2025).

The confusion analysis reported in the paper is acoustically interpretable. Errors are concentrated between speech_in_music and music, and between speech_in_traffic and in_traffic, particularly during pauses where only the background remains audible. Confusion also occurs between speech_in_vehicle and speech_in_wind_turbulence, because vehicle cabins may contain rushing-air sounds similar to wind turbulence (Zhong et al., 14 Aug 2025).

5. Edge deployment and runtime characteristics

YAMNet+ is explicitly framed as an edge-oriented baseline. The trained model is converted to TensorFlow Lite and to 16-bit floating point for smaller footprint, yielding a model size of 6.19 MiB (Zhong et al., 14 Aug 2025). Deployment is demonstrated on an Android smartphone, specifically a Google Pixel 3 running Android 13, with an application written in Flutter 3.24.5 and Dart 3.5.4 (Zhong et al., 14 Aug 2025).

Measured runtime on the Pixel 3 is 200.4 ms on average to process 5 s of audio. The paper summarizes the processing-time scaling as

$10^{-5}$0

where $10^{-5}$1 is seconds of audio and $10^{-5}$2 is processing time in milliseconds. This corresponds to roughly 48–50 ms startup latency to load the model and an approximate linear increase of about 30 ms per 1 s of audio (Zhong et al., 14 Aug 2025).

The authors present this as evidence of real-time sound-based scene recognition capability on modest mobile hardware. At the same time, the paper is careful to describe YAMNet+ as a baseline model rather than a finalized production system (Zhong et al., 14 Aug 2025).

The efficiency argument is comparative rather than absolute. The paper notes that PANN and AST have higher reported AudioSet mAP than YAMNet—0.44 for PANN, 0.46 for AST, and 0.31 for YAMNet/YAMNet+ from prior YAMNet reporting—but require more than 80M parameters, whereas YAMNet/YAMNet+ has approximately 3.7M parameters (Zhong et al., 14 Aug 2025). This suggests that YAMNet+ is positioned primarily around parameter efficiency, reproducibility, and deployability rather than maximal AudioSet-scale accuracy.

6. Broader literature and the extended meaning of “YAMNet+”

Outside AHEAD-DS, the term is often used informally for YAMNet-based transfer-learning systems. "Improving Pretrained YAMNet for Enhanced Speech Command Detection via Transfer Learning" adapts pretrained YAMNet to a 12-class speech command task by replacing the original 521-class output with a task-specific classifier; the reported result is accuracy 95.28%, precision 95.08%, recall 94.43%, F1 94.57%, and specificity 99.49% (Lachenani et al., 26 Apr 2025). "A Recurrent Neural Network Approach to the Answering Machine Detection Problem" uses YAMNet as a frozen pretrained frontend feeding a GRU-based classifier for streaming telecom inference, reporting 96.67% test accuracy without silence detection and 98.10% with silence detection (Altwlkany et al., 2024). "Context-Adaptive Hearing Aid Fitting Advisor through Multi-turn Multimodal LLM Conversation" uses 1024-dimensional YAMNet embeddings with temporal mean pooling and a lightweight feed-forward classifier to distinguish conversation, noise, and quiet, reporting 91.2% accuracy and macro F1 0.908 (Ding et al., 8 Sep 2025). "EgoTrigger" replaces YAMNet’s original classifier with a custom multilayer head for binary hand-object interaction detection and then converts those scores into camera-trigger intervals, achieving about 54% fewer frames on average while preserving comparable memory-QA performance relative to full capture (Paruchuri et al., 3 Aug 2025).

Other work extends the same pattern by embedding YAMNet into larger hybrid systems. "LEAN: Light and Efficient Audio Classification Network" uses pretrained YAMNet as one branch of a dual-stream model together with a raw-waveform temporal encoder and cross-attention-based temporal realignment; LEAN reaches mAP 0.4677 on FSD50K with a 4.58M-parameter model and a quantized TFLite size of 4.5 MB (Choudhary et al., 2023). Conversely, some comparative studies emphasize that vanilla YAMNet is a competent but non-dominant transfer baseline. In speech emotion recognition, the model is used as a frozen feature extractor producing frame-level embeddings that are average pooled into 1024-dimension clip-level embeddings, and it occupies a middle tier below speaker-recognition PTMs such as x-vector and ECAPA (Phukan et al., 2023). In low-data clinical audio, YAMNet is reported with participant-level AUC 0.69, sensitivity 0.71, and specificity 0.79, trailing stronger transfer strategies (Mahdi et al., 2024).

Taken together, these papers support a stable interpretation. In the narrow sense, YAMNet+ is the named AHEAD-DS hearing-aid scene-recognition baseline (Zhong et al., 14 Aug 2025). In the broader methodological sense, it denotes a family of systems in which pretrained YAMNet is retained as an efficient acoustic prior and extended with task-specific heads, temporal models, bias correction, or deployment logic. This suggests that the enduring significance of YAMNet+ is less a singular architecture than a recurring transfer-learning pattern: YAMNet as a compact, AudioSet-pretrained backbone, repurposed for downstream audio understanding under application-specific constraints (Zhong et al., 14 Aug 2025, Lachenani et al., 26 Apr 2025, Altwlkany et al., 2024).