RT-Tango: Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices

Abstract: Real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication, yet existing efficient solutions predominantly address single-channel settings. In this paper, we introduce RT-Tango, a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids. RT-Tango relies on a two-stage distributed architecture combining perceptually motivated ERB feature compression, lightweight grouped recurrent mask estimation, and temporal sparsification to reduce computational cost. Stringent latency constraints are addressed by decoupling spectral resolution from algorithmic delay using an asymmetric STFT, together with causal recurrent inference and online estimation of spatial statistics. Experimental results show that RT-Tango achieves competitive speech enhancement while significantly reducing MACs operations and functioning at ultra-low latencies as low as 8 ms.

• The paper introduces RT-Tango, a distributed streaming framework that achieves ultra-low latency (8 ms) binaural speech enhancement in hearing aids.
• It employs perceptually motivated feature compression and grouped recurrent neural networks to reduce computational complexity while preserving high speech quality.
• Through temporal sparsification and an asymmetric STFT, RT-Tango reduces computation by 6x compared to state-of-the-art methods, balancing efficiency and performance.

RT-Tango: Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aids

Introduction

Binaural speech enhancement on hearing aid platforms is characterized by severe latency, computational, and inter-device communication constraints. Most previous research in neural speech enhancement (SE) optimizes single-channel models for efficiency, with little attention to distributed, multi-microphone scenarios, especially under real-time and low-power requirements. The paper introduces RT-Tango, a distributed streaming framework for binaural SE specifically targeting the operational envelope of modern hearing aids. RT-Tango extends the two-stage distributed architecture of Tango [9466439], integrating perceptually motivated feature compression, grouped recurrent mask estimation, and temporal sparsification for high efficiency and ultra-low latency, and introduces a strictly causal variant RT-Tango-OS for 8 ms algorithmic latency.

System Architecture and Efficiency-Driven Design

RT-Tango retains the two-stage processing pipeline of Tango, where ear-nodes perform local DNN-based mask estimation and initial spatial filtering, exchange compressed representations, and then refine masks via a secondary DNN, finally applying an SDW-MWF to generate binaural output. To meet embedded system constraints, RT-Tango introduces a suite of complementary optimizations.

Figure 1: Block diagram of RT-Tango, highlighting new or optimized modules for efficiency and low-latency streaming.

Perceptual Feature Compression

The frontend employs ERB-scaled filterbanks to reduce input spectral dimensionality while preserving energy and cues critical for human speech intelligibility. This compresses upstream neural model inputs, reducing parameter count and MACs without sacrificing enhancement fidelity.

Grouped Recurrent Neural Network Mask Estimation

Both single-node (SN-DNN) and multi-node (MN-DNN) mask estimators are implemented as grouped recurrent neural networks (GRNNs), exploiting the locality of spectral dependencies in speech for further complexity reduction. Feature-space partitioning allows for parallel sub-network processing and reduces total layer complexity from $\mathcal{O}(H^2)$ to $\mathcal{O}(H^2/G)$ for hidden state size $H$ and $G$ groups, justified by ablation studies showing negligible performance tradeoff in grouping the SN-DNN.

Temporal Sparsification

To minimize redundant computation due to high temporal input rates in streaming scenarios, RT-Tango leverages both fixed-rate skipping (FRS) and learned gating strategies (e.g., SkipRNN, TinyLSTM). FRS, operating at $1/4$ and $1/2$ update rates for SN-DNN and MN-DNN, yields stable enhancement performance (within 0.2 dB SI-SDR of baseline) at substantially reduced per-frame MACs.

Low-Latency Streaming via Asymmetric STFT

Frequency resolution and algorithmic latency are decoupled by employing an asymmetric STFT—long analysis window for resolution, short synthesis window for low reconstruction delay. An 8 ms synthesis window with asymmetric Hann is shown to balance performance and latency tradeoff optimally. Online spatial statistics (SCM) are recursively estimated with exponential moving average, ensuring fully causal operation in RT-Tango-OS.

Experimental Results

Experiments use simulated binaural datasets (LibriSpeech, BinauRec, PHL) with hearing aid microphone configurations, evaluating SI-SDR, SI-SIR, SI-SAR, PESQ, STOI, and computational efficiency in MACs/s. Benchmarks include Tango, Tango-RNN, and GTCRN [rong_gtcrn_2024], with all models assessed under both standard and streaming hop configurations.

Computational Efficiency

RT-Tango achieves 33.4 MMACs/s, a 6x reduction in computation relative to GTCRN at comparable frame rates (197.5 MMACs/s). Further, RT-Tango-OS, with full temporal sparsification, operates at 35.14 MMACs/s at 8 ms algorithmic latency.

Enhancement Quality

Despite dramatic reductions in DNN complexity, RT-Tango matches or exceeds baseline PESQ, STOI, and SI-SIR across ears. Notably:

• Left/Right ear SI-SIR: 20.8 / 24.6 dB (RT-Tango) vs. 20.8 / 24.1 dB (Tango).
• PESQ: 1.66 / 1.71 (RT-Tango) vs. 1.61 / 1.64 (Tango). Left-right performance balance is preserved, important for spatial perception in hearing aids.

Performance tradeoffs from streaming and causal SCM updates in RT-Tango-OS result in slight degradations in SI-SDR and SI-SAR, but PESQ and STOI remain competitive with GTCRN, validating the robustness of the hybrid filtering architecture in online conditions.

Ablation and Trade-off Analyses

Ablations demonstrate:

• Grouping SN-DNN to 8 groups (GRNN) maintains SI-SDR/SAR while reducing MACs/frame by 44%.
• Fixed-rate skipping is preferable to learned skip gating for reliability and computational predictability.
• Asymmetric STFT with an 8 ms synthesis window yields the best delay-quality compromise.

Theoretical and Practical Implications

The proposed hybrid system architecture shows that strict decoupling between neural mask estimation and spatial filtering offers superior robustness to model compression and quantization, addressing challenges unique to resource-constrained distributed hearing aids. Use of ERB compression and grouped RNNs points towards models that are both human-perception aligned and hardware-friendly.

From a practical standpoint, RT-Tango (and RT-Tango-OS) introduces a demonstrably deployable solution for next-generation hearing aids that must deliver SE in real-world noisy environments at strict latency and power budgets, an application for which few alternatives exist. The framework’s design principles can be extended to multi-microphone, distributed SE in augmented reality, teleconferencing, or low-power edge scenarios requiring causal, efficient audio processing.

Future Directions

Further work could address:

• Full-integer quantization and hardware co-design for sub-mW operation,
• Robustness to device communication loss and packet jitter,
• End-to-end trainability with differentiable STFT frontends and spatial filtering,
• Generalization to larger array topologies or complex listening environments.

Conclusion

RT-Tango and its streaming variant represent a substantive advance in distributed binaural speech enhancement under real-world device constraints. Through architectural compression, grouped modeling, and algorithmic latency optimization, RT-Tango achieves competitive enhancement quality at an order-of-magnitude lower computational load while supporting real-time, strictly causal streaming—a critical requirement for modern hearing aids and other perceptually sensitive edge applications.