Near-End Listening Enhancement (NLE)

• NLE is a speech signal processing strategy that improves intelligibility by adaptively applying minimum gain to subbands based on noise and speech statistics.
• It minimizes distortion by only boosting the speech signal when the subband SNR falls below a set intelligibility threshold, preserving naturalness in favorable conditions.
• Empirical results show that NLE outperforms traditional methods by striking a better balance between objective measures and subjective listening quality in various noise scenarios.

Near-End Listening Enhancement (NLE) encompasses a class of speech signal processing techniques designed to improve the intelligibility and perceptual quality of speech for listeners exposed to additive noise at the listening point—such as via mobile phones or public announcement systems. NLE refers exclusively to signal enhancement on the “near-end” playback side (i.e., after source transmission) and is typically implemented as a real-time pre-processing module operating under knowledge of the noise statistics measured at the listener’s environment (Fuglsig et al., 2022).

1. Theoretical Foundations and Problem Formulation

Minimum-processing NLE systems are formulated in the short-time Fourier transform (STFT) domain, partitioning the frequency axis into perceptually motivated subbands. Given clean speech spectral coefficients $S_k$, additive noise coefficients $N_k$, and an empirically estimated per-bin signal variance $\sigma_{S_k}^2$, the played-out signal is represented as

$Z_k = v_k S_k + N_k$

where the gain coefficients $v_k \geq 0$ are applied per frequency bin, constrained to be real and non-negative.

Two primary quantities are defined within each subband $j$:

• Processing penalty (or distortion):

$$\mathcal{D}_j = \sum_{k \in \mathbb{B}_j} \omega_{j,k}(1-v_k)^2 \sigma_{S_k}^2$$

where $\omega_{j,k}$ is a normalized subband weight.

• Intelligibility constraint: via the approximated Speech Intelligibility Index (ASII)

$$\xi_j = \frac{\sum_{k \in \mathbb{B}_j} \omega_{j,k} v_k^2 \sigma_{S_k}^2}{\sigma_{N_j}^2}$$

with noise variance $$\sigma_{N_j}^2 = \sum_{k \in \mathbb{B}_j} \omega_{j,k} \sigma_{N_k}^2$$.

The goal is to minimize distortion subject to a lower-bounded intelligibility constraint: $N_k$0 where the SNR floor is set by $N_k$1 and $N_k$2 is chosen per subband to enforce a global ASII floor $N_k$3.

2. Closed-Form Solution and Gain Adaptation

Solving this convex constraint optimization yields a gain rule with two regimes: $N_k$4

• If the clean subband SNR exceeds the floor, unity gain is applied: the input is passed through without modification.
• If the SNR is below the desired floor, the minimum gain that brings $N_k$5 exactly to $N_k$6 is used, limiting unnecessary amplification and distortion.

3. Adaptive and Noise-Condition-Dependent Processing

This minimum-processing NLE scheme is inherently adaptive:

• In subbands with severe noise ($N_k$7), the speech is amplified just enough to meet intelligibility requirements, avoiding under-enhancement.
• In mild-noise or clean conditions ($N_k$8), no gain is applied, yielding zero added distortion and preserving naturalness.
• As overall SNR improves, the fraction of subbands requiring processing declines, with the system defaulting to a pass-through, ensuring minimal perceptual artifact.

4. Empirical Performance: Objective and Subjective Metrics

The minimum-processing NLE has been benchmarked against established baselines—Sauert10, Taal13, and Niermann21—using both objective and subjective evaluation (Fuglsig et al., 2022).

• Objective intelligibility (ASII, ESTOI): At low SNRs (e.g., –10 dB, 0 dB), minimum-processing NLE matches or slightly exceeds prior methods (e.g., in car noise at 0 dB SNR, ESTOI ≈ 0.60 vs. NoiseProp ≈ 0.55).
• Objective speech quality (SegSNR, PESQ): At high SNRs (≥5 dB), minimum-processing NLE tracks the unprocessed upper bound (e.g., SegSNR ≈ 15 dB, PESQ ≈ 3.5), whereas reference NLEs often incur substantial quality loss (up to a 0.8 PESQ point drop).
• Subjective listening (MUSHRA): In a 21-listener test, median MUSHRA scores in 0 dB car noise were ≈65 for minimum-processing NLE, substantially above NoiseProp (≈45) and unprocessed (≈40). In less adverse conditions (10 dB car noise), the margin widens further (80 for NLE vs. 42 for NoiseProp).

5. Comparison to Traditional, Maximum-Intelligibility NLE Methods

Traditional optimization-based NLEs employ a max-SI (maximize speech intelligibility) strategy, pushing subband gains as high as possible, often irrespective of necessity in benign noise conditions. This results in continuous processing, hence increased speech distortion and quality degradation, even when perceptual benefit saturates. Minimum-processing NLE applies only the gain required to meet the user-specified intelligibility floor $N_k$9, automatically switching off in favorable conditions and conserving perceptual quality. This approach achieves a more favorable tradeoff between intelligibility and naturalness, offering superior user experience in mixed-noise scenarios.

6. Practical Implications and Extensions

The minimum-processing NLE framework directly addresses the overprocessing problem inherent to prior NLE solutions, providing mathematical guarantees:

• The solution is globally optimal (per subband) under MSE/perceptual distortion and ASII/ESTOI constraints.
• Processing is intrinsically bounded: no unnecessary modification is performed if the intelligibility target is naturally met.
• The closed-form gain rule is amenable to low-power, real-time DSP implementation with low computational overhead.

A plausible implication is that such minimum-distortion enhancement strategies could be generalized to other audio enhancement domains whenever perceptual floor constraints (intelligibility, audibility, detection probability) must be met with the least processing intrusion possible.

---

References: All technical details, empirical results, and comparisons are sourced from "Minimum Processing Near-end Listening Enhancement" (Fuglsig et al., 2022).