ERB-Domain Envelope Stage

• ERB-Domain Envelope Stage is a signal processing module that applies smooth, band-limited spectral gain modifications based on the human auditory system.
• It features a compact U-Net style architecture with separable convolutions and grouped GRUs to efficiently predict ERB-band gains.
• Integrated into deep filtering pipelines, it optimizes compressed spectral loss for low-complexity, real-time speech enhancement.

The ERB-domain envelope stage is a specialized signal processing module central to perceptually motivated speech enhancement systems such as DeepFilterNet. Its purpose is to apply smooth, band-limited spectral gain modifications to noisy audio, operating in a frequency decomposition aligned with the human auditory system’s critical bands. Specifically, it predicts and applies gain values on ERB (Equivalent Rectangular Bandwidth)–scaled frequency bands, facilitating effective envelope shaping with a compact, low-complexity neural architecture (Schröter et al., 2021).

1. ERB Frequency Decomposition and Feature Extraction

The ERB scale provides a perceptually meaningful linearization of the frequency axis, frequently used for auditory modeling. The ERB rate as a function of frequency $f$ (Hz) is defined as:

$E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$

The instantaneous ERB bandwidth is

$$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$

To construct the filter bank, the frequency interval $[0, f_s/2]$ is partitioned into $N_\mathrm{ERB}$ contiguous intervals, each corresponding to an ERB band. Each band $b$ has a center frequency $f_b$, calculated by inverting $E(f)$ at uniformly spaced ERB-rate positions. For each band, all discrete STFT bins $f_i$ satisfying

$$f_b - \frac{1}{2}\mathrm{ERB}(f_b) \leq f_i < f_b + \frac{1}{2}\mathrm{ERB}(f_b)$$

are assigned to band $E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$0, establishing a rectangular filter bank $E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$1.

Given a time-frequency representation $E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$2 (where $E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$3 indexes the time frame), ERB-band log-power features are computed as:

$E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$4

A small constant $E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$5 ensures numerical stability. Each ERB band undergoes per-band normalization via an exponential moving average over $E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$61 s timescales before neural processing.

2. Envelope Subnetwork: Architecture and Gain Prediction

The ERB-domain envelope stage employs a neural network module with a compact U-Net–style architecture, with the following components:

• Encoder: Separable 2D convolutions (kernel $E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$7 in time × ERB), $E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$8 channels, batch normalization, ReLU activations, and $E(f) = 21.4 \log_{10}\big(1 + 0.00437\,f\big)$9 add-skip convolutions.
• Latent Pathway: Grouped linear layers and grouped GRUs (input split into $$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$0 groups, each processed by a GRU of size 64, then shuffled and recombined).
• Decoder: Expands the representation back to $$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$1 outputs.

The final output is a real-valued gain vector:

$$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$2

where values are clipped to $$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$3 via nonlinearity for stability and interpretability.

3. Gain Application: Reconstruction of Full-Band Gain

The predicted ERB-band gains are distributed back to STFT bins through an inverse filter-bank (transpose of $$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$4):

$$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$5

The gain-enhanced spectrogram is generated by element-wise multiplication:

$$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$6

A summary of the mapping flow is shown below.

Stage	Input Dimension	Output Dimension
ERB feature extraction	$$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$7 (noisy STFT)	$$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$8 (ERB log-power, $$\mathrm{ERB}(f) = 24.7\big(4.37 \times 10^{-3} f + 1\big)$$9)
Envelope subnetwork	$[0, f_s/2]$0	$[0, f_s/2]$1 (gains)
Gain application	$[0, f_s/2]$2 + $[0, f_s/2]$3	$[0, f_s/2]$4 (enhanced STFT)

All steps are designed for low computational overhead, allowing real-time execution.

4. Optimization and Training Objective

The ERB-domain envelope stage is not trained in isolation but receives gradients from a unified loss that integrates both the envelope and subsequent deep-filter stages. The principal objective is the compressed spectral loss:

$[0, f_s/2]$5

where $[0, f_s/2]$6 compresses magnitudes to match perceptual loudness scaling. Here, $[0, f_s/2]$7 denotes the system output, $[0, f_s/2]$8 the clean reference, and $[0, f_s/2]$9 the phase.

When trained in attenuation-limited mode, $N_\mathrm{ERB}$0 is clamped so the minimum SNR is preserved, avoiding over-suppression.

5. Algorithmic Workflow and Implementation Details

The algorithmic sequence for the ERB-domain envelope stage consists of:

1. Acquisition of a block of waveform audio $N_\mathrm{ERB}$1.
2. STFT computation: $N_\mathrm{ERB}$2, with configurable $N_\mathrm{ERB}$3 and 50% overlap.
3. ERB filter bank application and log-power feature computation: $N_\mathrm{ERB}$4.
4. Normalization of $N_\mathrm{ERB}$5 via exponential moving average.
5. Envelope network inference to predict $N_\mathrm{ERB}$6.
6. Reconstruction of per-bin gains $N_\mathrm{ERB}$7 via the inverse filter bank.
7. Application of $N_\mathrm{ERB}$8 to $N_\mathrm{ERB}$9 for the enhanced envelope STFT $b$0.
8. Output $b$1 passed to the subsequent deep-filter stage for periodic/harmonic enhancement.

Key implementation specifications:

• $b$2.
• STFT window lengths $b$3 samples (5–30 ms at 48 kHz).
• Model size of the envelope stage: $b$40.88 M parameters (full network: $b$51.78 M).
• Multiply–accumulate count: $b$6250 M MAC/s for envelope stage; $b$70.35 G MAC/s total.
• All convolutions in the envelope stage are causal post-initial look-ahead of $b$8 frames (adds latency~2x frame-shift).
• Efficiency is further improved by fusing batch norm with convolutions, using separable convolutions, and leveraging grouped linear and GRU layers; no exotic operations are employed.

6. Significance and Limitations

The ERB-domain envelope stage is a critical component for imposing perceptually meaningful spectral shaping while maintaining a low parameter count and low computational burden. By restricting gain prediction to ERB bands, the system achieves robust smoothing and reduction of artifacts, particularly under resource-limited constraints or real-time needs. There is no separate loss on the envelope stage; all parameter updates serve the joint objective of maximizing spectral fidelity post all stages (Schröter et al., 2021).

A plausible implication is that this envelope stage’s operation aligns enhancement with human auditory perception, especially for background noise reduction and intelligibility, but fine periodic structure must be restored by following harmonic enhancement modules. The envelope stage’s flexibility (by adjusting $b$9, window sizes, and normalization) also offers adaptability across deployment scenarios.

7. Related Methodological Context

The ERB-domain envelope stage forms only one part of DeepFilterNet’s two-stage architecture; subsequent deep filtering modules operate on the periodic elements preserved and clarified by the envelope enhancement. The use of ERB-domain gains is a direct response to the limitations of conventional full-band or sub-band masking, providing improved perceptual quality and efficiency. No separate stage-loss or hand-tuned SNR heuristics are required, as all learning is integrated under the compressed spectral loss. This approach exemplifies the trend in deep speech enhancement of tightly coupling perceptual modeling and computational efficiency (Schröter et al., 2021).