GTCRN: Lightweight Speech Enhancement

• GTCRN is a lightweight neural model that integrates grouped temporal convolutions with dual-path recurrent networks to robustly enhance speech in low SNR conditions.
• It reduces frequency-domain redundancy using an ERB filter bank and stacked subband feature extraction, optimizing performance for resource-constrained devices.
• Its dual-channel extension leverages IVA-based auxiliary estimation to further boost speech extraction accuracy in challenging acoustic environments.

The Grouped Temporal Convolutional Recurrent Network (GTCRN) is a lightweight neural architecture designed for efficient speech enhancement, especially under low signal-to-noise ratio (SNR) conditions. It integrates frequency-domain redundancy reduction, grouped temporal convolutions, and a grouped dual-path recurrent module, achieving robust speech extraction while maintaining low parameter and computational cost. The dual-channel variant enhances performance by leveraging independent vector analysis (IVA) as a coarse estimator, injecting auxiliary information from IVA outputs directly into the network. This system has demonstrated efficacy for dual-microphone setups deployed on resource-constrained devices (Wang et al., 26 May 2025).

1. Input Representation and Feature Extraction

GTCRN processes complex-valued short-time Fourier transforms (STFTs) of noisy signals, denoted $Y\in\mathbb{C}^{C\times T\times F}$ (where $C$ is the number of channels, $T$ frames, $F$ frequency bins). Initially, GTCRN applies an Equivalent Rectangular Bandwidth (ERB) filter bank as a "band merging" (BM) step, mapping the $F$ frequency bins into $B$ bands. Typically, low-frequency bands are left unmerged (e.g., 65), with higher frequencies merged into fewer bands (e.g., 64 ERB bands). This step yields $Y_B\in\mathbb{C}^{C\times T\times B}$.

A subsequent Subband Feature Extraction (SFE) module applies a $1\times3$ convolution to each band, aggregating information from neighboring bands. The output is reshaped, stacking all $B$ bands (and $K$ generated features per band) along the channel dimension, resulting in a tensor $C$0 where $C$1.

2. GTCRN Core Architecture

The GTCRN encoder consists of two cascaded blocks, each comprising:

• A 2D point-wise convolution (P-Conv2D) followed by batch normalization and PReLU activation.
• A second 2D convolution with kernel size $C$2 and stride $C$3, again followed by normalization and activation.
• Three Grouped Temporal Convolution (GT-Conv) sub-blocks per encoder block.

GT-Conv Block:

1. The input channels are split into two equal groups.
2. Branch A passes through identically.
3. Branch B processes the input via:
   • Two serial $C$4 point-wise convolutions, each followed by batch normalization and PReLU.
   • A 2D depth-wise dilated convolution (kernel $C$5, dilation $C$6).
4. Branch outputs are concatenated and channels are shuffled to enable inter-branch information exchange.
5. The decoder mirrors the encoder, using transpose convolutions and a final $C$7 activation to constrain the output to $C$8.

After encoding, features are processed by a Grouped Dual-Path RNN (G-DPRNN):

• The encoded tensor is rearranged such that the frequency axis becomes “segments” and time becomes “frames.”
• $C$9 channels are split into $T$0 disjoint groups.
• Within each group $T$1:
  • Intra-frame (frequency-wise): $T$2 for $T$3.
  • Inter-frame (time-wise): $T$4 for $T$5.
• Outputs from all groups are concatenated and projected back to $T$6 channels.

3. Dual-Channel Extensions and IVA Integration

The dual-channel GTCRN (DC-GTCRN) is designed to exploit two microphone channels and IVA-derived auxiliary features for improved speech enhancement under challenging acoustic conditions.

Input Feature Stacking:

• The system receives $T$7, representing stereo or dual-microphone recordings.
• An auxiliary IVA module (Aux-IVA) produces two separated spectrograms $T$8 (speech) and $T$9 (noise), each in $F$0.
• IVA features considered include the real and imaginary parts of these spectrograms, as well as the log-power spectrum ($F$1).
• The real and imaginary parts of $F$2 are concatenated with chosen IVA features along the channel axis to form the input tensor $F$3.

Dual-Encoder Option:

• Optionally, a dual-encoder architecture processes original $F$4 features and auxiliary IVA features in parallel. Their encoded outputs are concatenated before input to the G-DPRNN module.

Masking Strategies:

• The decoder outputs two real-valued masks $F$5, $F$6.
• These masks can be applied in two ways:
  • Masking 1: $F$7.
  • Masking 2: $F$8.
• Empirically, Masking 2, which multiplies the mask with the original noisy STFT, yields superior performance in low-SNR conditions.

4. Mathematical Formulation and Training Criteria

Given $F$9, its real and imaginary parts are stacked as $F$0. If the LPS of the IVA speech output is used, $F$1. Feature stacking results in $F$2.

Following BM and SFE, the tensor has shape $F$3 where $F$4.

The training loss is a weighted hybrid of time-domain and spectrogram-domain objectives:

$F$5

where $F$6, $F$7, and the loss terms are explicitly defined in (Wang et al., 26 May 2025). The total loss incorporates scale-invariant signal-to-noise ratio (SISNR), compressed magnitude loss, and normalized real/imaginary loss.

5. Computational Complexity and Parameterization

GTCRN emphasizes efficiency through architectural grouping:

• 2D convolutional layers: Standard cost is $F$8 multiply-accumulate operations (MACs). With $F$9 groups, cost is reduced by $B$0.
• GRU layers: For input dimension $B$1 and hidden size $B$2, one GRU cell requires about $B$3 operations per time step. Grouping into $B$4 parallel GRUs reduces per-group input and hidden size to $B$5, $B$6.

Measured parameter counts and complexity:

Variant	Parameters	Complexity (MMAC/s)
Single-channel GTCRN	≈23.4 k	≈32
Dual-channel GTCRN	≈23.9 k	≈35.6
Hybrid (with IVA)	24.39 k	≈43.2

The Aux-IVA module adds negligible parameters and just ≈0.2 MMAC/s per iteration (Wang et al., 26 May 2025).

6. IVA Auxiliary Estimation and Feature Fusion

Aux-IVA employs a classic independent vector analysis approach to obtain demixing vectors $B$7, estimating both speech and noise spectrograms $B$8. Update rules involve:

1. Computing $B$9.
2. Calculating $Y_B\in\mathbb{C}^{C\times T\times B}$0.
3. Updating $Y_B\in\mathbb{C}^{C\times T\times B}$1 by inverting $Y_B\in\mathbb{C}^{C\times T\times B}$2 and normalizing.
4. Generating $Y_B\in\mathbb{C}^{C\times T\times B}$3.

Feature fusion is conducted by concatenating either $Y_B\in\mathbb{C}^{C\times T\times B}$4 and optionally $Y_B\in\mathbb{C}^{C\times T\times B}$5, or the log-power spectrum $Y_B\in\mathbb{C}^{C\times T\times B}$6 with the original $Y_B\in\mathbb{C}^{C\times T\times B}$7 across channels before feeding to the encoding stage.

7. Practical Configuration and Training Protocols

GTCRN and its variants are trained with the following hyperparameters:

• STFT: 512-point Hanning window, 256-sample hop.
• Band merging: 65 low bands + 64 ERB bands.
• SFE: Convolution kernel $Y_B\in\mathbb{C}^{C\times T\times B}$8.
• Encoder block: Output channels $Y_B\in\mathbb{C}^{C\times T\times B}$9 (single encoder) or $1\times3$0 (dual encoder), kernel $1\times3$1, stride $1\times3$2. Second layer uses group size $1\times3$3.
• GT-Conv: Channel count $1\times3$4, kernel $1\times3$5, dilations $1\times3$6.
• G-DPRNN: $1\times3$7 groups, intra-frame BiGRU, inter-frame UniGRU, hidden units adjusted to keep total parameters $1\times3$8 k.
• Optimizer: Adam, linear warm-up (25k steps), cosine annealing (up to 250k steps), learning rates from $1\times3$9 to $B$0. Batch size $B$1, 1250 steps/epoch.
• SNRs: Trained on $B$2 dB, tested at $B$3 dB, $B$4 dB, $B$5 dB (Wang et al., 26 May 2025).

8. Summary of Contributions and Significance

GTCRN delivers high-performance speech enhancement within strict computational budgets, achieving parameter counts in the tens of thousands and sustaining real-time efficiency. The introduction of dual-channel contextual features and IVA-based auxiliary information results in robust performance even in extremely low-SNR conditions. Empirical results demonstrate that mask application to the original noisy STFT (Masking 2) provides superior enhancement in adverse environments. The hybrid system's lightweight design, modular architecture, and effectiveness for dual-microphone arrays position GTCRN as an efficient solution for embedded and resource-limited audio applications (Wang et al., 26 May 2025).