--- title: Dilated FAVOR Conformer Architecture url: https://www.emergentmind.com/topics/dilated-favor-conformer type: topic --- # Dilated FAVOR Conformer Architecture The Dilated FAVOR Conformer (DF-Conformer) is an efficient neural architecture for speech enhancement that unifies Conv-TasNet’s filterbank and the Conformer’s sophisticated masking module. DF-Conformer attains linear computational complexity via two central mechanisms: (i) FAVOR+ linear-attention in place of standard multi-head self-attention, and (ii) 1D depthwise-separable dilated convolution. This enables joint local and global context modeling in end-to-end denoising pipelines and allows practical scaling to long input sequences. Subsequent advancements employ bidirectional structured state-space models, notably the Hydra mixer, in place of FAVOR+ to eliminate random-feature approximations while maintaining linear complexity and further improving global sequence modeling [2106.15813], [2511.02454]. ## 1. Architectural Overview DF-Conformer integrates the Conv-TasNet filterbank encoder–decoder with a Conformer-inspired, block-stacked mask prediction network. The dataflow is as follows: - **Encoder**: A 1D convolutional analysis filterbank ($\mathrm{Enc}(x)$) extracts non-negative features $X\in\mathbb{R}^{N\times D_e}$ from the input waveform $x(t)$. Window length is 2.5 ms (40 samples), stride is 1.25 ms (20 samples), and number of filters $D_e=256$. - **Mask Prediction Network**: The feature map $X$ is projected to $D_b$ channels and passed through $L$ DF-Conformer blocks. Each block uses exponentially increasing dilation $d_i=2^{(i-1~\mathrm{mod}~L_s)}$ for local context, and applies FAVOR+ self-attention for global context. The output mask $M\in[0,1]^{N\times D_e}$ is obtained via a final dense layer and sigmoid activation. - **Decoder**: The mask is applied elementwise to the encoded features, and a transposed convolutional synthesis filterbank reconstructs the enhanced waveform $\hat{y}(t)$. Each DF-Conformer block interleaves macaron-style feedforward layers, FAVOR+ multi-head self-attention, and 1D depthwise dilated convolution. The block-wise design allows the receptive field to grow exponentially, periodically resetting, and covers long-range dependencies [2106.15813]. ## 2. FAVOR+ Linear-Complexity Attention Traditional multi-head self-attention in the Conformer and Transformer incurs $O(N^2)$ time and memory. FAVOR+ addresses this bottleneck by replacing the softmax kernel in scaled dot-product attention with a positive random-feature map $\phi:\mathbb{R}^{d_k}\to\mathbb{R}^{D_r}_+$. This map satisfies \[ \mathrm{softmax}(q^\mathsf{T}k)\approx \phi(q)^\mathsf{T}\phi(k) \] where each $\phi(x)_i=\exp(\omega_i^\mathsf{T}x - \frac12\|x\|^2)$ for orthogonal Gaussians $\omega_i$. The FAVOR+ approximation to attention is \[ \mathrm{Attention}(Q, K, V) \approx D^{-1}\phi(Q)[\phi(K)^\mathsf{T}V] \] with a diagonal normalization $D$. The computational cost is reduced to $O(ND_rH)$ for $N$ sequence length, $D_r$ random features, and $H$ heads, ensuring linear time and memory. The DF-Conformer utilizes $D_r=384$ and six attention heads, with random features fixed at initialization [2511.02454], [2106.15813]. ## 3. Dilated Depthwise Separable Convolution Local context aggregation is performed by depthwise separable 1D convolutions with exponentially increasing dilation. Within a block, hidden state $Z$ is batch-normalized and projected through a Gated Linear Unit (GLU) and depthwise convolution: \[ r(t) = \sum_{\tau=-(k-1)/2}^{(k-1)/2} w_\tau\,\mathrm{GLU}(Z)_{t-d_i\tau} \] Kernel size is 3 (default) or 5 (as in TDCN++). Dilation doubles every four blocks ($L_s=4$), so the receptive field grows exponentially: \[ R_B = 1 + (k-1)\sum_{b=1}^B 2^{\lfloor (b-1)/4\rfloor} \] This design achieves joint fine-grained and long-range sequence modeling with low overhead. Residual connections and interleaving with FAVOR+ attention complete each convolutional module [2106.15813], [2511.02454]. ## 4. Hydra State-Space Model Integration A recent refinement replaces FAVOR+ attention in the DF-Conformer with Hydra, a bidirectional, selective state-space sequence model (SSM) structured as a matrix mixer. While FAVOR+ yields a low-rank attention matrix via random features, Hydra constructs a bidirectional quasi-separable matrix: \[ \breve m_{ij} = \begin{cases} \overrightarrow{c}_{i-1}^\mathsf{T}(\prod_{k=j+1}^{i-1}\overrightarrow{A}_k)\overrightarrow{b}_j, & i>j \\ \delta_i, & i=j \\ \overleftarrow{c}_{i+1}^\mathsf{T}(\prod_{k=i+1}^{j-1}\overleftarrow{A}_k)\overleftarrow{b}_j, & i