---
title: 'SE-Res2Net Block: Multi-Scale Channel Attention'
url: https://www.emergentmind.com/topics/se-res2net-block
type: topic
---

# SE-Res2Net Block: Multi-Scale Channel Attention

The SE-Res2Net block is a neural building block that synergistically combines fine-grained multi-scale feature extraction (Res2Net) with adaptive channel recalibration (Squeeze-and-Excitation, SE) within a residual learning framework. It has demonstrated strong empirical performance and parameter efficiency across voice anti-spoofing and related audio forensics benchmarks, especially in the context of robust detection against replay and synthetic speech attacks [2010.15006], [2109.00913], [2210.03581].

## 1. Architectural Overview

The SE-Res2Net block extends the traditional ResNet bottleneck design through two major modifications: multi-scale residual learning across channel groups and explicit channel-wise gating. For an input tensor $X \in \mathbb{R}^{C \times H \times W}$:

- **Multi-scale decomposition**: The input is first projected by a $1 \times 1$ convolution to a channel size $D$ and split into $s$ equally sized subgroups $\{x_i\}_{i=1}^s$ along the channel dimension.
- **Hierarchical residual convolutions**: Each $x_i$ is recursively combined with the output from the previous path and transformed by a shared-parameter $3\times 3$ convolution, constructing $y_1 = K_1(x_1), y_i = K_i(x_i + y_{i-1})$ for $i=2,\dots,s$.
- **Re-aggregation**: The $s$ output streams are concatenated and projected back to $C$ channels via another $1 \times 1$ convolution to yield $Y$.
- **SE recalibration**: The SE module is applied to $Y$, computing channel attention weights using global average pooling and a two-layer MLP; this output $S$ is then channel-wise multiplied to recalibrate $Y$.
- **Residual addition**: The block output is the sum $X + S$ (plus optional normalization and/or non-linearity).

Explicit formulaic description [2010.15006], [2109.00913]:
\[
\begin{align*}
U &= \mathrm{Conv}_{1 \times 1}(X) \\
x_i &= \mathrm{Slice}_i(U), \quad i = 1, \dots, s \\
y_1 &= K_1(x_1) \\
y_i &= K_i(x_i + y_{i-1}), \quad i = 2, \dots, s \\
Y' &= \mathrm{Concat}(y_1, \dots, y_s) \\
Y &= \mathrm{Conv}_{1 \times 1}(Y') \\
S &= \mathrm{SE}(Y) \\
\mathrm{Output} &= X + S
\end{align*}
\]

## 2. Functional Components

### Res2Net Multi-Scale Convolution

By splitting the channel space and deploying hierarchical residual $3 \times 3$ convolutions per group, the block realizes diverse effective receptive fields in parallel. For scale factor $s$, the $i$th group’s convolutional path covers up to $i$ cascaded $3 \times 3$ convolutions, emulating receptive fields of size $3 \times 3$, $5 \times 5$, up to $(2s-1) \times (2s-1)$ within a single block [2010.15006], [2109.00913], [2210.03581]. This design enables highly granular feature extraction across both local and more global contexts.

### Squeeze-and-Excitation (SE) Module

The SE unit performs global-average “squeeze” pooling along spatial dimensions to compute a channel descriptor $z \in \mathbb{R}^C$. This is followed by two linear transformations with a reduction ratio $r$ (typically 8 or 16), first compressing to $C/r$ and then expanding back, interleaved with ReLU and sigmoid activations:
\[
a = \mathrm{ReLU}(W_1 z), \quad e = \sigma(W_2 a)
\]
where $e \in (0,1)^C$ serves as per-channel attention coefficients. Each feature channel is scaled by the corresponding entry in $e$ before the residual addition [2010.15006], [2109.00913], [2210.03581].

## 3. Mathematical Formulation

Given input $X \in \mathbb{R}^{C \times H \times W}$, the process is as follows (for $s$ channel groups, reduction ratio $r$):

1. **Bottleneck projection**:
    \[
    U = \mathrm{Conv}_{1\times1}(X)
    \]
2. **Split into channel groups**:
    \[
    U = [x_1, x_2, \dots, x_s], \quad x_i \in \mathbb{R}^{d \times H \times W}, \quad d = D/s
    \]
3. **Hierarchical convolution**:
    \[
    y_1 = K_1(x_1), \quad y_i = K_i(x_i + y_{i-1}), \quad i = 2, ..., s
    \]
4. **Concatenate and fuse**:
    \[
    Y' = [y_1, ..., y_s], \quad Y = \mathrm{Conv}_{1\times1}(Y')
    \]
5. **SE recalibration**:
    \[
    z_c = \frac{1}{H W} \sum_{u=1}^H \sum_{v=1}^W Y_c(u, v) \\
    a = \mathrm{ReLU}(W_1 z), \quad e = \sigma(W_2 a) \\
    S_c(u,v) = e_c \cdot Y_c(u, v)
    \]
6. **Residual addition**:
    \[
    \mathrm{Output} = X + S
    \]

## 4. Empirical Behavior and Parameter Efficiency

Empirical comparisons on the ASVspoof 2019 corpus demonstrate that SE-Res2Net architectures can achieve improved generalizability and accuracy in both physical access (PA) and logical access (LA) spoofing detection scenarios [2010.15006], [2109.00913], [2210.03581]. The block provides these benefits with lower parameter counts than canonical ResNet variants:

| Model             | Parameters (M) | Comparative Size                  |
|-------------------|:-------------:|-----------------------------------|
| ResNet34          | 1.33          | Baseline                          |
| ResNet50          | 1.05          | –21% vs ResNet34                  |
| Res2Net50         | 0.88          | –16.2% vs ResNet50                |
| SE-Res2Net50      | 0.92          | +0.04M (SE overhead) vs Res2Net50 |

The modest overhead of the SE block ($\sim$0.04M parameters) further amplifies performance gains by introducing channel interdependencies not captured by the spatial convolutions alone [2010.15006].

## 5. Design and Implementation Considerations

Key architectural and implementation aspects include:

- **Scale factor $s$**: Typically set between 2 and 8; increasing $s$ provides finer multi-scale granularity but incurs more $3\times3$ convolution paths.
- **SE reduction ratio $r$**: Commonly set to 8 or 16, balancing adaptivity and parameter growth.
- **Normalization**: Batch normalization follows each convolution; ReLU activations are applied after convolutional layers and after the final addition [2210.03581].
- **Residual matching**: When input and output channel counts differ, the residual branch is projected with a $1\times1$ convolution plus batch normalization before addition to ensure dimensional consistency [2210.03581].
- **Initialization**: Weight initialization for the SE MLP is recommended (e.g., He initialization); the bias of the second SE FC layer is often initialized near zero [2210.03581].

A summarized pseudocode sketch is given in [2109.00913]:
```python
def SE_Res2Net_Block(X, s, r):
    X0 = Conv1x1(X)
    Xs = Split(X0, s)
    Y1 = Conv3x3_1(Xs[0])
    Ys = [Y1]
    for i in range(1, s):
        Yi = Conv3x3_i(Xs[i] + Ys[i-1])
        Ys.append(Yi)
    U = Concat(Ys)
    R = Conv1x1_fuse(U)
    z = GlobalAvgPool(R)
    e = sigmoid(W2 @ ReLU(W1 @ z))
    R_se = R * expand(e)
    return R_se + X
```

## 6. Applications and Integration

The SE-Res2Net block is used as the main convolutional backbone in anti-spoofing systems, speech synthesis detection, and audio forensics. For example, in the SE-Res2Net-Conformer architecture, stacked SE-Res2Net blocks extract local and multi-scale time-frequency patterns from spectro-temporal features (e.g., CQT, log-mel), which are then passed to Conformer layers that capture long-range temporal dependencies [2210.03581]. In multi-modal voice spoofing detection pipelines, SE-Res2Net acts as the physical (acoustic) feature extractor, integrated with other modules such as densely connected CNNs with SE for physiological signal processing [2109.00913].

The combination of diverse local receptive fields, explicit cross-channel modeling, and residual learning confers robustness to previously unseen spoofing artifacts and allows for the efficient deployment of high-performing models on resource-constrained platforms [2010.15006], [2109.00913], [2210.03581].

Source: https://www.emergentmind.com/topics/se-res2net-block