---
title: 'TF-SepNet: Efficient Acoustic Scene CNN'
url: https://www.emergentmind.com/topics/tf-sepnet
type: topic
---

# TF-SepNet: Efficient Acoustic Scene CNN

TF-SepNet is a convolutional neural network architecture specifically designed for efficient and accurate acoustic scene classification (ASC). Its principal innovation is the explicit separation of time-domain and frequency-domain feature extraction via one-dimensional (1D) depthwise convolutions, followed by lightweight channel fusion. This structural disentanglement contrasts with standard and sequential 1D CNN designs and yields reduced computational complexity, an expanded effective receptive field (ERF), and empirically superior classification performance on resource-constrained tasks such as the TAU Urban Acoustic Scene 2022 Mobile benchmark [2309.08200].

## 1. Motivation and Design Principles

Acoustic scene classification systems require high accuracy and low resource use for on-device, real-time applications. Conventional CNNs for ASC utilize stacked two-dimensional (2D) kernels (\(k \times k\)) that convolve across both time and frequency simultaneously, resulting in significant parameter and computation overhead. Prior approaches attempting efficiency improvements replaced 2D kernels with sequences of 1D kernels (\(k \times 1\) then \(1 \times k\)), but still process time-frequency dependencies sequentially. TF-SepNet introduces a stricter decomposition: the intermediate feature map is divided along the channel dimension, with one half processed along frequency by depthwise \(3 \times 1\) convolutions and the other half along time via \(1 \times 3\) convolutions. These are recombined (channel concatenation and shuffle) after pool-projection-broadcast operations, preserving decoupling until the fusion stage. This explicitly factorized process yields lower MACs/parameters and a larger ERF compared to both standard 2D and sequential 1D models [2309.08200].

## 2. Detailed Network Architecture

TF-SepNet operates on log-Mel spectrogram inputs (\(1 \times 256 \times 64\), frequency \(\times\) time frames). The architecture is as follows:

| Output Shape                  | Architecture                         | \(k\)    | \(s\)   | \(p\)   |
|-------------------------------|--------------------------------------|----------|---------|---------|
| \(1 \times F \times T\)       | Input                                | –        | –       | –       |
| \(C/2 \times F/2 \times T/2\) | Conv–BN–ReLU                         | \(3\times3\)| 2       | 1       |
| \(2C \times F/4 \times T/4\)  | Conv–BN–ReLU (groups \(=C/2\))       | \(3\times3\)| 2       | 1       |
| \(C \times F/4 \times T/4\)   | TF-SepConvs × 2                      | –        | –       | –       |
| \(C \times F/8 \times T/8\)   | MaxPool                              | \(2\times2\)| 2       | 0       |
| \(1.5C \times F/8 \times T/8\)| TF-SepConvs × 2                      | –        | –       | –       |
| \(1.5C \times F/16 \times T/16\)| MaxPool                            | \(2\times2\)| 2       | 0       |
| \(2C \times F/16 \times T/16\)| TF-SepConvs × 2                      | –        | –       | –       |
| \(2.5C \times F/16 \times T/16\)| TF-SepConvs × 3                    | –        | –       | –       |
| \(10 \times F/16 \times T/16\)| Conv–BN–ReLU                         | \(1\times1\)| 1       | 0       |
| \(10 \times 1 \times 1\)      | GlobalAvgPool                        | –        | –       | –       |

Each TF-SepConvs block begins with a \(1\times1\) pointwise convolution for channel adjustment, channel shuffle, then equal partitioning into frequential \(x^{(f)}\) and temporal \(x^{(t)}\) halves. \(x^{(f)}\) receives a depthwise \(3\times1\) convolution, averaging, pointwise convolution, and broadcast. \(x^{(t)}\) follows the same scheme with \(1\times3\) convolutions. Outputs \(\hat x^{(f)}, \hat x^{(t)}\) are concatenated to form the block result. Adaptive residual normalization (AdaResNorm) follows each block for training stabilization [2309.08200].

## 3. Mathematical Formulation: 1D Versus 2D Convolutions

A 1D convolution on input \(x[t]\) and kernel \(w[i]\) (length \(k\)) is:
\[
(x*w)[t] = \sum_{i=0}^{k-1} x[t+i]\,w[i].
\]
A 2D convolution over feature map \(X[m,n]\) with filter \(W[i,j]\) (\(k \times k\)), is:
\[
(X*W)[m,n] = \sum_{i=0}^{k-1}\sum_{j=0}^{k-1} X[m+i, n+j]\, W[i, j].
\]
A 2D filter has \(k^2\) parameters per input-output channel pair, while 1D has only \(k\). TF-SepNet employs depthwise \(3\times1\) and \(1\times3\) kernels and pointwise \(1\times1\) fusions, splitting channels for further efficiency [2309.08200].

## 4. Complexity Analysis

Parameter and operation counts for conventional \(k \times k\) 2D, sequential 1D, and TF-SepNet decompositions are:

| Model/Block                | Params                                  | Approx. MACs              |
|----------------------------|-----------------------------------------|---------------------------|
| 2D Conv (\(C_{in}\), \(C_{out}\))    | \(C_{in}C_{out}k^2\)                | \(2F T C_{in} C_{out}k^2\)|
| Sequential 1D (see text)   | \(2C_{in}C_{out}k\)                     | \(4F T C_{in} C_{out}k\)  |
| TF-SepNet-40 (empirical)   | 53.4 K                                  | 7.0 M                     |
| BC-ResNet-40               | 88.1 K                                  | 17.2 M                    |
| BC-Res2Net-40              | 85.8 K                                  | 17.2 M                    |

TF-SepNet-40 achieves a MAC and parameter reduction of ~60% relative to BC-ResNet/Res2Net-40 baselines at similar or better accuracy [2309.08200].

## 5. Effective Receptive Field (ERF) Properties

In CNNs, the effective receptive field is the input region most influencing a neuron's output, quantified via the spatial distribution of backpropagated gradients. TF-SepNet displays a broader and more uniform ERF, with a larger top-30% contributing area (\(r=22.5\%\) for TF-SepNet-40, versus \(17.3\%\) for BC-ResNet-40 and \(18.9\%\) for BC-Res2Net-40). This increase in ERF spatial extent enables the capture of longer-range time-frequency dependencies at fixed depth, which plausibly underpins observed accuracy gains [2309.08200].

## 6. Experimental Methodology and Results

TF-SepNet was evaluated on the TAU Urban Acoustic Scene 2022 Mobile development set (10 classes, multiple devices), using the official 70/30 train/test split. Preprocessing included resampling to 32 kHz, STFT (window 3072, hop 500), and Mel filterbank (256 bins, 4096-point FFT, log transform). The training protocol comprised 100 epochs with Adam optimizer, batch size 32, learning-rate warmup then cosine-annealing, mixup and Freq-MixStyle regularization, and AdaResNorm. Performance metrics:

| Model               | Acc (%) | MACs (M) | Param (K) |
|---------------------|---------|----------|-----------|
| DCASE Baseline      | 42.9    | 29.2     | 46.5      |
| BC-ResNet-40        | 57.1    | 17.2     | 88.1      |
| BC-Res2Net-40       | 59.1    | 17.2     | 85.8      |
| TF-SepNet-40        | 60.0    | 7.0      | 53.4      |
| BC-ResNet-80        | 58.4    | 45.8     | 315.0     |
| BC-Res2Net-80       | 59.6    | 42.7     | 307.0     |
| TF-SepNet-80        | 61.6    | 24.2     | 196.7     |

TF-SepNet outperformed all compared models in accuracy, with marked reductions in both MACs and parameter count [2309.08200].

## 7. Ablation and Component Analysis

Ablation experiments established that removal of either the temporal or frequential pathway results in a 2.5–3.3% drop in accuracy, with paradoxical increases in MACs and parameters due to loss of channel compression via the \(1\times1\) bottleneck. Omitting channel shuffle reduced accuracy by 0.5%. Elimination of AdaResNorm led to a 1.5% accuracy decrease with only a 2% parameter saving. This confirms that separated 1D convolutional paths, AdaResNorm, and channel shuffle collectively underpin the full performance and efficiency of TF-SepNet [2309.08200].

Source: https://www.emergentmind.com/topics/tf-sepnet