DISPatch: Selective Patches for Speech Enhancement

• The paper introduces DISPatch, which applies KD loss only on spectrogram patches where the teacher outperforms the student as measured by a Knowledge Gap Score.
• It refines distillation by employing a patch-level approach and extends it with Multi-Scale Selective Patches to cater to varying spectral characteristics.
• Experiments reveal that DISPatch consistently improves WB-PESQ, NB-PESQ, and STOI, outperforming traditional full-spectrogram distillation methods.

Searching arXiv for the cited papers to ground the article in current records. Distilling Selective Patches (DISPatch) is a knowledge distillation framework for speech enhancement that applies the distillation loss only to spectrogram patches where the teacher outperforms the student, as determined by a Knowledge Gap Score. In contrast to conventional response-based KD, which enforces full-spectrogram imitation, DISPatch targets regions with the largest potential for student improvement and suppresses distillation in regions where the student already performs well or where the teacher is unreliable. The framework is defined at the patch level in the time–frequency domain and is extended by Multi-Scale Selective Patches (MSSP), which uses different patch sizes for low- and high-frequency bands to account for spectral heterogeneity (Kim et al., 19 Sep 2025).

1. Motivation and problem setting

Conventional response-based KD for speech enhancement applies a distillation loss over the entire time–frequency output of the teacher. The reported motivation for DISPatch is that such full-spectrogram imitation suffers from two problems: the student may already match the teacher, or even the clean target, in many regions, and the teacher itself is not perfect and contains low-quality regions. Under those conditions, indiscriminate KD can either yield little benefit or propagate teacher errors (Kim et al., 19 Sep 2025).

The framework is formulated on complex spectrograms. A clean target $\mathbf{S}$, a student estimate $\hat{\mathbf{S}}^S$, and a teacher estimate $\hat{\mathbf{S}}^T$ are split into $P$ non-overlapping patches of size $2C\times N\times 1$ along the frequency axis:

$$\{\mathbf{s}_p,\hat{\mathbf{s}}^S_p,\hat{\mathbf{s}}^T_p\}_{p=0}^{P-1}.$$

Patch selection is therefore frequency-structured rather than image-like: the unit of distillation is a spectrogram subregion, not a full utterance or a single frame.

A common misconception is to treat DISPatch as a mechanism for selecting the teacher’s most confident patches. The formal criterion is different. Selection is based on the difference between student and teacher patch-wise errors relative to the clean target, so a patch is emphasized only when the teacher is demonstrably better than the student on that region (Kim et al., 19 Sep 2025).

2. Knowledge Gap Score

DISPatch defines patch-wise student and teacher errors using a flexible error metric $\varepsilon(\mathbf{s}_p,\hat{\mathbf{s}}_p)$, with examples including $\ell_1$, $\ell_2$ magnitude distance, or the DFKD objective. For each patch $p$,

$\hat{\mathbf{S}}^S$0

The Knowledge Gap Score (KGS) is then

$\hat{\mathbf{S}}^S$1

A larger $\hat{\mathbf{S}}^S$2 indicates that the student’s error greatly exceeds the teacher’s, so distilling that patch should yield maximal improvement. The score is therefore a patch-wise estimate of actionable teacher superiority rather than a generic saliency or confidence measure (Kim et al., 19 Sep 2025).

This definition has two immediate consequences. First, patches where the student has already converged to low error are naturally deprioritized. Second, teacher regions with high error are also suppressed, because they reduce or reverse the gap. This suggests that DISPatch functions as a selective transfer operator that couples KD strength to local teacher–student discrepancy, rather than to teacher output magnitude alone.

3. Selective-patch distillation objective

After computing $\hat{\mathbf{S}}^S$3 over all patches, DISPatch ranks patches by their scores and retains only the top $\hat{\mathbf{S}}^S$4. With the indicator

$\hat{\mathbf{S}}^S$5

the distillation loss becomes

$\hat{\mathbf{S}}^S$6

In practice, the authors set $\hat{\mathbf{S}}^S$7, so 80% of the patches with highest $\hat{\mathbf{S}}^S$8 contribute to $\hat{\mathbf{S}}^S$9 (Kim et al., 19 Sep 2025). The selection ratio is therefore not extremely sparse; it removes the lowest-value fifth of the spectrogram patches while keeping the majority of informative regions. This distinguishes the method from top-$\hat{\mathbf{S}}^T$0 hard-mining schemes that retain only a very small fraction of patches.

DISPatch is agnostic to the underlying KD objective $\hat{\mathbf{S}}^T$1. The overall student training loss is

$\hat{\mathbf{S}}^T$2

where $\hat{\mathbf{S}}^T$3 is the standard speech enhancement loss and $\hat{\mathbf{S}}^T$4 balances the two terms. The reported setting fixes $\hat{\mathbf{S}}^T$5 throughout (Kim et al., 19 Sep 2025).

Operationally, one training iteration proceeds by computing teacher and student outputs on the noisy input, evaluating $\hat{\mathbf{S}}^T$6, splitting teacher output, student output, and clean target into patches, computing $\hat{\mathbf{S}}^T$7, $\hat{\mathbf{S}}^T$8, and $\hat{\mathbf{S}}^T$9, selecting the top $P$0 patches, forming $P$1, and then backpropagating $P$2 to update only the student. The method is thus a selective masking layer over an otherwise conventional KD pipeline.

4. Multi-Scale Selective Patches (MSSP)

MSSP extends DISPatch by recognizing that speech spectrograms exhibit different structures at low and high frequencies. The stated motivation is that vowel formants are fine-grained and require small patches, whereas fricatives and noise bursts are broader and require larger patches. MSSP therefore uses two patch sizes,

$P$3

The framework dynamically partitions each frame into low- and high-frequency bands using the DFKD frequency adapter. The paper’s best setting for ConvTasNet and DCCRN-CL is

$P$4

so that low-frequency patches are finer than high-frequency patches (Kim et al., 19 Sep 2025).

Patch selection and distillation then proceed exactly as in DISPatch, but over a heterogeneous set of patch sizes. When MSSP is integrated into a frequency-dependent KD method such as DFKD, the uniform patching step is replaced by separate low/high patching, and the per-band loss formulations of DFKD are retained: phase-only loss for the low band, and amplitude-plus-phase loss for the high band (Kim et al., 19 Sep 2025).

The conceptual significance of MSSP is that patch granularity becomes frequency-dependent rather than globally fixed. This suggests a domain-specific adaptation of selective KD: the patching geometry is aligned with the nonuniform structure of speech spectra instead of being imposed uniformly across all bands.

5. Experimental configuration and reported results

The reported experiments use two datasets: DNS2020, with 500+ h speech and 100+ h noise for training and the DNS2020 challenge set for testing, and VoiceBank+DEMAND with the standard 16 kHz test set. Two model families are evaluated: ConvTasNet, with a teacher of 9.71 M parameters and 8.05 G MAC and a student of 1.75 M parameters and 0.57 G MAC, and DCCRN-CL, with a teacher of 9.22 M and 15.33 G and a student of 2.24 M and 3.84 G. Training uses 100 epochs, Adam with learning rate $P$5 halved on plateau, batch size 16, gradient clip 5, STFT with Hann 32 ms and hop 8 ms, and patch selection ratio $P$6 (Kim et al., 19 Sep 2025).

The evaluation metrics are Wideband PESQ (WB-PESQ), Narrowband PESQ (NB-PESQ), and STOI (%). For brevity, the table below reproduces the ConvTasNet student results explicitly listed for DNS2020-test and VoiceBank+DEMAND.

Method	DNS2020-test	VoiceBank+DEMAND
Scratch	2.563 / 3.122 / 96.05	2.585 / 3.424 / 93.44
L1-KD	2.584 / 3.135 / 96.11	2.584 / 3.429 / 93.57
L1 + DISPatch	2.602 / 3.165 / 96.16	2.614 / 3.450 / 93.79
L2-KD	2.563 / 3.127 / 95.98	2.556 / 3.400 / 93.37
L2 + DISPatch	2.602 / 3.157 / 96.10	2.603 / 3.431 / 93.39
DFKD	2.623 / 3.170 / 96.08	2.583 / 3.458 / 93.70
DFKD + DISPatch	2.650 / 3.208 / 96.12	2.589 / 3.490 / 93.81
DFKD + MSSP(10/40) + DISPatch	2.677 / 3.226 / 96.17	2.618 / 3.505 / 94.02

The reported interpretation is consistent across methods. Adding DISPatch to L1 or L2 gives consistent WB-PESQ, NB-PESQ, and STOI gains over the corresponding KD methods without DISPatch. Even DFKD benefits from DISPatch, with a reported $P$7 WB-PESQ gain. The full combination DFKD+MSSP(10/40)+DISPatch yields the best overall ConvTasNet results on both DNS2020 and VoiceBank+DEMAND. The paper also states that analogous gains appear for DCCRN-CL (Kim et al., 19 Sep 2025).

6. Position within the broader patch-selection literature

The acronym “DISPatch” is not unique to speech enhancement. In dataset distillation, "Foreground-Aware Dataset Distillation via Dynamic Patch Selection" defines a method that routes each image through a cropping path or a resize path according to foreground occupancy derived from Grounded SAM2, using a class-specific threshold $P$8, typically with $P$9; it reports improvements over RDED, MTT, IDM, DATM, D$2C\times N\times 1$0M, G-VBSM, and SRe$2C\times N\times 1$1L across CIFAR-10, CIFAR-100, ImageNette, and ImageWoof (Li et al., 6 Jan 2026).

A second image-domain method, "Efficient Dataset Distillation via Diffusion-Driven Patch Selection for Improved Generalization," also uses the name DISPatch and scores patches by the difference between conditional and unconditional diffusion-model noise-prediction losses,

$2C\times N\times 1$2

then enforces diversity through intra-class clustering on DIFT features; it reports, for example, about $2C\times N\times 1$3 on ImageNet-1K at IPC$2C\times N\times 1$4 with a ResNet-18 teacher, compared with about $2C\times N\times 1$5 for Minimax and $2C\times N\times 1$6 for RDED (Zhong et al., 2024).

Related patch distillation ideas also appear outside dataset distillation. In computational pathology, AFD-MIL filters diagnostically informative patches before fusion, uses an attention mechanism to retain the top-$2C\times N\times 1$7 patches by attention score, and optimizes the process with a global loss; the reported results are 91.47% ACC and 94.29% AUC on Camelyon16, and 93.33% ACC and 98.17% AUC on TCGA-NSCLC (Nan et al., 2024).

These parallel usages indicate that “selective patches” has become a recurring design motif rather than a single modality-specific technique. This suggests a broader methodological pattern: patch-level selectivity is being used to suppress redundant or harmful supervision while preserving the subregions most relevant to the downstream objective. In speech enhancement, DISPatch instantiates that pattern through teacher–student error gaps in the time–frequency domain; in image distillation and MIL, analogous roles are played by foreground occupancy, diffusion-based class relevance, or attention-derived diagnostic value.