---
title: 'SuDoRM-RF++: Efficient Universal Audio Separation'
url: https://www.emergentmind.com/topics/sudorm-rf
type: topic
---

# SuDoRM-RF++: Efficient Universal Audio Separation

Searching arXiv for the core SuDoRM-RF and SuDoRM-RF++ papers to ground the article in the relevant literature.
SuDoRM-RF++ is a family of end-to-end, time-domain neural architectures for universal sound source separation built around the SuDoRM-RF separator backbone, where **SuDoRM-RF** denotes **SUccessive DOwnsampling and Resampling of Multi-Resolution Features**. In the literature, the “++” designation refers specifically to a variant that retains the multiresolution **U-ConvBlock** separator philosophy of the original SuDoRM-RF model while altering the estimation head, normalization, decoder arrangement, and activation parameterization [2103.02644]. It is designed for **single-channel** mixtures and targets both source-domain universality—speech, environmental sounds, and open-domain mixtures—and, in later extensions, broader robustness to recording conditions such as sampling frequency [2103.02644; 2309.12581].

## 1. Nomenclature and lineage

The architectural lineage begins with the model introduced under the stylized title **“Sudo rm -rf: Efficient Networks for Universal Audio Source Separation”**, whose body defines the architecture as **SuDoRM-RF**, not SuDoRM-RF++ [2007.06833]. That 2020 work establishes the essential waveform separation pipeline: a learned encoder, a separator composed of repeated **U-ConvBlocks**, and a learned decoder. It also frames the main efficiency rationale: high-quality separation with limited floating point operations, memory requirements, number of parameters, and latency [2007.06833].

The explicit introduction of **SuDoRM-RF++** appears in the 2021 paper **“Compute and memory efficient universal sound source separation”** [2103.02644]. That paper presents a small family of related models: **SuDoRM-RF**, **SuDoRM-RF++**, **SuDoRM-RF++ GC**, and **C-SuDoRM-RF++**. The relationship is architectural continuity rather than replacement by an unrelated design. SuDoRM-RF++ preserves the same core separator idea—successive downsampling and resampling of multi-resolution features—but changes the output formulation and several implementation choices [2103.02644].

A common misconception is that the 2020 paper introduces “SuDoRM-RF++.” It does not. The 2020 paper is the architectural precursor; the 2021 paper is the source that explicitly defines the “++” variants [2007.06833; 2103.02644].

## 2. Separation setting and model family

SuDoRM-RF++ addresses **single-channel end-to-end audio source separation** in a deliberately broad sense. In the 2021 formulation, “universal” has two distinct dimensions: universality with respect to **source domain** and universality with respect to **source cardinality** [2103.02644]. The evaluated tasks include:

- **WSJ0-2mix** for two-speaker speech separation.
- **ESC50-based mixtures** for two-source environmental or non-speech separation.
- **FUSS** for open-domain universal sound separation, where mixtures may contain **speech, music, engines, wind, rain, and many other real-world sounds**, with **1 to 4** active sources [2103.02644].

The fixed-maximum-output formulation used for variable source count assumes a known maximum number of output slots \(N\), while allowing only \(N' < N\) active sources in a given mixture. For **FUSS**, the model uses \(N=4\) output slots and handles \(N' \in \{1,2,3,4\}\) by learning to emit near-zero estimates in inactive slots under a permutation-invariant objective [2103.02644].

The family structure is summarized below.

| Variant | Defining change | Intended use |
|---|---|---|
| **SuDoRM-RF** | Mask-based latent estimation with separate decoders | Original non-causal model |
| **SuDoRM-RF++** | Direct latent target estimation, single decoder, GLN | Refined non-causal variant |
| **SuDoRM-RF++ GC** | SuDoRM-RF++ with group communication | Parameter-reduced variant |
| **C-SuDoRM-RF++** | Causal convolutions, no normalization | Streamable real-time use |

This organization suggests that “SuDoRM-RF++” is best understood not as a single frozen architecture but as a refined branch within a broader SuDoRM-RF design family.

## 3. Core architecture: encoder, U-ConvBlocks, and multiresolution processing

Like the original SuDoRM-RF, SuDoRM-RF++ is a **time-domain learned encoder–separator–decoder** system rather than an STFT-based separator [2103.02644]. For an input waveform mixture \(x \in \mathbb{R}^T\), the encoder is a 1D convolution followed by ReLU:
\[
v = \mathcal{E}(x) = \operatorname{ReLU}\!\left( \operatorname{Conv1D}_{N,K,K/2}(x) \right) \in \mathbb{R}^{N \times L}.
\]
The 2021 paper specifies \(N=512\), \(K=21\) at 8 kHz, \(K=41\) at 16 kHz, and stride \(K/2\) [2103.02644]. As in the earlier paper, this learned front-end produces a nonnegative latent mixture representation and reduces temporal resolution immediately through the encoder stride [2007.06833].

The separator first projects the latent representation into an internal channel space:
\[
y_0 = \operatorname{Conv1D}_{C,1,1}\big(\operatorname{LN}(v)\big) \in \mathbb{R}^{C \times L},
\]
with default \(C=128\) for non-causal models and \(C=256\) for the causal model [2103.02644]. The main separator then applies \(B\) repeated **U-ConvBlocks**:
\[
y_i = \text{UConvBlock}(y_{i-1}), \qquad i=1,\dots,B.
\]
The scale naming convention is explicit: **2.0x** uses \(B=32\), **1.0x** uses \(B=16\), **0.5x** uses \(B=8\), and **0.25x** uses \(B=4\) [2103.02644].

Within each U-ConvBlock, the input is first expanded from \(C\) channels to \(C_U\), processed by a depthwise convolution at full temporal resolution, then passed through repeated **stride-2 depthwise 1D convolutions** to create a hierarchy of lower-resolution feature maps. In the default non-causal configuration, \(C=128\), \(C_U=512\), \(Q=4\), \(K_U=5\), and \(S_U=2\) [2103.02644]. The coarser features are then returned to the original temporal resolution by **nearest-neighbor interpolation** and fused by additive top-down aggregation:
\[
\mathbf{u}^{(i)} = \mathbf{d}^{(i)} + \mathcal{I}_{S_U}\big(\mathbf{u}^{(i+1)}\big).
\]
Finally, the block projects back to \(C\) channels and applies a residual connection:
\[
y^{(i+1)} = \operatorname{PReLU}_C\big(y^{(i)} + \mathbf{o}\big).
\]

Operationally, this is the meaning of **SUccessive DOwnsampling and Resampling of Multi-Resolution Features**. The separator grows effective receptive field through multiscale processing inside each residual block rather than through very deep stacks of dilated convolutions or recurrent dual-path modules [2007.06833; 2103.02644]. The U-ConvBlock therefore occupies the same conceptual role across the entire family, including later GC and causal variants.

## 4. What distinguishes SuDoRM-RF++ from SuDoRM-RF

The principal architectural change is the removal of **mask estimation**. In the original SuDoRM-RF, the separator predicts source-specific latent pre-mask codes, converts them to masks through a source-wise softmax, and multiplies those masks with the encoded mixture representation \(v\) to obtain latent source estimates [2007.06833]. In SuDoRM-RF++, by contrast, the model **directly estimates the latent target signals** after the final separator output [2103.02644].

The 2021 paper identifies four defining changes for SuDoRM-RF++ [2103.02644]:

| Component | SuDoRM-RF | SuDoRM-RF++ |
|---|---|---|
| Estimation head | Mask estimation | Direct latent estimation |
| Decoder arrangement | Different decoders per source | One trainable decoder module |
| Normalization | LN | GLN |
| PReLU parameterization | One learnable parameter per channel | One learnable parameter per activation layer |

The decoder distinction is explicit. Original SuDoRM-RF uses **different decoders for different sources**, whereas SuDoRM-RF++ uses a **single shared decoder**:
\[
\hat{s}_i = \mathcal{D}(\hat{\mathbf{v}}_i) = \operatorname{ConvTr1D}_{1,K,K/2}(\hat{\mathbf{v}}_i).
\]
The paper reports that replacing LN with **GLN** significantly improves convergence, and that simplifying PReLU to one learnable slope parameter per activation layer reduces parameter count [2103.02644].

The paper does not claim that SuDoRM-RF++ is uniformly superior in every possible scenario. It explicitly notes that original SuDoRM-RF may still be preferable when **mask estimation is easier than direct latent target estimation**, when bounded masks in \([0,1]\) provide a useful inductive bias, or when separate source-specific decoders are advantageous for **heterogeneous source classes** [2103.02644]. This point matters because later work sometimes treats “++” as an unqualified replacement. The published account is more specific: the “++” design is a refined variant with a different inductive bias, not a categorical repudiation of mask-based separation.

## 5. Efficiency profile and empirical results

Efficiency is central to the SuDoRM-RF++ program. The 2021 paper reports forward-pass CPU results for **1 second at 8 kHz** and backward-pass GPU profiling, comparing the SuDoRM-RF family against **ConvTasNet**, **DPRNN**, and **Two-Step TDCN** [2103.02644].

For forward CPU inference at 8 kHz, **SuDoRM-RF++ 1.0x** reports **17.0 dB SI-SDRi** on speech and **8.6 dB** on non-speech, with **2.11 GFLOPs**, **0.79 GB**, and **0.17 s** runtime [2103.02644]. The corresponding **SuDoRM-RF 1.0x** reports **17.0 / 8.4 dB**, **2.45 GFLOPs**, **0.79 GB**, and **0.17 s**; **ConvTasNet** reports **15.3 / 7.7 dB** at **5.16 GFLOPs**; **DPRNN** reports **18.8 / 7.2 dB** at **48.81 GFLOPs**; and **Two-Step TDCN** reports **16.1 / 8.2 dB** at **7.11 GFLOPs** [2103.02644].

For backward-pass GPU profiling, **SuDoRM-RF++ 1.0x** uses **2.72M** parameters, **16.23 GFLOPs**, **0.99 GB**, and **0.27 s**, while **SuDoRM-RF++ 1.0x GC** reduces parameters to **0.30M** at **2.72 GFLOPs**, but with **1.21 GB** memory and **0.35 s** wall-clock time [2103.02644]. The paper emphasizes that parameter count alone does not determine deployment efficiency: **group communication** can reduce parameters substantially while not necessarily improving memory or runtime proportionally [2103.02644].

For **FUSS**, the strongest SuDoRM-RF-family result reported in the 2021 paper is **SuDoRM-RF++ 2.0x**, with **25.9 dB SI-SDR** for \(N'=1\), **10.9 dB SI-SDRi** for \(N'=2\), **10.6 dB** for \(N'=3\), **7.8 dB** for \(N'=4\), and average **9.8 dB** across \(2\)–\(4\) active sources [2103.02644]. The paper states that it nearly matches **TDCN++** on average and **outperforms it for the hardest \(N'=4\) case** [2103.02644].

Ablations in the same paper clarify why the “++” formulation took the shape it did. With \(K=17, C=128, B=16, Q=4\), replacing **LN** with **GLN** improves WSJ0-2mix performance from **15.9 dB** to **16.8 dB**. Larger separator width, deeper stacks, and larger multiresolution depth \(Q\) also improve performance; for example, \(K=21, C=512, B=34, Q=4\), GLN reaches **18.9 dB** [2103.02644]. This suggests that the SuDoRM-RF backbone is not only an efficiency-oriented design but also a scalable one.

## 6. Variants, causal streaming, and sampling-frequency-independent extension

The paper’s two principal extensions of SuDoRM-RF++ are **group communication** and **causal streaming**. In **SuDoRM-RF++ GC**, channels are split into **16 groups**, processed independently with shared parameters, and then coupled through a **self-attention module** for communication across groups [2103.02644]. The purpose is to reduce parameters, particularly in the bottleneck \(1\times1\) channel-mixing layers.

In **C-SuDoRM-RF++**, all non-causal convolutions are replaced by **causal convolutions**, all normalization layers are removed, and the model is configured for real-time operation [2103.02644]. On a laptop CPU for **1 second / 8000 samples**, Table 5 reports: \(B=4, K_U=5\) achieves **9.1 dB SI-SDRi** in **50.4 ms**; \(B=8, K_U=5\) achieves **10.1 dB** in **88.2 ms**; and \(B=8, K_U=11\) achieves **10.3 dB** in **165.9 ms**. The paper summarizes this as about **10–20× faster than real time** for some causal settings [2103.02644].

A later extension addresses **sampling-frequency independence** by adopting **SuDoRM-RF++ 1.0x GC** as the baseline architecture [2309.12581]. The 2023 paper does not introduce a new separator core. Instead, it replaces only the **encoder** and **decoder** with **sampling-frequency-independent (SFI) convolutional layers**, while keeping the separator core architecturally the same as standard SuDoRM-RF++ [2309.12581]. The key mechanism is to generate digital convolution kernels from a latent analog filter \(G(\omega;\theta)\) as a function of the input sampling frequency \(F_s\), using a least-squares design:
\[
\bm{b}=\arg\min_{\bm{b'}\in\mathbb{R}^{K}}\|\bm{G}-\bm{D}\bm{b'}\|^2.
\]
The front-end kernel size and stride are then scaled with sampling frequency:
\[
K^{(\mathrm{target})}=\frac{F_s^{(\mathrm{target})}}{F_s^{(\mathrm{train})}}K^{(\mathrm{train})}, \quad
S^{(\mathrm{target})}=\frac{F_s^{(\mathrm{target})}}{F_s^{(\mathrm{train})}}S^{(\mathrm{train})}.
\]

This extension is noteworthy because it preserves the central SuDoRM-RF++ assumption that most sample-rate dependence resides in the analysis/synthesis filterbanks and in latent time resolution, rather than in the U-ConvBlock separator itself [2309.12581]. The paper reports that resampling-based handling of unseen sampling frequencies degrades performance, whereas the proposed SFI method works more consistently across **8, 12, \ldots, 44, 48 kHz** test conditions after training only at **48 kHz** [2309.12581].

The same paper also records an important practical limitation: attempts to train the **original SuDoRM-RF** on **48 kHz** data with \(K=240\), \(S=120\) led to output collapse, with **all output signals becoming zeros after the first epoch**, and similar numerical instability across several \(K,S\) choices [2309.12581]. This makes the 2023 study especially relevant for understanding SuDoRM-RF++ as a practically viable high-sample-rate branch of the family.

## 7. Interpretation, trade-offs, and technical significance

SuDoRM-RF++ occupies a specific position in the design space of time-domain separation architectures. Its distinctive separator is neither a large dilated-TCN stack nor a dual-path recurrent module, but a stack of lightweight residual U-shaped blocks that exploit **depthwise convolutions**, **stride-2 temporal downsampling**, **nearest-neighbor interpolation**, and **additive multiscale fusion** to expand receptive field at comparatively low cost [2007.06833; 2103.02644]. The efficiency argument is therefore architectural rather than merely implementational.

Three trade-offs recur across the literature. First, **direct latent estimation versus mask estimation**: the “++” design removes masking, but later work built on SuDoRM-RF++ 1.0x GC reverts to **mask estimation** and notes that direct prediction had been observed to make training numerically unstable [2309.12581]. Second, **parameter count versus realized efficiency**: the GC variant dramatically reduces parameters, but may increase memory and fail to reduce wall-clock time proportionally [2103.02644]. Third, **universality versus specialization**: the family is explicitly framed for universal sound separation, yet the original mask-based SuDoRM-RF may still be preferable when strong bounded-mask inductive bias or source-specific decoders are desirable [2103.02644].

Taken together, these results suggest that SuDoRM-RF++ is best characterized as a refined and extensible architectural template for universal source separation rather than as a single canonical instantiation. Its stable identity across papers lies in the SuDoRM-RF backbone—**successive downsampling and resampling of multi-resolution features** via repeated **U-ConvBlocks**—while the “++” label denotes a particular head-level and normalization-level reformulation that proved competitive in both fixed-source and variable-source-count settings [2103.02644]. Subsequent work extends this template toward parameter reduction, causality, and sampling-frequency robustness without discarding the underlying separator philosophy [2103.02644; 2309.12581].

Source: https://www.emergentmind.com/topics/sudorm-rf