Generative Remixing in SURF for Source Separation

• Generative remixing in SURF is a technique that employs stochastic and algebraic transformations within invertible frameworks to generate high-quality synthetic training data.
• It utilizes a teacher–student model and flow-matching interpolation to overcome the lack of ground-truth source pairs in unsupervised source separation.
• The method also extends to event stream synthesis by manipulating latent-noise domains, ensuring efficient data augmentation and robust domain alignment.

Generative remixing in SURF refers to a mechanism for constructing new, training-compatible data from existing observed data via explicit stochastic or algebraic transformations within learned generative or invertible frameworks. In the context of source separation (SURF: Separation via Unsupervised Remixing Flow), generative remixing bootstraps high-quality pseudo-mixture/source pairs from mixtures only, enabling flow-based models to learn expressive priors in fully unsupervised regimes. In time series modeling (SurF: A Generative Model for Multivariate Irregular Time Series Forecasting), generative remixing operates in the latent-noise domain, enabling manipulation and synthesis of new event streams via bijective mappings to and from canonical Exp(1) noise. Both approaches exploit invertible mappings for data augmentation and domain alignment, significantly improving generative modeling when ground-truth supervision is scarce (Li et al., 3 Jun 2026, Rezaei et al., 13 May 2026).

1. Generative Remixing in Unsupervised Source Separation

In the context of single-channel source separation, generative remixing in SURF is designed to address the absence of ground-truth tuples of clean sources. The method operates via a teacher–student framework, with data augmentation achieved through a structured stochastic remix of the teacher’s source estimates. Given real mixtures $\{m_b\}_{b=1}^B$, the teacher $f_\mathcal{T}(m_b)$ produces its best estimate of the separated sources. All teacher outputs are stacked and globally shuffled with a random permutation $\Pi \in S_{BK}$: $\widetilde X_1 = \Pi \bar X,$ where $\bar X$ is the stacked set of $B$ mixtures’ $K$ source estimates. Synthetic mixtures are then constructed by summing contiguous blocks: $$\widetilde M = (I_B \otimes \mathbf{1}^\top) \widetilde X_1.$$ This process breaks direct correspondence with the teacher’s inputs, forcing subsequent models to generalize beyond simple memorization, and enables creation of arbitrarily many (mixture, pseudo-source) pairs without supervision. These synthetic pairs serve as a foundation for flow-based generative training (Li et al., 3 Jun 2026).

2. Mathematical Structure of the Remixing Flow

SURF’s generative remixing defines an explicit interpolation path for conditional flow-matching between a noise-initialized pseudo-source state and permutation-invariant pseudo-sources. For each pseudo-mixture $\widetilde m$:

• Initialization:

$$\widetilde X_0 = \tfrac{1}{K}\mathbf{1}\widetilde m + P^\perp Z,$$

with $f_\mathcal{T}(m_b)$0 and $f_\mathcal{T}(m_b)$1 the orthogonal projector onto the sum-zero subspace.

• Interpolation:

$f_\mathcal{T}(m_b)$2

where $f_\mathcal{T}(m_b)$3 block-diagonally aligns pseudo-sources via PIT assignment for unbiased flow matching.

Two loss variants are supported:

• ReMixIT-FM: Flow-matching on the pseudo-sources.

$f_\mathcal{T}(m_b)$4

• Self-Remixing-FM: Matching the remixed sum back to the original mixtures.

Pseudocode for a full iteration is explicitly provided and includes mixture collection, teacher estimation, permutation, mixture/sources recomposition, interpolation path construction, PIT assignment, loss evaluation, student update, and EMA-based teacher parameter update (Li et al., 3 Jun 2026).

3. Wake–Sleep Interpretation

SURF’s generative remixing loop is closely analogous to the Wake–Sleep algorithm:

• Sleep (student) phase: Synthetic data are generated from the teacher's implicit generative model $f_\mathcal{T}(m_b)$5, paired with a mixture $f_\mathcal{T}(m_b)$6, and used to minimize

$f_\mathcal{T}(m_b)$7

• Wake (teacher) phase: The ideal objective would also minimize the reverse KL, aligning the teacher’s prior to the aggregate posterior defined by the student.

The practical parameter update utilizes EMA of the student parameters to maintain stability. This loop enables iterative refinement, in which the generative student can surpass the initial regression-based teacher (Li et al., 3 Jun 2026).

4. Empirical Protocol and Stability

Key empirical considerations for generative remixing in SURF include:

• Batch size: $f_\mathcal{T}(m_b)$8 is necessary to obtain sufficient remixed source diversity and stable PIT alignment.
• EMA update rate: Values $f_\mathcal{T}(m_b)$9–$\Pi \in S_{BK}$0 prevent collapse of the teacher toward noisy student updates.
• Hybrid-teacher schedule: Linearly annealing from MixIT to EMA teachers over $\Pi \in S_{BK}$1200k steps improves convergence stability.

Empirical benchmarks demonstrate strong performance: On CIFAR-10/SURREAL, PSNR $\Pi \in S_{BK}$219.5 dB, LPIPS $\Pi \in S_{BK}$30.037, and FID $\Pi \in S_{BK}$412.5; on Libri2Mix, unsupervised SI-SDR $\Pi \in S_{BK}$516.5 dB—substantially outperforming MixIT and closely approaching supervised flow models. Across universal separation tasks, source-count accuracy improvements as large as $\Pi \in S_{BK}$6 are reported (Li et al., 3 Jun 2026).

5. Generative Remixing for Event Streams

In time series forecasting, the SurF model leverages the Time Rescaling Theorem (TRT) to create an invertible bijection between event times and canonical Exp(1) noise. Given a sequence $\Pi \in S_{BK}$7, SurF encodes each inter-event interval as

$\Pi \in S_{BK}$8

where $\Pi \in S_{BK}$9 is a parameterized cumulative intensity function, invertible under guaranteed monotonicity. Remixing is performed in noise space, where multiple event streams’ $\widetilde X_1 = \Pi \bar X,$0 sequences are subject to stochastic or deterministic transformations—linear interpolation, shuffling, or cross-fading—yielding new latent representations. Decoding employs safeguarded Newton steps for invertibility: $\widetilde X_1 = \Pi \bar X,$1 This framework supports diverse remixing operations, including partial prefix conditioning, stream merging, and handling of censored intervals. Zero-shot remix transfer is enabled by universality of the Exp(1) mapping (Rezaei et al., 13 May 2026).

6. Implementation and Efficiency

SURF and SurF implement highly efficient generative remixing:

• For source separation, all batched operations—permutation, summation, and flow path interpolation—are parallelizable.
• In event streams, batching is leveraged for all inter-event $\widetilde X_1 = \Pi \bar X,$2 and Gauss–Legendre quadrature computations.
• SurF-MoE and CSB models operate in closed form; SurF-GLQ requires $\widetilde X_1 = \Pi \bar X,$3 per event for $\widetilde X_1 = \Pi \bar X,$4, with negligible error.

Both systems guarantee invertibility and stability by enforcing positive intensity lower bounds ($\widetilde X_1 = \Pi \bar X,$5), with negligible statistical bias for practical $\widetilde X_1 = \Pi \bar X,$6. The design supports multi-dataset and zero-shot remixing due to the canonical noise domain (Li et al., 3 Jun 2026, Rezaei et al., 13 May 2026).

7. Summary and Significance

Generative remixing in SURF establishes a protocol for unsupervised generative modeling that is agnostic to ground-truth sources or event labels. By leveraging invertible transformations in either sample or latent-noise spaces, SURF and SurF realize state-of-the-art separation and time series synthesis with strong empirical robustness to domain shift. This framework enables creation of arbitrarily large, self-consistent pseudo-paired data, rigorous flow-based learning, and improved generalization over regression-based or supervised-only systems (Li et al., 3 Jun 2026, Rezaei et al., 13 May 2026). A plausible implication is that generative remixing will remain central in future data-limited, domain-heterogeneous generative modeling settings.