Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolants

Abstract: Transport-based methods have emerged as a leading paradigm for building generative models from large, clean datasets. However, in many scientific and engineering domains, clean data are often unavailable: instead, we only observe measurements corrupted through a noisy, ill-conditioned channel. A generative model for the original data thus requires solving an inverse problem at the level of distributions. In this work, we introduce a novel approach to this task based on Stochastic Interpolants: we iteratively update a transport map between corrupted and clean data samples using only access to the corrupted dataset as well as black box access to the corruption channel. Under appropriate conditions, this iterative procedure converges towards a self-consistent transport map that effectively inverts the corruption channel, thus enabling a generative model for the clean data. We refer to the resulting method as the self-consistent stochastic interpolant (SCSI). It (i) is computationally efficient compared to variational alternatives, (ii) highly flexible, handling arbitrary nonlinear forward models with only black-box access, and (iii) enjoys theoretical guarantees. We demonstrate superior performance on inverse problems in natural image processing and scientific reconstruction, and establish convergence guarantees of the scheme under appropriate assumptions.

• The paper introduces a Self-Consistent Stochastic Interpolants (SCSI) method that recovers clean data from corrupted samples using only black-box simulators.
• It reformulates inverse generative modeling as a fixed-point problem by iteratively updating neural transport maps to enforce distributional self-consistency.
• Empirical results on synthetic, image, and astronomical tasks demonstrate SCSI's competitive restoration performance and accelerated convergence compared to baselines.

Generative Modeling from Black-Box Corruptions via Self-Consistent Stochastic Interpolants

Problem Formulation and Motivations

The paper addresses the fundamental challenge of learning generative models when only corrupted measurements are available, which is typical in many scientific and engineering scenarios where the forward measurement process is noisy, ill-conditioned, and often non-invertible. Standard transport-based generative approaches, such as diffusion models and normalizing flows, require direct access to samples from the clean underlying data distribution, which is unrealistic in such settings.

Formally, the scenario involves an inaccessible clean data distribution $\pi$ and a black-box corruption operator $\mathcal{F}$ inducing a pushforward distribution $\mu = \mathcal{K}\pi$ observed through samples $y \sim \mu$. The operator $\mathcal{F}$ can be highly nonlinear, non-differentiable, and even stochastic, precluding direct likelihood-based or gradient-based inversion schemes. The central task is therefore to construct a generative model for the clean data $\pi$, using as resources only: (1) samples from the corrupted distribution $\mu$; and (2) the ability to produce new corrupted samples via a black-box simulator of $\mathcal{F}$. No knowledge of the analytic form or gradients of the corruption channel is assumed.

Methodology: Self-Consistent Stochastic Interpolants (SCSI)

The proposed approach reformulates the inverse generative modeling task as a fixed-point problem at the distributional level. The core conceptual tool is the Stochastic Interpolant (SI) framework, which allows for constructing transport maps between probability distributions by learning velocity (and optionally, score) fields that interpolate between two distributions via forward and reverse diffusion or flow processes.

The Self-Consistent Stochastic Interpolant (SCSI) method iteratively updates a transport map $\Phi_\Theta$ (parameterized by neural networks) via the following outer- and inner-loop structure:

• Outer iteration: For current parameters $\Theta^{(k)}$, define the empirical prior as $\pi_{\Theta^{(k)}} = (\Phi_{\Theta^{(k)}})_\#\mu$.
• Inner loop: Construct the SI between $\pi_{\Theta^{(k)}}$ (pseudo-clean samples) and $\mathcal{K}\pi_{\Theta^{(k)}}$ (corrupted pseudo-samples), and update $\Theta$ to minimize the SI loss so that the transport map, when applied to $\mu$, leads to a distribution whose pushforward through $\mathcal{F}$ matches $\mu$ (self-consistency).
• This process enforces distributional self-consistency: at the fixed point, $\mathcal{K}\pi_{\Theta^*} = \mu$, and under injectivity of $\mathcal{K}$, convergence to the ground-truth $\pi$ is guaranteed.

Crucially, this process requires no access to clean samples, no backpropagation through the corruption channel, and handles settings with only black-box forward operators.

Theoretical Guarantees

The authors provide a rigorous mathematical analysis of the convergence properties of the SCSI scheme:

• Distributional Injectivity: If the observation operator $\mathcal{K}$ is injective at the distributional level, the only fixed point of the SCSI iteration is the ground-truth $\pi$.
• Metric Contraction Properties: Under suitable regularity conditions and a Lipschitz stability assumption on the SI mappings, the procedure enjoys:
  • Linear contraction in Wasserstein distance when the reverse transport is Lipschitz (with contraction rate parameter $R < 1$), yielding $$W_2^2(\pi, \pi^{(k)}) \leq R^k W_2^2(\pi, \pi^{(0)})$$.
  • Linear contraction in KL-divergence, with a rate controlled by the SI Lipschitz constant and the condition number of the channel $\chi$, under mild regularity of the SI parameterizations.
• Fokker-Planck Channel Specialization: For channels expressible as diffusion processes (e.g., AWGN), the contraction rate can be strictly quantified, and exponential convergence can be established for all SNR regimes.

These results are substantiated by both abstract analysis and explicit closed-form calculations in the Gaussian/AWGN setting.

Figure 2: Convergence of $|| \Sigma - \Sigma_k ||^2$ for the SCSI ODE (red) and EM algorithm (green) in Gaussian/AWGN, empirically highlighting the accelerated, quadratically-convergent rate of the ODE approach.

Empirical Evaluation

The paper presents extensive empirical evaluations on both synthetic and real-data tasks involving a diversity of nonlinear, non-invertible, and non-Gaussian corruption channels:

• Synthetic Benchmarks: The ODE-based SCSI yields faster and more stable convergence in moderate-noise settings, while SDE-based variants offer greater robustness under extreme corruption but are less stable and more hyperparameter sensitive.
• Imaging Tasks (CIFAR-10, CelebA): SCSI is applied to challenging forward models, including:
  • Random masking with Gaussian noise,
  • Gaussian blur and additive (and Poisson) noise,
  • Motion blur (nonlinear operator),
  • JPEG compression (non-differentiable, nonlinear corruption).

SCSI consistently demonstrates sample restoration performance competitive with or exceeding strong baselines—including Diffusion Posterior Sampling (DPS) (which requires clean pretraining and forward gradients), and oracle SI models (trained with clean/corrupted pairs). Notably, SCSI achieves strong LPIPS and FID metrics using architectures with fewer parameters and significantly reduced computational resource requirements.

Figure 1: Example restoration of severely JPEG-compressed, noisy images using SCSI. Top: corrupted samples, Bottom: SCSI-restored samples.

Figure 5: Random samples drawn from the large diffusion model retrained on SCSI-restored data (CIFAR-10), demonstrating high generative fidelity.

Figure 7: Restoration under random masking ($25\%$ pixels masked); SCSI successfully infers plausible clean images despite massive information loss.

• Physical Science Task (Quasar Spectra Recovery): SCSI is deployed on real astronomical spectra measured under complex, unmodeled noise, calibration uncertainty, and spectral resolution variation. Compared to Wiener filtering, SCSI significantly improves the recovery of physically salient spectral features relevant for scientific inference.

Implications and Limitations

Practical Impact:

• SCSI provides a scalable, modular, and black-box-compatible solution for generative modeling under corrupted data. It does not require explicit analytic access to the corruption process, enabling its application to scientific and industrial data acquisition chains with only simulators or experimental access.
• Empirical results evidence performance competitive with or superior to both model-based and (semi-)supervised baselines, with dramatic computational efficiency gains.

Theoretical Significance:

• The SCSI framework clarifies and extends the class of inverse problems in which distribution-level generative modeling is solvable—even when corruption is stochastic, nonlinear, or non-differentiable—by harnessing the representation power of transport-based models and iterative self-consistency.
• The analysis further highlights a nontrivial phenomenon: in some regimes (notably ODE-based, noise-free transport for Gaussian/AWGN channels), marginal transport (SCSI) can yield faster convergence rates than posterior-based EM or MLE-type schemes.

Limitations and Future Directions:

• The necessity of injectivity at the distributional (not sample) level circumscribes the set of applicable corruption channels, although this still includes many scientifically relevant cases such as AWGN, tomographic projection, and randomized inpainting.
• The practical effectiveness of SDE-based SIs is highly contingent on hyperparameter tuning; ODE-based models generally offer more stability but can underperform in extreme-noise situations.
• Extensions to settings with mixed clean/corrupted supervision or discrete data have natural paths within the SCSI formalism, tying into very recent advances in stochastic interpolant theory.
• Comparative exploration of marginal vs. conditional transport (EM vs. SCSI) merits further investigation, particularly regarding trade-offs between convergence rates and robustness.

Conclusion

This work introduces a theoretically justified and empirically validated framework for generative modeling from corrupted observations, requiring only black-box access to the corruption process. The SCSI algorithm combines the expressivity of stochastic interpolants with a scalable, iterative, self-consistent training scheme, achieving provable convergence to the true data distribution under broad conditions. Empirical results across synthetic, image, and scientific data domains demonstrate that SCSI is competitive with or outperforms baselines needing stronger supervision or analytic access. The results suggest new directions in both the theory of inverse problems and the practice of generative learning under severe data corruption.