Conjecture: SIMS with a different auxiliary model

Determine whether Self-IMproving diffusion models with Synthetic data (SIMS) maintains similar self-improvement and MADness‑prophylactic performance when the auxiliary diffusion model differs from the base diffusion model but has comparable generation performance across the data domain; specifically, evaluate SIMS when Algorithm 1 employs two different state-of-the-art diffusion models for the base and auxiliary components and characterize conditions under which the negative‑guidance extrapolation remains effective.

Background

SIMS combines a base diffusion model trained on real data with an auxiliary diffusion model fine‑tuned on self‑generated synthetic data and uses a linear extrapolation of their score functions to provide negative guidance during generation.

In all experiments in the paper, the auxiliary model is obtained by fine‑tuning the base model, so both models share the same architecture. The authors conjecture that SIMS might work similarly even if the auxiliary model is architecturally different from the base model, provided both achieve similar performance over the data domain. Confirming this would broaden SIMS into a general negative‑guidance framework robust to model heterogeneity.

References

First, we conjecture that SIMS's performance might be similar if the auxiliary model differs from the base model but matches the base model's performance across the data domain (e.g., employ two different state-of-the-art diffusion models in Algorithm~\ref{alg:sims}). Confirming this could result in new negative-guidance-based training algorithms that are broad spectrum prophylactics against synthetic data from a range of different generative models.

— Self-Improving Diffusion Models with Synthetic Data  (2408.16333 - Alemohammad et al., 2024) in Discussion (Section 5)