SimDPS: Similarity-Guided Diffusion Sampling

• The paper introduces a novel hybrid approach that combines retrieval-based similarity guidance with diffusion posterior sampling for long-gap music inpainting.
• It employs contextual similarity matching and a modified likelihood to inject high-level musical structure into missing segments.
• Empirical evaluation on MAESTRO piano tracks shows that SimDPS enhances plausibility compared to standard diffusion methods, especially with moderately matching candidates.

Similarity-Guided Diffusion Posterior Sampling (SimDPS) is a hybrid long-gap music inpainting method that combines diffusion-based inference with similarity search. In the formulation introduced for piano music inpainting, an auxiliary candidate segment is first retrieved from a corpus using contextual similarity and is then incorporated into a modified likelihood so that diffusion posterior sampling is guided by both the observed context and the retrieved gap content (Turland et al., 19 Sep 2025). The method targets a regime in which standard diffusion posterior sampling is too weakly constrained for multi-second gaps, while pure retrieval is only reliable when a very good match exists; SimDPS is intended to use retrieval to inject high-level musical structure into the missing region while letting the diffusion model refine transitions and local waveform plausibility (Turland et al., 19 Sep 2025).

1. Problem setting and conceptual basis

SimDPS is formulated for music inpainting with an observed signal $\mathbf{y}\in\mathbb{R}^n$ that is a masked version of a clean signal $\mathbf{x}\in\mathbb{R}^n$: $$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$ where $\mathbf{M}$ is a diagonal binary mask and the missing region is a single compact interval (Turland et al., 19 Sep 2025). The inpainting difficulty is structural: $\mathbf{M}$ is non-injective, so the samples inside the gap lie in the null space $\mathrm{Ker}(\mathbf{M})$, and the observations contain no direct information there (Turland et al., 19 Sep 2025).

The paper’s motivation is specific to long-gap music reconstruction. Diffusion models can provide realistic local waveform structure, but for multi-second gaps the observed-only likelihood is too weak to constrain melody, rhythm, loudness trajectory, or phrase structure; pure retrieval, by contrast, can be musically coherent only when a very good match exists (Turland et al., 19 Sep 2025). SimDPS is therefore positioned between these two extremes. Relative to unguided diffusion posterior sampling, it adds stronger semantic structure inside the gap; relative to direct insertion of a retrieved segment, it retains a generative refinement stage that can smooth transitions and adapt the reconstruction to the observed boundaries (Turland et al., 19 Sep 2025).

This design inherits the posterior-sampling perspective of diffusion inverse solvers more generally. In the standard DPS framework, the reverse process combines a diffusion prior with a measurement-guidance term derived from the posterior score decomposition (Chung et al., 2022). SimDPS retains that logic, but alters the likelihood so that the missing region is constrained not only by the observed samples outside the gap but also by a retrieved auxiliary hypothesis inside the gap (Turland et al., 19 Sep 2025).

2. Retrieval stage and similarity construction

The retrieval component operates on a corpus

$$\mathcal{S}=\{\mathbf{s}_1,\mathbf{s}_2,\dots,\mathbf{s}_N\},$$

with each $\mathbf{s}_i\in\mathbb{R}^{L_i}$ and $L_i>n$ (Turland et al., 19 Sep 2025). In the reported experiments, the corpus is not an external music library but the remainder of the same track before and after the gap, so the method exploits within-track repetition rather than cross-track retrieval (Turland et al., 19 Sep 2025). A candidate segment is an excerpt $$\mathbf{s}_i(\hat t)=\mathbf{s}_i[\hat t:\hat t+n-1]$$ of the same total length $\mathbf{x}\in\mathbb{R}^n$0 as the inference window (Turland et al., 19 Sep 2025).

Candidate selection is based on contextual similarity rather than full-window waveform matching. The paper defines context windows around the left and right mask boundaries, selected and weighted by matrices $\mathbf{x}\in\mathbb{R}^n$1, and then minimizes

$\mathbf{x}\in\mathbb{R}^n$2

with

$\mathbf{x}\in\mathbb{R}^n$3

Here, $\mathbf{x}\in\mathbb{R}^n$4 are feature maps, $\mathbf{x}\in\mathbb{R}^n$5 are metric-space distances, and $\mathbf{x}\in\mathbb{R}^n$6 balance the features (Turland et al., 19 Sep 2025).

In the experiments, $\mathbf{x}\in\mathbb{R}^n$7: STFT features are used for short-range timbral matching, and chromagram features are used for longer-range melodic and rhythmic matching (Turland et al., 19 Sep 2025). The maximum context length is $\mathbf{x}\in\mathbb{R}^n$8 s, retrieval audio is resampled to 12 kHz for efficiency, and both STFT and chromagram use Hann windows with hop size 256, window length 1024, and FFT size 1024 (Turland et al., 19 Sep 2025). The paper also specifies $\mathbf{x}\in\mathbb{R}^n$9 (Turland et al., 19 Sep 2025).

The search itself is two-stage. First, a coarse scan is performed over candidate start times with hop $$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$0. Second, local refinement is done using waveform continuity at the boundaries: $$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$1 with $$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$2, after which the final guide is

$$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$3

Only the top-ranked candidate is used; the paper does not use multiple candidates, reranking ensembles, or weighted mixtures (Turland et al., 19 Sep 2025).

3. Modified posterior and similarity-guided likelihood

The diffusion backbone follows score-based posterior sampling. Using Tweedie’s formula, the denoised estimate at time $$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$4 is

$$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$5

and standard DPS uses a likelihood term defined only on the observed region (Turland et al., 19 Sep 2025). SimDPS changes the probabilistic model by treating the retrieved guide as an auxiliary noisy observation inside the gap.

The retrieved segment is interpreted as a noisy estimate of the missing samples: $$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$6 while the observed samples remain governed by the original observation outside the mask (Turland et al., 19 Sep 2025). This leads to the synthetic observation

$$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$7

Equivalently, the model can be written as

$$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$8

with diagonal covariance entries

$$\mathbf{y}=\mathbf{M}\mathbf{x}+\boldsymbol{\varepsilon}_y,$$9

The observed region and the retrieved gap thus enter as two noisy measurements with different uncertainties (Turland et al., 19 Sep 2025).

The resulting modified likelihood score is a sum of two gradient terms: $\mathbf{M}$0 Standard DPS is recovered when $\mathbf{M}$1, since the auxiliary gap term is then disabled (Turland et al., 19 Sep 2025).

The guidance strengths are controlled by $\mathbf{M}$2 and $\mathbf{M}$3. The paper states that $\mathbf{M}$4 and $\mathbf{M}$5 are set from normalized gradient magnitudes and controlled by these scalar weights (Turland et al., 19 Sep 2025). Two variants are reported: SimDPS-l with $\mathbf{M}$6 and SimDPS-h with $\mathbf{M}$7, while $\mathbf{M}$8 is fixed across methods (Turland et al., 19 Sep 2025).

This construction makes the role of “similarity guidance” precise. The retrieved segment does not replace the gap by hard copying; rather, it modifies the posterior through a non-uniform synthetic likelihood whose uncertainty is lower or higher depending on the chosen auxiliary guidance weight (Turland et al., 19 Sep 2025).

4. Inference procedure and implementation profile

Inference proceeds in six stages. First, a corrupted waveform excerpt is defined, with a central missing interval in the reported listening-test setup (Turland et al., 19 Sep 2025). Second, the retrieval stage computes the single auxiliary guide $\mathbf{M}$9 from the same track’s surrounding material (Turland et al., 19 Sep 2025). Third, the synthetic observation $\mathbf{M}$0 is formed. Fourth, diffusion sampling is initialized from the largest noise level. Fifth, reverse diffusion is performed with the modified posterior gradient. Sixth, the final time-domain waveform is returned (Turland et al., 19 Sep 2025).

The diffusion model itself is score-based, with the Karras et al. parameterization $\mathbf{M}$1, and uses the MR-CQTdiff backbone; although the architecture leverages multi-resolution CQT internally, diffusion operates in the time domain (Turland et al., 19 Sep 2025). The paper reports $\mathbf{M}$2 inference steps, $\mathbf{M}$3, $\mathbf{M}$4, logarithmic time discretization, and the second-order stochastic sampler of Karras et al. with $\mathbf{M}$5 (Turland et al., 19 Sep 2025).

The experimental domain is narrowly specified. Audio is mono at 44.1 kHz, the training and evaluation material comes from MAESTRO, the excerpt length is 6 s, and the inpainted gap is 2 s (Turland et al., 19 Sep 2025). The backbone score model is trained for 180k iterations on the full MAESTRO training split and then 320k iterations on higher-quality recordings from 2017–2018, with a reported total training time of about two days on a single NVIDIA H200 GPU (Turland et al., 19 Sep 2025).

Several implementation features are consequential. Retrieval runs on 12 kHz audio for efficiency, whereas generation is full-rate (Turland et al., 19 Sep 2025). Gradients for the posterior update are obtained by automatic differentiation through the denoiser and the likelihood terms (Turland et al., 19 Sep 2025). The model is convolutional and can generalize to longer sequences than the 6 s training length (Turland et al., 19 Sep 2025). At the same time, the paper does not report full pseudocode, exact retrieval hop $\mathbf{M}$6, batch size, or detailed optimizer settings, which leaves part of the method underspecified for strict reproduction (Turland et al., 19 Sep 2025).

5. Empirical findings in long-gap piano inpainting

Evaluation is primarily subjective. The paper uses a webMUSHRA listening test with 6 excerpts, each 6 s long and containing a 2 s central gap, rated by 17 volunteers of whom 1 was excluded; listeners scored plausibility on a 0–100 scale (Turland et al., 19 Sep 2025). The compared conditions are Original, LPC, DPS, Sim, SimDPS-h, and SimDPS-l (Turland et al., 19 Sep 2025).

The broad empirical pattern is consistent with the method’s stated motivation. Unguided DPS was “generally rated in the lower part of the scale,” indicating that standard diffusion posterior sampling is insufficiently constrained for long-gap musical reconstruction (Turland et al., 19 Sep 2025). SimDPS-l achieved the highest mean plausibility score across all excerpts (Turland et al., 19 Sep 2025).

Performance, however, depends strongly on candidate quality. When the retrieved segment was already highly coherent, pure similarity search could be excellent: in excerpt S1, Sim was excellent and SimDPS-l achieved equivalent ratings (Turland et al., 19 Sep 2025). The more characteristic regime is moderate candidate quality. In S2, SimDPS-l was significantly better than Sim with $\mathbf{M}$7; in S3, it again significantly outperformed Sim with $\mathbf{M}$8; and in S4, it was marginally better, with $\mathbf{M}$9 (Turland et al., 19 Sep 2025). In poor-candidate cases, the hybrid offered no consistent advantage over retrieval alone (Turland et al., 19 Sep 2025). An exception occurred in S6, where DPS and SimDPS-h outperformed both Sim and SimDPS-l, though all methods were still weak there (Turland et al., 19 Sep 2025).

These results support a narrow but important claim: SimDPS is most useful when a candidate is informative but imperfect. In that regime, retrieval contributes phrase-level or meso-scale structure, while diffusion refines the result into a more seamless waveform consistent with the actual left and right context (Turland et al., 19 Sep 2025).

6. Interpretation, relation to DPS, and limitations

SimDPS is structurally a modified-likelihood DPS method. In standard DPS, the reverse process uses a pretrained diffusion prior together with a likelihood surrogate evaluated at the denoised estimate $\mathrm{Ker}(\mathbf{M})$0, thereby guiding the reverse chain toward measurement consistency (Chung et al., 2022). SimDPS preserves that scaffold but changes the observation model so that the gap contains an auxiliary noisy measurement derived from retrieval (Turland et al., 19 Sep 2025). In that sense, its “similarity guidance” is not an extra classifier-like reward added outside the posterior; it is built directly into the posterior geometry through the synthetic likelihood.

This positioning matters because the broader DPS literature has raised doubts about whether such methods should be interpreted as faithful posterior samplers. One study argues that standard DPS behaves more like posterior maximization than conditional-score estimation, emphasizing that DPS generates high-quality samples with significantly lower diversity (Xu et al., 31 Jan 2025). Another study proposes a benchmark in which good reconstructions do not imply correct posterior calibration, and reports that existing DPS algorithms can be miscalibrated (Zach et al., 16 Sep 2025). A plausible implication is that SimDPS is best understood primarily as a hybrid guided inpainting method rather than as a demonstrated sampler of the full posterior.

The limitation profile in the SimDPS paper is correspondingly narrow. The experiments are only on piano music from MAESTRO, retrieval is performed within the same track rather than from a broad external corpus, and no objective semantic metric is introduced (Turland et al., 19 Sep 2025). The paper does not test arbitrary genres or instruments, and it does not establish that the modified likelihood yields calibrated posterior samples (Turland et al., 19 Sep 2025). Its own results show strong dependence on candidate quality: if the candidate is excellent, direct similarity insertion can already be competitive, whereas if the candidate is poor, the hybrid gives little consistent benefit (Turland et al., 19 Sep 2025).

At a more general level, posterior sampling with diffusion priors is computationally intractable in the fully general setting, even when unconditional diffusion sampling is fast (Gupta et al., 2024). That worst-case result does not negate the empirical utility of SimDPS on structured music inpainting, but it does frame the method as a task-specific hybrid posterior heuristic rather than a general, provably correct posterior sampler. Within that narrower role, SimDPS provides a concrete example of how retrieval-based similarity can be incorporated into diffusion posterior sampling by treating the retrieved content as a second, uncertainty-weighted measurement in the missing region (Turland et al., 19 Sep 2025).