Dual-branch Score Distillation Sampling (SDS)

• The paper introduces a dual-branch approach that separates the diffusion gradient into a reconstruction (consistency) branch and a text prompt branch, improving stability and identity preservation in 3D synthesis.
• It leverages methodologies like DDIM inversion and Tweedie’s estimate to balance positive and negative guidance, resulting in higher CLIP scores and better user preferences compared to prior techniques.
• The dual-branch formulation unifies 3D generation and editing by factorizing the optimization signal into interpretable components, enabling precise control over appearance and geometry in tasks such as NeRF inpainting.

Searching arXiv for the cited papers and topic-related context. arxiv_search(query="Dual-branch Score Distillation Sampling UDS BSD 3D editing generation inpainting", max_results=10) arxiv_search(query="(Miao et al., 3 May 2025)", max_results=5) Dual-branch Score Distillation Sampling (SDS) denotes a family of formulations in which the guidance used to optimize a 3D generator under a frozen 2D diffusion prior is decomposed into two complementary terms rather than treated as a single monolithic score. In one line of work, the decomposition is internal to SDS itself: a reconstruction or consistency branch is separated from a text or prompt branch, and this view is used to unify 3D generation and 3D editing through Unified Distillation Sampling (UDS) (Miao et al., 3 May 2025). In another line of work, dual-branch guidance refers to two parallel modalities—appearance and geometry—optimized with Balanced Score Distillation (BSD) for NeRF inpainting (Zhang et al., 2024). Across these formulations, the central idea is that the optimization signal from diffusion can be factorized into more interpretable components, and that this factorization can reduce instability, improve identity preservation or geometric consistency, and better align 2D priors with 3D objectives.

1. Definition and scope

In the UDS formulation, “dual-branch SDS” is a decomposition of the practical SDS gradient into a consistency or reconstruction branch and a text or prompt branch. Let $x_0$ be a clean image latent, let $x_t$ be its noisy version under a DDPM-style schedule,

$$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$

and let $\epsilon_\phi(x_t,t,y)$ be a pre-trained noise-prediction UNet trained by denoising score matching. For text-to-3D distillation, a differentiable 3D representation $g(\theta,c)$ renders a view $x := g(\theta,c)$, and SDS optimizes $\theta$ through

$$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$

Ignoring the UNet Jacobian, the practical gradient is

$$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$

The key observation is that, under classifier-free guidance (CFG), this update can be decomposed into two interpretable branches (Miao et al., 3 May 2025).

In GB-NeRF, “dual-branch” has a different but related meaning. The same score-distillation principle is applied to two rendered modalities of the same NeRF generator: RGB appearance and surface normals. BSD then supplies a geometry-aware guidance rule that removes the unconditional term and balances positive and negative conditional prompts in both branches (Zhang et al., 2024).

A common misconception is that dual-branch SDS names a single standardized algorithm. The literature summarized here indicates two distinct but compatible uses of the phrase: branch decomposition within SDS guidance itself, and branch decomposition across multiple supervision modalities.

2. Decomposition of SDS into reconstruction and prompt branches

The UDS paper defines

$\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$

With CFG weight $x_t$0, this becomes

$x_t$1

This yields two branches:

• reconstruction branch:

$x_t$2

• classifier branch:

$x_t$3

This decomposition clarifies that vanilla SDS is driven simultaneously by unconditional denoising consistency and by text-conditional steering. The first term stabilizes the optimization by tying the rendered sample to the unconditional denoiser; the second pushes the sample toward the conditional manifold specified by the prompt (Miao et al., 3 May 2025).

The same paper argues that several editing methods can be rewritten in this two-branch form. For Delta Denoising Score (DDS), the paper gives

$x_t$4

and rewrites its guidance as

$x_t$5

Posterior Distillation Sampling (PDS) introduces an explicit latent-matching term. Using Tweedie’s formula,

$x_t$6

the paper re-expresses PDS in simplified form as

$x_t$7

The paper identifies the term $x_t$8 as crucial for identity preservation in 3D editing.

This suggests that dual-branch SDS is not merely an analytical convenience. It is used as a unifying lens through which generation-oriented and editing-oriented score-distillation methods can be compared.

3. Unified Distillation Sampling as a dual-branch generalization

UDS replaces the reconstruction branch with differences in clean-latent predictions $x_t$9 and retains CFG as the text branch. The resulting unified update is

$$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$0

with gradient

$$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$1

The task dependence enters only through the $$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$2 terms (Miao et al., 3 May 2025).

For editing,

$$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$3

and

$$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$4

Hence

$$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$5

For generation,

$$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$6

and

$$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$7

optionally with negative CFG:

$$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$8

The paper gives two approximations for $$x_t = \sqrt{\bar{\alpha}_t}\,x_0 + \sqrt{1-\bar{\alpha}_t}\,\epsilon,\qquad \epsilon \sim \mathcal{N}(0,I),$$9. A single-step Tweedie estimate is

$\epsilon_\phi(x_t,t,y)$0

while a multi-step unconditional DDIM inverse computes

$\epsilon_\phi(x_t,t,y)$1

iterated to $\epsilon_\phi(x_t,t,y)$2 for a higher-fidelity $\epsilon_\phi(x_t,t,y)$3.

The theoretical claim advanced by the paper is that generation and editing differ only in what “consistency” means. In editing, consistency is identity preservation between source and target clean latents; in generation, consistency is temporal coherence between nearby denoising states. A plausible implication is that the dual-branch view shifts the emphasis from handcrafting separate objectives toward specifying the appropriate notion of latent consistency.

4. Dual-branch guidance in geometry-aware NeRF inpainting

GB-NeRF formulates NeRF inpainting as optimization of a differentiable generator under 2D diffusion priors and introduces Balanced Score Distillation (BSD), which also adopts a dual-branch structure, but here the branches are appearance RGB and geometry normals (Zhang et al., 2024). The overall objective is

$\epsilon_\phi(x_t,t,y)$4

For the appearance branch, an RGB rendering $\epsilon_\phi(x_t,t,y)$5 is encoded as latent $\epsilon_\phi(x_t,t,y)$6, then noised as

$\epsilon_\phi(x_t,t,y)$7

BSD removes the unconditional term and balances positive and negative prompts:

$\epsilon_\phi(x_t,t,y)$8

The gradient is

$\epsilon_\phi(x_t,t,y)$9

For the geometry branch, a normal map $g(\theta,c)$0 is encoded as $g(\theta,c)$1, and

$g(\theta,c)$2

The corresponding BSD direction is

$g(\theta,c)$3

with

$g(\theta,c)$4

The paper contrasts BSD with SDS and CSD. In its notation,

$g(\theta,c)$5

while CFG gives

$g(\theta,c)$6

GB-NeRF analyzes a CSD form

$g(\theta,c)$7

and reports that the unconditional prediction term $g(\theta,c)$8 introduces high variability: positive $g(\theta,c)$9 blurs reconstructions, negative $x := g(\theta,c)$0 causes artifacts, and best results arise near $x := g(\theta,c)$1. BSD therefore eliminates the unconditional term entirely.

This use of dual branches differs from UDS. In UDS, the two branches are a reconstruction or consistency term and a text term; in BSD, the two branches are appearance and geometry, each using the same positive-versus-negative conditional balancing principle. The commonality is the attempt to make score distillation more structured and less stochastic.

5. Optimization procedures and implementation regimes

In UDS, the paper gives explicit per-iteration procedures for editing and generation (Miao et al., 3 May 2025). For editing, a camera $x := g(\theta,c)$2 is sampled, the current target view $x := g(\theta,c)$3 is rendered, and a source view $x := g(\theta,c)$4 is prepared. Noise is added to both latents with the same $x := g(\theta,c)$5,

$x := g(\theta,c)$6

after which unconditional and conditional predictions are evaluated for target and source, $x := g(\theta,c)$7 is approximated by Tweedie or DDIM inverse, and the gradient

$x := g(\theta,c)$8

is used to update $x := g(\theta,c)$9.

For generation, the procedure samples camera, timestep, and noise; forms $\theta$0 from the rendered view; evaluates $\theta$1 and $\theta$2; optionally constructs $\theta$3 and $\theta$4; approximates $\theta$5 and $\theta$6; then applies the same UDS template with generation-specific $\theta$7 terms. The paper states that UDS is mask-free by default, though localized edits can be implemented by restricting the image-space gradient to a region of interest.

The implementation details reported for UDS are specific. Stable Diffusion 2.1 is used for 3D generation, Stable Diffusion 1.5 for SVG editing, and both NeRF and 3D Gaussian Splatting are supported. The paper lists Threestudio and DreamFusion-style volumetric radiance fields for NeRF, LucidDreamer-style 3D Gaussian Splatting, random or stratified camera sampling, timestep sampling $\theta$8, stride $\theta$9 for generation in the range $$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$0–$$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$1, and guidance weight $$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$2. All reported experiments used a single NVIDIA 3090 GPU.

GB-NeRF likewise specifies a concrete optimization pipeline (Zhang et al., 2024). NeRF maps $$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$3, with volumetric rendering

$$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$4

For unmasked regions, the method uses

$$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$5

and optionally

$$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$6

For masked regions, it encodes rendered RGB and normals through the Stable Diffusion VAE, applies BSD only within the NeRF mask $$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$7, and uses

$$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$8

$$L_{\mathrm{SDS}}(\theta) := \mathbb{E}_{t,c,\epsilon}\big[\omega(t)\|\epsilon_\phi(x_t,t,y)-\epsilon\|_2^2\big].$$9

The final loss is

$$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$0

with $$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$1 and $$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$2.

The paper also specifies a fine-tuned Stable Diffusion teacher with LoRA adapters inserted into both U-Net and text encoder, rank $$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$3, trained on DIODE RGB–normal pairs. BLIP captions from RGB are reused for normals, each caption prepended with a modality token, “RGB image” or “normal map.” Training uses 10,000 iterations, Adam with learning rate $$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$4, a single NVIDIA A100, latent size $$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$5, and timestep sampling $$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$6. BSD scales are $$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$7 for appearance and $$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$8 for geometry.

6. Empirical behavior, comparisons, and limitations

The UDS paper reports that, in 3D editing on a NeRF-based benchmark of 8 scenes and 37 prompt pairs, UDS achieves CLIP $$\nabla_\theta L_{\mathrm{SDS}} = \mathbb{E}_{t,\epsilon,c} \Big[ \omega(t)\big(\epsilon_\phi(x_t,t,y)-\epsilon\big)\frac{\partial g(\theta,c)}{\partial \theta} \Big].$$9 and user preference $\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$0, outperforming IN2N ($\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$1, $\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$2), DDS ($\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$3, $\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$4), and PDS ($\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$5, $\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$6). For 3D generation with Stable Diffusion 2.1 on NeRF and 3D Gaussian Splatting under a single 3090 GPU, UDS reports higher CLIP and user preference than DreamFusion, Fantasia3D, and ProlificDreamer, and reaches CLIP up to $\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$7 and user preference $\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$8 relative to LucidDreamer SDS and ISM baselines. For SVG editing with Stable Diffusion 1.5, it attains LPIPS $\delta_{x_t}:=\epsilon_\phi(x_t,t,y)-\epsilon.$9, CLIP $x_t$00, and user preference $x_t$01 (Miao et al., 3 May 2025).

The same paper attributes part of this behavior to the choice of reconstruction branch. Using DDIM inversion for $x_t$02 preserves identity better than single-step Tweedie in editing, while Tweedie may reflect text edits more aggressively but risks identity drift. In generation, adding DDIM reverse-process noise improves quality but increases compute and resource cost. The paper also states that UDS shows lower variability and more stable gradient norms than SDS, DDS, PDS, and ISM in 3D.

GB-NeRF reports improvements on SPIn-NeRF and LLFF. On SPIn-NeRF, it reports FID $x_t$03 versus $x_t$04 for MVIP-NeRF, D-FID $x_t$05 versus $x_t$06, D-PSNR $x_t$07 versus $x_t$08, SSIM $x_t$09 versus $x_t$10, NIMA $x_t$11 versus $x_t$12, and BRISQUE $x_t$13 versus $x_t$14. On LLFF, it reports FID $x_t$15 versus approximately $x_t$16 for SDS baselines, NIMA $x_t$17, and BRISQUE $x_t$18. Ablations state that BSD alone reduces FID to $x_t$19 versus $x_t$20 for the origin, while LoRA fine-tuning significantly lowers D-FID to $x_t$21 versus $x_t$22 and improves BRISQUE and normal/detail reconstruction (Zhang et al., 2024).

The limitations described in the two papers are also consistent with a dual-branch view. UDS identifies failure modes for large semantic gaps, an identity-versus-prompt trade-off controlled by $x_t$23, sensitivity to timestep stride $x_t$24, DDIM inversion overhead, and residual risk of oversaturation or color artifacts under poorly tuned $x_t$25 or negative guidance. GB-NeRF reports increased training time from the fine-tuned teacher and two-branch setup, sensitivity of hyperparameters $x_t$26 to dataset choice, inability to remove shadows reliably, and oversmoothing when the geometry branch is over-regularized. Taken together, these results suggest that dual-branch SDS improves control and stability, but does not remove the underlying dependence on diffusion priors, guidance schedules, and 3D initialization quality.