---
title: HFS-SDEdit Texture Refinement
url: https://www.emergentmind.com/topics/hfs-sdedit
type: topic
---

# HFS-SDEdit Texture Refinement

HFS-SDEdit (High-Frequency-Swapping SDEdit) is a texture enhancement method designed to refine low-quality 3D asset textures in modern score-based diffusion frameworks. Developed as the core of the Elevate3D pipeline, HFS-SDEdit directly addresses the fidelity–quality trade-off inherent in classical diffusion-based editing, providing state-of-the-art refinement capabilities for both 2D images and 3D model textures while preserving crucial high-frequency structural detail [2507.11465].

## 1. Theoretical Foundation: Score-Based Diffusion Models

HFS-SDEdit operates within the score-based diffusion model framework established by Song & Ermon. The generative process is modeled as a continuous-time stochastic differential equation (SDE) transforming a clean image $x_0$ into Gaussian noise via

$$
d\mathbf{x}_t = f(\mathbf{x}_t, t)\,dt + g(t)\,d\mathbf{w}_t,
$$

where the drift $f(\mathbf{x}, t) = -\tfrac12 \beta(t)\mathbf{x}$ and the diffusion $g(t) = \sqrt{\beta(t)}$, with $\beta(t)$ as the variance schedule and $\mathbf{w}_t$ standard Brownian motion. Sampling is achieved via the reverse-time SDE,

$$
d\mathbf{x}_t = \left[f(\mathbf{x}_t, t) - g(t)^2\nabla_\mathbf{x} \log p_t(\mathbf{x}_t)\right] dt + g(t)\, d\bar{\mathbf{w}}_t,
$$

with the score $\nabla_\mathbf{x} \log p_t(\mathbf{x})$ approximated by a network $\mathbf{s}_\theta$. In discrete terms (as in DDPM), the denoising update is

$$
\mathbf{x}_{t-1} = \frac{1}{\alpha_t}\left(\mathbf{x}_t - \frac{\beta_t}{\sqrt{1-\bar\alpha_t}}\mathbf{s}_\theta(\mathbf{x}_t, t)\right) + \sigma_t\mathbf{z},
$$

where $\alpha_t, \bar\alpha_t$ are cumulative-product schedule parameters and $\mathbf{z} \sim \mathcal N(\mathbf{0}, \mathbf{I})$ [2507.11465].

## 2. Reference-Guided SDEdit and Its Limitations

SDEdit provides a training-free anchoring of the diffusion process to a reference image $z_r$. The user chooses a noise-level $t_s$, controlling the trade-off between output quality and fidelity to $z_r$. The process involves:

- Forming the noised latent $z_{t_s} = \alpha(t_s) z_r + \beta(t_s)\epsilon$, with $\epsilon \sim \mathcal N(\mathbf{0}, \mathbf{I})$.
- Running reverse diffusion from $t = t_s$ down to $t = 0$.

Lower $t_s$ preserves the fidelity of $z_r$ but does not substantially improve perceptual quality, while higher $t_s$ removes low-frequency domain artifacts but diminishes alignment with the input. This coupling between fidelity and quality is a primary limitation of classical SDEdit [2507.11465].

## 3. High-Frequency-Swapping Mechanism

HFS-SDEdit introduces high-frequency injection during reverse diffusion to decouple fidelity from perceptual quality enhancement. The core insight is that low-frequency bands encode domain appearance, whereas high-frequency components represent perceptual edge and fine structure details.

The procedure is as follows:

- **Initialization**: At the chosen $t_s$, generate $\tilde{z}_{t_s} = \alpha(t_s) z_r + \beta(t_s)\epsilon$.
- **Iterative Denoising and Swapping** (for $t = t_s, t_s-1, ..., t_\mathrm{stop}+1$):
  1. Denoise: $\tilde{z}_t \rightarrow \hat{z}_{t-1}$ (reverse-DDPM).
  2. Freshly re-noise the reference: $\tilde{z}_t = \alpha(t) z_r + \beta(t)\epsilon$.
  3. **High-Frequency Swap**: Update latent via
     $$
     z_t' = (\delta - G_\sigma) * \tilde{z}_t + G_\sigma * \hat{z}_t,
     $$
     where $G_\sigma$ is a Gaussian low-pass filter and $*$ denotes convolution; high frequencies from $\tilde{z}_t$ (the reference) replace those of $\hat{z}_t$ (the diffused latent).
  4. **Mask Blending** (optional): For partial refinement, blend via
     $$
     \hat{z}_t = \tilde{m} \odot z_t' + (1 - \tilde{m}) \odot \tilde{z}_t,
     $$
     with $\tilde{m}$ as the refinement mask.
  5. Use $\hat{z}_t$ as the latent for the next step.
- **Final Unaltered Decoding**: For $t_\mathrm{stop}$, continue with vanilla reverse diffusion.

This mechanism ensures that the final sample $z_0$ conforms to the diffusion model distribution in the low frequencies and perfectly aligns high-frequency details (such as edges) with the reference input [2507.11465].

## 4. Implementation and Hyperparameter Choices

HFS-SDEdit requires no new learning objectives or additional losses; its efficacy is driven by the unmodified, pretrained diffusion model and the parameterization of the swapping mechanism. Mask blending uses hard mixing based on a down-sampled binary mask, not a learned penalty. The main trade-off hyperparameters are $t_s$ (starting noise index), $t_\mathrm{stop}$ (swap stop index), and $\sigma$ (Gaussian filter std), permitting flexible adjustment for fidelity/quality optimization.

Key implementation details in Elevate3D [2507.11465]:

| Parameter          | Value                        | Description                               |
|--------------------|-----------------------------|-------------------------------------------|
| Backbone           | FLUX (UNet w/ self-attention)| Diffusion-model architecture              |
| Reverse Steps $T$  | 30                          | Total denoising steps                     |
| $t_s$              | 29                          | Starting noise index                      |
| $t_\mathrm{stop}$  | 18                          | High-frequency swap stop                  |
| $\sigma$           | 4                           | Std for Gaussian filter                   |
| Mask threshold     | 0.5                         | For new-pixel detection                   |
| Geometry camera    | Orthographic                | For view synthesis and refinement         |

## 5. Texture–Geometry Refinement and Regularization

Following texture refinement, Elevate3D leverages images enhanced by HFS-SDEdit to improve geometry. A normal predictor estimates per-pixel surface orientation, from which a depth map is computed via minimization of a regularized normal-integration energy:

$$
E[z] = \iint \left[ (z_u + \tfrac{n_x}{n_z})^{2} + (z_v + \tfrac{n_y}{n_z})^{2} \right]\,du dv + \lambda \iint (z(u,v) - d(u,v))^{2}\,du dv,
$$

where $z_u, z_v$ are surface gradients, $[n_x, n_y, n_z]^T$ are predicted normals, and $d(u,v)$ is the prior mesh depth. The regularizer $\lambda=0.008$. The geometry refinement aims to match predicted per-view normals while remaining close to the original geometry in $L^2$ distance, following a semi-smooth integration approach with bilateral weights [2507.11465].

## 6. Evaluation Metrics and Empirical Performance

On the GSO real-scan benchmark (59 objects; $80\%$ geometry downsampling, heavy texture blur), Elevate3D with HFS-SDEdit establishes new state-of-the-art results in 3D model refinement:

| Metric  | Elevate3D | DreamGaussian | MagicBoost | DiSR-NeRF |
|---------|-----------|--------------|------------|-----------|
| MUSIQ   | 66.53     | 61.67        | 51.65      | 48.94     |
| LIQE    | 2.77      | 2.12         | 2.11       | 1.29      |
| TOPIQ   | 0.53      | 0.47         | 0.39       | 0.39      |
| Q-Align | 3.22      | 2.74         | 2.50       | 2.68      |

In 2D enhancement (LSDIR validation), HFS-SDEdit achieves the highest no-reference scores (MUSIQ 39.52 vs. best SDEdit 29.19) and superior LPIPS (0.598 vs. 0.746), in exchange for some reduction in PSNR/SSIM, reflecting higher perceptual fidelity [2507.11465].

Qualitatively, HFS-SDEdit produces images with sharp, accurate edges, preserves crisp structural details, and eliminates artifacts such as blur and scanlines. 3D meshes reconstructed following HFS-SDEdit-refined textures exhibit accurate silhouettes, fine bump details, and tight texture–geometry alignment.

## 7. Significance and Implications

HFS-SDEdit replaces SDEdit’s pure noising and denoising at high noise levels with targeted high-frequency injections from the reference. This modification removes the conventional fidelity–quality trade-off, enabling aggressive removal of low-frequency scan noise while anchoring critical high-frequency features—edges, corners, and lettering—to the input.

Embedded within Elevate3D’s alternating texture and geometry refinement pipeline, HFS-SDEdit underpins the highest texture and geometric consistency currently attained in open research for 3D asset enhancement [2507.11465]. A plausible implication is that the high-frequency swapping paradigm can generalize to other structured-editing tasks within diffusion-based restoration or translation, where content fidelity at select scales is paramount.

Source: https://www.emergentmind.com/topics/hfs-sdedit