DVFace: One-Step Diffusion for Face Restoration

• The paper introduces DVFace, a video face restoration framework that leverages a one-step diffusion formulation with spatio-temporal dual-codebook priors to efficiently recover high-quality facial details.
• It employs an asymmetric spatio-temporal fusion module that injects global temporal modulation and local spatial residuals, ensuring robust identity preservation and coherence across frames.
• Evaluations on synthetic and real-world datasets demonstrate that DVFace outperforms multi-step diffusion methods in fidelity, perceptual quality, and temporal consistency.

Searching arXiv for the DVFace paper and closely related context papers. arXiv search query: DVFace Spatio-Temporal Dual-Prior Diffusion for Video Face Restoration (Chen et al., 16 Apr 2026) DVFace is a video face restoration framework that targets the recovery of high-quality face videos from degraded inputs affected by blur, noise, compression artifacts, and other real-world degradations. It is defined by a one-step diffusion formulation coupled to explicit facial priors extracted through a spatio-temporal dual-codebook, with the stated objective of producing realistic facial details, stable identity, and temporal coherence in restored videos (Chen et al., 16 Apr 2026). In contrast to approaches that rely on generic diffusion priors and multi-step sampling, DVFace is designed around face-specific prior modeling and a single denoising step, using an asymmetric spatio-temporal fusion strategy to inject spatial and temporal information into a diffusion transformer backbone (Chen et al., 16 Apr 2026).

1. Problem setting and motivating constraints

Video face restoration (VFR) is presented as a stricter problem than single-image restoration because restoration errors accumulate not only spatially within a frame but also temporally across frames (Chen et al., 16 Apr 2026). Three constraints are emphasized. First, identity stability is critical because tiny structural errors across frames translate into identity drift. Second, temporal consistency is required because fine details must remain coherent under motion and expression changes. Third, severe degradations destroy both high-frequency textures and mid-level structures, making recovery dependent on strong priors rather than local enhancement alone (Chen et al., 16 Apr 2026).

Within this setting, diffusion models provide strong generative priors, but the paper argues that existing video restoration methods still rely heavily on multi-step sampling. That dependence is characterized as computationally heavy and as a source of stochasticity that can amplify temporal flicker. The same discussion also identifies a second limitation: generic diffusion priors are not tailored to faces and often fail to preserve identity under strong degradations (Chen et al., 16 Apr 2026).

DVFace is introduced specifically to address those two limitations. Its proposed remedies are explicit in the formulation: one-step diffusion is used to remove iterative sampling overhead and reduce temporal stochasticity, and a spatio-temporal dual-codebook is used to encode facial appearance and dynamics separately, then fuse them asymmetrically into the diffusion backbone (Chen et al., 16 Apr 2026). A plausible implication is that the method treats facial recovery not as generic video denoising but as restoration under strong structural priors, where identity and motion are modeled as distinct but coupled constraints.

2. End-to-end architecture

The input to DVFace is a degraded face video $$x_{lq} \in \mathbb{R}^{T \times H \times W \times 3}$$. The system first applies a VAE encoder $E$ to obtain latent representations,

$z_{lq} = E(x_{lq}).$

A dual-prior extraction module then derives complementary spatial and temporal priors, denoted $f_s$ and $f_t$, from the degraded video via learned codebooks $C_s$ and $C_t$ (Chen et al., 16 Apr 2026).

The restoration backbone is a diffusion transformer, specifically a DiT from Wan 2.1 T2V, which predicts a velocity field $v_\theta$ conditioned on the latent input, timestep, optional text condition, and the two facial priors. The one-step latent update is

$$\hat{z}_{hq} = z_{lq} - t^* v_\theta(z_{lq}, t^*, c_{\text{text}}, f_s, f_t),$$

and the restored video is obtained by decoding the predicted high-quality latent through the VAE decoder $D$:

$E$0

The notation and modular decomposition are explicit: $E$1 and $E$2 are the VAE encoder and decoder; DiT is the diffusion backbone; $E$3 and $E$4 are VQ-style codebooks; $E$5 and $E$6 are transformers used for robust code prediction; and $E$7 is a fixed timestep in $E$8 used to align $E$9 as a noisy latent (Chen et al., 16 Apr 2026).

The role of text conditioning is deliberately secondary. The variable $z_{lq} = E(x_{lq}).$0 is optional and is often empty or fixed in practice (Chen et al., 16 Apr 2026). This placement indicates that DVFace is not organized around prompt-driven generation; the central conditioning signal is the pair of facial priors extracted from the degraded sequence itself.

3. Spatio-temporal dual-codebook priors

A defining characteristic of DVFace is the use of two distinct codebooks rather than a single shared latent vocabulary. The spatial codebook

$z_{lq} = E(x_{lq}).$1

is intended to capture structures and textures, whereas the temporal codebook

$z_{lq} = E(x_{lq}).$2

is intended to capture dynamic priors, including motion, expression evolution, and identity transitions (Chen et al., 16 Apr 2026).

The degraded video is encoded to a latent tensor $z_{lq} = E(x_{lq}).$3. From this tensor, DVFace constructs two branches. The temporal latent is

$z_{lq} = E(x_{lq}).$4

where temporal self-attention and adjacent-frame differencing are combined. The spatial latent is

$z_{lq} = E(x_{lq}).$5

which is intended to emphasize local appearance details (Chen et al., 16 Apr 2026).

Robust code prediction under degradations is handled by applying transformers $z_{lq} = E(x_{lq}).$6 and $z_{lq} = E(x_{lq}).$7 to the reshaped token sequences derived from $z_{lq} = E(x_{lq}).$8 and $z_{lq} = E(x_{lq}).$9. Quantization is then performed by nearest-neighbor lookup for each token:

$f_s$0

and

$f_s$1

The quantized priors are then defined as

$f_s$2

This decomposition makes the priors explicit rather than implicit, and the paper attributes the method’s identity preservation and temporal coherence to this explicit spatio-temporal facial prior modeling (Chen et al., 16 Apr 2026).

Codebook learning uses a VQ-style code-level loss,

$f_s$3

equated in the paper with the standard

$f_s$4

Here $f_s$5 denotes stop-gradient (Chen et al., 16 Apr 2026). The use of discrete prototypical embeddings suggests a reusable prior bank for facial structures and dynamics, rather than direct regression of restoration features.

4. One-step diffusion and asymmetric fusion

DVFace builds on a DiT backbone but compresses denoising to a single step using supervised objectives, and the paper explicitly states that no teacher–student distillation is required (Chen et al., 16 Apr 2026). The generic diffusion forward process is recalled as

$f_s$6

In the latent space used by DVFace, the paper adopts a linear interpolation form

$f_s$7

and identifies the degraded latent with a fixed-timestep noisy latent,

$f_s$8

The learned velocity field then maps the degraded latent to a restored latent in one step:

$f_s$9

This formulation replaces iterative denoising with a direct supervised mapping conditioned on explicit priors (Chen et al., 16 Apr 2026).

The injection of priors is handled by the asymmetric spatio-temporal fusion module (ASTF), inserted into each DiT block. The asymmetry is functional rather than cosmetic. The temporal prior is used as a global modulation signal: a pooled descriptor $f_t$0 is processed by an MLP to produce shared scale and shift parameters,

$f_t$1

which modulate the DiT token features as

$f_t$2

The paper states that sharing $f_t$3 across all DiT layers provides a consistent temporal bias for stability and coherence (Chen et al., 16 Apr 2026).

The spatial prior is then injected as a temporally guided local residual. Cross-attention uses queries from the temporal prior and keys and values from the spatial prior:

$f_t$4

followed by

$f_t$5

The projection layers in this branch are zero-initialized, described as Zero-Linear, so that training begins from an identity mapping (Chen et al., 16 Apr 2026). The paper summarizes the fused block algebraically as

$f_t$6

This design also clarifies a conceptual distinction. Temporal priors act as a globally shared conditioning signal, whereas spatial priors are filtered through temporal compatibility before local injection. The ablations reported in the paper state that shared $f_t$7 across layers beats independent per-layer modulation, that temporally pre-refined spatial injection outperforms simple spatial injection, and that the full ASTF variant is best (Chen et al., 16 Apr 2026).

5. Training regime and implementation

Training is divided into three stages. Stage-1 learns the codebooks and autoencoder on high-quality video. The reconstruction pipeline decodes

$f_t$8

to $f_t$9 and is optimized by

$C_s$0

The component losses are specified as reconstruction, perceptual using VGG19, adversarial, and the VQ-style code-level feature loss (Chen et al., 16 Apr 2026).

Stage-1′ aligns low-quality inputs with high-quality code assignments while freezing the codebooks and decoder. In this phase, $C_s$1 and $C_s$2 are trained, with light encoder refinement, using cross-entropy over predicted code indices and a code-level feature loss:

$C_s$3

$C_s$4

and

$C_s$5

The stated purpose is robust code assignment from degraded inputs (Chen et al., 16 Apr 2026).

Stage-2 performs one-step diffusion restoration while freezing the prior extractor and VAE encoder and optimizing the DiT, ASTF, and VAE decoder. The supervised objective is

$C_s$6

with

$C_s$7

$C_s$8

and the temporal consistency term

$C_s$9

The flows are computed on high-quality video (Chen et al., 16 Apr 2026).

Several implementation details are explicit. The backbone is Wan 2.1, pretrained for text-to-video, and ASTF is inserted into each DiT block. The prior extractor uses TemporalAttn plus FrameDiff for the temporal path and two convolution blocks for the spatial path. The reported training schedule is 250k iterations for Stage-1 codebook learning on HQ, 50k iterations for LQ alignment, and 15k iterations for Stage-2, with learning rates $C_t$0 and $C_t$1 respectively; $C_t$2, $C_t$3, $C_t$4, and $C_t$5 (Chen et al., 16 Apr 2026). Optimization uses AdamW with batch size 4 on 4×NVIDIA A6000. Degradations follow a synthetic pipeline consistent with RealBasicVSR/OSDFace practice, and stabilization includes zero-initialization of ASTF projections (Chen et al., 16 Apr 2026).

The paper also states that DVFace does not require an explicit identity loss such as ArcFace; identity preservation is said to emerge from the spatial codebook prior and temporal regularization (Chen et al., 16 Apr 2026). This directly counters the common assumption that identity retention in face restoration necessarily requires a recognition-specific supervisory term.

6. Evaluation, comparisons, and limits

Training is conducted on VFHQ, described as 16k HQ face videos. Evaluation covers both synthetic and real-world settings: VFHQ-Test and HDTF, each with 100 videos and the same synthetic degradations as training, and RFV-LQ with 50 videos plus VoxCeleb2 with 100 videos for real-world testing (Chen et al., 16 Apr 2026). The reported metrics span fidelity (PSNR, SSIM), perceptual quality (LPIPS, DISTS, CLIP-IQA, MUSIQ, NIQE, MANIQA, LIQE), identity (AKD), video quality (FVD, DOVER), and temporal consistency ($C_t$6, VIDD) (Chen et al., 16 Apr 2026).

On VFHQ-Test, the paper reports PSNR 31.81, SSIM 0.9007, LPIPS 0.0776, DISTS 0.0805, CLIP-IQA 0.6289, MUSIQ 71.89, MANIQA 0.4195, LIQE 4.05, AKD 0.0026, FVD 60.11, DOVER 0.8703, $C_t$7 0.91, and VIDD 0.4339 (Chen et al., 16 Apr 2026). On HDTF, it reports PSNR 29.54, LPIPS 0.1074, DISTS 0.0931, CLIP-IQA 0.5972, MUSIQ 72.58, MANIQA 0.3985, LIQE 3.71, AKD 0.0020, FVD 60.47, DOVER 0.8085, $C_t$8 0.51, and VIDD 0.3681 (Chen et al., 16 Apr 2026). On RFV-LQ, the reported values are CLIP-IQA 0.6256, MUSIQ 63.69, MANIQA 0.3422, LIQE 2.80, DOVER 0.7165, and VIDD 0.3159. On VoxCeleb2, the reported values are CLIP-IQA 0.6411, MUSIQ 53.19, NIQE 6.59, LIQE 4.13, $C_t$9 1.45, and VIDD 0.4795 (Chen et al., 16 Apr 2026).

The comparison set includes KEEP, AverNet, PGTFormer, BFVR, DicFace, and SVFR. The paper states that DVFace outperforms these strong baselines on synthetic and real datasets across fidelity, perceptual, identity, and temporal metrics (Chen et al., 16 Apr 2026). It also states that the one-step design offers practical efficiency advantages over multi-step diffusion by eliminating iterative denoising and reducing video latency, while noting that exact runtime and FLOPs are not reported (Chen et al., 16 Apr 2026). That omission is important: the efficiency claim is architectural and qualitative rather than benchmarked with explicit system-level throughput numbers.

The ablations are tightly aligned with the claimed design rationale. On VFHQ-Test, adding the spatial prior improves PSNR and LPIPS, adding the temporal prior improves $v_\theta$0 and FVD, and using both yields the best overall combination, including PSNR 31.81, LPIPS 0.0776, $v_\theta$1 0.91, and FVD 60.11 (Chen et al., 16 Apr 2026). The paper also reports that text-based priors, including empty or fixed prompts and DAPE, yield only marginal gains, whereas dual codebooks bring clear improvements (Chen et al., 16 Apr 2026). This suggests that explicit spatio-temporal facial priors are more consequential than generic textual conditioning for this task.

The stated limitations are also specific. Extreme degradations, including very strong blur, noise, and heavy occlusions, can still impair accurate code prediction and lead to identity drift or local artifacts. Very rare identities or out-of-distribution motions may not be fully captured by the codebooks. The optical-flow-based temporal loss assumes reliable flow on high-quality video, so strong motion or nonrigid occlusions may reduce flow accuracy. Generalization to unseen degradations is described as robust but not guaranteed, with performance depending on the breadth of the synthetic degradation pipeline (Chen et al., 16 Apr 2026).

7. Inference, reproducibility, and projected directions

The inference procedure is a direct instantiation of the training-time architecture. Given an input video $v_\theta$2, DVFace first computes $v_\theta$3. It then extracts $v_\theta$4 and $v_\theta$5, predicts code indices with $v_\theta$6 and $v_\theta$7, and looks up the quantized priors $v_\theta$8 and $v_\theta$9. One-step denoising produces

$$\hat{z}_{hq} = z_{lq} - t^* v_\theta(z_{lq}, t^*, c_{\text{text}}, f_s, f_t),$$0

and the restored output is obtained as

$$\hat{z}_{hq} = z_{lq} - t^* v_\theta(z_{lq}, t^*, c_{\text{text}}, f_s, f_t),$$1

The practical recommendations reported with the method are to process long videos in clips with overlap and simple cross-fade at boundaries, keep $$\hat{z}_{hq} = z_{lq} - t^* v_\theta(z_{lq}, t^*, c_{\text{text}}, f_s, f_t),$$2 empty or fixed, use the same frame size and normalization as training, and keep a consistent $$\hat{z}_{hq} = z_{lq} - t^* v_\theta(z_{lq}, t^*, c_{\text{text}}, f_s, f_t),$$3 (Chen et al., 16 Apr 2026).

Reproducibility is supported through released code and checkpoints at the stated repository, and the implementation summary ties reproducibility to matching preprocessing, synthetic degradations, the two-stage training schedule, and the provided loss weights (Chen et al., 16 Apr 2026). The model is explicitly described as based on Wan 2.1 and trained on 4×A6000 GPUs with AdamW and batch size 4 (Chen et al., 16 Apr 2026).

The future directions listed in the paper outline several extensions rather than settled conclusions. These include stronger temporal modeling through causal transformers, long-range memory, or flow-free temporal constraints; adaptive or larger codebooks with dynamic allocation or hierarchical structure; dynamic fusion with content-aware routing or gating; recognition-aligned priors through joint training with a face recognition backbone; and specialized face diffusion priors that reduce reliance on generic text-to-video pretraining (Chen et al., 16 Apr 2026). A plausible implication is that DVFace defines a modular template in which one-step diffusion, discrete prior learning, and asymmetric conditioning can be varied independently, particularly for settings where identity faithfulness and temporal coherence must be jointly optimized.