FastVSR: One-Step Diffusion VSR

• The paper introduces an asymmetric VAE codec with a spatial compression ratio of 16, achieving up to 111.9× speedup and a 46.3% reduction in peak memory compared to multi-step models.
• FastVSR is a codec-centered acceleration method that utilizes pixel shuffle, channel replication, and lower-bound-guided training to stabilize high-compression outputs in one-step diffusion.
• The method integrates with existing latent diffusion backbones, maintaining comparable or superior perceptual and fidelity metrics on both synthetic and real-world video benchmarks.

FastVSR is a one-step diffusion-based video super-resolution method introduced in "Asymmetric VAE for One-Step Video Super-Resolution Acceleration" (Li et al., 29 Sep 2025). It targets the residual efficiency bottleneck that remains after one-step diffusion, namely the high-resolution VAE codec, by implementing a high compression VAE with spatial compression ratio of 16, denoted as $f16$, together with pixel shuffle, channel replication, and a lower-bound-guided training strategy. Reported results show speedups of $111.9\times$ compared to multi-step models and $3.92\times$ compared to existing one-step models, with a $46.3\%$ reduction in peak memory, while maintaining comparable or superior performance in perceptual, fidelity, and video-quality metrics on synthetic and real-world benchmarks (Li et al., 29 Sep 2025).

1. Scope and problem formulation

FastVSR is situated in the real-world video super-resolution setting, where diffusion models are used because of their ability to restore high-frequency textures and maintain generalization to unknown, complex degradations found in naturally captured videos. Within this setting, the method distinguishes between multi-step diffusion frameworks, which require many iterative denoising steps, and one-step diffusion models, which compress the denoising trajectory into a single forward pass. The central claim of FastVSR is that recent one-step models greatly reduce latency but leave the VAE’s high-resolution encoding and decoding as the main bottleneck (Li et al., 29 Sep 2025).

A common simplification in efficient diffusion VSR is that once sampling is reduced to one step, the denoiser dominates inference cost. FastVSR is explicitly motivated by the opposite observation in this regime: the codec becomes the primary source of compute and memory pressure. This focus shifts acceleration away from sampling schedules and toward the latent video representation itself. In that sense, FastVSR is a codec-centered acceleration method rather than a denoiser-centered one.

2. Asymmetric codec architecture

The architectural core of FastVSR is an asymmetric VAE design. Let $V = T \times H \times W$ denote the HR video volume, and let the VAE encoder stride be $(s_t, s_s)$. The method models compute and activation costs as

$$\mathrm{FLOPs} \approx \kappa_E V + \kappa_T \frac{V}{s_t s_s^2} + \kappa_D V$$

and

$$\mathrm{Act}_{\max} \approx \max \left\{ \mu_E V, \mu_T \frac{V}{s_t s_s^2}, \mu_D V \right\}.$$

It further characterizes the decoder-to-denoiser cost ratio as

$$\frac{\mathrm{FLOPs}_D}{\mathrm{FLOPs}_T} \approx \frac{\kappa_D}{\kappa_T} s_t s_s^2 \gg 1,$$

with the example $s_t = 4, s_s = 8 \Rightarrow s_t s_s^2 = 256$ (Li et al., 29 Sep 2025).

The specific instantiation is an $111.9\times$0 asymmetric VAE. The encoder is fixed, pretrained, and uses stride $111.9\times$1; the decoder is trained and uses stride $111.9\times$2; and the indirect upsampling ratio is $111.9\times$3. The pipeline is

$V = T \times H \times W$6

Only the decoder is retrained, while the encoder and transformer remain fixed. The paper states that this reduces spatial tokens processed by the VAE and transformer, keeps latent distribution unchanged, and concentrates compute in a single decode pass. A plausible implication is that FastVSR preserves compatibility with an existing one-step latent diffusion backbone while shifting the efficiency frontier through the codec alone.

3. Lower-bound-guided optimization

FastVSR does not treat the high-compression decoder as a drop-in replacement. The paper reports that direct supervised training for the $111.9\times$4 VAE tends to result in non-convergence and pseudo-textures. To address this, it introduces a lower-bound-guided training strategy built around two auxiliary VAEs: a frozen reference VAE $111.9\times$5 trained on real HR data, and a trainable lower-bound VAE $111.9\times$6 fit to current generator outputs (Li et al., 29 Sep 2025).

For a VAE, the free energy is defined as

$111.9\times$7

with $111.9\times$8. The method then uses the likelihood-ratio bound

$111.9\times$9

and defines

$3.92\times$0

The reconstruction objective is

$3.92\times$1

and the full training objective is

$3.92\times$2

where $3.92\times$3 can be TV or color regularization. Optimization alternates between two stages: updating the $3.92\times$4 decoder while freezing both auxiliary VAEs, and updating the lower-bound VAE to fit current outputs. The stated purpose is to reduce pseudo-textures, improve convergence, and maintain temporal consistency (Li et al., 29 Sep 2025).

This training design is central to FastVSR’s claim. The method is not merely “high compression VAE plus one-step diffusion”; rather, it is a high-compression decoder coupled to a stabilization mechanism that converts an otherwise unstable regime into a trainable one.

4. Decoder upsampling and experimental protocol

The decoder performs additional upsampling through pixel shuffle and channel replication. PixelShuffle(2) is applied before the output layer, rearranging feature-map channels into higher spatial dimensions by $3.92\times$5, after which channels are expanded by duplication or lightweight $3.92\times$6 projection, and a standard convolution produces the final output. The paper states that PixelShuffle is fast and introduces no learnable upsampling parameters, minimizing inference time and pseudo-textures compared to bicubic or deconvolution approaches, while channel replication ensures sufficient output channels for RGB image generation (Li et al., 29 Sep 2025).

The implementation is based on the CogVideoX1.5 T2V diffusion backbone. The $3.92\times$7 encoder and transformer are frozen, and only the $3.92\times$8 decoder is trained. Training uses REDS with 239 video sequences and LSDIR with 85k images. Evaluation is performed at $3.92\times$9 scaling on UDM10, SPMCS, and YouHQ40 as synthetic benchmarks, and on RealVSR, MVSR4x, and VideoLQ as real-world benchmarks. Reported metrics include PSNR, SSIM, LPIPS, DISTS, MUSIQ, CLIP-IQA, DOVER, $46.3\%$0, sampling steps, end-to-end latency, MACs, and peak memory (Li et al., 29 Sep 2025).

These implementation details clarify the intended use case. FastVSR is not introduced as a new end-to-end diffusion backbone; it is an acceleration layer over an existing one-step latent video diffusion pipeline, with the asymmetry isolated to the codec.

5. Empirical performance, implications, and limitations

FastVSR reports the following efficiency profile (Li et al., 29 Sep 2025):

Variant	Sampling steps	E2E latency / MACs / Peak memory
Multi-step methods	15–50	120–425 s / 4,281–9k T / 11.7–19.6 GB
One-step (f8 VAE)	1	14.90 s / 504.8 T / 60.78 GB
FastVSR	1	3.80 s / 125.7 T / 32.59 GB

The reported headline numbers are a $46.3\%$1 speedup over multi-step diffusion models, a $46.3\%$2 speedup over previous one-step models, and a $46.3\%$3 reduction in peak memory. The paper further states that FastVSR is comparable or superior to baselines in most perceptual, fidelity, and video-quality metrics and improves temporal consistency (Li et al., 29 Sep 2025).

The stated implications are deployment-oriented. The reduction in inference time and memory is presented as making one-step diffusion VSR more viable for real-time, edge, or mobile deployments. Because only the decoder is trained while the encoder and transformer are frozen, the method is also described as plug-and-play with existing backbones. The paper additionally suggests that the asymmetric codec idea can generalize to other video or image generation and restoration tasks.

The limitations are explicit. The paper notes that compression ratios beyond $46.3\%$4 may require new regularization or training advances; some failure cases appear in very fast, nonrigid scenarios; there may be a theoretical bound on achievable fidelity under very high compression; and compatibility depends on keeping encoder and transformer latent spaces aligned, with the transformer largely assumed to remain unchanged (Li et al., 29 Sep 2025). These caveats are important because they delimit FastVSR’s contribution: it is an acceleration method for a particular one-step latent diffusion regime, not a universal replacement for full-resolution codecs.

6. Position within efficient video super-resolution research

FastVSR belongs to a broader research line on efficient video super-resolution, but its mechanism differs from several earlier acceleration strategies. FAST, a framework for compressed videos, applies SR only to a subset of frames and transfers SR pixels to the remaining frames using motion vectors and residuals already embedded in the compressed stream, yielding up to $46.3\%$5 acceleration with a visual quality loss of $46.3\%$6dB (Zhang et al., 2016). FSTRN reduces the cost of 3D-convolutional VSR by factorizing a heavy $46.3\%$7 convolution into $46.3\%$8 and $46.3\%$9 operations (Li et al., 2019). RSTT instead unifies temporal interpolation and spatial super-resolution in a single transformer and reports 26.19 fps with 4.49M parameters for RSTT-S on $V = T \times H \times W$0 frames (Geng et al., 2022). These methods accelerate propagation, alignment, or unified reconstruction, but they do not target the one-step diffusion codec bottleneck that motivates FastVSR.

A second cluster of work accelerates video restoration through compression or flow-free temporal modeling. FLAVR eliminates explicit optical flow and depth from frame interpolation through 3D space-time convolutions and is described as a useful template for fast video super-resolution pipelines because it bypasses explicit motion estimation (Kalluri et al., 2020). FCA2 introduces compression-driven dimensionality reduction for compressed video super-resolution and reports up to $V = T \times H \times W$1–$V = T \times H \times W$2 faster inference depending on video length and resolution (Wang et al., 13 Jun 2025). NanoVSR, a fully convolutional and TensorRT-compatible architecture for edge devices, reaches 27.2 FPS on the NVIDIA Jetson Orin NX 16GB (25W) with NanoVSR-644k at 28.64 dB on REDS4 (Pawlicki et al., 11 Jul 2026). This suggests that efficient VSR has progressively moved from algorithmic frame transfer, to lightweight spatio-temporal modeling, to explicit compression of the temporal input space.

Within diffusion-based VSR specifically, TurboVSR employs an autoencoder with a compression ratio of $V = T \times H \times W$3, factorized conditioning, and a shortcut model, and is reported to be 100+ times faster, taking only 7 seconds to process a 2-second 1080p video (Wang et al., 30 Jun 2025). FlashVSR extends the efficiency objective to streaming inference and is described as the first diffusion-based one-step streaming framework towards real-time VSR, running at approximately 17 FPS for $V = T \times H \times W$4 videos on a single A100 GPU (Zhuang et al., 14 Oct 2025). Against this backdrop, FastVSR occupies a distinct position: it does not primarily distill a streaming student or reduce diffusion steps below one, but accelerates the already one-step regime by replacing the symmetric codec with an asymmetric $V = T \times H \times W$5 VAE and stabilizing that replacement through lower-bound-guided training (Li et al., 29 Sep 2025).

In that narrower sense, FastVSR is best understood as a codec-acceleration method for one-step real-world diffusion VSR. Its central thesis is that once denoising has been collapsed to a single pass, the most consequential remaining inefficiency lies in high-resolution latent decoding; the method’s architecture, optimization strategy, and reported gains are all organized around that thesis.