Papers
Topics
Authors
Recent
Search
2000 character limit reached

SoL-Refiner: A high resolution video refiner

Updated 1 October 2026
  • SoL-Refiner is a model that converts low resolution or low fidelity video output into high resolution, including 4K. Doing so reduces computational cost by performing semantic and motion tasks at low resolution before refining high-resolution specifics.

SoL-Refiner is a one-step latent-video refinement model that converts low-resolution or low-fidelity video outputs into high-resolution video, including 4K outputs, using a single denoising evaluation. It addresses the computational cost of high-resolution video generation by shifting semantic composition and motion formation to a lower-resolution base generator and reserving high-resolution computation for targeted refinement. Its training combines high-resolution continual training, reinforcement-learning post-training, and one-step distillation. At approximately 2K resolution, SoL-Refiner outperforms the evaluated external refiners on VBench and UniPercept averages; at 3840×21763840\times2176, it improves both metrics over the three-step LTX-2.3 Refiner. With the complete acceleration stack, it achieves an 8.91×8.91\times refinement-latency speedup over that baseline (Liu et al., 29 Sep 2026).

1. Computational motivation and design objectives

High-resolution video generation is expensive because the number of spatiotemporal latent tokens increases with frame count and spatial resolution. If FF denotes the number of frames, HH and WW the spatial dimensions, and sts_t and shs_h the temporal and spatial compression factors, the latent token count is approximately

L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.

For a diffusion or flow-based model requiring NN denoising steps, computational cost scales approximately with the product of the number of denoising evaluations, model cost, and spatiotemporal token count. Direct high-resolution generation therefore incurs both large per-step cost and repeated high-resolution model evaluations.

SoL-Refiner adopts a cascaded formulation:

xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),

where a relatively inexpensive base generator produces a low-resolution video and the refiner reconstructs high-resolution detail. The base stage establishes scene identity, composition, motion, and semantic content; the refinement stage targets local texture, edges, facial details, fine geometry, imaging quality, and artifacts introduced by low-resolution generation.

Conventional cascaded refiners can nevertheless require three, eight, or more high-resolution denoising steps. SoL-Refiner treats the refiner itself as a one-step generation problem. Its objective is not restricted to spatial upsampling: the model can also improve artifacts at an existing resolution, although resolution enhancement is a principal application.

The method is initialized from the pretrained LTX-2.3 video model. Its conditioning includes the low-resolution or low-fidelity video latent, text conditioning, and reference-image features. Reference-image tokens guide generation but are excluded from the refinement reconstruction loss.

2. Latent formulation and flow-matching refinement path

SoL-Refiner operates in latent video space. An encoder 8.91×8.91\times0 maps a video 8.91×8.91\times1 to a clean latent:

8.91×8.91\times2

Let 8.91×8.91\times3 denote the low-quality or low-resolution latent and 8.91×8.91\times4 the high-quality target latent. Rather than initiating refinement from independent Gaussian noise, the method constructs a noisy source endpoint that retains information from the low-resolution input:

8.91×8.91\times5

Stage I uses

8.91×8.91\times6

A flow-matching path connects the high-quality target to the noisy low-resolution-conditioned source:

8.91×8.91\times7

The target velocity is

8.91×8.91\times8

The refiner predicts a velocity field 8.91×8.91\times9, where FF0 includes the low-resolution video, text, and reference-image conditioning. At the target endpoint FF1, the state equals FF2; at FF3, it equals the noisy source FF4. This source-conditioned path distinguishes refinement from unconstrained generation from noise.

The model uses a multiscale residual video architecture with base channel width 64, channel multipliers FF5, and two residual blocks per stage. Inference and end-to-end training use an eight-step Euler sampler in the multi-step configuration:

FF6

3. Three-stage training procedure

Stage I: high-resolution continual training

The first stage teaches a pretrained high-capacity video model to transform a low-fidelity video into a high-fidelity target while preserving motion and semantic content. The primary data source is an internal real-video dataset. The original high-quality video provides FF7, while spatial downsampling followed by upsampling produces the aligned degraded input FF8.

A smaller synthetic-pair dataset is used for warm-up. A supported base generator creates a low-resolution video, and the LTX-2.3 Refiner produces the corresponding high-quality target. The training objective is the flow-matching loss

FF9

The noise level is sampled from a shifted logit-normal distribution truncated to HH0. This truncation ensures that training states remain connected to the low-resolution input rather than requiring the model to solve an unconstrained generation problem. Stage I yields a multi-step high-resolution refiner and does not yet produce the final one-step model.

Stage II: reinforcement-learning post-training

Flow-matching supervision does not directly optimize perceptual qualities such as human preference, facial detail, or image degradation. Stage II therefore applies Reward Feedback Learning (ReFL) to the multi-step refiner.

The final training configuration uses HH1 effective Euler updates. Reward supervision is applied only at later, lower-noise states. A timestep is sampled as

HH2

Earlier steps are executed without retaining gradients. At the selected state, the model estimates the clean latent using

HH3

The decoded video is divided into beginning, middle, and end segments, with one frame sampled uniformly from each segment. The reward models are HPSv3++, which measures prompt-conditioned human preference and perceptual quality, and DeQA, which measures degradation-aware visual quality. If HH4 and HH5 are their scores, the combined reward is

HH6

The clipping terms limit reward exploitation. The multi-step model reaches a VBench average of HH7 and a UniPercept average of HH8 after this stage.

Stage III: one-step distillation

Stage III compresses the multi-step Stage II model into a one-step model through few-step DMD distillation followed by one-step DMD-GAN distillation. The intermediate three-step model stabilizes the final one-step optimization.

In the three-step stage, the student generator, frozen real-score teacher, and trainable fake-score model are initialized from the Stage II checkpoint, with the real-score teacher frozen. The student noise levels are

HH9

The student predicts the high-quality latent WW0. The DMD gradient is formed from the normalized difference between fake and real score-model predictions:

WW1

The student is optimized with the surrogate DMD loss

WW2

where WW3 denotes stop-gradient. The fake-score model is updated from detached student samples, and implicit distribution alignment uses

WW4

For the final one-step stage,

WW5

and inference uses the schedule WW6. The model directly maps the noisy low-resolution-conditioned latent to the high-quality latent in one generator evaluation.

The final student also receives adversarial supervision from a projected DiT discriminator. The discriminator uses features from blocks WW7, WW8, and WW9 of the frozen real-score DiT, together with noise- and text-conditioned query heads. Approximate R1 regularization is used:

sts_t0

This regularization encourages local discriminator smoothness without requiring second-order gradients through the DiT feature extractor.

4. Inference acceleration and implementation stack

SoL-Refiner’s reported latency improvement combines three mechanisms:

  1. One-step distillation, which reduces high-resolution denoising evaluations.
  2. Tiny Autoencoder (TAE), which replaces the full video variational autoencoder and reduces encoding and decoding latency.
  3. Sol-Engine, which applies kernel fusion and sparse Sol-Attn attention to accelerate transformer computation.

For multi-step refinement, Sol-Engine can reuse cached intermediate results across denoising steps.

The principal 2K latency experiment uses one H100 GPU and 241-frame, 24-fps videos at sts_t1:

Configuration Latency
Three-step LTX-2.3 Refiner sts_t2 s
One-step SoL-Refiner Baseline divided by sts_t3
One-step plus TAE Previous latency divided by sts_t4
One-step plus TAE and Sol-Engine sts_t5 s

The complete speedup is

sts_t6

The adjacent acceleration factors are sts_t7 from one-step distillation, sts_t8 from TAE, and sts_t9 from Sol-Engine.

For longer videos, Sol-Engine reduces multi-step refinement latency from shs_h0 to shs_h1 seconds for 81 frames at 2K and from shs_h2 to shs_h3 seconds for 161 frames. At 4K, the corresponding reductions are from shs_h4 to shs_h5 seconds for 81 frames and from shs_h6 to shs_h7 seconds for 161 frames.

A separate end-to-end example uses distilled four-step MiniMax H3 generation followed by one-step SoL-Refiner. A shs_h8 five-second, 24-fps output takes shs_h9 seconds on one GB200, compared with L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.0 seconds for published full-resolution 50-step H3 generation, a L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.1 end-to-end improvement. This result is distinct from the L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.2 refiner-only comparison.

5. Refiner-Bench and evaluation methodology

Refiner-Bench contains 150 videos: 50 generated by WAN 2.1 1.3B, 50 by SANA-Video 2B, and 50 by LTX-2 Stage 1. The use of multiple base generators tests whether refinement quality depends on a particular upstream model. The benchmark includes 12 content groups, three motion levels, and six camera-motion types.

Its content categories include human daily activity, animals, nature and landscape, sports and dance, cinematic portraits, urban and architecture, vehicles and machines, multi-subject interaction, camera-motion stress, fantasy and science fiction, food and cooking, and product or object close-ups. The benchmark contains 50 low-motion, 64 medium-motion, and 36 high-motion videos. Camera-motion labels include dolly, orbit, pan, handheld, tilt, and static.

A shared-input protocol aligns the upstream outputs:

Source Native resolution Aligned crop Shared input
WAN 2.1 1.3B L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.3 L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.4 L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.5
SANA-Video 2B L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.6 L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.7 L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.8
LTX-2 Stage 1 L≈FHWstsh2.L \approx \frac{FHW}{s_t s_h^2}.9 NN0 NN1

The evaluated refiners are LingBot Stage-2 Refiner, LTX-2.3 Refiner, LTX-2.0 Refiner, SEEDVR2, one-step SoL-Refiner, and the 23-step SoL-Refiner.

VBench measures subject consistency, background consistency, motion smoothness, dynamic degree, aesthetic quality, and imaging quality. UniPercept measures image aesthetic assessment, image quality assessment, and image structure or temporal assessment. Its scores use eight uniformly sampled frames per video.

At approximately 2K resolution, the main results are:

Method Steps VBench average UniPercept average
Resized input — NN2 NN3
LingBot Stage-2 8 NN4 NN5
LTX-2.3 3 NN6 NN7
LTX-2.0 3 NN8 NN9
SEEDVR2 1 xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),0 xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),1
SoL-Refiner, 23-step 23 xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),2 xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),3
SoL-Refiner, one-step 1 xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),4 xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),5

The one-step model obtains the best external-refiner result on both reported averages. Its VBench aesthetic-quality and imaging-quality scores are xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),6 and xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),7, respectively. The 23-step model remains stronger, demonstrating a quality–speed trade-off.

At xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),8, the one-step model improves over the three-step LTX-2.3 Refiner by xlow=Gϕ(c),x^high=Rθ(xlow),x_{\mathrm{low}}=G_\phi(c), \qquad \hat{x}_{\mathrm{high}}=R_\theta(x_{\mathrm{low}}),9 on VBench average and 8.91×8.91\times00 on UniPercept average.

6. Transfer, ablations, and limitations

Transfer across base generators

SoL-Refiner can be applied without modifying or retraining the base generator. The experiments use WAN-5B, WAN-1.3B, and Cosmos-Nano:

Base generator Latency reduction
WAN-5B 8.91×8.91\times01
WAN-1.3B 8.91×8.91\times02
Cosmos-Nano 8.91×8.91\times03

For WAN-5B, direct high-resolution generation scores 8.91×8.91\times04 on VBench and 8.91×8.91\times05 on UniPercept with 8.91×8.91\times06 seconds of latency; the low-resolution-plus-refiner pipeline scores 8.91×8.91\times07 and 8.91×8.91\times08 with 8.91×8.91\times09 seconds. For WAN-1.3B, scores increase from 8.91×8.91\times10 to 8.91×8.91\times11 on VBench and from 8.91×8.91\times12 to 8.91×8.91\times13 on UniPercept, while latency decreases from 8.91×8.91\times14 to 8.91×8.91\times15 seconds. For Cosmos-Nano, scores increase from 8.91×8.91\times16 to 8.91×8.91\times17 and from 8.91×8.91\times18 to 8.91×8.91\times19, while latency decreases from 8.91×8.91\times20 to 8.91×8.91\times21 seconds.

Training-stage ablations

Stage VBench average UniPercept average
Stage I: continual multi-step training 8.91×8.91\times22 8.91×8.91\times23
Stage II: RL post-training 8.91×8.91\times24 8.91×8.91\times25
Stage III: one-step DMD-GAN 8.91×8.91\times26 8.91×8.91\times27

Stage I establishes the refinement mapping, Stage II provides the largest perceptual improvement, and Stage III sacrifices some quality to obtain one-step inference while remaining superior to the Stage I baseline.

The complete reward recipe reaches a VBench average of 8.91×8.91\times28. Removing the regularization recipe lowers this to 8.91×8.91\times29; using only HPSv3++ lowers it to 8.91×8.91\times30. The two-reward configuration balances prompt-conditioned preference and degradation-aware quality.

Sequential RL followed by DMD is also superior to jointly optimizing RL and DMD: the DMD-R strategy reaches 8.91×8.91\times31, whereas RL followed by DMD reaches 8.91×8.91\times32.

Quality–speed trade-offs

The one-step model is faster but remains below the 23-step teacher in perceptual quality. Its principal advantage is operational: it allows a low-resolution base generator to establish semantic and temporal structure while restricting high-resolution computation to one denoising evaluation.

The method is primarily designed for local visual refinement. Large semantic, geometric, or motion errors in the base video may persist. Stage II evaluates three sampled frames with image-quality and preference rewards rather than a direct long-range temporal-consistency objective. Consequently, temporal consistency is handled through the joint video latent representation and temporally aligned training data, but is not directly optimized by a dedicated temporal reward.

The benchmark consists of AI-generated videos from three source generators and does not establish performance on broad camera-captured degradation, arbitrary video restoration, or general video editing. Source fidelity also constrains the correction process: SoL-Refiner is trained to enhance rather than substantially reinterpret the input video.

SoL-Refiner therefore represents a refinement-aware generative architecture in which the low-resolution generator and high-resolution corrector are separated computationally but coupled through training. Its defining contribution is the combination of a low-resolution-conditioned flow path, reward-optimized multi-step refinement, and one-step DMD-GAN distillation. The result is a high-resolution video refiner that retains the quality advantages of a learned multi-step corrector while substantially reducing the second-stage sampling bottleneck.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (1)

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to SoL-Refiner.