Compressed Video Quality Enhancement (CVQE)

• CVQE is a decoder-side restoration technique that reduces artifacts (blocking, ringing, blurring) from lossy video compression.
• It employs single- and multi-frame architectures leveraging spatial, temporal, and frequency domain methods to enhance visual quality.
• Empirical studies report gains in PSNR, SSIM, and bitrate reductions while balancing fidelity, perceptual quality, and computational efficiency.

Compressed Video Quality Enhancement (CVQE) denotes decoder-side or post-decoder restoration of decoded video frames to reduce artifacts introduced by lossy coding tools and to recover visually faithful content without changing the encoder or bitstream. In AVC, HEVC, and VVC, the degradation arises from block-based transforms, quantization, motion-compensated prediction, in-loop filtering, and rate control, producing blocking, ringing, blurring, banding, mosquito noise, flicker, and breathing. Within the broader literature, CVQE spans post-decoder enhancement, in-loop restoration/filtering, and, less prominently, pre-encoder preprocessing, but post-decoder enhancement remains dominant (Meng et al., 2019, HoangVan et al., 12 Sep 2025).

1. Scope, signal model, and artifact structure

CVQE is commonly formulated as residual restoration on decoded frames. A unified architectural view describes three stages—Feature Encoding, Feature Propagation, and Residual Decoding—written as

$$X_t^{res} = RD(FP(FE(X_{t-m}^{LQ}, \ldots, X_t^{LQ}, \ldots, X_{t+m}^{LQ}))),$$

followed by

$X_t^{HQ} = X_t^{LQ} + X_t^{res}.$

Single-frame models use only $X_t^{LQ}$, whereas multi-frame models incorporate adjacent decoded frames to exploit temporal correlations (HoangVan et al., 12 Sep 2025).

Artifact behavior depends on coding type and standard. Intra coding is dominated by spatial defects tied to transform and quantization, whereas inter coding adds temporal errors from motion vectors, reference mismatch, residual propagation, and GOP-level decisions. The survey literature further notes that screen content coding induces sharp-edge ringing and false textures due to palette quantization and intra copy mismatch. Even when codecs apply deblocking, SAO, and ALF, residual ringing, over-smoothing, temporal instability, and quality fluctuation remain, which is why decoder-side learned restoration persists as a separate research direction (HoangVan et al., 12 Sep 2025).

This framing also explains why CVQE papers repeatedly emphasize compatibility. SDTS is explicitly a decoder-side post-processing approach applied after VVC decoding and does not modify the VVC bitstream or coding tools, and analogous claims appear in later codec-aware frameworks that operate on decoded frames while reusing metadata already available in the bitstream (Meng et al., 2019, Zeng et al., 21 Apr 2025).

2. Architectural paradigms

The earliest deep CVQE systems are predominantly spatial. Single-frame methods such as ARCNN, DnCNN, MemNet, VRCNN, and DCAD focus on spatial priors per frame, which limits performance on videos because temporal exploitation is absent (Meng et al., 2019). Multi-frame methods then introduced explicit temporal aggregation. QG-ConvLSTM models long-range frame dependencies with a bi-directional recurrent structure and learns the “forget” and “input” gates in the ConvLSTM cell from quality-related features, so that higher-quality frames overwrite memory more aggressively and lower-quality frames rely more on existing memory (Yang et al., 2019). The contribution analysis in that work formalizes the quality–correlation trade-off by weighting each frame through products of learned forget and input gates, rather than by fixed temporal neighborhoods alone (Yang et al., 2019).

Alignment-centric CNN designs remain a major paradigm. SDTS for VVC uses a temporal structure fusion subnet and a spatial detail enhancement subnet. Its motion compensation module modifies spatial transformer motion compensation to jointly predict coarse flow, fine flow, and still scenes flow, and warps neighboring frames through bilinear interpolation,

$$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$

before slow fusion and ENet-based artifact suppression (Meng et al., 2019). TGAFNet instead constructs three temporal groups $G_i = \{I_{k-i}, I_k, I_{k+i}\}$ for $i \in \{1,2,3\}$, applies deformable alignment with different kernel sizes $K=1,3,5$ for short-, mid-, and long-term context, and then performs staged inter-group fusion plus a feature enhancement head built from Spatial Dual Contextual Blocks (Zhu et al., 2024). TVQE replaces deformable offset learning with a Swin-AutoEncoder based Spatio-Temporal feature Fusion module and a Channel-wise Attention based Quality Enhancement module, explicitly arguing that offset fields in deformable convolution are difficult to train and that windowed transformer attention improves execution efficiency while avoiding offset overflow (Yu et al., 2022).

A parallel line emphasizes frequency structure. MW-GAN applies multi-level Wavelet Packet Transform, a motion compensation pyramid, and wavelet-dense residual blocks to target perceptual quality rather than PSNR, on the premise that perceptual restoration depends primarily on recovering high-frequency sub-bands (Wang et al., 2020). HFUR, by contrast, models the quantization loss in the DCT domain as $\xi = \Theta - \Theta^*$, reconstructs it through Implicit Frequency Upsampling with expanded IDCT sampling, and then refines high- and low-frequency branches through Hierarchical and Iterative Refinement (Zhang et al., 2024). OVQE-VVC integrates spatio-temporal feature fusion, bidirectional propagation in a grid manner, and Omni-Frequency Adaptive Enhancement for VVC post-processing, thereby combining multi-frame propagation with explicit low-, mid-, and high-frequency enhancement (HoangVan et al., 28 Apr 2025).

3. Codec-aware conditioning and compressed-domain priors

A defining shift in recent CVQE research is the transition from pixel-only enhancement to codec-aware conditioning. Prediction-aware VVC enhancement conditions a residual CNN on the reconstructed component image $C$, the prediction signal $P$, and a normalized QP-map $X_t^{HQ} = X_t^{LQ} + X_t^{res}.$0, using

$X_t^{HQ} = X_t^{LQ} + X_t^{res}.$1

This framework uses one model for all QPs with a QP-map, shares the QP-map between luma and chroma, and reports that the prediction-aware aspect improves the coding efficiency gain of the default CNN-based QE method by $X_t^{HQ} = X_t^{LQ} + X_t^{res}.$2 in BD-BR at the same network complexity (Nasiri et al., 2021).

Bitstream-aware models generalize this idea. MetaBit leverages decoder motion vectors instead of optical flow, exploits GOP structure, and conditions a single model on spatially and temporally varying quantization data, thereby handling CRF, CBR, and CQP without per-QP ensembles (Ehrlich et al., 2022). CPGA goes further by explicitly ingesting motion vectors, predictive frames, and residual frames through an Inter-frame Temporal Aggregation module and a Multi-scale Non-local Aggregation module, and accompanies this with the VCP dataset, comprising 300 videos with coding priors extracted from corresponding bitstreams (Zhu et al., 2024). Plug-and-Play Versatile Compressed Video Enhancement introduces Compression-Aware Adaptation and Bitstream-Aware Enhancement: sequence-wise CRF, frame-wise CRF or slice type, motion vectors, and partition maps are reused to estimate frame-adaptive enhancement parameters and to drive region-aware sparse refinement (Zeng et al., 21 Apr 2025).

The most structurally codec-aligned formulation appears in CRDS. That model introduces three inductive biases aligned with predictive video compression: residual extraction, domain transformation, and quantization. MuNA performs motion-aware residual estimation in a codec-oriented latent domain, LDRAE learns a latent degradation residual space, and Progressive Denoising via Intermediate Supervision decomposes enhancement into staged denoising sub-tasks consistent with uniform quantization noise increments (Sun et al., 2024). This suggests a broad methodological transition: later CVQE systems no longer treat compression artifacts as generic image noise, but as distortions governed by codec mechanics and therefore amenable to codec-conditioned restoration.

4. Optimization strategies, supervision, and adaptation

Training protocols in CVQE vary substantially with architectural assumptions. SDTS uses a three-phase strategy: motion compensation pretraining with raw frames, ENet training with Euclidean reconstruction loss, and joint fine-tuning with

$X_t^{HQ} = X_t^{LQ} + X_t^{res}.$3

It also trains separate models for low-quality frames and high-quality frames, exploiting adjacent HQFs to enhance LQFs under the VVC Low-Delay P configuration (Meng et al., 2019). TVQE adopts two-stage optimization as well, beginning with a Charbonnier objective and then switching to $X_t^{HQ} = X_t^{LQ} + X_t^{res}.$4 fine-tuning, with

$X_t^{HQ} = X_t^{LQ} + X_t^{res}.$5

where Stage-1 sets $X_t^{HQ} = X_t^{LQ} + X_t^{res}.$6 and Stage-2 sets $X_t^{HQ} = X_t^{LQ} + X_t^{res}.$7 (Yu et al., 2022).

More recent systems use intermediate supervision and latent priors. CRDS pre-trains LDRAE on multi-level hybrid targets $X_t^{HQ} = X_t^{LQ} + X_t^{res}.$8, activates intermediate supervision for the first 150k iterations, freezes the flow for the first 5k iterations, and freezes the LDRAE encoder for the first 20k iterations to preserve latent structure (Sun et al., 2024). HFUR minimizes a Charbonnier penalty between reconstructed and raw frames while coupling transformer-based pixel-domain processing with an implicit DCT-domain branch and quantization-aware modulation (Zhang et al., 2024). PeQuENet is trained adversarially with LSGAN, VGG perceptual loss, and feature matching, using

$X_t^{HQ} = X_t^{LQ} + X_t^{res}.$9

to favor perceptual quality and to support a single QP-conditional model across multiple HEVC QPs (Zhang et al., 2022). MW-GAN likewise emphasizes perceptual restoration through a weighted wavelet-domain content loss, motion loss, and adversarial supervision in wavelet space (Wang et al., 2020).

Adaptation to unknown compression strength has become a separate topic. A diffusion-based HEVC post-processor estimates a latent feature prior $X_t^{LQ}$0 from the decoded frame alone and injects it into CBAM-based UNet blocks, explicitly targeting mixed-QP and blind-QP deployment (Liu et al., 2023). A later blind method replaces global degradation vectors with pretrained multiscale degradation representations and introduces hierarchical termination, dynamically adjusting the number of artifact reduction stages according to the predicted compression level (Yu et al., 20 Nov 2025). The literature therefore spans QP-specific specialization, QP-conditioned single-model designs, and fully blind adaptation, with each choice tied to different deployment assumptions.

5. Benchmarks, datasets, and empirical findings

The empirical record is heterogeneous because evaluation protocols differ by codec, dataset, and objective. On VVC, SDTS_MC reports average $X_t^{LQ}$1PSNR of $X_t^{LQ}$2 dB at QP 37 and $X_t^{LQ}$3 dB at QP 32 over the VTM 3.0 baseline, with a best per-sequence gain of $X_t^{LQ}$4 dB on BasketballPass at QP 37; the same study highlights BasketballDrive at QP 37 as a challenging case where motion compensation reduces the negative impact to $X_t^{LQ}$5 dB (Meng et al., 2019). OVQE-VVC reports average $X_t^{LQ}$6PSNR of $X_t^{LQ}$7 dB at QP 32, $X_t^{LQ}$8 dB at QP 37, $X_t^{LQ}$9 dB at QP 42, and $$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$0 dB at QP 47, together with an average BD-rate of $$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$1 relative to STD-VVC (HoangVan et al., 28 Apr 2025).

On HEVC benchmarks, several models exceed $$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$2 dB on standard settings. HFUR reports average $$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$3PSNR of $$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$4 dB at QP 37 and $$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$5 dB at QP 27 in CQP mode, and $$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$6 dB at 800 kbps in CBR mode (Zhang et al., 2024). TVQE reports average $$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$7PSNR of $$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$8 dB, $$I_{t+1}^\prime(x, y) = \mathcal{I}\left(I_{t+1}(x + \Delta^x_{t+1},\; y + \Delta^y_{t+1})\right),$$9SSIM of $G_i = \{I_{k-i}, I_k, I_{k+i}\}$0, and a BD-rate reduction of $G_i = \{I_{k-i}, I_k, I_{k+i}\}$1 at QP 37, while also reducing memory relative to RFDA at 1080p (Yu et al., 2022). TGAFNet reports average $G_i = \{I_{k-i}, I_k, I_{k+i}\}$2PSNR of $G_i = \{I_{k-i}, I_k, I_{k+i}\}$3 dB and $G_i = \{I_{k-i}, I_k, I_{k+i}\}$4SSIM of $G_i = \{I_{k-i}, I_k, I_{k+i}\}$5 at QP 37 on MFQE 2.0, and improves up to about $G_i = \{I_{k-i}, I_k, I_{k+i}\}$6 dB over CF-STIF (Zhu et al., 2024). QG-ConvLSTM, an earlier recurrent method, reports average gains of $G_i = \{I_{k-i}, I_k, I_{k+i}\}$7 dB at QP 42 and $G_i = \{I_{k-i}, I_k, I_{k+i}\}$8 dB at QP 37 with only 646,907 parameters (Yang et al., 2019).

Codec-aware latent and bitstream-informed systems obtain stronger gains on newer datasets. CRDS reports $G_i = \{I_{k-i}, I_k, I_{k+i}\}$9PSNR $i \in \{1,2,3\}$0 dB and $i \in \{1,2,3\}$1SSIM $i \in \{1,2,3\}$2 on LDV 2.0, and is described as the first to exceed 2 dB average $i \in \{1,2,3\}$3PSNR on that benchmark (Sun et al., 2024). CPGA reaches $i \in \{1,2,3\}$4 dB and $i \in \{1,2,3\}$5 average $i \in \{1,2,3\}$6SSIM at QP 37 under LDB, while the full combination of motion vectors, predictive frames, and residuals improves by $i \in \{1,2,3\}$7 dB over the no-prior baseline (Zhu et al., 2024). MetaBit reports $i \in \{1,2,3\}$8 dB PSNR, $i \in \{1,2,3\}$9 SSIM, and $K=1,3,5$0 LPIPS at CRF 40, using one model across CRF 35, 40, and 50 (Ehrlich et al., 2022). The plug-and-play bitstream-aware framework reports REDS4 performance of $K=1,3,5$1 dB / $K=1,3,5$2 SSIM at CRF15 and $K=1,3,5$3 dB / $K=1,3,5$4 SSIM at CRF35, and additionally reports gains on downstream tasks such as video super-resolution, optical flow estimation, and video object segmentation (Zeng et al., 21 Apr 2025).

Challenge benchmarks reinforce these trends. In the NTIRE 2021 fixed-QP fidelity track, the baseline compressed videos averaged $K=1,3,5$5 dB PSNR and $K=1,3,5$6 MS-SSIM, while the top submission reached $K=1,3,5$7 dB and $K=1,3,5$8. In the fixed-bitrate fidelity track, the baseline averaged $K=1,3,5$9 dB and $\xi = \Theta - \Theta^*$0, while the top submission reached $\xi = \Theta - \Theta^*$1 dB and $\xi = \Theta - \Theta^*$2 (Yang et al., 2021). These results show that decoder-side CVQE can yield sizeable gains under both reference-software and practical x265 settings, although the absolute gains depend strongly on compression mode and target metric.

6. Trade-offs, blind enhancement, and open directions

A recurrent theme in CVQE is the tension between fidelity, perceptual quality, and efficiency. MW-GAN demonstrates the perception–distortion trade-off explicitly: it lowers LPIPS and PI across all classes and sequences, yields the highest average $\xi = \Theta - \Theta^*$3MOS of $\xi = \Theta - \Theta^*$4 at QP 37, but reduces PSNR by $\xi = \Theta - \Theta^*$5 dB at QP 32 and $\xi = \Theta - \Theta^*$6 dB at QP 37 (Wang et al., 2020). PeQuENet makes the same point from a different angle: at QP 32 it attains the best average LPIPS/DISTS of $\xi = \Theta - \Theta^*$7 among the compared perceptual methods, while its average PSNR/SSIM is $\xi = \Theta - \Theta^*$8 dB / $\xi = \Theta - \Theta^*$9, below fidelity-oriented HEVC baselines such as STDF (Zhang et al., 2022). The literature therefore rejects the common misconception that higher PSNR and better visual quality are equivalent objectives.

Another persistent issue is model specialization versus generalization. Early and mid-period methods frequently train separate models for QPs, frame types, or frame-quality strata. SDTS trains separate models for LQFs and HQFs (Meng et al., 2019), and the prediction-aware VVC framework distinguishes inter, intra, and skip regions at inference (Nasiri et al., 2021). Later work shifts toward unified models conditioned on QP maps, CRF, slice type, or blind degradation priors (Ehrlich et al., 2022, Liu et al., 2023, Yu et al., 20 Nov 2025). This suggests that QP-aware or blind conditioning is becoming necessary for real deployments, where QP may be unknown, partially known, or variable across streams and devices.

Robustness under hard content remains unresolved. Fast motion, occlusions, scene cuts, inaccurate motion vectors, and very heavy compression are repeatedly reported as failure modes. SDTS shows negative gains without motion compensation on BasketballDrive (Meng et al., 2019); CRDS notes that attention windows may miss true correspondences under severe motion with occlusions (Sun et al., 2024); TGAFNet states that very fast motion or scene cuts reduce the utility of long-term groups (Zhu et al., 2024); OVQE-VVC reports a regression of $C$0 dB on BasketballDrill at QP 32 (HoangVan et al., 28 Apr 2025). Complexity reporting is also uneven: several influential papers do not provide parameters, FLOPs, memory, or runtime, complicating fair deployment-oriented comparison (Meng et al., 2019, HoangVan et al., 28 Apr 2025).

The field’s forward-looking agenda is now relatively clear. The benchmarking literature calls for VVC-specific compressed-domain features such as ALF signals and QTMT partition maps, multi-criteria evaluation that combines fidelity and complexity, cross-codec generalization, hardware-aware co-design, and hybrid compressed-domain plus pixel-domain approaches that stably fuse motion vectors, residuals, and block geometry (HoangVan et al., 12 Sep 2025). Individual method papers point to stronger alignment, recurrent or transformer-based temporal attention, adaptive scheduling tied to bitstream metadata, multi-scale fusion, and perceptual objectives beyond PSNR (Meng et al., 2019, Sun et al., 2024, Zeng et al., 21 Apr 2025). Taken together, these directions indicate that CVQE has moved from generic post-filtering toward a codec-aware, deployment-conscious discipline in which artifact restoration, temporal coherence, and computational adaptivity are treated as coupled design constraints rather than isolated goals.