Enhancing Quality for HEVC Compressed Videos (1709.06734v2)

Abstract: The latest High Efficiency Video Coding (HEVC) standard has been increasingly applied to generate video streams over the Internet. However, HEVC compressed videos may incur severe quality degradation, particularly at low bit-rates. Thus, it is necessary to enhance the visual quality of HEVC videos at the decoder side. To this end, this paper proposes a Quality Enhancement Convolutional Neural Network (QE-CNN) method that does not require any modification of the encoder to achieve quality enhancement for HEVC. In particular, our QE-CNN method learns QE-CNN-I and QE-CNN-P models to reduce the distortion of HEVC I and P frames, respectively. The proposed method differs from the existing CNN-based quality enhancement approaches, which only handle intra-coding distortion and are thus not suitable for P frames. Our experimental results validate that our QE-CNN method is effective in enhancing quality for both I and P frames of HEVC videos. To apply our QE-CNN method in time-constrained scenarios, we further propose a Time-constrained Quality Enhancement Optimization (TQEO) scheme. Our TQEO scheme controls the computational time of QE-CNN to meet a target, meanwhile maximizing the quality enhancement. Next, the experimental results demonstrate the effectiveness of our TQEO scheme from the aspects of time control accuracy and quality enhancement under different time constraints. Finally, we design a prototype to implement our TQEO scheme in a real-time scenario.

• The paper introduces a novel QE-CNN framework that enhances both I and P/B frames in HEVC videos.
• It employs specialized models (QE-CNN-I and QE-CNN-P) with optimized convolutional layers to mitigate intra- and inter-coding artifacts.
• The method achieves significant Y-PSNR gains and integrates a time-constrained optimization scheme for efficient real-time processing.

Enhancing Quality for HEVC Compressed Videos

The paper "Enhancing Quality for HEVC Compressed Videos" presents a method to address quality degradation in High Efficiency Video Coding (HEVC) compressed videos, particularly at low bit-rates, by leveraging convolutional neural networks (CNNs). The authors propose a novel Quality Enhancement CNN (QE-CNN) method designed to improve the quality of both I and P/B frames of HEVC compressed videos by enhancing intra- and inter-coding distortions.

Methodology Overview

The QE-CNN approach involves two specific models: QE-CNN-I and QE-CNN-P, targeting I frames and P/B frames, respectively. Unlike existing CNN-based quality enhancement techniques, which predominantly focus on intra-coding distortions, the proposed method integrates intra- and inter-coding features, thus offering comprehensive quality enhancement across different frame types.

QE-CNN-I Model: This model is structured to extract and mitigate intra-coding distortions. The architecture includes five convolutional layers designed to progressively refine features and improve video quality. Parameters such as number of filters and activation functions like Parametric Rectified Linear Unit (PReLU) are empirically optimized for superior performance.

QE-CNN-P Model: Addressing both intra- and inter-coding distortions, QE-CNN-P leverages a nine-layer architecture. The model employs concatenated features from intra-coding to enhance inter-coding frames specifically. Novel network configurations are fine-tuned to manage varying distortion characteristics between I and P/B frames.

Experimental Results

Numerically, the QE-CNN method achieves significant improvement in Y-PSNR compared to prior methods (AR-CNN, VRCNN, DCAD). For I frames, QE-CNN-I shows enhancement gains up to 0.5065 dB, markedly outperforming other conventional methods. Similarly, QE-CNN-P provides an average improvement of 0.3407 dB for P frames, demonstrating its capability to tackle inter-coding figures prominently.

In addition to objective measures, subjective evaluations and DMOS scores indicate marked betterment in video quality, further underpinning the efficacy of the proposed method.

Time-Constrained Optimization

The paper introduces a Time-constrained Quality Enhancement Optimization (TQEO) scheme to enable efficient real-time processing. This scheme controls computational time while maximizing quality enhancement, a crucial feature for practical applications, especially in resource-constrained environments. A detailed algorithm leverages look-up tables and polynomial regression to govern processing time and ensure real-time feasibility without compromising enhancement quality.

Practical Implications and Future Directions

The QE-CNN method and TQEO scheme offered in this paper provide a significant stride toward practical real-time enhancement of HEVC compressed videos. The prototype discussed highlights the method's applicability in real-world scenarios, despite needing substantial compute resources such as GPU arrays. Future work could explore the application of such methods on emerging hardware accelerators to achieve even faster processing speeds.

From a theoretical standpoint, this paper expands upon the versatility of deep learning architectures for video quality enhancement and intimates the feasibility of integrating intraframe and interframe distortion correction effectively within neural networks—potentially influencing future research trajectories in video coding and enhancement systems. Further research could explore the optimization for various resolutions and diverse artifacts presented by other forms of compression beyond HEVC. Overall, this paper makes a substantial contribution to the field of video coding enhancement through the integration of convolutional neural networks and time-constrained optimization methodologies.