Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Sensitivity Decouple Learning for Image Compression Artifacts Reduction (2405.09291v1)

Published 15 May 2024 in cs.CV, cs.AI, and eess.IV

Abstract: With the benefit of deep learning techniques, recent researches have made significant progress in image compression artifacts reduction. Despite their improved performances, prevailing methods only focus on learning a mapping from the compressed image to the original one but ignore the intrinsic attributes of the given compressed images, which greatly harms the performance of downstream parsing tasks. Different from these methods, we propose to decouple the intrinsic attributes into two complementary features for artifacts reduction,ie, the compression-insensitive features to regularize the high-level semantic representations during training and the compression-sensitive features to be aware of the compression degree. To achieve this, we first employ adversarial training to regularize the compressed and original encoded features for retaining high-level semantics, and we then develop the compression quality-aware feature encoder for compression-sensitive features. Based on these dual complementary features, we propose a Dual Awareness Guidance Network (DAGN) to utilize these awareness features as transformation guidance during the decoding phase. In our proposed DAGN, we develop a cross-feature fusion module to maintain the consistency of compression-insensitive features by fusing compression-insensitive features into the artifacts reduction baseline. Our method achieves an average 2.06 dB PSNR gains on BSD500, outperforming state-of-the-art methods, and only requires 29.7 ms to process one image on BSD500. Besides, the experimental results on LIVE1 and LIU4K also demonstrate the efficiency, effectiveness, and superiority of the proposed method in terms of quantitative metrics, visual quality, and downstream machine vision tasks.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. G. Zhai, W. Zhang, X. Yang, W. Lin, and Y. Xu, “Efficient deblocking with coefficient regularization, shape-adaptive filtering, and quantization constraint,” IEEE Trans. Multimedia, vol. 10, no. 5, pp. 735–745, 2008.
  2. S. B. Yoo, K. Choi, and J. B. Ra, “Post-processing for blocking artifact reduction based on inter-block correlation,” IEEE Trans. Multimedia, vol. 16, no. 6, pp. 1536–1548, 2014.
  3. A. Foi, V. Katkovnik, and K. Egiazarian, “Pointwise shape-adaptive dct for high-quality denoising and deblocking of grayscale and color images,” IEEE Trans. Image Process., vol. 16, no. 5, pp. 1395–1411, 2007.
  4. J. Zhang, R. Xiong, C. Zhao, Y. Zhang, S. Ma, and W. Gao, “Concolor: Constrained non-convex low-rank model for image deblocking,” IEEE Trans. Image Process., vol. 25, no. 3, pp. 1246–1259, 2016.
  5. X. Liu, X. Wu, J. Zhou, and D. Zhao, “Data-driven soft decoding of compressed images in dual transform-pixel domain,” IEEE Trans. Image Process., vol. 25, no. 4, pp. 1649–1659, 2016.
  6. K. Bredies and M. Holler, “A total variation–based jpeg decompression model,” SIAM Journal on Imaging Sciences, vol. 5, no. 1, pp. 366–393, 2012.
  7. H. Chang, M. K. Ng, and T. Zeng, “Reducing artifacts in jpeg decompression via a learned dictionary,” IEEE Trans. Signal Process., vol. 62, no. 3, pp. 718–728, 2013.
  8. Q. Song, R. Xiong, X. Fan, D. Liu, F. Wu, T. Huang, and W. Gao, “Compressed image restoration via artifacts-free pca basis learning and adaptive sparse modeling,” IEEE Trans. Image Process., vol. 29, pp. 7399–7413, 2020.
  9. X. Zhang, W. Lin, R. Xiong, X. Liu, S. Ma, and W. Gao, “Low-rank decomposition-based restoration of compressed images via adaptive noise estimation,” IEEE Trans. Image Process., vol. 25, no. 9, pp. 4158–4171, 2016.
  10. X. Zhang, R. Xiong, X. Fan, S. Ma, and W. Gao, “Compression artifact reduction by overlapped-block transform coefficient estimation with block similarity,” IEEE Trans. Image Process., vol. 22, no. 12, pp. 4613–4626, 2013.
  11. X. Liu, G. Cheung, X. Ji, D. Zhao, and W. Gao, “Graph-based joint dequantization and contrast enhancement of poorly lit jpeg images,” IEEE Trans. Image Process., vol. 28, no. 3, pp. 1205–1219, 2018.
  12. K. Zhang and R. Timofte, “Deep plug-and-play and deep unfolding methods for image restoration,” in Advanced Methods and Deep Learning in Computer Vision.   Elsevier, 2022, pp. 481–509.
  13. J. Jiang, K. Zhang, and R. Timofte, “Towards flexible blind jpeg artifacts removal,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), 2021, pp. 4997–5006.
  14. T. Vandal, E. Kodra, S. Ganguly, A. Michaelis, R. Nemani, and A. R. Ganguly, “Generating high resolution climate change projections through single image super-resolution: An abridged version,” in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 2018.
  15. Y. Zhang, K. Li, K. Li, B. Zhong, and Y. Fu, “Residual non-local attention networks for image restoration,” arXiv preprint arXiv:1903.10082, 2019.
  16. K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,” IEEE Trans. Image Process., vol. 26, no. 7, pp. 3142–3155, 2017.
  17. K. Li, B. Bare, and B. Yan, “An efficient deep convolutional neural networks model for compressed image deblocking,” in Proc. IEEE Int. Conf. Multimedia Expo (ICME).   IEEE, 2017, pp. 1320–1325.
  18. W. Wang, R. Guo, Y. Tian, and W. Yang, “Cfsnet: Toward a controllable feature space for image restoration,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), 2019, pp. 4140–4149.
  19. Y. Kim, J. W. Soh, and N. I. Cho, “Agarnet: adaptively gated jpeg compression artifacts removal network for a wide range quality factor,” IEEE Access, vol. 8, pp. 20 160–20 170, 2020.
  20. H. Chen, X. He, C. An, and T. Q. Nguyen, “Adaptive image coding efficiency enhancement using deep convolutional neural networks,” Inf. Sci., vol. 524, pp. 298–317, 2020.
  21. Z. Jin, M. Z. Iqbal, W. Zou, X. Li, and E. Steinbach, “Dual-stream multi-path recursive residual network for jpeg image compression artifacts reduction,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 2, pp. 467–479, 2020.
  22. H. Qiu, Q. Zheng, G. Memmi, J. Lu, M. Qiu, and B. Thuraisingham, “Deep residual learning-based enhanced jpeg compression in the internet of things,” IEEE Trans. Ind. Inform., vol. 17, no. 3, pp. 2124–2133, 2020.
  23. Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, “Residual dense network for image restoration,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 7, pp. 2480–2495, 2020.
  24. X. Fu, Z.-J. Zha, F. Wu, X. Ding, and J. Paisley, “Jpeg artifacts reduction via deep convolutional sparse coding,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2019, pp. 2501–2510.
  25. Z. Wang, D. Liu, S. Chang, Q. Ling, Y. Yang, and T. S. Huang, “D3: Deep dual-domain based fast restoration of jpeg-compressed images,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016, pp. 2764–2772.
  26. J. Guo and H. Chao, “Building dual-domain representations for compression artifacts reduction,” in Proc. Eur. Conf. Comput. Vis. (ECCV).   Springer, 2016, pp. 628–644.
  27. H. Chen, X. He, L. Qing, S. Xiong, and T. Q. Nguyen, “Dpw-sdnet: Dual pixel-wavelet domain deep cnns for soft decoding of jpeg-compressed images,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshop (CVPRW), 2018, pp. 711–720.
  28. X. Zhang, W. Yang, Y. Hu, and J. Liu, “Dmcnn: Dual-domain multi-scale convolutional neural network for compression artifacts removal,” in Proc. IEEE Int. Conf. Image Process. (ICIP).   IEEE, 2018, pp. 390–394.
  29. M. Sun, X. He, S. Xiong, C. Ren, and X. Li, “Reduction of jpeg compression artifacts based on dct coefficients prediction,” Neurocomputing, vol. 384, pp. 335–345, 2020.
  30. L. Galteri, L. Seidenari, M. Bertini, and A. Del Bimbo, “Deep generative adversarial compression artifact removal,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017, pp. 4826–4835.
  31. J. Guo and H. Chao, “One-to-many network for visually pleasing compression artifacts reduction,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017, pp. 3038–3047.
  32. L. Galteri, L. Seidenari, M. Bertini, and A. Del Bimbo, “Deep universal generative adversarial compression artifact removal,” IEEE Trans. Multimedia, vol. 21, no. 8, pp. 2131–2145, 2019.
  33. H. Chen, X. He, C. An, and T. Q. Nguyen, “Deep wide-activated residual network based joint blocking and color bleeding artifacts reduction for 4: 2: 0 jpeg-compressed images,” IEEE Signal Processing Lett., vol. 26, no. 1, pp. 79–83, 2018.
  34. B. Zheng, Y. Chen, X. Tian, F. Zhou, and X. Liu, “Implicit dual-domain convolutional network for robust color image compression artifact reduction,” IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 11, pp. 3982–3994, 2019.
  35. J. Li, Y. Wang, H. Xie, and K.-K. Ma, “Learning a single model with a wide range of quality factors for jpeg image artifacts removal,” IEEE Trans. Image Process., vol. 29, pp. 8842–8854, 2020.
  36. Y. Kim, J. W. Soh, J. Park, B. Ahn, H.-S. Lee, Y.-S. Moon, and N. I. Cho, “A pseudo-blind convolutional neural network for the reduction of compression artifacts,” IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 4, pp. 1121–1135, 2019.
  37. L. Ma, P. Peng, P. Xing, Y. Wang, and Y. Tian, “Reducing image compression artifacts for deep neural networks,” in Proc. Data Compress. Conf. (DCC).   IEEE, 2021, pp. 355–355.
  38. L. Ma, Y. Tian, P. Xing, and T. Huang, “Residual-based post-processing for hevc,” IEEE MultiMedia, vol. 26, no. 4, pp. 67–79, 2019.
  39. T. Wang, M. Chen, and H. Chao, “A novel deep learning-based method of improving coding efficiency from the decoder-end for hevc,” in Proc. Data Compress. Conf. (DCC).   IEEE, 2017, pp. 410–419.
  40. J. Liu, D. Liu, W. Yang, S. Xia, X. Zhang, and Y. Dai, “A comprehensive benchmark for single image compression artifact reduction,” IEEE Trans. Image Process., vol. 29, pp. 7845–7860, 2020.
  41. C. Dong, Y. Deng, C. Change Loy, and X. Tang, “Compression artifacts reduction by a deep convolutional network,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), 2015, pp. 576–584.
  42. J. Mu, R. Xiong, X. Fan, D. Liu, F. Wu, and W. Gao, “Graph-based non-convex low-rank regularization for image compression artifact reduction,” IEEE Trans. Image Process., vol. 29, pp. 5374–5385, 2020.
  43. H. Chen, X. He, H. Yang, L. Qing, and Q. Teng, “A feature-enriched deep convolutional neural network for jpeg image compression artifacts reduction and its applications,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 1, pp. 430–444, 2021.
  44. Z. Zha, B. Wen, X. Yuan, J. T. Zhou, J. Zhou, and C. Zhu, “Triply complementary priors for image restoration,” IEEE Trans. Image Process., vol. 30, pp. 5819–5834, 2021.
  45. H. Chen, Y. Wang, T. Guo, C. Xu, Y. Deng, Z. Liu, S. Ma, C. Xu, C. Xu, and W. Gao, “Pre-trained image processing transformer,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2021, pp. 12 299–12 310.
  46. J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool, and R. Timofte, “Swinir: Image restoration using swin transformer,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), 2021, pp. 1833–1844.
  47. J. Cao, Y. Li, K. Zhang, and L. Van Gool, “Video super-resolution transformer,” arXiv preprint arXiv:2106.06847, 2021.
  48. Z. Wang, X. Cun, J. Bao, and J. Liu, “Uformer: A general u-shaped transformer for image restoration,” arXiv preprint arXiv:2106.03106, 2021.
  49. K. Zhang, W. Zuo, and L. Zhang, “Deep plug-and-play super-resolution for arbitrary blur kernels,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2019, pp. 1671–1681.
  50. W. Dong, P. Wang, W. Yin, G. Shi, F. Wu, and X. Lu, “Denoising prior driven deep neural network for image restoration,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 10, pp. 2305–2318, 2018.
  51. Y. Jo, S. Y. Chun, and J. Choi, “Rethinking deep image prior for denoising,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), 2021, pp. 5087–5096.
  52. K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool, and R. Timofte, “Plug-and-play image restoration with deep denoiser prior,” IEEE Trans. Pattern Anal. Mach. Intell., 2021.
  53. X. Fu, X. Wang, A. Liu, J. Han, and Z.-J. Zha, “Learning dual priors for jpeg compression artifacts removal,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), 2021, pp. 4086–4095.
  54. X. Fu, M. Wang, X. Cao, X. Ding, and Z.-J. Zha, “A model-driven deep unfolding method for jpeg artifacts removal,” IEEE Trans. Neural Netw. Learn. Syst., 2021.
  55. R. Liu, L. Ma, Y. Zhang, X. Fan, and Z. Luo, “Underexposed image correction via hybrid priors navigated deep propagation,” IEEE Trans. Neural Netw. Learn. Syst., 2021.
  56. K. Zhang, W. Zuo, S. Gu, and L. Zhang, “Learning deep cnn denoiser prior for image restoration,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017, pp. 3929–3938.
  57. R. Liu, Z. Jiang, X. Fan, and Z. Luo, “Knowledge-driven deep unrolling for robust image layer separation,” IEEE Trans. Neural Netw. Learn. Syst., vol. 31, no. 5, pp. 1653–1666, 2019.
  58. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016.
  59. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2017, pp. 5998–6008.
  60. A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434, 2015.
  61. M. Ehrlich, L. Davis, S.-N. Lim, and A. Shrivastava, “Quantization guided jpeg artifact correction,” in Proc. Eur. Conf. Comput. Vis. (ECCV).   Springer, 2020, pp. 293–309.
  62. P. Liu, H. Zhang, K. Zhang, L. Lin, and W. Zuo, “Multi-level wavelet-cnn for image restoration,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshop (CVPRW), 2018, pp. 773–782.
  63. Y. Bengio et al., “Learning deep architectures for ai,” Found. Trends Mach. Learn., vol. 2, no. 1, pp. 1–127, 2009.
  64. A. Ng et al., “Sparse autoencoder,” CS294A Lecture notes, vol. 72, no. 2011, pp. 1–19, 2011.
  65. C.-Y. Liou, W.-C. Cheng, J.-W. Liou, and D.-R. Liou, “Autoencoder for words,” Neurocomputing, vol. 139, pp. 84–96, 2014.
  66. Q. Meng, D. Catchpoole, D. Skillicom, and P. J. Kennedy, “Relational autoencoder for feature extraction,” in Proc. IEEE Int. Joint Conf. Neural Netw. (IJCNN).   IEEE, 2017, pp. 364–371.
  67. M. Tschannen, O. Bachem, and M. Lucic, “Recent advances in autoencoder-based representation learning,” arXiv preprint arXiv:1812.05069, 2018.
  68. S. Lazebnik, C. Schmid, and J. Ponce, “Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), vol. 2, 2006, pp. 2169–2178.
  69. E. Agustsson and R. Timofte, “Ntire 2017 challenge on single image super-resolution: Dataset and study,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), 2017, pp. 1122–1131.
  70. R. e. Timofte, “Ntire 2017 challenge on single image super-resolution: Methods and results,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), 2017, pp. 1110–1121.
  71. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. Int. Conf. Learn.Represent. (ICLR), 2015.
  72. T. Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Doll¨¢r, “Microsoft coco: Common objects in context,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2014.
  73. S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” IEEE Trans. Pattern Anal. Mach. Intell., Jun 2017.
  74. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” Int. J. Comput. Vis., vol. 88, no. 2, pp. 303–338, 2010.
  75. H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017.
  76. Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, 2004.
  77. C. Yim and A. C. Bovik, “Quality assessment of deblocked images,” IEEE Trans. Image Process., vol. 20, no. 1, pp. 88–98, 2011.
  78. W. Gao, S. Ma, L. Duan, Y. Tian, P. Xing, Y. Wang, S. Wang, H. Jia, and T. Huang, “Digital retina: A way to make the city brain more efficient by visual coding,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 11, pp. 4147–4161, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Li Ma (153 papers)
  2. Yifan Zhao (66 papers)
  3. Peixi Peng (24 papers)
  4. Yonghong Tian (184 papers)
Citations (1)
View on Semantic Scholar