Deep Network for Image Compressed Sensing Coding Using Local Structural Sampling (2402.19111v1)
Abstract: Existing image compressed sensing (CS) coding frameworks usually solve an inverse problem based on measurement coding and optimization-based image reconstruction, which still exist the following two challenges: 1) The widely used random sampling matrix, such as the Gaussian Random Matrix (GRM), usually leads to low measurement coding efficiency. 2) The optimization-based reconstruction methods generally maintain a much higher computational complexity. In this paper, we propose a new CNN based image CS coding framework using local structural sampling (dubbed CSCNet) that includes three functional modules: local structural sampling, measurement coding and Laplacian pyramid reconstruction. In the proposed framework, instead of GRM, a new local structural sampling matrix is first developed, which is able to enhance the correlation between the measurements through a local perceptual sampling strategy. Besides, the designed local structural sampling matrix can be jointly optimized with the other functional modules during training process. After sampling, the measurements with high correlations are produced, which are then coded into final bitstreams by the third-party image codec. At last, a Laplacian pyramid reconstruction network is proposed to efficiently recover the target image from the measurement domain to the image domain. Extensive experimental results demonstrate that the proposed scheme outperforms the existing state-of-the-art CS coding methods, while maintaining fast computational speed.
- Generative Adversarial Networks for Extreme Learned Image Compression. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 221–231. https://doi.org/10.1109/ICCV.2019.00031
- Emmanuel J Candès and Michael B Wakin. 2008. An introduction to compressive sampling. IEEE Signal Processing Magazine 25, 2 (2008), 21–30.
- Bin Chen and Jian Zhang. 2022. Content-aware scalable deep compressed sensing. IEEE Transactions on Image Processing 31 (2022), 5412–5426.
- End-to-End Learnt Image Compression via Non-Local Attention Optimization and Improved Context Modeling. IEEE Transactions on Image Processing 30 (2021), 3179–3191. https://doi.org/10.1109/TIP.2021.3058615
- Efficient and Robust Image Coding and Transmission Based on Scrambled Block Compressive Sensing. IEEE Transactions on Multimedia 20, 7 (2018), 1610–1621.
- Compressive Sensing Multi-Layer Residual Coefficients for Image Coding. IEEE Transactions on Circuits and Systems for Video Technology 30, 4 (2020), 1109–1120. https://doi.org/10.1109/TCSVT.2019.2898908
- Wonwoo Cho and Nam Yul Yu. 2020. Secure and Efficient Compressed Sensing-Based Encryption With Sparse Matrices. IEEE Transactions on Information Forensics and Security 15 (2020), 1999–2011. https://doi.org/10.1109/TIFS.2019.2953383
- An Efficient Deep Quantized Compressed Sensing Coding Framework of Natural Images. ACM International Conference on Multimedia (MM) (2018), 1777–1785. https://doi.org/10.1145/3240508.3240706
- An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences 57, 11 (2004), 1413–1457.
- K. Q. Dinh and B. Jeon. 2017. Iterative Weighted Recovery for Block-Based Compressive Sensing of Image/Video at a Low Subrate. IEEE Transactions on Circuits and Systems for Video Technology 27, 11 (2017), 2294–2308. https://doi.org/10.1109/TCSVT.2016.2587398
- Measurement coding for compressive imaging using a structural measuremnet matrix. In 2013 IEEE International Conference on Image Processing. 10–13. https://doi.org/10.1109/ICIP.2013.6738003
- Compressive sensing via nonlocal low-rank regularization. IEEE Transactions on Image Processing (TIP) 23, 8 (2014), 3618–3632.
- David L Donoho. 2006. Compressed sensing. IEEE Transactions on Information Theory 52, 4 (2006), 1289–1306.
- Single-pixel imaging via compressive sampling. IEEE Signal Processing Magazine 25, 2 (2008), 83–91. https://doi.org/10.1109/MSP.2007.914730
- Mansoor Ebrahim and Wai Chong Chia. 2015. Multiview Image Block Compressive Sensing with Joint Multiphase Decoding for Visual Sensor Network. ACM Trans. Multimedia Comput. Commun. Appl. 12, 2, Article 30 (oct 2015), 23 pages. https://doi.org/10.1145/2818712
- Mark Everingham and John Winn. 2011. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Development Kit.
- Lu Gan. 2007. Block compressed sensing of natural images. Proceedings of the international conference on digital signal processing (2007), 403–406.
- Block-based compressive sensing coding of natural images by local structural measurement matrix. IEEE Data Compression Conference (DCC) (2015), 133–142.
- Hierarchical Perception Adversarial Learning Framework for Compressed Sensing MRI. IEEE Transactions on Medical Imaging (2023), 1–1. https://doi.org/10.1109/TMI.2023.3240862
- Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. IEEE International Conference on Computer Vision (ICCV) (2015), 1026–1034.
- Lensless imaging by compressive sensing. In 2013 IEEE International Conference on Image Processing. 2101–2105. https://doi.org/10.1109/ICIP.2013.6738433
- Dequantizing compressed sensing: When oversampling and non-gaussian constraints combine. IEEE Transactions on Information Theory 57, 1 (2011), 559–571.
- Light Field Image Compression Using Generative Adversarial Network-Based View Synthesis. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9, 1 (2019), 177–189. https://doi.org/10.1109/JETCAS.2018.2886642
- An End-to-End Compression Framework Based on Convolutional Neural Networks. IEEE Transactions on Circuits and Systems for Video Technology 28, 10 (2018), 3007–3018. https://doi.org/10.1109/TCSVT.2017.2734838
- Tval3: Tv minimization by augmented lagrangian and alternating direction agorithm 2009.
- Efficient and Effective Context-Based Convolutional Entropy Modeling for Image Compression. IEEE Transactions on Image Processing 29 (2020), 5900–5911. https://doi.org/10.1109/TIP.2020.2985225
- Learning Content-Weighted Deep Image Compression. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020), 1–1. https://doi.org/10.1109/TPAMI.2020.2983926
- Learning a Convolutional Neural Network for Image Compact-Resolution. IEEE Transactions on Image Processing 28, 3 (2019), 1092–1107. https://doi.org/10.1109/TIP.2018.2872876
- Convolutional Neural Network-Based Block Up-Sampling for Intra Frame Coding. IEEE Transactions on Circuits and Systems for Video Technology 28, 9 (2018), 2316–2330. https://doi.org/10.1109/TCSVT.2017.2727682
- Image encryption based on compressive sensing and scrambled index for secure multimedia transmission. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 12, 4s (2016), 1–22.
- Convolutional Neural Network-Based Block Up-Sampling for HEVC. IEEE Transactions on Circuits and Systems for Video Technology 29, 12 (2019), 3701–3715. https://doi.org/10.1109/TCSVT.2018.2884203
- Compressive Sampling-Based Image Coding for Resource-Deficient Visual Communication. IEEE Transactions on Image Processing 25, 6 (2016), 2844–2855.
- Deep Compressive Imaging With Meta-Learning. IEEE Transactions on Instrumentation and Measurement 72 (2022), 1–9.
- Compressed Sensing MRI. IEEE Signal Processing Magazine 25, 2 (2008), 72–82.
- Stéphane G Mallat and Zhifeng Zhang. 1993. Matching pursuits with time-frequency dictionaries. IEEE Transactions on signal processing 41, 12 (1993), 3397–3415.
- From Denoising to Compressed Sensing. IEEE Transactions on Information Theory 62, 9 (2016), 5117–5144.
- A deep learning approach to structured signal recovery. IEEE Allerton Conference on Communication, Control, and Computing (Allerton) (2015), 1336–1343.
- Sungkwang Mun and James E Fowler. 2012. DPCM for quantized blockbased compressed sensing of images. IEEE Signal Processing Conference (2012), 1424–1428.
- Scalable Convolutional Neural Network for Image Compressed Sensing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 12290–12299.
- Image Compressed Sensing Using Convolutional Neural Network. IEEE Transactions on Image Processing 29, 1 (2020), 375–388.
- Video Compressed Sensing Using a Convolutional Neural Network. IEEE Transactions on Circuits and Systems for Video Technology 31, 2 (2021), 425–438. https://doi.org/10.1109/TCSVT.2020.2978703
- Structure-Aware Deep Networks and Pixel-Level Generative Adversarial Training for Single Image Super-Resolution. IEEE Transactions on Instrumentation and Measurement 72 (2023), 1–14. https://doi.org/10.1109/TIM.2023.3246523
- Memory-augmented deep unfolding network for compressive sensing. In Proceedings of the 29th ACM International Conference on Multimedia. 4249–4258.
- Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology 22, 12 (2012), 1649–1668.
- Dual-Path Attention Network for Compressed Sensing Image Reconstruction. IEEE Transactions on Image Processing 29 (2020), 9482–9495. https://doi.org/10.1109/TIP.2020.3023629
- Feature-supervised compressed sensing for microwave imaging systems. IEEE Transactions on Instrumentation and Measurement 69, 8 (2019), 5287–5297.
- Bi-directional intra prediction based measurement coding for compressive sensing images. In 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP). 1–6. https://doi.org/10.1109/MMSP48831.2020.9287074
- Joel A Tropp and Anna C Gilbert. 2007. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on information theory 53, 12 (2007), 4655–4666.
- APMC: Adjacent Pixels Based Measurement Coding System for Compressively Sensed Images. IEEE Transactions on Multimedia 24 (2022), 3558–3569. https://doi.org/10.1109/TMM.2021.3102394
- Versatile Denoising-Based Approximate Message Passing for Compressive Sensing. IEEE Transactions on Image Processing 32 (2023), 2761–2775.
- Overview of the H. 264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology 13, 7 (2003), 560–576.
- Sparse reconstruction by separable approximation. IEEE Transactions on Signal Processing 57, 7 (2009), 2479–2493.
- Privacy-preserving multimedia big data aggregation in large-scale wireless sensor networks. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 12, 4s (2016), 1–19.
- Compressive blind mixing matrix estimation of audio signals. IEEE Transactions on Instrumentation and Measurement 63, 5 (2014), 1253–1261.
- LAPRAN: A Scalable Laplacian Pyramid Reconsructive Adversarial Network for Flexible Compressive Sensing Reconstruction. Springer European Conference on Computer Vision (ECCV) (2018), 491–507.
- Shrinkage-based alternating projection algorithm for efficient measurement matrix construction in compressive sensing. IEEE Transactions on Instrumentation and Measurement 63, 5 (2014), 1073–1084.
- Compressed Imaging Reconstruction with Sparse Random Projection. ACM Trans. Multimedia Comput. Commun. Appl. 17, 1, Article 26 (apr 2021), 25 pages. https://doi.org/10.1145/3447431
- Variational Bayesian Algorithm for Quantized Compressed Sensing. IEEE Transactions on Signal Processing 61, 11 (2013), 2815–2824.
- DR2-Net: Deep Residual Reconstruction Network for Image Compressive Sensing. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017).
- COAST: COntrollable Arbitrary-Sampling NeTwork for Compressive Sensing. IEEE Transactions on Image Processing 30 (2021), 6066–6080. https://doi.org/10.1109/TIP.2021.3091834
- Xin Yuan and Raziel Haimi-Cohen. 2020. Image Compression Based on Compressive Sensing: End-to-End Comparison With JPEG. IEEE Transactions on Multimedia 22, 11 (2020), 2889–2904. https://doi.org/10.1109/TMM.2020.2967646
- Jian Zhang and Bernard Ghanem. 2018. ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018), 1828–1837.
- Optimization-Inspired Compact Deep Compressive Sensing. IEEE Journal of Selected Topics in Signal Processing 14, 4 (2020), 765–774. https://doi.org/10.1109/JSTSP.2020.2977507
- Group-based Sparse Representation for Image Restoration. IEEE Transactions on Image Processing (TIP) 23, 8 (2014), 3336–3351.
- Spatially Directional Predictive Coding for Block-based Compressive Sensing of Natural Images. IEEE International Conference on Image Processing (ICIP) (2013), 1021–1025.
- Compressed sensing recovery via collaborative sparsity. In Data Compression Conference (DCC), 2012. IEEE, 287–296.
- AMP-Net: Denoising-Based Deep Unfolding for Compressive Image Sensing. IEEE Transactions on Image Processing 30 (2021), 1487–1500. https://doi.org/10.1109/TIP.2020.3044472
- Nonconvex lp nuclear norm based admm framework for compressed sensing. In 2016 Data Compression Conference (DCC). IEEE, 161–170.
- Multiple Description Convolutional Neural Networks for Image Compression. IEEE Transactions on Circuits and Systems for Video Technology 29, 8 (2019), 2494–2508. https://doi.org/10.1109/TCSVT.2018.2867067
- Efficient Chroma Sub-Sampling and Luma Modification for Color Image Compression. IEEE Transactions on Circuits and Systems for Video Technology 29, 5 (2019), 1559–1563. https://doi.org/10.1109/TCSVT.2019.2895840
- Wenxue Cui (10 papers)
- Xingtao Wang (20 papers)
- Xiaopeng Fan (39 papers)
- Shaohui Liu (54 papers)
- Xinwei Gao (6 papers)
- Debin Zhao (33 papers)