Compressed learning based onboard semantic compression for remote sensing platforms (2409.01988v1)
Abstract: Earth observation (EO) plays a crucial role in creating and sustaining a resilient and prosperous society that has far reaching consequences for all life and the planet itself. Remote sensing platforms like satellites, airborne platforms, and more recently dones and UAVs are used for EO. They collect large amounts of data and this needs to be downlinked to Earth for further processing and analysis. Bottleneck for such high throughput acquisition is the downlink bandwidth. Data-centric solutions to image compression is required to address this deluge. In this work, semantic compression is studied through a compressed learning framework that utilizes only fast and sparse matrix-vector multiplication to encode the data. Camera noise and a communication channel are the considered sources of distortion. The complete semantic communication pipeline then consists of a learned low-complexity compression matrix that acts on the noisy camera output to generate onboard a vector of observations that is downlinked through a communication channel, processed through an unrolled network and then fed to a deep learning model performing the necessary downstream tasks; image classification is studied. Distortions are compensated by unrolling layers of NA-ALISTA with a wavelet sparsity prior. Decoding is thus a plug-n-play approach designed according to the camera/environment information and downstream task. The deep learning model for the downstream task is jointly fine-tuned with the compression matrix and the unrolled network through the loss function in an end-to-end fashion. It is shown that addition of a recovery loss along with the task dependent losses improves the downstream performance in noisy settings at low compression ratios.
- J. Bacca, T. Gelvez-Barrera, and H. Arguello, “Deep coded aperture design: An end-to-end approach for computational imaging tasks,” IEEE Trans. Comput. Imag., vol. 7, pp. 1148–1160, 2021.
- J. Sauder, M. Genzel, and P. Jung, “Gradient-based learning of discrete structured measurement operators for signal recovery,” IEEE J. Sel. Areas Inf. Theory, vol. 3, no. 3, pp. 481–492, 2022.
- V. Monga, Y. Li, and Y. C. Eldar, “Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,” IEEE Signal Process. Mag., vol. 38, no. 2, pp. 18–44, 2021.
- M. A. Davenport, M. F. Duarte, M. B. Wakin, J. N. Laska, D. Takhar, K. F. Kelly, and R. G. Baraniuk, “The smashed filter for compressive classification and target recognition,” in Proc. SPIE 6498, Computational Imaging V, vol. 6498, 2007, p. 64980H.
- S. Lohit, K. Kulkarni, and P. Turaga, “Direct inference on compressive measurements using convolutional neural networks,” in 2016 IEEE Int. Conf. on Image Processing (ICIP), 2016, pp. 1913–1917.
- E. Zisselmann, A. Adler, and M. Elad, “Chapter 1 - compressed learning for image classification: A deep neural network approach,” in Processing, Analyzing and Learning of Images, Shapes, and Forms: Part 1, ser. Handbook of Numerical Analysis, R. Kimmel and X.-C. Tai, Eds. Elsevier, 2018, vol. 19, pp. 3–17.
- F. Behrens, J. Sauder, and P. Jung, “Neurally augmented ALISTA,” in Proc. Int. Conf. Learning Representations (ICLR), 2021.
- J. Ansel et al., “PyTorch 2: Faster machine learning through dynamic python bytecode transformation and graph compilation,” in Proc. of the 29th ACM Int. Conf. on Architectural Support for Programming Languages and Operating Systems, Volume 2, 2024, p. 929–947.
- Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
- E. Larsen, D. Noever, K. MacVittie, and J. Lilly, “Overhead-MNIST: Machine learning baselines for image classification,” arXiv preprint arXiv:2107.00436v2, 2021.
- S. Basu, M. Karki, S. Ganguly, R. DiBiano, S. Mukhopadhyay, S. Gayaka, R. Kannan, and R. Nemani, “Learning sparse feature representations using probabilistic quadtrees and deep belief nets,” Neural Processing Letters, vol. 45, pp. 855–867, 6 2017.
- D. Lam et al., “xView: Objects in context in overhead imagery,” arXiv preprint arXiv:1802.07856v1, 2018.
- Y. Yang and S. Newsam, “Bag-of-visual-words and spatial extensions for land-use classification,” in Proceedings of the 18th SIGSPATIAL Int. Conf. on Advances in Geographic Information Systems. Association for Computing Machinery, 2010, p. 270–279.
- G.-S. Xia, X. Bai, J. Ding, Z. Zhu, S. Belongie, J. Luo, M. Datcu, M. Pelillo, and L. Zhang, “Dota: A large-scale dataset for object detection in aerial images,” in 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 2018, pp. 3974–3983.
- A. V. Atten, D. Lindenbaum, and T. M. Bacastow, “SpaceNet: : A remote sensing dataset and challenge series,” arXiv preprint arXiv:1807.01232v3, 2019.
- D. Hendrycks and T. Dietterich, “Benchmarking neural network robustness to common corruptions and perturbations,” in Proc. Int. Conf. Learning Representations (ICLR), 2019.