Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

D$^\text{2}$UF: Deep Coded Aperture Design and Unrolling Algorithm for Compressive Spectral Image Fusion (2205.12158v1)

Published 24 May 2022 in eess.IV, cs.LG, and math.OC

Abstract: Compressive spectral imaging (CSI) has attracted significant attention since it employs synthetic apertures to codify spatial and spectral information, sensing only 2D projections of the 3D spectral image. However, these optical architectures suffer from a trade-off between the spatial and spectral resolution of the reconstructed image due to technology limitations. To overcome this issue, compressive spectral image fusion (CSIF) employs the projected measurements of two CSI architectures with different resolutions to estimate a high-spatial high-spectral resolution. This work presents the fusion of the compressive measurements of a low-spatial high-spectral resolution coded aperture snapshot spectral imager (CASSI) architecture and a high-spatial low-spectral resolution multispectral color filter array (MCFA) system. Unlike previous CSIF works, this paper proposes joint optimization of the sensing architectures and a reconstruction network in an end-to-end (E2E) manner. The trainable optical parameters are the coded aperture (CA) in the CASSI and the colored coded aperture in the MCFA system, employing a sigmoid activation function and regularization function to encourage binary values on the trainable variables for an implementation purpose. Additionally, an unrolling-based network inspired by the alternating direction method of multipliers (ADMM) optimization is formulated to address the reconstruction step and the acquisition systems design jointly. Finally, a spatial-spectral inspired loss function is employed at the end of each unrolling layer to increase the convergence of the unrolling network. The proposed method outperforms previous CSIF methods, and experimental results validate the method with real measurements.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. G. R. Arce, D. J. Brady, L. Carin, H. Arguello, and D. S. Kittle, “Compressive coded aperture spectral imaging: An introduction,” IEEE Signal Processing Magazine, vol. 31, no. 1, pp. 105–115, 2014.
  2. P. Vouras, K. V. Mishra, A. Artusio-Glimpse, S. Pinilla, A. Xenaki, D. W. Griffith, and K. Egiazarian, “An overview of advances in signal processing techniques for classical and quantum wideband synthetic apertures,” arXiv preprint arXiv:2205.05602, 2022.
  3. R. Jacome, J. Bacca, and H. Arguello, “Deep-fusion: An end-to-end approach for compressive spectral image fusion,” in 2021 IEEE International Conference on Image Processing (ICIP), 2021, pp. 2903–2907.
  4. P.-J. Lapray, X. Wang, J.-B. Thomas, and P. Gouton, “Multispectral filter arrays: Recent advances and practical implementation,” Sensors, vol. 14, no. 11, pp. 21 626–21 659, 2014.
  5. B. Geelen, N. Tack, and A. Lambrechts, “A compact snapshot multispectral imager with a monolithically integrated per-pixel filter mosaic,” in Advanced Fabrication Technologies for Micro/Nano Optics and Photonics VII, G. von Freymann, W. V. Schoenfeld, and R. C. Rumpf, Eds., vol. 8974, International Society for Optics and Photonics.   SPIE, 2014, pp. 80 – 87.
  6. Y. Li, A. Majumder, H. Zhang, and M. Gopi, “Optimized multi-spectral filter array based imaging of natural scenes,” Sensors, vol. 18, no. 4, p. 1172, 2018.
  7. A. Wagadarikar, R. John, R. Willett, and D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Appl. Opt., vol. 47, no. 10, pp. B44–B51, Apr 2008.
  8. H. Rueda, A. Parada, and H. Arguello, “Spectral resolution enhancement of hyperspectral imagery by a multiple-aperture compressive optical imaging system,” Ingenieria e investigacion, vol. 34, no. 3, pp. 50–55, 2014.
  9. R. Jácome, C. López, H. Garcia, and H. Arguello, “Deep learning-based object classification for spectral images,” in Applications of Computational Intelligence, A. D. Orjuela-Cañón, J. Lopez, J. D. Arias-Londoño, and J. C. Figueroa-García, Eds.   Cham: Springer International Publishing, 2021, pp. 147–159.
  10. C. Hinojosa, J. Bacca, and H. Arguello, “Coded aperture design for compressive spectral subspace clustering,” IEEE Journal of Selected Topics in Signal Processing, vol. 12, no. 6, pp. 1589–1600, 2018.
  11. S. Karthik, H. Supreetha, and S. Sandhya, “Detection of anomalies in time series data,” in 2021 IEEE International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS).   IEEE, 2021, pp. 1–5.
  12. E. Vargas, O. Espitia, H. Arguello, and J.-Y. Tourneret, “Spectral image fusion from compressive measurements,” IEEE Transactions on Image Processing, vol. 28, no. 5, pp. 2271–2282, 2018.
  13. T. Gelvez and H. Arguello, “Nonlocal low-rank abundance prior for compressive spectral image fusion,” IEEE Transactions on Geoscience and Remote Sensing, 2020.
  14. J. M. Ramirez, J. I. Martínez-Torre, and H. Arguello, “Ladmm-net: An unrolled deep network for spectral image fusion from compressive data,” Signal Processing, vol. 189, p. 108239, 2021.
  15. H. Rueda-Chacon, F. Rojas, and H. Arguello, “Compressive spectral image fusion via a single aperture high throughput imaging system,” Scientific Reports, vol. 11, no. 1, pp. 1–12, 2021.
  16. J. Bacca, C. V. Correa, and H. Arguello, “Noniterative hyperspectral image reconstruction from compressive fused measurements,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 4, pp. 1231–1239, 2019.
  17. W. He, N. Yokoya, and X. Yuan, “Fast hyperspectral image recovery via non-iterative fusion of dual-camera compressive hyperspectral imaging,” arXiv preprint arXiv:2012.15104, 2020.
  18. N. Diaz, J. Ramirez, E. Vera, and H. Arguello, “Adaptive multisensor acquisition via spatial contextual information for compressive spectral image classification,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 9254–9266, 2021.
  19. J. M. Ramirez, J. I. M. Torre, and H. Arguello, “Feature fusion via dual-resolution compressive measurement matrix analysis for spectral image classification,” Signal Processing: Image Communication, vol. 90, p. 116014, 2021.
  20. T. Gelvez, H. Rueda, and H. Arguello, “Joint sparse and low rank recovery algorithm for compressive hyperspectral imaging,” Appl. Opt., vol. 56, no. 24, pp. 6785–6795, Aug 2017.
  21. X. Miao, X. Yuan, Y. Pu, and V. Athitsos, “l-net: Reconstruct hyperspectral images from a snapshot measurement,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 4059–4069.
  22. Z. Meng, J. Ma, and X. Yuan, “End-to-end low cost compressive spectral imaging with spatial-spectral self-attention,” in European Conference on Computer Vision.   Springer, 2020, pp. 187–204.
  23. B. Monroy, J. Bacca, and H. Arguello, “Deep low-dimensional spectral image representation for compressive spectral reconstruction,” in 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP).   IEEE, 2021, pp. 1–6.
  24. L. Wang, C. Sun, Y. Fu, M. H. Kim, and H. Huang, “Hyperspectral image reconstruction using a deep spatial-spectral prior,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 8032–8041.
  25. L. Wang, C. Sun, M. Zhang, Y. Fu, and H. Huang, “Dnu: Deep non-local unrolling for computational spectral imaging,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  26. T. Huang, W. Dong, X. Yuan, J. Wu, and G. Shi, “Deep gaussian scale mixture prior for spectral compressive imaging,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 16 216–16 225.
  27. L. Wang, C. Sun, M. Zhang, Y. Fu, and H. Huang, “Dnu: Deep non-local unrolling for computational spectral imaging,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 1661–1671.
  28. Z. Cheng, B. Chen, R. Lu, Z. Wang, H. Zhang, Z. Meng, and X. Yuan, “Recurrent neural networks for snapshot compressive imaging,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  29. C. V. Correa, H. Arguello, and G. R. Arce, “Spatiotemporal blue noise coded aperture design for multi-shot compressive spectral imaging,” JOSA A, vol. 33, no. 12, pp. 2312–2322, 2016.
  30. H. Arguello and G. R. Arce, “Colored coded aperture design by concentration of measure in compressive spectral imaging,” IEEE Transactions on Image Processing, vol. 23, no. 4, pp. 1896–1908, 2014.
  31. J. Bacca, T. Gelvez-Barrera, and H. Arguello, “Deep coded aperture design: An end-to-end approach for computational imaging tasks,” IEEE Transactions on Computational Imaging, vol. 7, pp. 1148–1160, 2021.
  32. N. Diaz, C. Hinojosa, and H. Arguello, “Adaptive grayscale compressive spectral imaging using optimal blue noise coding patterns,” Optics & Laser Technology, vol. 117, pp. 147 – 157, 2019.
  33. Y. Mejia and H. Arguello, “Binary codification design for compressive imaging by uniform sensing,” IEEE Transactions on Image Processing, vol. 27, no. 12, pp. 5775–5786, 2018.
  34. V. Sitzmann, S. Diamond, Y. Peng, X. Dun, S. Boyd, W. Heidrich, F. Heide, and G. Wetzstein, “End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging,” ACM Transactions on Graphics (TOG), vol. 37, no. 4, pp. 1–13, 2018.
  35. L. Wang, T. Zhang, Y. Fu, and H. Huang, “Hyperreconnet: Joint coded aperture optimization and image reconstruction for compressive hyperspectral imaging,” IEEE Transactions on Image Processing, vol. 28, no. 5, pp. 2257–2270, 2019.
  36. J. Bacca, L. Galvis, and H. Arguello, “Coupled deep learning coded aperture design for compressive image classification,” Optics express, vol. 28, no. 6, pp. 8528–8540, 2020.
  37. G. Ongie, A. Jalal, C. A. Metzler, R. G. Baraniuk, A. G. Dimakis, and R. Willett, “Deep learning techniques for inverse problems in imaging,” IEEE Journal on Selected Areas in Information Theory, vol. 1, no. 1, pp. 39–56, 2020.
  38. Z. Xiong, Z. Shi, H. Li, L. Wang, D. Liu, and F. Wu, “Hscnn: Cnn-based hyperspectral image recovery from spectrally undersampled projections,” in 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), 2017, pp. 518–525.
  39. V. Monga, Y. Li, and Y. C. Eldar, “Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,” IEEE Signal Processing Magazine, vol. 38, no. 2, pp. 18–44, 2021.
  40. Y. Sogabe, S. Sugimoto, T. Kurozumi, and H. Kimata, “Admm-inspired reconstruction network for compressive spectral imaging,” in 2020 IEEE International Conference on Image Processing (ICIP).   IEEE, 2020, pp. 2865–2869.
  41. E. Vargas, H. Arguello, and J.-Y. Tourneret, “Spectral image fusion from compressive measurements using spectral unmixing and a sparse representation of abundance maps,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 7, pp. 5043–5053, 2019.
  42. L. Galvis, H. Arguello, and G. R. Arce, “Coded aperture design in mismatched compressive spectral imaging,” Appl. Opt., vol. 54, no. 33, pp. 9875–9882, Nov 2015.
  43. H. Arguello, H. Rueda, Y. Wu, D. W. Prather, and G. R. Arce, “Higher-order computational model for coded aperture spectral imaging,” Appl. Opt., vol. 52, no. 10, pp. D12–D21, Apr 2013.
  44. H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Transactions on Computational Imaging, vol. 3, no. 1, pp. 47–57, 2017.
  45. T. Blumensath and M. E. Davies, “Iterative hard thresholding for compressed sensing,” Applied and computational harmonic analysis, vol. 27, no. 3, pp. 265–274, 2009.
  46. A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM journal on imaging sciences, vol. 2, no. 1, pp. 183–202, 2009.
  47. S. Zhang, L. Wang, L. Zhang, and H. Huang, “Learning tensor low-rank prior for hyperspectral image reconstruction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 12 006–12 015.
  48. B. Arad and O. Ben-Shahar, “Sparse recovery of hyperspectral signal from natural rgb images,” in European Conference on Computer Vision.   Springer, 2016, pp. 19–34.
  49. B. Arad, R. Timofte, O. Ben-Shahar, Y.-T. Lin, and G. D. Finlayson, “Ntire 2020 challenge on spectral reconstruction from an rgb image,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020, pp. 446–447.
  50. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  51. A. Horé and D. Ziou, “Image quality metrics: Psnr vs. ssim,” in 2010 20th International Conference on Pattern Recognition, 2010, pp. 2366–2369.
  52. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004.
  53. R. H. Yuhas, A. F. Goetz, and J. W. Boardman, “Discrimination among semi-arid landscape endmembers using the spectral angle mapper (sam) algorithm,” in Proc. Summaries 3rd Annu. JPL Airborne Geosci. Workshop, vol. 1, 1992, pp. 147–149.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now