Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ProxNF: Neural Field Proximal Training for High-Resolution 4D Dynamic Image Reconstruction (2403.03860v3)

Published 6 Mar 2024 in eess.IV

Abstract: Accurate spatiotemporal image reconstruction methods are needed for a wide range of biomedical research areas but face challenges due to data incompleteness and computational burden. Data incompleteness arises from the undersampling often required to increase frame rates, while computational burden emerges due to the memory footprint of high-resolution images with three spatial dimensions and extended time horizons. Neural fields (NFs), an emerging class of neural networks that act as continuous representations of spatiotemporal objects, have previously been introduced to solve these dynamic imaging problems by reframing image reconstruction as a problem of estimating network parameters. Neural fields can address the twin challenges of data incompleteness and computational burden by exploiting underlying redundancies in these spatiotemporal objects. This work proposes ProxNF, a novel neural field training approach for spatiotemporal image reconstruction leveraging proximal splitting methods to separate computations involving the imaging operator from updates of the network parameters. Specifically, ProxNF evaluates the (subsampled) gradient of the data-fidelity term in the image domain and uses a fully supervised learning approach to update the neural field parameters. This method is demonstrated in two numerical phantom studies and an in-vivo application to tumor perfusion imaging in small animal models using dynamic contrast-enhanced photoacoustic computed tomography (DCE PACT).

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. A. Taruttis, S. Morscher, N. C. Burton, D. Razansky, and V. Ntziachristos, “Fast multispectral optoacoustic tomography (MSOT) for dynamic imaging of pharmacokinetics and biodistribution in multiple organs,” PloS one, vol. 7, no. 1, p. e30491, 2012.
  2. E. J. Stern and W. R. Webb, “Dynamic imaging of lung morphology with ultrafast high-resolution computed tomography.” Journal of thoracic imaging, vol. 8, no. 4, pp. 273–282, 1993.
  3. U. Gamper, P. Boesiger, and S. Kozerke, “Compressed sensing in dynamic MRI,” Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, vol. 59, no. 2, pp. 365–373, 2008.
  4. Z.-P. Liang, “Spatiotemporal imaging with partially separable functions,” ser. 4th IEEE international symposium on biomedical imaging: from nano to macro.   Berlin, Germany: IEEE, 2007.
  5. C. Kortman, “Redundancy reduction—A practical method of data compression,” Proceedings of the IEEE, vol. 55, no. 3, pp. 253–263, 1967.
  6. Z.-P. Liang and P. C. Lauterbur, “An efficient method for dynamic magnetic resonance imaging,” IEEE Transactions on medical imaging, vol. 13, no. 4, pp. 677–686, 1994.
  7. J.-F. Cai, X. Jia, H. Gao, S. B. Jiang, Z. Shen, and H. Zhao, “Cine cone beam CT reconstruction using low-rank matrix factorization: algorithm and a proof-of-principle study,” IEEE transactions on medical imaging, vol. 33, no. 8, pp. 1581–1591, 2014.
  8. M. N. Wernick, E. J. Infusino, and M. Milosevic, “Fast spatio-temporal image reconstruction for dynamic PET,” IEEE transactions on medical imaging, vol. 18, no. 3, pp. 185–195, 1999.
  9. J. P. Haldar and Z.-P. Liang, “Spatiotemporal imaging with partially separable functions: A matrix recovery approach,” in 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.   IEEE, 2010, pp. 716–719.
  10. R. M. Cam, C. Wang, W. Thompson, S. A. Ermilov, M. A. Anastasio, and U. Villa, “Low-rank matrix estimation-based spatiotemporal image reconstruction from few tomographic measurements per frame for dynamic photoacoustic computed tomography,” in Medical Imaging 2023: Physics of Medical Imaging, vol. 12463.   SPIE, 2023, p. 124630R.
  11. ——, “Spatiotemporal image reconstruction to enable high-frame-rate dynamic photoacoustic tomography with rotating-gantry volumetric imagers,” Journal of Biomedical Optics, vol. 29, no. S1, p. S11516, 2024. [Online]. Available: https://doi.org/10.1117/1.JBO.29.S1.S11516
  12. R. M. Cam, C. Wang, S. Park, W. Thompson, S. A. Ermilov, M. A. Anastasio, and U. Villa, “Dynamic image reconstruction to monitor tumor vascular perfusion in small animals using 3D photoacoustic computed-tomography imagers with rotating gantries,” in Photons Plus Ultrasound: Imaging and Sensing 2023, vol. 12379.   SPIE, 2023, pp. 78–83.
  13. L. Lozenski, M. A. Anastasio, and U. Villa, “A memory-efficient self-supervised dynamic image reconstruction method using neural fields,” IEEE Transactions on Computational Imaging, vol. 8, pp. 879–892, 2022.
  14. A. W. Reed, H. Kim, R. Anirudh, K. A. Mohan, K. Champley, J. Kang, and S. Jayasuriya, “Dynamic CT reconstruction from limited views with implicit neural representations and parametric motion fields,” pp. 2258–2268, 2021.
  15. A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous, and Y. LeCun, “The loss surfaces of multilayer networks,” in Artificial intelligence and statistics.   PMLR, 2015, pp. 192–204.
  16. A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon, “A survey on contrastive self-supervised learning,” Technologies, vol. 9, no. 1, p. 2, 2020.
  17. E. Dupont, A. Goliński, M. Alizadeh, Y. W. Teh, and A. Doucet, “Coin: Compression with implicit neural representations,” arXiv preprint arXiv:2103.03123, 2021.
  18. J. Martel, D. Lindell, C. Lin, E. Chan, M. Monteiro, and G. Wetzstein, “ACORN: Adaptive coordinate networks for neural scene representation,” arXiv preprint arXiv:2105.02788, 2021.
  19. V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein, “Implicit neural representations with periodic activation functions,” Advances in Neural Information Processing Systems, vol. 33, pp. 7462–7473, 2020.
  20. Y. Xie, T. Takikawa, S. Saito, O. Litany, S. Yan, N. Khan, F. Tombari, J. Tompkin, V. Sitzmann, and S. Sridhar, “Neural fields in visual computing and beyond,” in Computer Graphics Forum, vol. 41, no. 2.   Wiley Online Library, 2022, pp. 641–676.
  21. R. Liu, Y. Sun, J. Zhu, L. Tian, and U. S. Kamilov, “Recovery of continuous 3D refractive index maps from discrete intensity-only measurements using neural fields,” Nature Machine Intelligence, vol. 4, no. 9, pp. 781–791, 2022.
  22. M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng, “Fourier features let networks learn high frequency functions in low dimensional domains,” Advances in Neural Information Processing Systems, vol. 33, pp. 7537–7547, 2020.
  23. L. Shen, J. Pauly, and L. Xing, “NeRP: implicit neural representation learning with prior embedding for sparsely sampled image reconstruction,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
  24. D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv e-prints, 2014.
  25. Y. Sun, J. Liu, M. Xie, B. Wohlberg, and U. S. Kamilov, “CoIL: Coordinate-based internal learning for imaging inverse problems,” arXiv preprint arXiv:2102.05181, 2021.
  26. E. Y. Sidky, C.-M. Kao, and X. Pan, “Accurate image reconstruction from few-views and limited-angle data in divergent-beam CT,” Journal of X-ray Science and Technology, vol. 14, no. 2, pp. 119–139, 2006.
  27. J. V. Velikina and A. A. Samsonov, “Reconstruction of dynamic image series from undersampled MRI data using data-driven model consistency condition (MOCCO),” Magnetic resonance in medicine, vol. 74, no. 5, pp. 1279–1290, 2015.
  28. J. P. Haldar and Z.-P. Liang, “Low-rank approximations for dynamic imaging,” in 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.   IEEE, 2011, pp. 1052–1055.
  29. K. A. Juhl, X. Morales, O. De Backer, O. Camara, and R. R. Paulsen, “Implicit neural distance representation for unsupervised and supervised classification of complex anatomies,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part II 24.   Springer, 2021, pp. 405–415.
  30. K. Lee, N. Trask, R. G. Patel, M. A. Gulian, and E. C. Cyr, “Partition of unity networks: deep hp-approximation,” arXiv e-prints, 2021.
  31. U. S. Kamilov, H. Mansour, and B. Wohlberg, “A plug-and-play priors approach for solving nonlinear imaging inverse problems,” IEEE Signal Processing Letters, vol. 24, no. 12, pp. 1872–1876, 2017.
  32. Y. Sun, B. Wohlberg, and U. S. Kamilov, “An online plug-and-play algorithm for regularized image reconstruction,” IEEE Transactions on Computational Imaging, vol. 5, no. 3, pp. 395–408, 2019.
  33. X. Xu, Y. Sun, J. Liu, B. Wohlberg, and U. S. Kamilov, “Provable convergence of plug-and-play priors with MMSE denoisers,” IEEE Signal Processing Letters, vol. 27, pp. 1280–1284, 2020.
  34. Y. Sun, Z. Wu, X. Xu, B. Wohlberg, and U. S. Kamilov, “Scalable plug-and-play admm with convergence guarantees,” IEEE Transactions on Computational Imaging, vol. 7, pp. 849–863, 2021.
  35. A. Nitanda, “Stochastic proximal gradient descent with acceleration techniques,” Advances in Neural Information Processing Systems, vol. 27, 2014.
  36. D. Kim and J. A. Fessler, “Ordered subsets acceleration using relaxed momentum for X-ray CT image reconstruction,” in 2013 IEEE Nuclear Science Symposium and Medical Imaging Conference (2013 NSS/MIC).   IEEE, 2013, pp. 1–5.
  37. D. Kim, S. Ramani, and J. A. Fessler, “Combining ordered subsets and momentum for accelerated X-ray CT image reconstruction,” IEEE Transactions on Medical Imaging, vol. 34, no. 1, pp. 167–178, 2014.
  38. V. Haase, K. Stierstorfer, K. Hahn, H. Schöndube, A. Maier, and F. Noo, “An improved combination of ordered subsets and momentum for fast model-based iterative CT reconstruction,” in Medical Imaging 2020: Physics of Medical Imaging, vol. 11312.   SPIE, 2020, pp. 391–397.
  39. A. Sisniega, J. Stayman, S. Capostagno, C. Weiss, T. Ehtiati, and J. Siewerdsen, “Accelerated 3D image reconstruction with a morphological pyramid and noise-power convergence criterion,” Physics in Medicine & Biology, vol. 66, no. 5, p. 055012, 2021.
  40. C. W. Hupple, S. Morscher, N. C. Burton, M. D. Pagel, L. R. McNally, and J. Cárdenas-Rodríguez, “A light-fluence-independent method for the quantitative analysis of dynamic contrast-enhanced multispectral optoacoustic tomography (DCE MSOT),” Photoacoustics, vol. 10, pp. 54–64, 2018.
  41. L. V. Wang and J. Yao, “A practical guide to photoacoustic tomography in the life sciences,” Nature methods, vol. 13, no. 8, pp. 627–638, 2016.
  42. T. Matthews and M. Anastasio, “Joint reconstruction of the initial pressure and speed of sound distributions from combined photoacoustic and ultrasound tomography measurements,” Inverse Problems, vol. 30, no. 12, 2017.
  43. K. Wang and M. Anastasio, “Photoacoustic and thermoacoustic tomography: image formation principles,” Handbook of Mathematical Methods in Imaging, 2015.
  44. W. P. Segars, B. M. Tsui, E. C. Frey, G. A. Johnson, and S. S. Berr, “Development of a 4-D digital mouse phantom for molecular imaging research,” Molecular Imaging & Biology, vol. 6, no. 3, pp. 149–159, 2004.
  45. W. Segars and M. Tsui, “MCAT to XCAT: The evolution of 4-D computerized phantoms for imaging research,” Proceedings of the IEEE, vol. 97, no. 12, 2009.
  46. P. S. Tofts, “Modeling tracer kinetics in dynamic Gd-DTPA MR imaging,” Journal of magnetic resonance imaging, vol. 7, no. 1, pp. 91–101, 1997.
  47. P. S. Tofts, G. Brix, D. L. Buckley, J. L. Evelhoch, E. Henderson, M. V. Knopp, H. B. Larsson, T.-Y. Lee, N. A. Mayr, G. J. Parker et al., “Estimating kinetic parameters from dynamic contrast-enhanced T1-weighted MRI of a diffusable tracer: standardized quantities and symbols,” Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine, vol. 10, no. 3, pp. 223–232, 1999.
  48. S. P. Sourbron and D. L. Buckley, “On the scope and interpretation of the Tofts models for DCE-MRI,” Magnetic resonance in medicine, vol. 66, no. 3, pp. 735–745, 2011.
  49. S. L. Jacques, “Optical properties of biological tissues: a review,” Physics in Medicine & Biology, vol. 58, no. 11, p. R37, 2013.
  50. J. T. Alander, I. Kaartinen, A. Laakso, T. Pätilä, T. Spillmann, V. V. Tuchin, M. Venermo, and P. Välisuo, “A review of indocyanine green fluorescent imaging in surgery,” Journal of Biomedical Imaging, vol. 2012, pp. 7–7, 2012.
  51. T. Tarvainen, B. T. Cox, J. Kaipio, and S. R. Arridge, “Reconstructing absorption and scattering distributions in quantitative photoacoustic tomography,” Inverse Problems, vol. 28, no. 8, p. 084009, 2012.
  52. P. C. Hansen and J. S. Jørgensen, “AIR Tools II: algebraic iterative reconstruction methods, improved implementation,” Numerical Algorithms, vol. 79, no. 1, pp. 107–137, 2018.
  53. J. R. Gilbert, C. Moler, and R. Schreiber, “Sparse matrices in MATLAB: Design and implementation,” SIAM journal on matrix analysis and applications, vol. 13, no. 1, pp. 333–356, 1992.
  54. L. Lozenski, R. Cam, M. Anastasio, and U. Villa, “Technical note: An efficient implementation of the spherical radon transform with cylindrical apertures,” arXiv, p. 2402.15641, 2024.
  55. M. Haltmeier and S. Moon, “The spherical radon transform with centers on cylindrical surfaces,” Journal of Mathematical Analysis and Applications, vol. 448, no. 1, pp. 567–579, 2017.
  56. K. Wang, J. Xia, C. Li, L. V. Wang, and M. A. Anastasio, “Fast spatiotemporal image reconstruction based on low-rank matrix estimation for dynamic photoacoustic computed tomography,” J Biomed Opt, vol. 19, no. 5, p. 056007, May 2014.
  57. V. A. Morozov, “On the solution of functional equations by the method of regularization,” in Soviet Math. Dokl, vol. 7, no. 1, 1966, pp. 414–417.
  58. Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004.
  59. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,” 2019, vol. 32.
  60. A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM Journal on Imaging Sciences, vol. 2, no. 1, 2009.
  61. N. Perraudin, V. Kalofolias, D. Shuman, and P. Vandergheynst, “UNLocBoX: A MATLAB convex optimization toolbox for proximal-splitting methods,” arXiv preprint arXiv:1402.0779, 2016.
  62. N. Halko, P. G. Martinsson, and J. A. Tropp, “Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions,” SIAM Review, vol. 53, no. 2, pp. 217–288, 2011.
  63. J. Ahrens, B. Geveci, C. Law, C. Hansen, and C. Johnson, “Paraview: An end-user tool for large-data visualization,” The visualization handbook, vol. 717, pp. 50 038–1, 2005.
  64. R. Frostig, M. J. Johnson, and C. Leary, “Compiling machine learning programs via high-level tracing,” Systems for Machine Learning, vol. 4, no. 9, 2018.
  65. L. Lozenski and U. Villa, “Neural Field CRT Dynamic Imaging,” 2022. [Online]. Available: {https://doi.org/10.5281/zenodo.7007589}
  66. U. Villa, L. Lozenski, S. Park, and R. Mert Cam, “2D tumor perfusion numerical phantom for dynamic contrast enhanced photoacoustic tomography virtual imaging studies,” 2023. [Online]. Available: https://doi.org/10.7910/DVN/P7ZVJO
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now