Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Low-light phase retrieval with implicit generative priors (2402.17745v2)

Published 27 Feb 2024 in physics.comp-ph, cs.CV, and physics.optics

Abstract: Phase retrieval (PR) is fundamentally important in scientific imaging and is crucial for nanoscale techniques like coherent diffractive imaging (CDI). Low radiation dose imaging is essential for applications involving radiation-sensitive samples. However, most PR methods struggle in low-dose scenarios due to high shot noise. Recent advancements in optical data acquisition setups, such as in-situ CDI, have shown promise for low-dose imaging, but they rely on a time series of measurements, making them unsuitable for single-image applications. Similarly, data-driven phase retrieval techniques are not easily adaptable to data-scarce situations. Zero-shot deep learning methods based on pre-trained and implicit generative priors have been effective in various imaging tasks but have shown limited success in PR. In this work, we propose low-dose deep image prior (LoDIP), which combines in-situ CDI with the power of implicit generative priors to address single-image low-dose phase retrieval. Quantitative evaluations demonstrate LoDIP's superior performance in this task and its applicability to real experimental scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Y. H. Lo, L. Zhao, M. Gallagher-Jones, A. Rana, J. J. Lodico, W. Xiao, B. Regan, and J. Miao, “In situ coherent diffractive imaging,” Nature communications, vol. 9, no. 1, p. 1826, 2018.
  2. J. Miao, P. Charalambous, J. Kirz, and D. Sayre, “Extending the methodology of x-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens,” Nature, vol. 400, no. 6742, pp. 342–344, 1999.
  3. J. Miao, T. Ishikawa, I. K. Robinson, and M. M. Murnane, “Beyond crystallography: Diffractive imaging using coherent x-ray light sources,” Science, vol. 348, no. 6234, pp. 530–535, 2015.
  4. G. N. George, I. J. Pickering, M. J. Pushie, K. Nienaber, M. J. Hackett, I. Ascone, B. Hedman, K. O. Hodgson, J. B. Aitken, A. Levina, C. Glover, and P. A. Lay, “X-ray-induced photo-chemistry and X-ray absorption spectroscopy of biological samples,” Journal of Synchrotron Radiation, vol. 19, no. 6, pp. 875–886, Nov 2012. [Online]. Available: https://doi.org/10.1107/S090904951203943X
  5. E. F. Garman and M. Weik, “X-ray radiation damage to biological macromolecules: further insights,” Journal of Synchrotron Radiation, vol. 24, no. 1, pp. 1–6, Jan 2017. [Online]. Available: https://doi.org/10.1107/S160057751602018X
  6. C. T. Putkunz, J. N. Clark, D. J. Vine, G. J. Williams, M. A. Pfeifer, E. Balaur, I. McNulty, K. A. Nugent, and A. G. Peele, “Phase-diverse coherent diffractive imaging: High sensitivity with low dose,” Physical review letters, vol. 106, no. 1, p. 013903, 2011.
  7. T.-Y. Lan, P.-N. Li, and T.-K. Lee, “Method to enhance the resolution of x-ray coherent diffraction imaging for non-crystalline bio-samples,” New Journal of Physics, vol. 16, no. 3, p. 033016, 2014.
  8. X. Lu, M. Pham, E. Negrini, D. Davis, S. J. Osher, and J. Miao, “Computational microscopy beyond perfect lenses,” arXiv preprint arXiv:2306.11283, 2023.
  9. A. Goy, K. Arthur, S. Li, and G. Barbastathis, “Low photon count phase retrieval using deep learning,” Physical review letters, vol. 121, no. 24, p. 243902, 2018.
  10. M. Deng, S. Li, A. Goy, I. Kang, and G. Barbastathis, “Learning to synthesize: robust phase retrieval at low photon counts,” Light: Science & Applications, vol. 9, no. 1, p. 36, 2020.
  11. D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Deep image prior,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 9446–9454.
  12. U. Dmitry, A. Vedaldi, and L. Victor, “Deep image prior,” International Journal of Computer Vision, vol. 128, no. 7, pp. 1867–1888, 2020.
  13. F. Wang, Y. Bian, H. Wang, M. Lyu, G. Pedrini, W. Osten, G. Barbastathis, and G. Situ, “Phase imaging with an untrained neural network,” Light: Science & Applications, vol. 9, no. 1, p. 77, 2020.
  14. R. Heckel and P. Hand, “Deep decoder: Concise image representations from untrained non-convolutional networks,” arXiv preprint arXiv:1810.03982, 2018.
  15. E. Bostan, R. Heckel, M. Chen, M. Kellman, and L. Waller, “Deep phase decoder: self-calibrating phase microscopy with an untrained deep neural network,” Optica, vol. 7, no. 6, pp. 559–562, 2020.
  16. K. Tayal, C.-H. Lai, V. Kumar, and J. Sun, “Inverse problems, deep learning, and symmetry breaking,” arXiv preprint arXiv:2003.09077, 2020.
  17. Z. Zhuang, D. Yang, F. Hofmann, D. Barmherzig, and J. Sun, “Practical phase retrieval using double deep image priors,” arXiv:2211.00799, 2022.
  18. J. Miao, D. Sayre, and H. Chapman, “Phase retrieval from the magnitude of the fourier transforms of nonperiodic objects,” JOSA A, vol. 15, no. 6, pp. 1662–1669, 1998.
  19. Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev, “Phase retrieval with application to optical imaging: A contemporary overview,” IEEE Signal Processing Magazine, vol. 32, no. 3, pp. 87–109, may 2015.
  20. J. R. Fienup, “Reconstruction of an object from the modulus of its fourier transform,” Optics letters, vol. 3, no. 1, pp. 27–29, 1978.
  21. J. Miao, J. Kirz, and D. Sayre, “The oversampling phasing method,” Acta Crystallographica Section D: Biological Crystallography, vol. 56, no. 10, pp. 1312–1315, 2000.
  22. H. H. Bauschke, P. L. Combettes, and D. R. Luke, “Phase retrieval, error reduction algorithm, and fienup variants: a view from convex optimization,” Journal of the Optical Society of America A, vol. 19, no. 7, p. 1334, jul 2002.
  23. D. R. Luke, “Relaxed averaged alternating reflections for diffraction imaging,” Inverse problems, vol. 21, no. 1, p. 37, 2004.
  24. J. A. Rodriguez, R. Xu, C.-C. Chen, Y. Zou, and J. Miao, “Oversampling smoothness: an effective algorithm for phase retrieval of noisy diffraction intensities,” Journal of applied crystallography, vol. 46, no. 2, pp. 312–318, 2013.
  25. M. Pham, P. Yin, A. Rana, S. Osher, and J. Miao, “Generalized proximal smoothing (gps) for phase retrieval,” Optics Express, vol. 27, no. 3, pp. 2792–2808, 2019.
  26. S. Marchesini, H. He, H. N. Chapman, S. P. Hau-Riege, A. Noy, M. R. Howells, U. Weierstall, and J. C. Spence, “X-ray image reconstruction from a diffraction pattern alone,” Physical Review B, vol. 68, no. 14, p. 140101, 2003.
  27. A. Sinha, J. Lee, S. Li, and G. Barbastathis, “Lensless computational imaging through deep learning,” Optica, vol. 4, no. 9, p. 1117, sep 2017.
  28. A. Goy, K. Arthur, S. Li, and G. Barbastathis, “Low photon count phase retrieval using deep learning,” Physical Review Letters, vol. 121, no. 24, dec 2018.
  29. T. Uelwer, A. Oberstraß, and S. Harmeling, “Phase retrieval using conditional generative adversarial networks,” arXiv:1912.04981, 2019.
  30. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2017.
  31. D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
  32. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020.
  33. D. P. Kingma and P. Dhariwal, “Glow: Generative flow with invertible 1x1 convolutions,” Advances in neural information processing systems, vol. 31, 2018.
  34. A. Bora, A. Jalal, E. Price, and A. G. Dimakis, “Compressed sensing using generative models,” in International Conference on Machine Learning.   PMLR, 2017, pp. 537–546.
  35. Z. Cheng, M. Gadelha, S. Maji, and D. Sheldon, “A bayesian perspective on the deep image prior,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5443–5451.
  36. H. Wang, T. Li, Z. Zhuang, T. Chen, H. Liang, and J. Sun, “Early stopping for deep image prior,” arXiv preprint arXiv:2112.06074, 2021.
  37. Z. Shi, P. Mettes, S. Maji, and C. G. Snoek, “On measuring and controlling the spectral bias of the deep image prior,” International Journal of Computer Vision, vol. 130, no. 4, pp. 885–908, 2022.
  38. Y. Gandelsman, A. Shocher, and M. Irani, “” double-dip”: unsupervised image decomposition via coupled deep-image-priors,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 11 026–11 035.
  39. Y. Lu, Y. Lin, H. Wu, Y. Luo, X. Zheng, and L. Wang, “All one needs to know about priors for deep image restoration and enhancement: A survey,” arXiv preprint arXiv:2206.02070, 2022.
  40. M. Z. Darestani and R. Heckel, “Accelerated mri with un-trained neural networks,” IEEE Transactions on Computational Imaging, vol. 7, pp. 724–733, 2021.
  41. A. Qayyum, I. Ilahi, F. Shamshad, F. Boussaid, M. Bennamoun, and J. Qadir, “Untrained neural network priors for inverse imaging problems: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  42. G. Ongie, A. Jalal, C. A. M. R. G. Baraniuk, A. G. Dimakis, and R. Willett, “Deep learning techniques for inverse problems in imaging,” IEEE Journal on Selected Areas in Information Theory, 2020.
  43. K. Tayal, R. Manekar, Z. Zhuang, D. Yang, V. Kumar, F. Hofmann, and J. Sun, “Phase retrieval using single-instance deep generative prior,” in Applied Industrial Spectroscopy.   Optica Publishing Group, 2021, pp. JW2A–37.
  44. R. Manekar, Z. Zhuang, K. Tayal, V. Kumar, and J. Sun, “Deep learning initialized phase retrieval,” in NeurIPS 2020 Workshop on Deep Learning and Inverse Problems, 2020.
  45. R. Hyder, Z. Cai, and M. S. Asif, “Solving phase retrieval with a learned reference,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16.   Springer, 2020, pp. 425–441.
  46. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention.   Springer, 2015, pp. 234–241.
  47. D. A. Barmherzig, J. Sun, P.-N. Li, T. J. Lane, and E. J. Candes, “Holographic phase retrieval and reference design,” Inverse Problems, vol. 35, no. 9, p. 094001, 2019.
  48. I. McNulty, J. Kirz, C. Jacobsen, E. H. Anderson, M. R. Howells, and D. P. Kern, “High-resolution imaging by fourier transform x-ray holography,” Science, vol. 256, no. 5059, pp. 1009–1012, 1992.
  49. D. J. Chang, C. M. O’Leary, C. Su, D. A. Jacobs, S. Kahn, A. Zettl, J. Ciston, P. Ercius, and J. Miao, “Deep-learning electron diffractive imaging,” Physical review letters, vol. 130, no. 1, p. 016101, 2023.
  50. J. R. Fienup, “Phase retrieval algorithms: a comparison,” Applied Optics, vol. 21, no. 15, p. 2758, aug 1982.
  51. C. A. Metzler, P. Schniter, A. Veeraraghavan, and R. G. Baraniuk, “prdeep: Robust phase retrieval with a flexible deep network,” arXiv preprint arXiv:1803.00212, 2018.
  52. K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,” IEEE transactions on image processing, vol. 26, no. 7, pp. 3142–3155, 2017.
  53. H. Lawrence, D. A. Barmherzig, M. Eickenberg, and M. Gabrie, “Low-photon holographic phase retrieval via a deep decoder neural network,” in Optical Sensors.   Optica Publishing Group, 2021, pp. JTu5A–19.
  54. M. van Heel and M. Schatz, “Fourier shell correlation threshold criteria,” Journal of Structural Biology, vol. 151, no. 3, pp. 250–262, 2005. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1047847705001292

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now