Ring Deconvolution Microscopy: An Exact Solution for Spatially-Varying Aberration Correction (2206.08928v3)
Abstract: The most ubiquitous form of computational aberration correction for microscopy is deconvolution. However, deconvolution relies on the assumption that the point spread function is the same across the entire field-of-view. It is well recognized that this assumption is often inadequate, but space-variant deblurring techniques generally require impractical amounts of calibration and computation. We present a new imaging pipeline, ring deconvolution microscopy (RDM), that leverages the rotational symmetry of most optical systems to provide simple and fast spatially-varying aberration correction. We formally derive theory and algorithms for exact image recovery and additionally propose a neural network based on Seidel coefficients as a fast alternative. We showcase significant enhancements both visually and quantitatively compared to standard deconvolution and existing spatially-varying deconvolution across a diverse range of microscope modalities, including miniature microscopy, multicolor fluorescence microscopy, and point-scanning multimode fiber micro-endoscopy. Our approach enables near-isotropic, subcellular resolution in each of these applications.
- Daniel Aharoni “UCLA Miniscope” In UCLA Miniscope v3 URL: http://www.miniscope.org/index.php/Main_Page
- “Preliminary cost model for space telescopes” In UV/Optical/IR Space Telescopes: Innovative Technologies and Concepts IV 7436 SPIE, 2009, pp. 11–22
- “Image restoration of space variant blurs by sectioned methods” In ICASSP ’78. IEEE International Conference on Acoustics, Speech, and Signal Processing 3, 1978, pp. 196–198 DOI: 10.1109/ICASSP.1978.1170472
- Shing-Hong Lin, Thomas F. Krile and John F. Walkup “Piecewise isoplanatic modeling of space-variant linear systems” In J. Opt. Soc. Am. A 4.3 OSA, 1987, pp. 481–487 DOI: 10.1364/JOSAA.4.000481
- A. W. Lohmann and D. P. Paris “Space-Variant Image Formation” In J. Opt. Soc. Am. 55.8 OSA, 1965, pp. 1007–1013 DOI: 10.1364/JOSA.55.001007
- L. Denis, E. Thiébaut and F. Soulez “Fast model of space-variant blurring and its application to deconvolution in astronomy” In 2011 18th IEEE International Conference on Image Processing, 2011, pp. 2817–2820 DOI: 10.1109/ICIP.2011.6116257
- James G. Nagy and Dianne P. O’Leary “Restoring Images Degraded by Spatially Variant Blur” In SIAM Journal on Scientific Computing 19.4, 1998, pp. 1063–1082 DOI: 10.1137/S106482759528507X
- “Fast approximations of shift-variant blur” In International Journal of Computer Vision 115.3 Springer, 2015, pp. 253–278
- Ralf C. Flicker and François J. Rigaut “Anisoplanatic deconvolution of adaptive optics images” In J. Opt. Soc. Am. A 22.3 OSA, 2005, pp. 504–513 DOI: 10.1364/JOSAA.22.000504
- “An additive convolution model for fast restoration of nonuniform blurred images” In International Journal of Computer Mathematics 91.11 TaylorFrancis Ltd., 2014, pp. 2446–2466 DOI: 10.1080/00207160.2013.811235
- “Efficient shift-variant image restoration using deformable filtering (Part I)” In EURASIP Journal on Advances in Signal Processing 2012.1 Springer, 2012, pp. 1–20
- Timothy Popkin, Andrea Cavallaro and David Hands “Accurate and efficient method for smoothly space-variant Gaussian blurring” In IEEE Transactions on image processing 19.5 IEEE, 2010, pp. 1362–1370
- Filip Sroubek, Jan Kamenicky and Yue M. Lu “Decomposition of Space-Variant Blur in Image Deconvolution” In IEEE Signal Processing Letters 23.3, 2016, pp. 346–350 DOI: 10.1109/LSP.2016.2519764
- Tod Lauer “Deconvolution with a spatially variant PSF” In Astronomical Data Analysis II 4847 SPIE, 2002, pp. 167 –173 International Society for OpticsPhotonics DOI: 10.1117/12.461035
- “Deconvolution for multimode fiber imaging: modeling of spatially variant PSF” In Biomed. Opt. Express 11.8 Optica Publishing Group, 2020, pp. 4759–4771
- “Generalized image deconvolution by exploiting the transmission matrix of an optical imaging system” In Scientific Reports 7.1 Nature Publishing Group, 2017, pp. 1–10
- “Deep learning for fast spatially varying deconvolution” In Optica 9.1 OSA, 2022, pp. 96–99 DOI: 10.1364/OPTICA.442438
- “Deep non-blind deconvolution via generalized low-rank approximation” In Advances in neural information processing systems 31, 2018
- “Spatially variant deblur and image enhancement in a single multimode fiber imaged by deep learning” In Opt. Lett. 47.19 Optica Publishing Group, 2022, pp. 5040–5043 DOI: 10.1364/OL.469034
- Jiangxin Dong, Stefan Roth and Bernt Schiele “Deep wiener deconvolution: Wiener meets deep learning for image deblurring” In Advances in Neural Information Processing Systems 33, 2020, pp. 1048–1059
- “Content-aware image restoration: pushing the limits of fluorescence microscopy” In Nature methods 15.12 Nature Publishing Group, 2018, pp. 1090–1097
- “A Robust Non-Blind Deblurring Method Using Deep Denoiser Prior” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 735–744
- “Incorporating the image formation process into deep learning improves network performance” In Nature Methods Nature Publishing Group US New York, 2022, pp. 1–11
- Christopher B. Webster and Stanley J. Reeves “Radial Deblurring with FFTs” In 2007 IEEE International Conference on Image Processing 1, 2007, pp. I –101–I –104 DOI: 10.1109/ICIP.2007.4378901
- “Deblurring noisy radial-blurred images: spatially adaptive filtering approach” In Image Processing: Algorithms and Systems VI 6812, 2008, pp. 68121D International Society for OpticsPhotonics
- “A space-variant deblur method for focal-plane microwave imaging” In Applied Sciences 8.11 Multidisciplinary Digital Publishing Institute, 2018, pp. 2166
- “Deblur of radially variant blurred image for single lens system” In IEEJ Transactions on Electrical and Electronic Engineering 6.S1, 2011, pp. S7–S16 DOI: https://doi.org/10.1002/tee.20615
- Yupeng Zhang, Ikumi Minema and Toshitsugu Ueda “Restoration of Radially Blurred Image Created by Spherical Single Lens System of Cell Phone Camera” In Proceedings of IEEE Sensors, 2010, pp. 1333–1337 DOI: 10.1109/ICSENS.2010.5690562
- “Analysis of Radially Restored Images for Spherical Single Lens Cellphone Camera” In IEEE Sensors Journal 11.11, 2011, pp. 2834–2844 DOI: 10.1109/JSEN.2011.2167504
- “Real scene capturing using spherical single-element lens camera and improved restoration algorithm for radially variant blur” In Opt. Express 20.25 OSA, 2012, pp. 27569–27588 DOI: 10.1364/OE.20.027569
- “Blind optical aberration correction by exploring geometric and visual priors”, 2015, pp. 1684–1692 DOI: 10.1109/CVPR.2015.7298777
- “Characterization of spatially varying aberrations for wide field-of-view microscopy” In Opt. Express 21.13 Optica Publishing Group, 2013, pp. 15131–15143 DOI: 10.1364/OE.21.015131
- “Spatially varying aberration calibration using a pair of matched periodic pinhole array masks” In Opt. Express 27.2 Optica Publishing Group, 2019, pp. 729–742 DOI: 10.1364/OE.27.000729
- “Principles of optics: electromagnetic theory of propagation, interference and diffraction of light” Elsevier, 2013
- David Y. H. Wang and Duncan T. Moore “Third-order aberration theory for weak gradient-index lenses” In Appl. Opt. 29.28 Optica Publishing Group, 1990, pp. 4016–4025 DOI: 10.1364/AO.29.004016
- “Extracting and composing robust features with denoising autoencoders” In Proceedings of the 25th international conference on Machine learning, 2008, pp. 1096–1103
- “Ntire 2017 challenge on single image super-resolution: Dataset and study” In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 126–135
- “Subcellular spatial resolution achieved for deep-brain imaging in vivo using a minimally invasive multimode fiber” In Light: science & applications 7.1 Nature Publishing Group UK London, 2018, pp. 110
- W. H. Southwell “Wave-front analyzer using a maximum likelihood algorithm” In J. Opt. Soc. Am. 67.3 Optica Publishing Group, 1977, pp. 396–399 DOI: 10.1364/JOSA.67.000396
- Lihong Yang, Xingxiang Zhang and Jianyue Ren “Adaptive Wiener filtering with Gaussian fitted point spread function in image restoration” In 2011 IEEE 2nd International Conference on Software Engineering and Service Science, 2011, pp. 208–212 DOI: 10.1109/ICSESS.2011.5982291
- “Modeling Point Spread Function in Fluorescence Microscopy With a Sparse Gaussian Mixture: Tradeoff Between Accuracy and Efficiency” In IEEE Transactions on Image Processing 28.8, 2019, pp. 3688–3702 DOI: 10.1109/TIP.2019.2898843
- Elie Maalouf, Bruno Colicchio and Alain Dieterlen “Fluorescence microscopy three-dimensional depth variant point spread function interpolation using Zernike moments” In J. Opt. Soc. Am. A 28.9 Optica Publishing Group, 2011, pp. 1864–1870 DOI: 10.1364/JOSAA.28.001864
- “On soft clipping of Zernike moments for deblurring and enhancement of optical point spread functions” In Computational Imaging IV 6065 SPIE, 2006, pp. 60650C International Society for OpticsPhotonics DOI: 10.1117/12.642272
- “Phase retrieval for high-numerical-aperture optical systems” In Optics letters 28.10 Optica Publishing Group, 2003, pp. 801–803
- “Phase-retrieved pupil functions in wide-field fluorescence microscopy” In Journal of Microscopy 216.1, 2004, pp. 32–48 DOI: https://doi.org/10.1111/j.0022-2720.2004.01393.x
- “Point-Spread Function retrieval for fluorescence microscopy” In 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2009, pp. 1095–1098 DOI: 10.1109/ISBI.2009.5193247
- “Parameterized Modeling of Spatially Varying PSF for Lens Aberration and Defocus” In J. Opt. Soc. Korea 19.2 OSA, 2015, pp. 136–143 URL: http://opg.optica.org/josk/abstract.cfm?URI=josk-19-2-136
- Jonathan Simpkins and Robert L. Stevenson “Mapping measurable quantities of point-spread function observations to Seidel aberration coefficients” In 2012 19th IEEE International Conference on Image Processing, 2012, pp. 369–372 DOI: 10.1109/ICIP.2012.6466872
- Joseph W Goodman “Introduction to Fourier optics. 3rd” In Roberts and Company Publishers, 2005
- David George Voelz “Computational fourier optics: a MATLAB tutorial” SPIE press Bellingham, Washington, 2011
- Diederik P Kingma and Jimmy Ba “Adam: A method for stochastic optimization” In arXiv:1412.6980, 2014
- David Ha, Andrew M. Dai and Quoc V. Le “HyperNetworks” In International Conference on Learning Representations, 2017 URL: https://openreview.net/forum?id=rkpACe1lx
- Ana M Lyons, Kevin T Roberts and Caroline M Williams “Survival of tardigrades (Hypsibius exemplaris) to subzero temperatures depends on exposure intensity, duration, and ice-nucleation—as shown by large-scale mortality dye-based assays” In bioRxiv Cold Spring Harbor Laboratory, 2024, pp. 2024–02
- “Pycro-Manager: open-source software for customized and reproducible microscope control” In Nature methods 18.3 Nature Publishing Group US New York, 2021, pp. 226–228
- François Orieux, Jean-François Giovannelli and Thomas Rodet “Bayesian estimation of regularization and point spread function parameters for Wiener–Hunt deconvolution” In J. Opt. Soc. Am. A 27.7 Optica Publishing Group, 2010, pp. 1593–1607 DOI: 10.1364/JOSAA.27.001593
- United States. National Bureau Standards “Miscellaneous Publication - National Bureau of Standards”, Miscellaneous Publication - National Bureau of Standards The Bureau, 1934 URL: https://books.google.com/books?id=CctOOyTHR3QC
- Alan V Oppenheim and Ronald W Schafer “Digital signal processing(Book)” In Research supported by the Massachusetts Institute of Technology, Bell Telephone Laboratories, and Guggenheim Foundation. Englewood Cliffs, N. J., Prentice-Hall, Inc., 1975. 598 p, 1975
- Alejandro Dominguez “A history of the convolution operation” In EMBS IEEE, 2015 URL: https://www.embs.org/pulse/articles/history-convolution-operation/
- Norbert Wiener “Extrapolation, interpolation, and smoothing of stationary time series: with engineering applications” MIT press Cambridge, MA:, 1964
- William Hadley Richardson “Bayesian-Based Iterative Method of Image Restoration” In J. Opt. Soc. Am. 62.1 OSA, 1972, pp. 55–59 DOI: 10.1364/JOSA.62.000055
- L.B. Lucy “An iterative technique for the rectification of observed distributions” In Astronomical Journal 79, 1974, pp. 745 DOI: 10.1086/111605