Reconstructing Interpretable Features in Computational Super-Resolution microscopy via Regularized Latent Search (2405.19112v1)
Abstract: Supervised deep learning approaches can artificially increase the resolution of microscopy images by learning a mapping between two image resolutions or modalities. However, such methods often require a large set of hard-to-get low-res/high-res image pairs and produce synthetic images with a moderate increase in resolution. Conversely, recent methods based on GAN latent search offered a drastic increase in resolution without the need of paired images. However, they offer limited reconstruction of the high-resolution image interpretable features. Here, we propose a robust super-resolution method based on regularized latent search~(RLS) that offers an actionable balance between fidelity to the ground-truth and realism of the recovered image given a distribution prior. The latter allows to split the analysis of a low-resolution image into a computational super-resolution task performed by deep learning followed by a quantification task performed by a handcrafted algorithm and based on interpretable biological features. This two-step process holds potential for various applications such as diagnostics on mobile devices, where the main aim is not to recover the high-resolution details of a specific sample but rather to obtain high-resolution images that preserve explainable and quantifiable differences between conditions.
- Content-aware image restoration: pushing the limits of fluorescence microscopy. Nature methods, 15(12):1090–1097, 2018.
- Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nature methods, 16(1):103–110, 2019.
- Deep learning massively accelerates super-resolution localization microscopy. Nature biotechnology, 36(5):460–468, 2018.
- The promise and peril of deep learning in microscopy. Nature methods, 18(2):131–132, 2021.
- A natural-scene gradient distribution prior and its application in light-microscopy image processing. IEEE Journal of Selected Topics in Signal Processing, 10(1):99–114, 2015.
- Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2223–2232, 2017.
- Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy. Optics letters, 19(11):780–782, 1994.
- Imaging intracellular fluorescent proteins at nanometer resolution. Science, 313(5793):1642–1645, 2006.
- Deep-storm: super-resolution single-molecule microscopy by deep learning. Optica, 5(4):458–464, 2018.
- A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401–4410, 2019.
- Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8110–8119, 2020.
- Image2stylegan: How to embed images into the stylegan latent space? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4432–4441, 2019.
- Improving inversion and generation diversity in stylegan using a gaussianized latent space. arXiv preprint arXiv:2009.06529, 2020.
- Compressed sensing using generative models. In Proceedings of the International Conference on Machine Learning, pages 537–546. PMLR, 2017.
- Pulse: Self-supervised photo upsampling via latent space exploration of generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2437–2445, 2020.
- Improved stylegan embedding: Where are the good latents? arXiv preprint arXiv:2012.09036, 2020.
- Deep learning. MIT press, 2016.
- Bayesian image reconstruction using deep generative models. In NeurIPS Workshop on Deep Generative Models and Downstream Applications, 2021.
- Normalizing flows: An introduction and review of current methods. 43(11):3964–3979, 2020.
- Normalizing flows for probabilistic modeling and inference. 22(57):1–64, 2021.
- Density estimation using real nvp. arXiv preprint arXiv:1605.08803, 2016.
- Masked autoregressive flow for density estimation. 2017.
- Training generative adversarial networks with limited data. pages 12104–12114, 2020.
- Weakly supervised learning of single-cell feature embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9309–9318, 2018.
- Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment. Journal of biomolecular screening, 18(10):1321–1329, 2013.
- High-content phenotypic profiling of drug response signatures across distinct cancer cells phenotypic profiling across cancer cell types. Molecular cancer therapeutics, 9(6):1913–1926, 2010.
- Gans trained by a two time-scale update rule converge to a local nash equilibrium. 30, 2017.
- Demystifying mmd gans. arXiv preprint arXiv:1801.01401, 2018.
- Enhancing high-content imaging for studying microtubule networks at large-scale. In Proceedings of Machine Learning for Healthcare Conference, pages 592–613. PMLR, 2019.
- The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 586–595, 2018.
- Multiscale structural similarity for image quality assessment. In Proceedings of Asilomar Conference on Signals, Systems & Computers, volume 2, pages 1398–1402. IEEE, 2003.