Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
120 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Super-resolution of biomedical volumes with 2D supervision (2404.09425v1)

Published 15 Apr 2024 in eess.IV and cs.CV

Abstract: Volumetric biomedical microscopy has the potential to increase the diagnostic information extracted from clinical tissue specimens and improve the diagnostic accuracy of both human pathologists and computational pathology models. Unfortunately, barriers to integrating 3-dimensional (3D) volumetric microscopy into clinical medicine include long imaging times, poor depth / z-axis resolution, and an insufficient amount of high-quality volumetric data. Leveraging the abundance of high-resolution 2D microscopy data, we introduce masked slice diffusion for super-resolution (MSDSR), which exploits the inherent equivalence in the data-generating distribution across all spatial dimensions of biological specimens. This intrinsic characteristic allows for super-resolution models trained on high-resolution images from one plane (e.g., XY) to effectively generalize to others (XZ, YZ), overcoming the traditional dependency on orientation. We focus on the application of MSDSR to stimulated Raman histology (SRH), an optical imaging modality for biological specimen analysis and intraoperative diagnosis, characterized by its rapid acquisition of high-resolution 2D images but slow and costly optical z-sectioning. To evaluate MSDSR's efficacy, we introduce a new performance metric, SliceFID, and demonstrate MSDSR's superior performance over baseline models through extensive evaluations. Our findings reveal that MSDSR not only significantly enhances the quality and resolution of 3D volumetric data, but also addresses major obstacles hindering the broader application of 3D volumetric microscopy in clinical diagnostics and biomedical research.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. A new generative adversarial network for medical images super resolution. Scientific Reports, 12(1):9533, 2022.
  2. The fine structure of ependymomas. CNS oncology, 3(1):49–59, 2014.
  3. A super-resolution diffusion model for recovering bone microstructure from ct images. Radiology: Artificial Intelligence, 5(6):e220251, 2023.
  4. Learning continuous image representation with local implicit image function. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8628–8638, 2021.
  5. Score-based diffusion models for accelerated mri. Medical image analysis, 80:102479, 2022.
  6. Mr image denoising and super-resolution using regularized reverse diffusion. IEEE Transactions on Medical Imaging, 42(4):922–934, 2022.
  7. Solving 3d inverse problems using pre-trained 2d diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22542–22551, 2023.
  8. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021.
  9. Deep residual dense u-net for resolution enhancement in accelerated mri acquisition. In Medical Imaging 2019: Image Processing, pages 110–117. SPIE, 2019.
  10. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2):295–307, 2015.
  11. Conditional diffusion models for semantic 3d brain mri synthesis. IEEE Journal of Biomedical and Health Informatics, 2024.
  12. Label-free biomedical imaging with high sensitivity by stimulated raman scattering microscopy. Science, 322(5909):1857–1861, 2008.
  13. Implicit diffusion models for continuous super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10021–10030, 2023.
  14. Light-sheet microscopy for slide-free non-destructive pathology of large clinical specimens. Nature biomedical engineering, 1(7):0084, 2017.
  15. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  16. Hayit Greenspan. Super-resolution in medical imaging. The computer journal, 52(1):43–63, 2009.
  17. Medsrgan: medical images super-resolution using generative adversarial networks. Multimedia Tools and Applications, 79:21815–21840, 2020.
  18. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
  19. Artificial-intelligence-based molecular classification of diffuse gliomas using rapid, label-free optical imaging. Nature Medicine, 29(4):828–832, 2023.
  20. Near real-time intraoperative brain tumor diagnosis using stimulated raman histology and deep neural networks. Nat. Med., 2020.
  21. A self-supervised framework for learning whole slide representations. arXiv preprint arXiv:2402.06188, 2024.
  22. Opensrh: optimizing brain tumor surgery using intraoperative stimulated raman histology. Advances in neural information processing systems, 35:28502–28516, 2022.
  23. Hierarchical discriminative learning improves visual representations of biomedical microscopy. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19798–19808, 2023.
  24. Denoising diffusion restoration models. In Advances in Neural Information Processing Systems, 2022.
  25. Unsupervised real-world super resolution with cycle generative adversarial network and domain discriminator. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 456–457, 2020.
  26. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  27. Virtual thin slice: 3d conditional gan-based super-resolution for ct slice interval. In Machine Learning for Medical Image Reconstruction: Second International Workshop, MLMIR 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 17, 2019, Proceedings 2, pages 91–100. Springer, 2019.
  28. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4681–4690, 2017.
  29. Improving 3d imaging with pre-trained perpendicular 2d diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10710–10720, 2023.
  30. Swinir: Image restoration using swin transformer. arXiv preprint arXiv:2108.10257, 2021.
  31. Enhanced deep residual networks for single image super-resolution. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2017.
  32. Mri super-resolution with ensemble learning and complementary priors. IEEE Transactions on Computational Imaging, 6:615–624, 2020.
  33. Step-calibrated diffusion for biomedical optical image restoration. arXiv preprint arXiv:2403.13680, 2024.
  34. Mr image super resolution by combining feature disentanglement cnns and vision transformers. In International Conference on Medical Imaging with Deep Learning, pages 858–878. PMLR, 2022.
  35. Image super resolution using generative adversarial networks and local saliency maps for retinal image analysis. In Medical Image Computing and Computer Assisted Intervention- MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part III 20, pages 382–390. Springer, 2017.
  36. Denoising of stimulated raman scattering microscopy images via deep learning. Biomed. Opt. Express, 10(8):3860–3874, 2019.
  37. Deep learning single-frame and multiframe super-resolution for cardiac mri. Radiology, 295(3):552–561, 2020.
  38. Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated raman scattering microscopy. Nat Biomed Eng, 1, 2017.
  39. Diffuseir: Diffusion models for isotropic reconstruction of 3d microscopic images. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 323–332. Springer, 2023.
  40. Saint: spatially aware interpolation network for medical slice synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7750–7759, 2020.
  41. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022.
  42. The digital brain tumour atlas, an open histopathology resource. Scientific Data, 9(1):55, 2022.
  43. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.
  44. Image super-resolution via iterative refinement. arXiv:2104.07636, 2021.
  45. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479–36494, 2022.
  46. Unsupervised super-resolution: creating high-resolution medical images from low-resolution anisotropic examples. In Medical Imaging 2021: Image Processing, pages 82–88. SPIE, 2021.
  47. Autoencoding low-resolution mri for semantically smooth interpolation of anisotropic mri. Medical Image Analysis, 78:102393, 2022.
  48. Review of image interpolation and super-resolution. In Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference, pages 1–10, 2012.
  49. Denoising diffusion implicit models. In International Conference on Learning Representations, 2020.
  50. 3d registration of pre-surgical prostate mri and histopathology images via super-resolution volume reconstruction. Medical image analysis, 69:101957, 2021.
  51. Fvd: A new metric for video generation. 2019.
  52. Inversesr: 3d brain mri super-resolution using a latent diffusion model. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 438–447. Springer, 2023.
  53. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nature methods, 15(12):1090–1097, 2018.
  54. An arbitrary scale super-resolution approach for 3d mr images via implicit neural representation. IEEE Journal of Biomedical and Health Informatics, 27(2):1004–1015, 2022.
  55. Super-resolution of cardiac mr cine imaging using conditional gans and unsupervised transfer learning. Medical Image Analysis, 71:102037, 2021.
  56. Single image super-resolution with denoising diffusion gans. Scientific Reports, 14(1):4272, 2024.
  57. Resshift: Efficient diffusion model for image super-resolution by residual shifting. Advances in Neural Information Processing Systems, 36, 2024.
  58. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 2223–2232, 2017.
  59. Csrgan: Medical image super-resolution using a generative adversarial network. In 2020 IEEE 17th International Symposium on Biomedical Imaging Workshops (ISBI Workshops), pages 1–4, 2020.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now