Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

RegWSI: Whole Slide Image Registration using Combined Deep Feature- and Intensity-Based Methods: Winner of the ACROBAT 2023 Challenge (2404.13108v2)

Published 19 Apr 2024 in eess.IV and cs.CV

Abstract: The automatic registration of differently stained whole slide images (WSIs) is crucial for improving diagnosis and prognosis by fusing complementary information emerging from different visible structures. It is also useful to quickly transfer annotations between consecutive or restained slides, thus significantly reducing the annotation time and associated costs. Nevertheless, the slide preparation is different for each stain and the tissue undergoes complex and large deformations. Therefore, a robust, efficient, and accurate registration method is highly desired by the scientific community and hospitals specializing in digital pathology. We propose a two-step hybrid method consisting of (i) deep learning- and feature-based initial alignment algorithm, and (ii) intensity-based nonrigid registration using the instance optimization. The proposed method does not require any fine-tuning to a particular dataset and can be used directly for any desired tissue type and stain. The method scored 1st place in the ACROBAT 2023 challenge. We evaluated using three open datasets: (i) ANHIR, (ii) ACROBAT, and (iii) HyReCo, and performed several ablation studies concerning the resolution used for registration and the initial alignment robustness and stability. The method achieves the most accurate results for the ACROBAT dataset, the cell-level registration accuracy for the restained slides from the HyReCo dataset, and is among the best methods evaluated on the ANHIR dataset. The method does not require any fine-tuning to a new datasets and can be used out-of-the-box for other types of microscopic images. The method is incorporated into the DeeperHistReg framework, allowing others to directly use it to register, transform, and save the WSIs at any desired pyramid level. The proposed method is a significant contribution to the WSI registration, thus advancing the field of digital pathology.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. ANHIR: automatic non-rigid histological image registration challenge. IEEE transactions on medical imaging, 39(10):3042–3052, 2020.
  2. Increasing the usefulness of already existing annotations through wsi registration. arXiv preprint arXiv:2303.06727, 2023a.
  3. 3d reconstruction of multiple stained histology images. Journal of pathology informatics, 4(2):7, 2013a.
  4. A review of artifacts in histopathology. Journal of oral and maxillofacial pathology: JOMFP, 22(2):279, 2018.
  5. HistoQC: an open-source quality control tool for digital pathology slides. JCO clinical cancer informatics, 3:1–7, 2019.
  6. M. Wodzinski. DeeperHistReg Library. https://github.com/MWod/DeeperHistReg/, 2023. [Online; accessed 14-December-2023].
  7. DeeperHistReg: Robust Whole Slide Images Registration Framework. In Review, 2024.
  8. The ACROBAT 2022 Challenge: Automatic Registration Of Breast Cancer Tissue. arXiv preprint arXiv:2305.18033, 2023b.
  9. ACROBAT - a multi-stain breast cancer histological whole-slide-image data set from routine diagnostics for computational pathology. arXiv preprint arXiv:2211.13621, 2022.
  10. Comparison of consecutive and restained sections for image registration in histopathology. Journal of Medical Imaging, 2023.
  11. HyReCo - Hybrid Re-Stained And Consecutive Histological Serial Sections. IEEE Data Port, 2021.
  12. End-to-end affine registration framework for histopathological images with weak annotations. Computer Methods and Programs in Biomedicine, 241:107763, 2023.
  13. Deep feature based cross-slide registration. Computerized Medical Imaging and Graphics, 104:102162, 2023.
  14. Deformable medical image registration: A survey. IEEE transactions on medical imaging, 32(7):1153–1190, 2013.
  15. Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning. IEEE Transactions on Medical Imaging, 42(3):697–712, 2022.
  16. Large deformation diffeomorphic image registration with laplacian pyramid networks. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part III 23, pages 211–221. Springer, 2020.
  17. Voxelmorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging, 38(8):1788–1800, 2019.
  18. Elastix: a toolbox for intensity-based medical image registration. IEEE transactions on medical imaging, 29(1):196–205, 2009.
  19. Deep learning in medical image registration: a survey. Machine Vision and Applications, 31(1):8, 2020.
  20. Deformable medical image registration using generative adversarial networks. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pages 1449–1453. IEEE, 2018.
  21. Simpleelastix: A user-friendly, multi-lingual library for medical image registration. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 134–142, 2016.
  22. Deformable medical image registration: setting the state of the art with discrete methods. Annual review of biomedical engineering, 13:219–244, 2011.
  23. Mind: Modality independent neighbourhood descriptor for multi-modal deformable registration. Medical image analysis, 16(7):1423–1435, 2012.
  24. Medical image registration using mutual information. Proceedings of the IEEE, 91(10):1699–1722, 2003.
  25. Benchmarking of image registration methods for differently stained histological slides. In 2018 25th IEEE International Conference on Image Processing (ICIP), pages 3368–3372. IEEE, 2018.
  26. Robust, fast and accurate: a 3-step method for automatic histological image registration. arXiv preprint arXiv:1903.12063, 2019.
  27. Virtual alignment of pathology image series for multi-gigapixel whole slide images. Nature communications, 14(1):4502, 2023.
  28. Accurate and robust alignment of variable-stained histologic images using a general-purpose greedy diffeomorphic registration tool. arXiv preprint arXiv:1904.11929, 2019.
  29. Accurate and robust alignment of differently stained histologic images based on greedy diffeomorphic registration. Applied Sciences, 11(4):1892, 2021.
  30. Multistep, automatic and nonrigid image registration method for histology samples acquired using multiple stains. Physics in Medicine & Biology, 66(2):025006, 2021.
  31. Unsupervised content classification based nonrigid registration of differently stained histology images. IEEE Transactions on Biomedical Engineering, 61(1):96–108, 2013b.
  32. Patch-based nonlinear image registration for gigapixel whole slide images. IEEE Transactions on Biomedical Engineering, 63(9):1812–1819, 2015.
  33. Robust quad-tree based registration on whole slide images. In MICCAI Workshop on Computational Pathology, pages 181–190. PMLR, 2021.
  34. Unsupervised 3d end-to-end medical image registration with volume tweening network. IEEE journal of biomedical and health informatics, 24(5):1394–1404, 2019.
  35. DeepHistReg: Unsupervised deep learning registration framework for differently stained histology samples. Computer methods and programs in biomedicine, 198:105799, 2021.
  36. Unsupervised learning-based nonrigid registration of high resolution histology images. In Machine Learning in Medical Imaging: 11th International Workshop, MLMI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Proceedings 11, pages 484–493. Springer, 2020a.
  37. Unsupervised histological image registration using structural feature guided convolutional neural network. IEEE Transactions on Medical Imaging, 41(9):2414–2431, 2022.
  38. VA Pyatov and DV Sorokin. Affine registration of histological images using transformer-based feature matching. Pattern Recognition and Image Analysis, 32(3):626–630, 2022a.
  39. Tahir: Transformer-based affine histological image registration. In International Conference on Pattern Recognition, pages 541–552. Springer, 2022b.
  40. Learning-based affine registration of histological images. In Biomedical Image Registration: 9th International Workshop, WBIR 2020, Portorož, Slovenia, December 1–2, 2020, Proceedings 9, pages 12–22. Springer, 2020b.
  41. PyVips. PyVips Library. https://www.libvips.org/, 2023. [Online; accessed 14-December-2023].
  42. Superpoint: Self-supervised interest point detection and description. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 224–236, 2018.
  43. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4938–4947, 2020.
  44. ANHIR Evaluation Platform. https://anhir.grand-challenge.org/evaluation/challenge/leaderboard/. [Online; accessed 14-December-2023].
  45. R. Fernandez-Gonzalez et al. System for combined three-dimensional morphological and molecular analysis of thick tissue specimens. Microscopy Research and Technique, 59(6):522–530, 2002.
  46. L. Gupta et al. Stain independent segmentation of whole slide images: A case study in renal histology. Proceedings - International Symposium on Biomedical Imaging, pages 1360–1364, 2018.
  47. The immune microenvironment of various histological types of ebv-associated gastric cancer. Virchows Archiv, 2018.
  48. G. Bueno and O. Deniz. AIDPATH: Academia and Industry Collaboration for Digital Pathology. http://aidpath.eu.
  49. ACROBAT Evaluation Platform. https://acrobat.grand-challenge.org/evaluation/model-development/leaderboard/. [Online; accessed 14-December-2023].
  50. Resolution-agnostic tissue segmentation in whole-slide histopathology images with convolutional neural networks. PeerJ, 7:e8242, 2019.
  51. Robust Multiresolution and Multistain Background Segmentation in Whole Slide Images. In Polish Conference on Biocybernetics and Biomedical Engineering, pages 29–40. Springer, 2023.
  52. BigPicture project. https://bigpicture.eu//. [Online; accessed 14-December-2023].
  53. Deformable Medical Image Registration Under Distribution Shifts with Neural Instance Optimization. In International Workshop on Machine Learning in Medical Imaging, pages 126–136. Springer, 2023.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now