Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Cross-attention learning enables real-time nonuniform rotational distortion correction in OCT (2306.04512v2)

Published 7 Jun 2023 in eess.IV, cs.CV, and physics.med-ph

Abstract: Nonuniform rotational distortion (NURD) correction is vital for endoscopic optical coherence tomography (OCT) imaging and its functional extensions, such as angiography and elastography. Current NURD correction methods require time-consuming feature tracking or cross-correlation calculations and thus sacrifice temporal resolution. Here we propose a cross-attention learning method for the NURD correction in OCT. Our method is inspired by the recent success of the self-attention mechanism in natural language processing and computer vision. By leveraging its ability to model long-range dependencies, we can directly obtain the correlation between OCT A-lines at any distance, thus accelerating the NURD correction. We develop an end-to-end stacked cross-attention network and design three types of optimization constraints. We compare our method with two traditional feature-based methods and a CNN-based method, on two publicly-available endoscopic OCT datasets and a private dataset collected on our home-built endoscopic OCT system. Our method achieved a $\sim3\times$ speedup to real time ($26\pm 3$ fps), and superior correction performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. D. Huang, E. A. Swanson, C. P. Lin, J. S. Schuman, W. G. Stinson, W. Chang, M. R. Hee, T. Flotte, K. Gregory, C. A. Puliafito et al., “Optical coherence tomography,” \JournalTitlescience 254, 1178–1181 (1991).
  2. W. Drexler, M. Liu, A. Kumar, T. Kamali, A. Unterhuber, and R. A. Leitgeb, “Optical coherence tomography today: speed, contrast, and multimodality,” \JournalTitleJournal of biomedical optics 19, 071412–071412 (2014).
  3. M. Adhi and J. S. Duker, “Optical coherence tomography–current and future applications,” \JournalTitleCurrent opinion in ophthalmology 24, 213 (2013).
  4. E. A. Swanson and J. G. Fujimoto, “The ecosystem that powered the translation of oct from fundamental research to clinical and commercial impact,” \JournalTitleBiomedical optics express 8, 1638–1664 (2017).
  5. E. Zagaynova, N. Gladkova, N. Shakhova, G. Gelikonov, and V. Gelikonov, “Endoscopic oct with forward-looking probe: clinical studies in urology and gastroenterology,” \JournalTitleJournal of biophotonics 1, 114–128 (2008).
  6. B. E. Bouma, M. Villiger, K. Otsuka, and W.-Y. Oh, “Intravascular optical coherence tomography,” \JournalTitleBiomedical optics express 8, 2660–2686 (2017).
  7. M. J. Gora, M. J. Suter, G. J. Tearney, and X. Li, “Endoscopic optical coherence tomography: technologies and clinical applications,” \JournalTitleBiomedical optics express 8, 2405–2444 (2017).
  8. M. Araki, S.-J. Park, H. L. Dauerman, S. Uemura, J.-S. Kim, C. Di Mario, T. W. Johnson, G. Guagliumi, A. Kastrati, M. Joner et al., “Optical coherence tomography in coronary atherosclerosis assessment and intervention,” \JournalTitleNature Reviews Cardiology 19, 684–703 (2022).
  9. O. O. Ahsen, H.-C. Lee, M. G. Giacomelli, Z. Wang, K. Liang, T.-H. Tsai, B. Potsaid, H. Mashimo, and J. G. Fujimoto, “Correction of rotational distortion for catheter-based en face oct and oct angiography,” \JournalTitleOptics letters 39, 5973–5976 (2014).
  10. T. Wang, T. Pfeiffer, E. Regar, W. Wieser, H. van Beusekom, C. T. Lancee, G. Springeling, I. Krabbendam, A. F. van der Steen, R. Huber et al., “Heartbeat oct: in vivo intravascular megahertz-optical coherence tomography,” \JournalTitleBiomedical optics express 6, 5021–5032 (2015).
  11. W. C. Lo, N. Uribe-Patarroyo, K. Hoebel, K. Beaudette, M. Villiger, N. S. Nishioka, B. J. Vakoc, and B. E. Bouma, “Balloon catheter-based radiofrequency ablation monitoring in porcine esophagus using optical coherence tomography,” \JournalTitleBiomedical Optics Express 10, 2067–2089 (2019).
  12. G. Cao, S. Li, S. Zhang, Z. Peng, Y. Wu, D. Wang, and C. Dai, “Improved fast algorithm for non-uniform rotational distortion correction in oct endoscopic imaging,” \JournalTitleOptics Express 31, 2754–2767 (2023).
  13. G. van Soest, J. G. Bosch, and A. F. van der Steen, “Azimuthal registration of image sequences affected by nonuniform rotation distortion,” \JournalTitleIEEE Transactions on Information Technology in Biomedicine 12, 348–355 (2008).
  14. L. Qi, Z. Zhuang, S. Zhang, S. Huang, Q. Feng, and W. Chen, “Automatic correction of the initial rotation angle error improves 3d reconstruction in endoscopic airway optical coherence tomography,” \JournalTitleBiomedical Optics Express 12, 7616–7631 (2021).
  15. Y. Miao, J. J. Jing, and Z. Chen, “Graph-based rotational nonuniformity correction for localized compliance measurement in the human nasopharynx,” \JournalTitleBiomedical Optics Express 12, 2508–2518 (2021).
  16. E. Abouei, A. M. Lee, H. Pahlevaninezhad, G. Hohert, M. Cua, P. Lane, S. Lam, and C. MacAulay, “Correction of motion artifacts in endoscopic optical coherence tomography and autofluorescence images based on azimuthal en face image registration,” \JournalTitleJournal of biomedical optics 23, 016004–016004 (2018).
  17. N. Uribe-Patarroyo and B. E. Bouma, “Rotational distortion correction in endoscopic optical coherence tomography based on speckle decorrelation,” \JournalTitleOptics letters 40, 5518–5521 (2015).
  18. S. Guo, S. Wei, S. Lee, M. Sheu, S. Kang, and J. U. Kang, “Intraoperative speckle variance optical coherence tomography for tissue temperature monitoring during cutaneous laser therapy,” \JournalTitleIEEE Journal of Translational Engineering in Health and Medicine 7, 1–8 (2019).
  19. G. Liao, O. Caravaca-Mora, B. Rosa, P. Zanne, D. Dall’Alba, P. Fiorini, M. de Mathelin, F. Nageotte, and M. J. Gora, “Distortion and instability compensation with deep learning for rotational scanning endoscopic optical coherence tomography,” \JournalTitleMedical Image Analysis 77, 102355 (2022).
  20. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” \JournalTitleAdvances in neural information processing systems 30 (2017).
  21. N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano, “Learning to summarize with human feedback,” \JournalTitleAdvances in Neural Information Processing Systems 33, 3008–3021 (2020).
  22. J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision, (2017), pp. 2223–2232.
  23. C. Sun, F. Nolte, K. H. Cheng, B. Vuong, K. K. Lee, B. A. Standish, B. Courtney, T. R. Marotta, A. Mariampillai, and V. X. Yang, “In vivo feasibility of endovascular doppler optical coherence tomography,” \JournalTitleBiomedical optics express 3, 2600–2610 (2012).
  24. S.-W. Lee, A. E. Heidary, D. Yoon, D. Mukai, T. Ramalingam, S. Mahon, J. Yin, J. Jing, G. Liu, Z. Chen et al., “Quantification of airway thickness changes in smoke-inhalation injury using in-vivo 3-d endoscopic frequency-domain optical coherence tomography,” \JournalTitleBiomedical optics express 2, 243–254 (2011).
  25. J. Li, M. de Groot, F. Helderman, J. Mo, J. M. Daniels, K. Grünberg, T. G. Sutedja, and J. F. de Boer, “High speed miniature motorized endoscopic probe for optical frequency domain imaging,” \JournalTitleOptics express 20, 24132–24138 (2012).
  26. T. Wang, W. Wieser, G. Springeling, R. Beurskens, C. T. Lancee, T. Pfeiffer, A. F. Van Der Steen, R. Huber, and G. Van Soest, “Intravascular optical coherence tomography imaging at 3200 frames per second,” \JournalTitleOptics letters 38, 1715–1717 (2013).
  27. S. H. Yun, G. J. Tearney, B. J. Vakoc, M. Shishkov, W. Y. Oh, A. E. Desjardins, M. J. Suter, R. C. Chan, J. A. Evans, I.-K. Jang et al., “Comprehensive volumetric optical microscopy in vivo,” \JournalTitleNature medicine 12, 1429–1433 (2006).
  28. M. J. Gora, J. S. Sauk, R. W. Carruth, K. A. Gallagher, M. J. Suter, N. S. Nishioka, L. E. Kava, M. Rosenberg, B. E. Bouma, and G. J. Tearney, “Tethered capsule endomicroscopy enables less invasive imaging of gastrointestinal tract microstructure,” \JournalTitleNature medicine 19, 238–240 (2013).
  29. G. Liao, O. Caravaca-Mora, B. Rosa, P. Zanne, A. Asch, D. Dall’Alba, P. Fiorini, M. de Mathelin, F. Nageotte, and M. J. Gora, “Data stream stabilization for optical coherence tomography volumetric scanning,” \JournalTitleIEEE Transactions on Medical Robotics and Bionics 3, 855–865 (2021).
  30. W. Kim, X. Chen, J. A. Jo, and B. E. Applegate, “Lensless, ultra-wideband fiber optic rotary joint for biomedical applications,” \JournalTitleOptics letters 41, 1973–1976 (2016).
  31. L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in Proceedings of COMPSTAT’2010: 19th International Conference on Computational StatisticsParis France, August 22-27, 2010 Keynote, Invited and Contributed Papers, (Springer, 2010), pp. 177–186.
  32. Z. A. Ali, U. Landmesser, A. Maehara, M. Matsumura, R. A. Shlofmitz, G. Guagliumi, M. J. Price, J. M. Hill, T. Akasaka, F. Prati et al., “Optical coherence tomography–guided versus angiography-guided pci,” \JournalTitleNew England Journal of Medicine (2023).
  33. Z. A. Ali, A. Maehara, P. Généreux, R. A. Shlofmitz, F. Fabbiocchi, T. M. Nazif, G. Guagliumi, P. M. Meraj, F. Alfonso, H. Samady et al., “Optical coherence tomography compared with intravascular ultrasound and with angiography to guide coronary stent implantation (ilumien iii: Optimize pci): a randomised controlled trial,” \JournalTitleThe Lancet 388, 2618–2628 (2016).
  34. H. Bogunović, F. Venhuizen, S. Klimscha et al., “Retouch: the retinal oct fluid detection and segmentation benchmark and challenge,” \JournalTitleIEEE transactions on medical imaging 38, 1858–1874 (2019).
  35. Z. Niu, G. Zhong, and H. Yu, “A review on the attention mechanism of deep learning,” \JournalTitleNeurocomputing 452, 48–62 (2021).
  36. N. Patwardhan, S. Marrone, and C. Sansone, “Transformers in the real world: A survey on nlp applications,” \JournalTitleInformation 14, 242 (2023).
  37. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations, (2020).
  38. K. Han, Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao, C. Xu, Y. Xu et al., “A survey on vision transformer,” \JournalTitleIEEE transactions on pattern analysis and machine intelligence 45, 87–110 (2022).
  39. J. Li, J. Chen, Y. Tang, C. Wang, B. A. Landman, and S. K. Zhou, “Transforming medical imaging with transformers? a comparative review of key properties, current progresses, and future perspectives,” \JournalTitleMedical image analysis p. 102762 (2023).
  40. P. Zaffino, S. Moccia, E. De Momi, and M. F. Spadea, “A review on advances in intra-operative imaging for surgery and therapy: imagining the operating room of the future,” \JournalTitleAnnals of Biomedical Engineering 48, 2171–2191 (2020).
  41. S. B. de Koning, A. Schaeffers, W. Schats, M. van den Brekel, T. Ruers, and M. Karakullukcu, “Assessment of the deep resection margin during oral cancer surgery: A systematic review,” \JournalTitleEuropean journal of surgical oncology 47, 2220–2232 (2021).
  42. L. Yunyao, F. Jinyu, J. Tianliang, T. Ning, and S. Guohua, “Review of the development of optical coherence tomography imaging navigation technology in ophthalmic surgery,” \JournalTitleOpto-Electronic Engineering 50, 220027–1 (2023).
  43. R. Leitgeb, F. Placzek, E. Rank, L. Krainz, R. Haindl, Q. Li, M. Liu, M. Andreana, A. Unterhuber, T. Schmoll et al., “Enhanced medical diagnosis for doctors: a perspective of optical coherence tomography,” \JournalTitleJournal of Biomedical Optics 26, 100601–100601 (2021).
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now