Papers
Topics
Authors
Recent
Search
2000 character limit reached

Graph Relation Distillation for Efficient Biomedical Instance Segmentation

Published 12 Jan 2024 in cs.CV and cs.AI | (2401.06370v1)

Abstract: Instance-aware embeddings predicted by deep neural networks have revolutionized biomedical instance segmentation, but its resource requirements are substantial. Knowledge distillation offers a solution by transferring distilled knowledge from heavy teacher networks to lightweight yet high-performance student networks. However, existing knowledge distillation methods struggle to extract knowledge for distinguishing instances and overlook global relation information. To address these challenges, we propose a graph relation distillation approach for efficient biomedical instance segmentation, which considers three essential types of knowledge: instance-level features, instance relations, and pixel-level boundaries. We introduce two graph distillation schemes deployed at both the intra-image level and the inter-image level: instance graph distillation (IGD) and affinity graph distillation (AGD). IGD constructs a graph representing instance features and relations, transferring these two types of knowledge by enforcing instance graph consistency. AGD constructs an affinity graph representing pixel relations to capture structured knowledge of instance boundaries, transferring boundary-related knowledge by ensuring pixel affinity consistency. Experimental results on a number of biomedical datasets validate the effectiveness of our approach, enabling student models with less than $ 1\%$ parameters and less than $10\%$ inference time while achieving promising performance compared to teacher models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. H. Chen, X. Qi, L. Yu, and P.-A. Heng, “Dcan: deep contour-aware networks for accurate gland segmentation,” in CVPR, 2016.
  2. M. Li, C. Chen, X. Liu, W. Huang, Y. Zhang, and Z. Xiong, “Advanced deep networks for 3d mitochondria instance segmentation,” in ISBI.   IEEE, 2022, pp. 1–5.
  3. N. Kumar, R. Verma, S. Sharma, S. Bhargava, A. Vahadane, and A. Sethi, “A dataset and a technique for generalized nuclear segmentation for computational pathology,” IEEE Trans. Med. Imag., vol. 36, no. 7, pp. 1550–1560, 2017.
  4. Z. Song, P. Wang, J. Zhou, Z. Yang, Y. Yang, Z. Gong, and N. Zheng, “Muscleparsenet: a novel framework for parsing muscles of drosophila larva in light-sheet fluorescence microscopy images,” IEEE Trans. Circuits Syst. Video Technol., 2023.
  5. K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” in Proc. Int. Conf. Comput. Vis., 2017, pp. 2961–2969.
  6. D. Liu, D. Zhang, Y. Song, C. Zhang, F. Zhang, L. O’Donnell, and W. Cai, “Nuclei segmentation via a deep panoptic model with semantic feature fusion.” in IJCAI, 2019, pp. 861–868.
  7. D. Zhang, Y. Song, D. Liu, H. Jia, S. Liu, Y. Xia, H. Huang, and W. Cai, “Panoptic segmentation with an end-to-end cell r-cnn for pathology image analysis,” in MICCAI.   Springer, 2018, pp. 237–244.
  8. L. Chen, M. Strauch, and D. Merhof, “Instance segmentation of biomedical images with an object-aware embedding learned with local constraints,” in MICCAI.   Springer, 2019, pp. 451–459.
  9. V. Kulikov and V. Lempitsky, “Instance segmentation of biological images using harmonic embeddings,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2020, pp. 3843–3851.
  10. K. Lee, R. Lu, K. Luther, and H. S. Seung, “Learning and segmenting dense voxel embeddings for 3d neuron reconstruction,” IEEE Trans. Med. Imag., vol. 40, no. 12, pp. 3801–3811, 2021.
  11. C. Payer, D. Štern, T. Neff, H. Bischof, and M. Urschler, “Instance segmentation and tracking with cosine embeddings and recurrent hourglass networks,” in MICCAI.   Springer, 2018, pp. 3–11.
  12. G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.
  13. A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, “Fitnets: Hints for thin deep nets,” arXiv preprint arXiv:1412.6550, 2014.
  14. F. Tung and G. Mori, “Similarity-preserving knowledge distillation,” in Proc. Int. Conf. Comput. Vis., 2019, pp. 1365–1374.
  15. N. Zagoruyko, Komodakis, “Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer,” arXiv preprint arXiv:1612.03928, 2016.
  16. D. Qin, J.-J. Bu, Z. Liu, X. Shen, S. Zhou, J.-J. Gu, Z.-H. Wang, L. Wu, and H.-F. Dai, “Efficient medical image segmentation based on knowledge distillation,” IEEE Trans. Med. Imag., vol. 40, no. 12, pp. 3820–3831, 2021.
  17. P. Chen, S. Liu, H. Zhao, and J. Jia, “Distilling knowledge via knowledge review,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2021, pp. 5008–5017.
  18. X. Liu, B. Hu, W. Huang, Y. Zhang, and Z. Xiong, “Efficient biomedical instance segmentation via knowledge distillation,” in MICCAI.   Springer, 2022, pp. 14–24.
  19. D. Liu, D. Zhang, Y. Song, H. Huang, and W. Cai, “Panoptic feature fusion net: a novel instance segmentation paradigm for biomedical and biological images,” IEEE Trans. Image Process., vol. 30, pp. 2045–2059, 2021.
  20. A. Kirillov, R. Girshick, K. He, and P. Dollár, “Panoptic feature pyramid networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2019, pp. 6399–6408.
  21. J. Yi, P. Wu, M. Jiang, Q. Huang, D. J. Hoeppner, and D. N. Metaxas, “Attentive neural cell instance segmentation,” Medical image analysis, vol. 55, pp. 228–240, 2019.
  22. R. Girshick, “Fast r-cnn,” in Proc. Int. Conf. Comput. Vis., 2015, pp. 1440–1448.
  23. X. Zhang, H. Li, F. Meng, Z. Song, and L. Xu, “Segmenting beyond the bounding box for instance segmentation,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 2, pp. 704–714, 2021.
  24. H. Zhang, Y. Tian, K. Wang, W. Zhang, and F.-Y. Wang, “Mask ssd: An effective single-stage approach to object instance segmentation,” IEEE Trans. Circuits Syst. Video Technol., vol. 29, pp. 2078–2093, 2019.
  25. L. Yang, H. Li, F. Meng, Q. Wu, and K. N. Ngan, “Task-specific loss for robust instance segmentation with noisy class labels,” IEEE Trans. Circuits Syst. Video Technol., 2021.
  26. B. De Brabandere, D. Neven, and L. Van Gool, “Semantic instance segmentation with a discriminative loss function,” arXiv preprint arXiv:1708.02551, 2017.
  27. M. Lalit, P. Tomancak, and F. Jug, “Embedseg: Embedding-based instance segmentation for biomedical microscopy data,” Medical image analysis, vol. 81, p. 102523, 2022.
  28. J.-H. Shi, Q. Zhang, Y.-H. Tang, and Z.-Q. Zhang, “Polyp-mixer: An efficient context-aware mlp-based paradigm for polyp segmentation,” IEEE Trans. Circuits Syst. Video Technol., vol. 33, no. 1, pp. 30–42, 2022.
  29. T. Beier, C. Pape, N. Rahaman, T. Prange, S. Berg, D. D. Bock, A. Cardona, G. W. Knott, S. M. Plaza, L. K. Scheffer et al., “Multicut brings automated neurite segmentation closer to human performance,” Nature methods, vol. 14, no. 2, pp. 101–102, 2017.
  30. K. Fukunaga and L. Hostetler, “The estimation of the gradient of a density function, with applications in pattern recognition,” IEEE Trans. Inf. Theory, vol. 21, no. 1, pp. 32–40, 1975.
  31. D. Comaniciu and P. Meer, “Mean shift: A robust approach toward feature space analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603–619, 2002.
  32. R. J. Campello, D. Moulavi, and J. Sander, “Density-based clustering based on hierarchical density estimates,” in Pacific-Asia conference on knowledge discovery and data mining.   Springer, 2013, pp. 160–172.
  33. A. Wolny, Q. Yu, C. Pape, and A. Kreshuk, “Sparse object-level supervision for instance segmentation with pixel embeddings,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2022, pp. 4402–4411.
  34. M. Lalit, P. Tomancak, and F. Jug, “Embedding-based instance segmentation in microscopy,” in MIDL, 2021.
  35. C. Payer, D. Štern, M. Feiner, H. Bischof, and M. Urschler, “Segmenting and tracking cell instances with cosine embeddings and recurrent hourglass networks,” Medical image analysis, vol. 57, pp. 106–119, 2019.
  36. W. Huang, S. Deng, C. Chen, X. Fu, and Z. Xiong, “Learning to model pixel-embedded affinity for homogeneous instance segmentation,” in AAAI, vol. 36, no. 1, 2022, pp. 1007–1015.
  37. X. Liu, W. Huang, Y. Zhang, and Z. Xiong, “Biological instance segmentation with a superpixel-guided graph.” in IJCAI, 2022.
  38. T. Fukuda, M. Suzuki, G. Kurata, S. Thomas, J. Cui, and B. Ramabhadran, “Efficient knowledge distillation from an ensemble of teachers.” in Interspeech, 2017, pp. 3697–3701.
  39. C.-H. Chao, B.-W. Cheng, and C.-Y. Lee, “Rethinking ensemble-distillation for semantic segmentation based unsupervised domain adaption,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2021, pp. 2610–2620.
  40. J. M. Noothout, N. Lessmann, M. C. Van Eede, L. D. van Harten, E. Sogancioglu, F. G. Heslinga, M. Veta, B. van Ginneken, and I. Išgum, “Knowledge distillation with ensembles of convolutional neural networks for medical image segmentation,” Journal of Medical Imaging, vol. 9, no. 5, pp. 052 407–052 407, 2022.
  41. Y. Liu, J. Cao, B. Li, C. Yuan, W. Hu, Y. Li, and Y. Duan, “Knowledge distillation via instance relationship graph,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2019, pp. 7096–7104.
  42. C. Li, G. Cheng, and J. Han, “Boosting knowledge distillation via intra-class logit distribution smoothing,” IEEE Trans. Circuits Syst. Video Technol., 2023.
  43. L. Xu, J. Ren, Z. Huang, W. Zheng, and Y. Chen, “Improving knowledge distillation via head and tail categories,” IEEE Trans. Circuits Syst. Video Technol., 2023.
  44. Y. Wen, L. Chen, S. Xi, Y. Deng, X. Tang, and C. Zhou, “Towards efficient medical image segmentation via boundary-guided knowledge distillation,” in ICME.   IEEE, 2021, pp. 1–6.
  45. Y. Chen, P. Chen, S. Liu, L. Wang, and J. Jia, “Deep structured instance graph for distilling object detectors,” in Proc. Int. Conf. Comput. Vis., 2021, pp. 4359–4368.
  46. Y. Fu, Y. Feng, and J. P. Cunningham, “Paraphrase generation with latent bag of words,” Adv. Neural Inform. Process. Syst., vol. 32, 2019.
  47. Y. Tian, D. Krishnan, and P. Isola, “Contrastive multiview coding,” in Proc. Eur. Conf. Comput. Vis.   Springer, 2020, pp. 776–794.
  48. C. Yang, H. Zhou, Z. An, X. Jiang, Y. Xu, and Q. Zhang, “Cross-image relational knowledge distillation for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2022, pp. 12 319–12 328.
  49. J. Funke, F. Tschopp, W. Grisaitis, A. Sheridan, C. Singh, S. Saalfeld, and S. C. Turaga, “Large scale image segmentation with structured loss based deep learning for connectome reconstruction,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 7, pp. 1669–1680, 2018.
  50. S. Wolf, A. Bailoni, C. Pape, N. Rahaman, A. Kreshuk, U. Köthe, and F. A. Hamprecht, “The mutex watershed and its objective: Efficient, parameter-free graph partitioning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 10, pp. 3724–3738, 2020.
  51. E. M. A. Anas, S. Nouranian, S. S. Mahdavi, I. Spadinger, W. J. Morris, S. E. Salcudean, P. Mousavi, and P. Abolmaesumi, “Clinical target-volume delineation in prostate brachytherapy using residual neural networks,” in MICCAI.   Springer, 2017, pp. 365–373.
  52. Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” in MICCAI worshops.   Springer, 2018, pp. 3–11.
  53. O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in MICCAI.   Springer, 2015, pp. 234–241.
  54. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2018, pp. 4510–4520.
  55. H. Scharr, M. Minervini, A. Fischbach, and S. A. Tsaftaris, “Annotated image datasets of rosette plants,” in ECCV, 2014, pp. 6–12.
  56. V. Ljosa, K. L. Sokolnicki, and A. E. Carpenter, “Annotated high-throughput microscopy image sets for validation.” Nature methods, vol. 9, no. 7, pp. 637–637, 2012.
  57. W. M. Rand, “Objective criteria for the evaluation of clustering methods,” Journal of the American Statistical association, vol. 66, no. 336, pp. 846–850, 1971.
  58. H. Chen, X. Qi, L. Yu, Q. Dou, J. Qin, and P.-A. Heng, “Dcan: Deep contour-aware networks for object instance segmentation from histology images,” Medical image analysis, vol. 36, pp. 135–146, 2017.
  59. N. Kasthuri, K. J. Hayworth, D. R. Berger, R. L. Schalek, J. A. Conchello, S. Knowles-Barley, D. Lee, A. Vázquez-Reina, V. Kaynig, T. R. Jones et al., “Saturated reconstruction of a volume of neocortex,” Cell, vol. 162, no. 3, pp. 648–661, 2015.
  60. M. Meilă, “Comparing clusterings by the variation of information,” in LTKM workshop.   Springer, 2003, pp. 173–187.
  61. CREMI, “Miccal challenge on circuit reconstruction from electron microscopy images,” https://cremi.org/, 2016.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

GitHub

  1. GitHub - liuxy1103/GRDBIS (21 stars)