Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Cell Graph Transformer for Nuclei Classification (2402.12946v1)

Published 20 Feb 2024 in cs.CV

Abstract: Nuclei classification is a critical step in computer-aided diagnosis with histopathology images. In the past, various methods have employed graph neural networks (GNN) to analyze cell graphs that model inter-cell relationships by considering nuclei as vertices. However, they are limited by the GNN mechanism that only passes messages among local nodes via fixed edges. To address the issue, we develop a cell graph transformer (CGT) that treats nodes and edges as input tokens to enable learnable adjacency and information exchange among all nodes. Nevertheless, training the transformer with a cell graph presents another challenge. Poorly initialized features can lead to noisy self-attention scores and inferior convergence, particularly when processing the cell graphs with numerous connections. Thus, we further propose a novel topology-aware pretraining method that leverages a graph convolutional network (GCN) to learn a feature extractor. The pre-trained features may suppress unreasonable correlations and hence ease the finetuning of CGT. Experimental results suggest that the proposed cell graph transformer with topology-aware pretraining significantly improves the nuclei classification results, and achieves the state-of-the-art performance. Code and models are available at https://github.com/lhaof/CGT

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Multi-class cell detection using spatial context representation. In ICCV, 4005–4014.
  2. NuCLS: A scalable crowdsourcing approach and dataset for nucleus classification and segmentation in breast cancer. GigaScience, 11.
  3. Histographs: graphs in histopathology. In Medical Imaging 2020: Digital Pathology, volume 11320, 150–155. SPIE.
  4. Learning whole-slide segmentation from inexact and incomplete labels using tissue graphs. In MICCAI, 636–646. Springer.
  5. Rccnet: An efficient convolutional neural network for histological routine colon cancer nuclei classification. In ICARCV, 1222–1227. IEEE.
  6. Disentangle your dense object detector. In ACM Multimedia, 4939–4948.
  7. Masked-attention mask transformer for universal image segmentation. In CVPR, 1290–1299.
  8. Augmented cell-graphs for automated cancer diagnosis. Bioinformatics, 21(suppl_2): ii7–ii12.
  9. SONNET: A self-guided ordinal regression neural network for segmentation and classification of nuclei in large-scale multi-tissue histology images. IEEE JBHI.
  10. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699.
  11. Benchmarking graph neural networks. arXiv preprint arXiv:2003.00982.
  12. Fast graph representation learning with PyTorch Geometric. ICLR Workshop.
  13. Pannuke dataset extension, insights and baselines. arXiv preprint arXiv:2003.10778.
  14. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430.
  15. Lizard: A large-scale dataset for colonic nuclear instance segmentation and classification. In ICCV Workshops, 684–693.
  16. Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. MIA, 58: 101563.
  17. Toward a shared vision for cancer genomic data. New England Journal of Medicine, 375(12): 1109–1112.
  18. Visual attention network. Computational Visual Media, 1–20.
  19. Nucleus Classification in Histology Images Using Message Passing Network. MIA, 102480.
  20. Prompt-based grouping transformer for nucleus detection and classification. In MICCAI, 569–579. Springer.
  21. Affine-Consistent Transformer for Multi-Class Cell Nuclei Detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 21384–21393.
  22. Cellular community detection for tissue phenotyping in colorectal cancer histology images. MIA, 63: 101696.
  23. Pure Transformers are Powerful Graph Learners. NeurIPS.
  24. Transformers generalize deepsets and can be extended to graphs & hypergraphs. NeurIPS, 34: 28016–28028.
  25. Panoptic segmentation. In CVPR, 9404–9413.
  26. Rethinking graph transformers with spectral attention. NeurIPS, 34: 21618–21629.
  27. Breast cancer detection, segmentation and classification on histopathology images analysis: a systematic review. Archives of Computational Methods in Engineering, 28: 2607–2619.
  28. A review and comparison of breast tumor cell nuclei segmentation performances using deep convolutional neural networks. Scientific Reports, 11(1): 8025.
  29. Deepgcns: Making gcns go as deep as cnns. TPAMI, 6923 – 6939.
  30. Deepergcn: All you need to train deeper gcns. arXiv preprint arXiv:2006.07739.
  31. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI, volume 32.
  32. Mesh graphormer. In ICCV, 12939–12948.
  33. Feature pyramid networks for object detection. In CVPR, 2117–2125.
  34. Features for cells and nuclei classification. In EMBC, 6601–6604.
  35. Pathological prognosis classification of patients with neuroblastoma using computational pathology analysis. CBM, 149: 105980.
  36. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 10012–10022.
  37. A convnet for the 2020s. In CVPR, 11976–11986.
  38. Which pixel to annotate: a label-efficient nuclei segmentation framework. IEEE Transactions on Medical Imaging, 42(4): 947–958.
  39. Structure Embedded Nucleus Classification for Histopathology Images. arXiv preprint arXiv:2302.11416.
  40. Multi-stream Cell Segmentation with Low-level Cues for Multi-modality Images. In Competitions in Neural Information Processing Systems, 1–10. PMLR.
  41. The Multi-modality Cell Segmentation Challenge: Towards Universal Solutions. arXiv preprint arXiv:2308.05864.
  42. Deep adversarial training for multi-organ nuclei segmentation in histopathology images. IEEE TMI, 39(11): 3257–3267.
  43. Graph Neural Networks Exponentially Lose Expressive Power for Node Classification. In ICLR.
  44. Automatic differentiation in pytorch. NIPS Workshop.
  45. Hierarchical graph representations in digital pathology. MIA, 75: 102264.
  46. Computer-aided classification of breast cancer nuclei. Technology and Health Care, 4(2): 147–161.
  47. A multi-resolution approach for combining visual information using nuclei segmentation and classification in histopathological images. In VISAPP (3), 37–46.
  48. Gland segmentation in colon histology images: The glas challenge contest. MIA, 35: 489–502.
  49. Attention is all you need. NIPS, 30.
  50. Representing long-range context for graph neural networks with global attention. NeurIPS, 34: 13266–13279.
  51. Do transformers really perform badly for graph representation? NeurIPS, 34: 28877–28888.
  52. Diffusion-based data augmentation for nuclei image segmentation. In MICCAI, 592–602. Springer.
  53. DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection. In ICLR.
  54. DeepPap: deep convolutional networks for cervical cell classification. IEEE JBHI, 21(6): 1633–1643.
  55. Predicting lymph node metastasis using histopathological images based on multiple instance learning with deep graph convolution. In CVPR, 4837–4846.
  56. A graph-transformer for whole slide image classification. IEEE TMI, 41(11): 3003–3015.
  57. Cgc-net: Cell graph convolutional network for grading of colorectal cancer histology images. In ICCV Workshops.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now