Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Compositionally Equivariant Representation Learning (2306.07783v2)

Published 13 Jun 2023 in cs.CV and cs.LG

Abstract: Deep learning models often need sufficient supervision (i.e. labelled data) in order to be trained effectively. By contrast, humans can swiftly learn to identify important anatomy in medical images like MRI and CT scans, with minimal guidance. This recognition capability easily generalises to new images from different medical facilities and to new tasks in different settings. This rapid and generalisable learning ability is largely due to the compositional structure of image patterns in the human brain, which are not well represented in current medical models. In this paper, we study the utilisation of compositionality in learning more interpretable and generalisable representations for medical image segmentation. Overall, we propose that the underlying generative factors that are used to generate the medical images satisfy compositional equivariance property, where each factor is compositional (e.g. corresponds to the structures in human anatomy) and also equivariant to the task. Hence, a good representation that approximates well the ground truth factor has to be compositionally equivariant. By modelling the compositional representations with learnable von-Mises-Fisher (vMF) kernels, we explore how different design and learning biases can be used to enforce the representations to be more compositionally equivariant under un-, weakly-, and semi-supervised settings. Extensive results show that our methods achieve the best performance over several strong baselines on the task of semi-supervised domain-generalised medical image segmentation. Code will be made publicly available upon acceptance at https://github.com/vios-s.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. O. Bernard, A. Lalande, C. Zotti, and et al., “Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved?” TMI, vol. 37, no. 11, pp. 2514–2525, 2018.
  2. P. Tokmakov, Y.-X. Wang, and M. Hebert, “Learning compositional representations for few-shot recognition,” in CVPR, 2019, pp. 6372–6381.
  3. N. Liu, S. Li, Y. Du, J. Tenenbaum, and A. Torralba, “Learning to compose visual relations,” in NeurIPS, vol. 34, 2021.
  4. D. Huynh and E. Elhamifar, “Compositional zero-shot learning via fine-grained dense feature composition,” in NeurIPS, vol. 33, 2020, pp. 19 849–19 860.
  5. A. Kortylewski, J. He, Q. Liu, and A. L. Yuille, “Compositional convolutional neural networks: A deep architecture with innate robustness to partial occlusion,” in CVPR, 2020, pp. 8940–8949.
  6. X. Liu, S. Thermos, A. O’Neil, and S. A. Tsaftaris, “Semi-supervised meta-learning with disentanglement for domain-generalised medical image segmentation,” in MICCAI.   Springer, 2021, pp. 307–317.
  7. X. Liu, S. Thermos, P. Sanchez, A. Q. O’Neil, and S. A. Tsaftaris, “vMFNet: Compositionality meets domain-generalised segmentation,” in MICCAI.   Springer, 2022, pp. 704–714.
  8. D. Li, Y. Yang, Y.-Z. Song, and T. Hospedales, “Learning to generalise: Meta-learning for domain generalisation,” in AAAI, 2018.
  9. V. M. Campello et al., “Multi-centre, multi-vendor and multi-disease cardiac segmentation: The M&Ms challenge,” TMI, 2021.
  10. F. Prados et al., “Spinal cord grey matter segmentation challenge,” NeuroImage, vol. 152, pp. 312–329, 2017.
  11. J. Tubiana and R. Monasson, “Emergence of compositional representations in restricted boltzmann machines,” Physical review letters, vol. 118, no. 13, p. 138301, 2017.
  12. D. Arad Hudson and L. Zitnick, “Compositional transformers for scene generation,” in NeurIPS, 2021.
  13. X. Yuan, A. Kortylewski et al., “Robust instance segmentation through reasoning about multi-object occlusion,” in CVPR, 2021, pp. 11 141–11 150.
  14. Y. Zhang, A. Kortylewski, Q. Liu et al., “A light-weight interpretable compositionalnetwork for nuclei detection and weakly-supervised segmentation,” arXiv:2110.13846, 2021.
  15. L. Zhang, X. Wang, D. Yang, T. Sanford et al., “Generalising deep learning for medical image segmentation to unseen domains via deep stacked transformation,” TMI, vol. 39, no. 7, pp. 2531–2540, 2020.
  16. C. Chen, K. Hammernik, C. Ouyang, C. Qin, W. Bai, and D. Rueckert, “Cooperative training and latent space data augmentation for robust medical image segmentation,” in MICCAI.   Springer, 2021, pp. 149–159.
  17. F. M. Carlucci, A. D’Innocente, S. Bucci, B. Caputo, and T. Tommasi, “Domain generalisation by solving jigsaw puzzles,” in CVPR, 2019, pp. 2229–2238.
  18. J. Huang, D. Guan, A. Xiao, and S. Lu, “FSDR: Frequency space domain randomization for domain generalization,” in CVPR, 2021, pp. 6891–6902.
  19. H. Li, Y. Wang, R. Wan, S. Wang et al., “Domain generalisation for medical imaging classification with linear-dependency regularization,” in NeurIPS, 2020.
  20. R. Gu, J. Zhang, R. Huang, W. Lei, G. Wang, and S. Zhang, “Domain composition and attention for unseen-domain generalizable medical image segmentation,” in MICCAI.   Springer, 2021, pp. 241–250.
  21. Q. Dou, D. C. Castro, K. Kamnitsas, and B. Glocker, “Domain generalisation via model-agnostic learning of semantic features,” in NeurIPS, 2019.
  22. Q. Liu, Q. Dou, and P.-A. Heng, “Shape-aware meta-learning for generalising prostate mri segmentation to unseen domains,” in MICCAI.   Springer, 2020, pp. 475–485.
  23. Q. Liu, C. Chen, J. Qin, Q. Dou, and P.-A. Heng, “FedDG: Federated domain generalization on medical image segmentation via episodic learning in continuous frequency space,” in CVPR, 2021, pp. 1013–1023.
  24. H. Yao, X. Hu, and X. Li, “Enhancing pseudo label quality for semi-supervised domain-generalized medical image segmentation,” in AAAI, 2022.
  25. Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” TPAMI, vol. 35, no. 8, pp. 1798–1828, 2013.
  26. B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio, “Toward causal representation learning,” Proceedings of the IEEE, vol. 109, no. 5, 2021.
  27. P. W. Koh, T. Nguyen, Y. S. Tang, S. Mussmann, E. Pierson, B. Kim, and P. Liang, “Concept bottleneck models,” in ICML.   PMLR, 2020, pp. 5338–5348.
  28. A. Stone, H. Wang, M. Stark, Y. Liu, D. Scott Phoenix, and D. George, “Teaching compositionality to CNNs,” in CVPR, 2017, pp. 5058–5067.
  29. X. Liu, P. Sanchez, S. Thermos, A. Q. O’Neil, and S. A. Tsaftaris, “Learning disentangled representations in the imaging domain,” Medical Image Analysis, p. 102516, 2022.
  30. W. Stammer, M. Memmel, P. Schramowski, and K. Kersting, “Interactive disentanglement: Learning concepts by interacting with their prototype representations,” in CVPR, 2022, pp. 10 317–10 328.
  31. F. Locatello et al., “Weakly-supervised disentanglement without compromises,” in ICML.   PMLR, 2020, pp. 6348–6359.
  32. I. Higgins, D. Amos, D. Pfau, S. Racaniere, L. Matthey, D. Rezende, and A. Lerchner, “Towards a definition of disentangled representations,” arXiv:1812.02230., 2018.
  33. T. Wang, Z. Yue, J. Huang, Q. Sun, and H. Zhang, “Self-supervised learning disentangled group representation as feature,” in NeurIPS, vol. 34, 2021, pp. 18 225–18 240.
  34. A. Achille and S. Soatto, “Emergence of invariance and disentanglement in deep representations,” JMLR, vol. 19, pp. 1–34, 2017.
  35. A. Chartsias, T. Joyce, G. Papanastasiou, S. Semple, M. Williams, D. E. Newby, R. Dharmakumar, and S. A. Tsaftaris, “Disentangled representation learning in cardiac image analysis,” Medical Image Analysis, vol. 58, 2019.
  36. X. Liu, S. Thermos, G. Valvano, A. Chartsias, A. O’Neil, and S. A. Tsaftaris, “Measuring the biases and effectiveness of content-style disentanglement,” in BMVC, 2021.
  37. F. Milletari, N. Navab, and S.-A. Ahmadi, “VNet: Fully convolutional neural networks for volumetric medical image segmentation,” in 3DV.   IEEE, 2016, pp. 565–571.
  38. X. Chen et al., “Semi-supervised semantic segmentation with cross pseudo supervision,” in CVPR, 2021, pp. 2613–2622.
  39. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in ICLR, 2015.
  40. O. Ronneberger, P. Fischer, and T. Brox, “UNet: Convolutional networks for biomedical image segmentation,” in MICCAI.   Springer, 2015, pp. 234–241.
  41. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in CVPR, 2016, pp. 770–778.
  42. A. Paszke et al., “Pytorch: An imperative style, high-performance deep learning library,” in NeurIPS, vol. 32, 2019.
  43. L. R. Dice, “Measures of the amount of ecologic association between species,” Ecology, vol. 26, no. 3, pp. 297–302, 1945.
  44. M.-P. Dubuisson and A. K. Jain, “A modified hausdorff distance for object matching,” in ICPR, vol. 1.   IEEE, 1994, pp. 566–568.
  45. F. Isensee, P. F. Jaeger et al., “nnUNet: a self-configuring method for deep learning-based biomedical image segmentation,” Nature Methods, vol. 18, no. 2, pp. 203–211, 2021.
  46. X. Liu, S. Thermos, A. Chartsias, A. O’Neil, and S. A. Tsaftaris, “Disentangled representations for domain-generalized cardiac segmentation,” in STACOM Workshop.   Springer, 2020, pp. 187–195.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Xiao Liu (402 papers)
  2. Pedro Sanchez (20 papers)
  3. Spyridon Thermos (16 papers)
  4. Alison Q. O'Neil (23 papers)
  5. Sotirios A. Tsaftaris (100 papers)
Citations (2)
View on Semantic Scholar