Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Exploring Image Augmentations for Siamese Representation Learning with Chest X-Rays (2301.12636v2)

Published 30 Jan 2023 in eess.IV, cs.AI, cs.CV, and cs.LG

Abstract: Image augmentations are quintessential for effective visual representation learning across self-supervised learning techniques. While augmentation strategies for natural imaging have been studied extensively, medical images are vastly different from their natural counterparts. Thus, it is unknown whether common augmentation strategies employed in Siamese representation learning generalize to medical images and to what extent. To address this challenge, in this study, we systematically assess the effect of various augmentations on the quality and robustness of the learned representations. We train and evaluate Siamese Networks for abnormality detection on chest X-Rays across three large datasets (MIMIC-CXR, CheXpert and VinDR-CXR). We investigate the efficacy of the learned representations through experiments involving linear probing, fine-tuning, zero-shot transfer, and data efficiency. Finally, we identify a set of augmentations that yield robust representations that generalize well to both out-of-distribution data and diseases, while outperforming supervised baselines using just zero-shot transfer and linear probes by up to 20%. Our code is available at https://github.com/StanfordMIMI/siaug.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Big self-supervised models advance medical image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3478–3488, 2021.
  2. Robust and efficient medical imaging with self-supervision. arXiv preprint arXiv:2205.09723, 2022.
  3. Signature verification using a “siamese” time delay neural network. Advances in Neural Information Processing Systems, 6, 1993.
  4. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9650–9660, 2021.
  5. Contrastive learning of global and local features for medical image segmentation with limited annotations. Advances in Neural Information Processing Systems, 33:12546–12558, 2020.
  6. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pages 1597–1607. PMLR, 2020a.
  7. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15750–15758, 2021.
  8. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020b.
  9. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv:1805.09501, 2018.
  10. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition workshops, pages 702–703, 2020.
  11. ViLMedic: a framework for research at the intersection of vision and language in medical AI. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 23–34, Dublin, Ireland, May 2022. Association for Computational Linguistics.
  12. Vortex: Physics-driven data augmentations for consistency training for robust accelerated mri reconstruction. Medical Imaging with Deep Learning, 2022.
  13. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552, 2017.
  14. Self-supervised deep convolutional neural network for chest x-ray classification. IEEE Access, 9:151972–151982, 2021.
  15. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728, 2018.
  16. Bootstrap your own latent-a new approach to self-supervised learning. Advances in Neural Information Processing Systems, 33:21271–21284, 2020.
  17. Dimensionality reduction by learning an invariant mapping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, volume 2, pages 1735–1742. IEEE, 2006.
  18. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, June 2016. 10.1109/cvpr.2016.90.
  19. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729–9738, 2020.
  20. Using self-supervised learning can improve model robustness and uncertainty. Advances in Neural Information Processing Systems, 32, 2019.
  21. Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3942–3951, 2021.
  22. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33-01, pages 590–597, 2019.
  23. Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs. arXiv preprint arXiv:1901.07042, 2019.
  24. Did you get what you paid for? rethinking annotation cost of deep learning based computer aided detection in chest radiographs. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 261–270. Springer, 2022.
  25. Self-supervised learning in medicine and healthcare. Nature Biomedical Engineering, pages 1–7, 2022.
  26. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90, 2017.
  27. Vindr-cxr: An open dataset of chest x-rays with radiologist’s annotations. Scientific Data, 9(1):1–7, 2022.
  28. Tormentor: Deterministic dynamic-path, data augmentations with fractals. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2707–2711, 2022.
  29. Unsupervised learning of visual representations by solving jigsaw puzzles. In European conference on computer vision, pages 69–84. Springer, 2016.
  30. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2536–2544, 2016.
  31. Transfusion: Understanding transfer learning for medical imaging. Advances in Neural Information Processing Systems, 32, 2019.
  32. Kornia: an open source differentiable computer vision library for pytorch. In Winter Conference on Applications of Computer Vision, 2020.
  33. Moco pretraining improves representation and transferability of chest x-ray models. In Medical Imaging with Deep Learning, pages 728–744. PMLR, 2021.
  34. Multimodal self-supervised learning for medical image analysis. In International Conference on Information Processing in Medical Imaging, pages 661–673. Springer, 2021.
  35. What makes for good views for contrastive learning? In Advances in Neural Information Processing Systems, NIPS’20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
  36. Expert-level detection of pathologies from unannotated chest x-ray images via self-supervised learning. Nature Biomedical Engineering, 6(12):1399–1406, 2022.
  37. Mining multi-label data. Data Mining and Knowledge Discovery Handbook, 2010. 10.1007/978-0-387-09823-4_34.
  38. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ, 369, 2020.
  39. Colorful image colorization. In European Conference on Computer Vision, pages 649–666. Springer, 2016.
  40. Contrastive learning of medical visual representations from paired images and text. arXiv preprint arXiv:2010.00747, 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Nandita Bhaskhar (5 papers)
  2. Daniel Rubin (32 papers)
  3. Curtis Langlotz (24 papers)
  4. Akshay Chaudhari (34 papers)
  5. Rogier Van der Sluijs (4 papers)
Citations (12)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now