Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Source Identification: A Self-Supervision Task for Dense Prediction (2307.02238v1)

Published 5 Jul 2023 in cs.CV

Abstract: The paradigm of self-supervision focuses on representation learning from raw data without the need of labor-consuming annotations, which is the main bottleneck of current data-driven methods. Self-supervision tasks are often used to pre-train a neural network with a large amount of unlabeled data and extract generic features of the dataset. The learned model is likely to contain useful information which can be transferred to the downstream main task and improve performance compared to random parameter initialization. In this paper, we propose a new self-supervision task called source identification (SI), which is inspired by the classic blind source separation problem. Synthetic images are generated by fusing multiple source images and the network's task is to reconstruct the original images, given the fused images. A proper understanding of the image content is required to successfully solve the task. We validate our method on two medical image segmentation tasks: brain tumor segmentation and white matter hyperintensities segmentation. The results show that the proposed SI task outperforms traditional self-supervision tasks for dense predictions including inpainting, pixel shuffling, intensity shift, and super-resolution. Among variations of the SI task fusing images of different types, fusing images from different patients performs best.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. A new learning algorithm for blind signal separation, in: Advances in neural information processing systems, Morgan Kaufmann Publishers. pp. 757–763.
  2. Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Scientific data 4, 170117.
  3. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv preprint arXiv:1811.02629 .
  4. An unsupervised learning model for deformable medical image registration, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9252–9260.
  5. A self-organizing neural network that discovers surfaces in random-dot stereograms. Nature 355, 161–163.
  6. An information-maximization approach to blind separation and blind deconvolution. Neural computation 7, 1129–1159.
  7. Image inpainting, in: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press/Addison-Wesley Publishing Co., USA. p. 417–424. URL: https://doi.org/10.1145/344779.344972, doi:10.1145/344779.344972.
  8. Contrastive learning of global and local features for medical image segmentation with limited annotations. arXiv preprint arXiv:2006.10511 .
  9. Monoaural audio source separation using deep convolutional neural networks, in: International conference on latent variable analysis and signal separation, Springer. pp. 258–266.
  10. Self-supervised learning for medical image analysis using image context restoration. Medical Image Analysis 58, 101539.
  11. A simple framework for contrastive learning of visual representations. URL: https://arxiv.org/abs/2002.05709.
  12. Blind source separation and independent component analysis: A review. Neural Information Processing-Letters and Reviews 6, 1–57.
  13. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 .
  14. Unsupervised visual representation learning by context prediction, in: International Conference on Computer Vision (ICCV).
  15. Integration of neural networks and probabilistic spatial models for acoustic blind source separation. IEEE Journal of Selected Topics in Signal Processing 13, 815–826.
  16. Deep attractor networks for speaker re-identification and blind source separation, in: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE. pp. 11–15.
  17. Improving data augmentation for medical image segmentation. Medical Imaging with Deep Learning URL: https://openreview.net/forum?id=rkBBChjiG.
  18. Nuc2vec: Learning representations of nuclei in histopathology images with contrastive loss, in: Medical Imaging with Deep Learning.
  19. Deep residual learning for image recognition. arXiv:1512.03385.
  20. Deep clustering: Discriminative embeddings for segmentation and separation, in: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE. pp. 31--35.
  21. Blind source separation by nonstationarity of variance: A cumulant-based approach. IEEE transactions on neural networks 12, 1471--1474.
  22. nnu-net: Self-adapting framework for u-net-based medical image segmentation. CoRR abs/1809.10486. URL: http://arxiv.org/abs/1809.10486, arXiv:1809.10486.
  23. A local learning rule for independent component analysis. Scientific reports 6, 1--17.
  24. Self-supervised contrastive video-speech representation learning for ultrasound, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 534--543.
  25. Self-supervised visual feature learning with deep neural networks: A survey. CoRR abs/1902.06162. URL: http://dblp.uni-trier.de/db/journals/corr/corr1902.html#abs-1902-06162.
  26. Region-of-interest guided supervoxel inpainting for self-supervision, in: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (Eds.), Medical Image Computing and Computer Assisted Intervention - MICCAI 2020 - 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part I, Springer. pp. 500--509. URL: https://doi.org/10.1007/978-3-030-59710-8_49, doi:10.1007/978-3-030-59710-8_49.
  27. Skip-thought vectors. arXiv preprint arXiv:1506.06726 .
  28. Standardized assessment of automatic segmentation of white matter hyperintensities; results of the wmh segmentation challenge. IEEE transactions on medical imaging .
  29. Colorization as a proxy task for visual understanding, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, IEEE Computer Society. pp. 840--849. URL: https://doi.org/10.1109/CVPR.2017.96, doi:10.1109/CVPR.2017.96.
  30. Photo-realistic single image super-resolution using a generative adversarial network, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105--114. doi:10.1109/CVPR.2017.19.
  31. Contrastive rendering for ultrasound image segmentation, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 563--572.
  32. Multi-task contrastive learning for automatic ct and x-ray diagnosis of covid-19. Pattern Recognition 114, 107848.
  33. Dense depth estimation in monocular endoscopy with self-supervised learning methods. IEEE transactions on medical imaging 39, 1438--1447.
  34. The multimodal brain tumor image segmentation benchmark (brats). IEEE transactions on medical imaging 34, 1993--2024.
  35. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 .
  36. Mathematics for Computer Graphics Applications: An Introduction to the Mathematics and Geometry of CAD/Cam, Geometric Modeling, Scientific Visualizati. 2nd ed., Industrial Press, Inc., USA.
  37. Unsupervised learning of visual representations by solving jigsaw puzzles, in: ECCV.
  38. Context encoders: Feature learning by inpainting, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 2536--2544.
  39. U-net: Convolutional networks for biomedical image segmentation, in: International Conference on Medical image computing and computer-assisted intervention, Springer. pp. 234--241.
  40. Self-supervised learning methods and applications in medical imaging analysis: A survey. arXiv:2109.08685.
  41. When does self-supervision improve few-shot learning?, in: European Conference on Computer Vision, Springer. pp. 645--666.
  42. Deep learning on image denoising: An overview. Neural Networks .
  43. Automated lesion detection by regressing intensity-based distance with a neural network, in: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.T., Khan, A. (Eds.), Medical Image Computing and Computer Assisted Intervention -- MICCAI 2019, Springer International Publishing, Cham. pp. 234--242.
  44. When does self-supervision help graph convolutional networks?, in: International Conference on Machine Learning, PMLR. pp. 10871--10880.
  45. mixup: Beyond empirical risk minimization. International Conference on Learning Representations URL: https://openreview.net/forum?id=r1Ddp1-Rb.
  46. Smore: A self-supervised anti-aliasing and super-resolution algorithm for mri using deep learning. IEEE transactions on medical imaging .
  47. Models genesis: Generic autodidactic models for 3d medical image analysis, in: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.T., Khan, A. (Eds.), Medical Image Computing and Computer Assisted Intervention -- MICCAI 2019, Springer International Publishing, Cham. pp. 384--393.
  48. Self-supervised feature learning for 3d medical images by playing a rubik’s cube, in: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P., Khan, A.R. (Eds.), Medical Image Computing and Computer Assisted Intervention - MICCAI 2019 - 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part IV, Springer. pp. 420--428. URL: https://doi.org/10.1007/978-3-030-32251-9_46, doi:10.1007/978-3-030-32251-9_46.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now