Papers
Topics
Authors
Recent
Search
2000 character limit reached

Scale-Space Hypernetworks for Efficient Biomedical Imaging

Published 11 Apr 2023 in cs.CV and cs.LG | (2304.05448v2)

Abstract: Convolutional Neural Networks (CNNs) are the predominant model used for a variety of medical image analysis tasks. At inference time, these models are computationally intensive, especially with volumetric data. In principle, it is possible to trade accuracy for computational efficiency by manipulating the rescaling factor in the downsample and upsample layers of CNN architectures. However, properly exploring the accuracy-efficiency trade-off is prohibitively expensive with existing models. To address this, we introduce Scale-Space HyperNetworks (SSHN), a method that learns a spectrum of CNNs with varying internal rescaling factors. A single SSHN characterizes an entire Pareto accuracy-efficiency curve of models that match, and occasionally surpass, the outcomes of training many separate networks with fixed rescaling factors. We demonstrate the proposed approach in several medical image analysis applications, comparing SSHN against strategies with both fixed and dynamic rescaling factors. We find that SSHN consistently provides a better accuracy-efficiency trade-off at a fraction of the training cost. Trained SSHNs enable the user to quickly choose a rescaling factor that appropriately balances accuracy and computational efficiency for their particular needs at inference.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. Automatic segmentation of mandible in panoramic x-ray. Journal of Medical Imaging, 2(4):044003, 2015.
  2. Voxelmorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging, 38(8):1788–1800, 2019.
  3. Deep learning, volume 1. MIT press Cambridge, MA, USA, 2017.
  4. A learning strategy for contrast-agnostic mri segmentation. In T. Arbel, I. Ben Ayed, M. de Bruijne, M. Descoteaux, H. Lombaert, and C. Pal, editors, Proceedings of the Third Conference on Medical Imaging with Deep Learning, volume 121 of Proceedings of Machine Learning Research, pages 75–93. PMLR, 06–08 Jul 2020. URL https://proceedings.mlr.press/v121/billot20a.html.
  5. What is the state of neural network pruning? arXiv preprint arXiv:2003.03033, 2020.
  6. Smash: one-shot model architecture search through hypernetworks. arXiv preprint arXiv:1708.05344, 2017.
  7. Principled weight initialization for hypernetworks. In International Conference on Learning Representations, 2019.
  8. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587, 2017.
  9. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), pages 801–818, 2018.
  10. F. Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1251–1258, 2017.
  11. Editing factual knowledge in language models. arXiv preprint arXiv:2104.08164, 2021.
  12. L. R. Dice. Measures of the amount of ecologic association between species. Ecology, 26(3):297–302, 1945.
  13. A. Dosovitskiy and J. Djolonga. You only train once: Loss-conditional training of deep networks. In International conference on learning representations, 2020.
  14. The importance of skip connections in biomedical image segmentation. In International Workshop on Deep Learning in Medical Image Analysis, International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis.
  15. The importance of skip connections in biomedical image segmentation. In Deep learning and data labeling for medical applications, pages 179–187. Springer, 2016.
  16. Feature-wise transformations. Distill, 3(7):e11, 2018.
  17. Neural architecture search: A survey. The Journal of Machine Learning Research, 20(1):1997–2017, 2019.
  18. B. Fischl. Freesurfer. Neuroimage, 62(2):774–781, 2012.
  19. The state of sparsity in deep neural networks, 2019.
  20. B. Graham. Fractional max-pooling. arXiv preprint arXiv:1412.6071, 2014.
  21. Hypernetworks. arXiv preprint arXiv:1609.09106, 2016.
  22. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. In Y. Bengio and Y. LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016. URL http://arxiv.org/abs/1510.00149.
  23. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015.
  24. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  25. Learning the effect of registration hyperparameters with hypermorph. Machine Learning for Biomedical Imaging, 1:1–30, 2022. ISSN 2766-905X. URL https://melba-journal.org/2022:003.
  26. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141, 2018.
  27. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
  28. Binarized neural networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 4114–4122, 2016.
  29. nnu-net: Self-adapting framework for u-net-based medical image segmentation. arXiv preprint arXiv:1809.10486, 2018.
  30. nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2):203–211, 2021a.
  31. nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2):203–211, 2021b.
  32. Image-to-image translation with conditional adversarial networks. CVPR, 2017.
  33. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2704–2713, 2018.
  34. Speeding up convolutional neural networks with low rank expansions. arXiv preprint arXiv:1405.3866, 2014.
  35. S. A. Janowsky. Pruning versus clipping in neural networks. Physical Review A, 39(12):6600–6603, June 1989. ISSN 0556-2791. doi: 10.1103/PhysRevA.39.6600. URL https://link.aps.org/doi/10.1103/PhysRevA.39.6600.
  36. Learning to downsample for segmentation of ultra-high resolution images. arXiv preprint arXiv:2109.11071, 2021.
  37. A survey of the recent architectures of deep convolutional neural networks. Artificial Intelligence Review, 53(8):5455–5516, 2020.
  38. D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  39. Hypernetwork functional image representation. In International Conference on Artificial Neural Networks, pages 496–510. Springer, 2019.
  40. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
  41. Bayesian hypernetworks. arXiv preprint arXiv:1710.04759, 2017.
  42. Stochastic downsampling for cost-adjustable inference and improved regularization in convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7929–7938, 2018.
  43. Deep learning for segmentation using an open large-scale dataset in 2d echocardiography. IEEE transactions on medical imaging, 38(9):2198–2210, 2019.
  44. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117–2125, 2017.
  45. Shape adaptor: A learnable resizing module. In Proceedings of the European Conference on Computer Vision (ECCV), 2020.
  46. Metapruning: Meta learning for automatic neural network channel pruning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3296–3305, 2019.
  47. J. Lorraine and D. Duvenaud. Stochastic hyperparameter optimization through hypernetworks. arXiv preprint arXiv:1802.09419, 2018.
  48. Rectifier nonlinearities improve neural network acoustic models. In Proc. icml, volume 30, page 3. Citeseer, 2013.
  49. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. arXiv preprint arXiv:1903.03088, 2019.
  50. Open access series of imaging studies (oasis): cross-sectional mri data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience, 19(9):1498–1507, 2007.
  51. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pages 565–571. IEEE, 2016.
  52. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999, 2018.
  53. Implicit weight uncertainty in neural networks. arXiv preprint arXiv:1711.01297, 2017.
  54. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  55. On network design spaces for visual recognition. In Proceedings of the IEEE International Conference on Computer Vision, pages 1882–1890, 2019.
  56. Xnor-net: Imagenet classification using binary convolutional neural networks. In European conference on computer vision, pages 525–542. Springer, 2016.
  57. Learning strides in convolutional neural networks. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=M752z9FKJP.
  58. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
  59. Use-net: Incorporating squeeze-and-excitation blocks into u-net for prostate zonal segmentation of multi-institutional mri datasets. Neurocomputing, 365:31–43, 2019.
  60. J. Schmidhuber. A ‘self-referential’weight matrix. In International Conference on Artificial Neural Networks, pages 446–450. Springer, 1993.
  61. Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion. arXiv preprint arXiv:1906.00794, 2019.
  62. U-net and its variants for medical image segmentation: theory and applications. arXiv preprint arXiv:2011.01118, 2020.
  63. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806, 2014.
  64. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: Third International Workshop, DLMIA 2017, and 7th International Workshop, ML-CDS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 14, Proceedings 3, pages 240–248. Springer, 2017.
  65. Efficient processing of deep neural networks: A tutorial and survey. arXiv preprint arXiv:1703.09039, 2017.
  66. M. Tan and Q. V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946, 2019.
  67. Hypernetwork-based implicit posterior estimation and model averaging of cnn. In Asian Conference on Machine Learning, pages 176–191. PMLR, 2018.
  68. Continual learning with hypernetworks. arXiv preprint arXiv:1906.00695, 2019.
  69. Hyperrecon: Regularization-agnostic cs-mri reconstruction with hypernetworks. In Machine Learning for Medical Image Reconstruction: 4th International Workshop, MLMIR 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, October 1, 2021, Proceedings 4, pages 3–13. Springer, 2021.
  70. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492–1500, 2017.
  71. Designing energy-efficient convolutional neural networks using energy-aware pruning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5687–5695, 2017.
  72. Graph hypernetworks for neural architecture search. arXiv preprint arXiv:1810.05749, 2018.
  73. Meta-learning via hypernetworks. 2020.
  74. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2881–2890, 2017a.
  75. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2881–2890, 2017b.
  76. Fast and robust segmentation of white blood cell images by self-supervised learning. Micron, 107:55–71, 2018. doi: https://doi.org/10.1016/j.micron.2018.01.010. URL https://www.sciencedirect.com/science/article/pii/S0968432817303037.
  77. Squeeze-and-attention networks for semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13065–13074, 2020.
  78. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8697–8710, 2018.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up