Insight Gained from Migrating a Machine Learning Model to Intelligence Processing Units (2404.10730v1)
Abstract: The discoveries in this paper show that Intelligence Processing Units (IPUs) offer a viable accelerator alternative to GPUs for ML applications within the fields of materials science and battery research. We investigate the process of migrating a model from GPU to IPU and explore several optimization techniques, including pipelining and gradient accumulation, aimed at enhancing the performance of IPU-based models. Furthermore, we have effectively migrated a specialized model to the IPU platform. This model is employed for predicting effective conductivity, a parameter crucial in ion transport processes, which govern the performance of multiple charge and discharge cycles of batteries. The model utilizes a Convolutional Neural Network (CNN) architecture to perform prediction tasks for effective conductivity. The performance of this model on the IPU is found to be comparable to its execution on GPUs. We also analyze the utilization and performance of Graphcore's Bow IPU. Through benchmark tests, we observe significantly improved performance with the Bow IPU when compared to its predecessor, the Colossus IPU.
- Benchmarking the performance of accelerators on national cyberinfrastructure resources for artificial intelligence/machine learning workloads. In Practice and Experience in Advanced Research Computing, pages 1–9. 2022.
- Porting ai/ml models to intelligence processing units (ipus). In Practice and Experience in Advanced Research Computing, pages 231–236. 2023.
- Learning matrices and their applications. IEEE Transactions on Electronic Computers, (6):846–862, 1963.
- Graphcore documents. Graphcore documents, 2024.
- A quantitative study of irregular programs on gpus. In 2012 IEEE International Symposium on Workload Characterization (IISWC), pages 141–151. IEEE, 2012.
- Using gpus for machine learning algorithms. In Eighth International Conference on Document Analysis and Recognition (ICDAR’05), pages 1115–1120. IEEE, 2005.
- Dissecting the nvidia volta gpu architecture via microbenchmarking. arXiv preprint arXiv:1804.06826, 2018.
- Accelerated materials design of lithium superionic conductors based on first-principles calculations and machine learning algorithms. Advanced Energy Materials, 3(8):980–985, 2013.
- Nathan Baker. Unlocking a new era for scientific discovery with ai: How microsoft’s ai screened over 32 million candidates to find a better battery, 2024.
- Predicting ionic conductivity of solid-state battery cathodes using machine learning, 2024.
- Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
- Challenges in speeding up solid-state battery development. Nature Energy, 8(3):230–240, 2023.
- Predicting ionic conductivity of solid-state battery cathodes using machine learning, in preparation, 2024.
- NVIDIA. A100 40gb pcie product brief, 2020.
- TAMU HPRC. Tamu hprc wiki, 2024.