Neural Architecture Codesign for Fast Bragg Peak Analysis (2312.05978v2)
Abstract: We develop an automated pipeline to streamline neural architecture codesign for fast, real-time Bragg peak analysis in high-energy diffraction microscopy. Traditional approaches, notably pseudo-Voigt fitting, demand significant computational resources, prompting interest in deep learning models for more efficient solutions. Our method employs neural architecture search and AutoML to enhance these models, including hardware costs, leading to the discovery of more hardware-efficient neural architectures. Our results match the performance, while achieving a 13$\times$ reduction in bit operations compared to the previous state-of-the-art. We show further speedup through model compression techniques such as quantization-aware-training and neural network pruning. Additionally, our hierarchical search space provides greater flexibility in optimization, which can easily extend to other tasks and domains.
- Aad, G.; et al. 2020. Operation of the ATLAS trigger system in Run 2. JINST, 15(10): P10004.
- Zero-Cost Proxies for Lightweight NAS. In International Conference on Learning Representations.
- Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2623–2631.
- UNIQ: Uniform Noise Injection for Non-Uniform Quantization of Neural Networks. ACM Trans. Comput. Syst., 37(1–4).
- Algorithms for Hyper-Parameter Optimization. In Shawe-Taylor, J.; Zemel, R.; Bartlett, P.; Pereira, F.; and Weinberger, K., eds., Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc.
- Significance of diffraction peak shapes in determining crystallite size distribution: a peak shape analysis procedure for pseudo-Voigt profiles and its application. Journal of Applied Crystallography, 56(5).
- What is the state of neural network pruning? Proceedings of machine learning and systems, 2: 129–146.
- Once for All: Train One Network and Specialize it for Efficient Deployment. In International Conference on Learning Representations.
- Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors. Nat. Mach. Intell., 3: 675.
- A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 6(2): 182.
- FastML Science Benchmarks: Accelerating Real-Time Scientific Edge Machine Learning. In 3rd Workshop on Benchmarking Machine Learning Workloads on Emerging Hardware (MLBench) at 5th Conference on Machine Learning and Systems (MLSys).
- Duarte, J.; et al. 2018. Fast inference of deep neural networks in FPGAs for particle physics. JINST, 13(07): P07027.
- Neural architecture search: A survey. The Journal of Machine Learning Research, 20(1): 1997–2017.
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. In Bengio, Y.; and LeCun, Y., eds., International Conference on Learning Representations.
- OpenHLS: High-Level Synthesis for Low-Latency Deep Neural Networks for Experimental Science.
- Bridging Data Center AI Systems with Edge Computing for Actionable Information Retrieval. In 2021 3rd Annual Workshop on Extreme-scale Experiment-in-the-Loop Computing (XLOOP), 15–23. IEEE.
- BraggNN: fast X-ray Bragg peak analysis using deep learning. IUCrJ, 9(1).
- Pappalardo, A. 2023. Xilinx/brevitas.
- Far-field high-energy diffraction microscopy: a non-destructive tool for characterizing the microstructure and micromechanical state of polycrystalline materials. Microscopy Today, 25(5): 36–45.
- A comprehensive survey of neural architecture search: Challenges and solutions. ACM Computing Surveys (CSUR), 54(4): 1–34.
- Fiducial marker application method for position alignment of in situ multimodal X-ray experiments and reconstructions. Journal of Applied Crystallography, 49(2): 700–704.
- Sirunyan, A. M.; et al. 2020. Performance of the CMS Level-1 trigger in proton-proton collisions at s=13𝑠13\sqrt{s}=13square-root start_ARG italic_s end_ARG = 13 TeV. JINST, 15(10): P10017.
- Weng, O. 2021. Neural network quantization for efficient inference: A survey. arXiv preprint arXiv:2112.06126.
- Learning best combination for efficient n: M sparsity. Advances in Neural Information Processing Systems, 35: 941–953.