Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
143 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Ultrafast jet classification on FPGAs for the HL-LHC (2402.01876v2)

Published 2 Feb 2024 in hep-ex, cs.LG, and physics.ins-det

Abstract: Three machine learning models are used to perform jet origin classification. These models are optimized for deployment on a field-programmable gate array device. In this context, we demonstrate how latency and resource consumption scale with the input size and choice of algorithm. Moreover, the models proposed here are designed to work on the type of data and under the foreseen conditions at the CERN LHC during its high-luminosity phase. Through quantization-aware training and efficient synthetization for a specific field programmable gate array, we show that $O(100)$ ns inference of complex architectures such as Deep Sets and Interaction Networks is feasible at a relatively low computational resource cost.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. ATLAS Collaboration, “The ATLAS Experiment at the CERN Large Hadron Collider”, JINST 3 (2008) S08003, doi:10.1088/1748-0221/3/08/S08003.
  2. D. Contardo et al., “Technical Proposal for the Phase-II Upgrade of the CMS Detector”, CMS Technical Proposal, 2015. doi:10.17181/CERN.VU8I.D59J.
  3. CMS Collaboration, “The Phase-2 Upgrade of the CMS Level-1 Trigger”, CMS Technical Design Report, 2020.
  4. CMS Collaboration, “Particle-flow reconstruction and global event description with the CMS detector”, JINST 12 (2017) P10003, doi:10.1088/1748-0221/12/10/P10003, arXiv:1706.04965.
  5. M. Cacciari, G. P. Salam, and G. Soyez, “The anti-kTsubscript𝑘Tk_{\mathrm{T}}italic_k start_POSTSUBSCRIPT roman_T end_POSTSUBSCRIPT jet clustering algorithm”, JHEP 04 (2008) 063, doi:10.1088/1126-6708/2008/04/063, arXiv:0802.1189.
  6. M. Cacciari, G. P. Salam, and G. Soyez, “FastJet User Manual”, Eur. Phys. J. C 72 (2012) 1896, doi:10.1140/epjc/s10052-012-1896-2, arXiv:1111.6097.
  7. E. A. Moreno et al., “JEDI-net: a jet identification algorithm based on interaction networks”, Eur. Phys. J. C 80 (2020) 58, doi:10.1140/epjc/s10052-020-7608-4, arXiv:1908.05318.
  8. H. Qu and L. Gouskos, “ParticleNet: Jet Tagging via Particle Clouds”, Phys. Rev. D 101 (2020), no. 5, 056019, doi:10.1103/PhysRevD.101.056019, arXiv:1902.08570.
  9. D. Guest et al., “Jet Flavor Classification in High-Energy Physics with Deep Neural Networks”, Phys. Rev. D 94 (2016), no. 11, 112002, doi:10.1103/PhysRevD.94.112002, arXiv:1607.08633.
  10. G. Kasieczka et al., “The Machine Learning landscape of top taggers”, SciPost Phys. 7 (2019) 014, doi:10.21468/SciPostPhys.7.1.014, arXiv:1902.09914.
  11. J. Duarte et al., “Fast inference of deep neural networks in FPGAs for particle physics”, JINST 13 (2018), no. 07, doi:10.1088/1748-0221/13/07/P07027, arXiv:1804.06913.
  12. Fast Machine Learning Lab Collaboration, “hls4ml”, 2018. https://fastmachinelearning.org/hls4ml/.
  13. M. Courbariaux, Y. Bengio, and J.-P. David, “BinaryConnect: Training deep neural networks with binary weights during propagations”, in Advances in Neural Information Processing Systems, C. Cortes et al., eds., volume 28, p. 3123. Curran Associates, Inc., 2015. arXiv:1511.00363.
  14. S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding”, in 4th International Conference on Learning Representations, San Juan, Puerto Rico, May 2, 2016, Y. Bengio and Y. LeCun, eds. 2016. arXiv:1510.00149.
  15. C. Coelho, “QKeras”, 2019. https://github.com/google/qkeras.
  16. P. T. Komiske, E. M. Metodiev, and J. Thaler, “Energy Flow Networks: Deep Sets for Particle Jets”, JHEP 01 (2019) 121, doi:10.1007/JHEP01(2019)121, arXiv:1810.05165.
  17. H. Qu, C. Li, and S. Qian, “Particle Transformer for Jet Tagging”, in Proceedings of the 39th International Conference on Machine Learning, K. Chaudhuri et al., eds., volume 162, p. 18281. 2022. arXiv:2202.03772.
  18. Y. Iiyama et al., “Distance-Weighted Graph Neural Networks on FPGAs for Real-Time Particle Reconstruction in High Energy Physics”, Front. Big Data 3 (2020) 598927, doi:10.3389/fdata.2020.598927, arXiv:2008.03601.
  19. A. Bogatskiy, T. Hoffman, D. W. Miller, and J. T. Offermann, “PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics”, arXiv:2211.00454.
  20. A. Bogatskiy et al., “Explainable Equivariant Neural Networks for Particle Physics: PELICAN”, arXiv:2307.16506.
  21. S. Gong et al., “An efficient Lorentz equivariant graph neural network for jet tagging”, JHEP 07 (2022) 030, doi:10.1007/JHEP07(2022)030, arXiv:2201.08187.
  22. M. Zaheer et al., “Deep sets”, in Advances in Neural Information Processing Systems, I. Guyon et al., eds., volume 30. Curran Associates, Inc., 2017. arXiv:1703.06114.
  23. P. W. Battaglia et al., “Relational inductive biases, deep learning, and graph networks”, 2018. arXiv:1806.01261.
  24. M. M. Bronstein, J. Bruna, T. Cohen, and P. Velvčković, “Geometric deep learning: Grids, groups, graphs, geodesics, and gauges”, 2021.
  25. J. Zhou et al., “Graph neural networks: A review of methods and applications”, AI Open 1 (2021) 57, doi:10.1016/j.aiopen.2021.01.001, arXiv:1812.08434.
  26. J. Shlomi, P. Battaglia, and J.-R. Vlimant, “Graph neural networks in particle physics”, Mach. Learn. Sci. Tech. 2 (2021) 021001, doi:10.1088/2632-2153/abbf9a, arXiv:2007.13681.
  27. K. Guo et al., “A survey of FPGA-based neural network inference accelerators”, ACM Trans. Reconfigurable Technol. Syst. 12l (2018) doi:10.1145/3289185.
  28. S. I. Venieris, A. Kouris, and C. Bouganis, “Toolflows for mapping convolutional neural networks on fpgas: A survey and future directions”, ACM Comput. Surv. 51 (2018), no. 3, doi:10.1145/3186332, arXiv:1803.05900.
  29. S. Summers et al., “Fast inference of boosted decision trees in FPGAs for particle physics”, JINST 15 (2020), no. 05, P05026, doi:10.1088/1748-0221/15/05/p05026, arXiv:2002.02534.
  30. T. M. Hong et al., “Nanosecond machine learning event classification with boosted decision trees in FPGA for high energy physics”, JINST 16 (2021) P08016, doi:10.1088/1748-0221/16/08/P08016, arXiv:2104.03408.
  31. B. Carlson, Q. Bayer, T. M. Hong, and S. Roche, “Nanosecond machine learning regression with deep boosted decision trees in FPGA for high energy physics”, JINST 17 (2022) P09039, doi:10.1088/1748-0221/17/09/P09039, arXiv:2207.05602.
  32. S. Roche et al., “Nanosecond anomaly detection with decision trees for high energy physics and real-time application to exotic Higgs decays”, arXiv:2304.03836.
  33. Z. Que et al., “LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physics”, ACM Trans. Embed. Comput. Syst. (2024) doi:10.1145/3640464.
  34. B. Bhattacherjee, P. Konar, V. S. Ngairangbam, and P. Solanki, “LLPNet: Graph Autoencoder for Triggering Light Long-Lived Particles at HL-LHC”, arXiv:2308.13611.
  35. J. M. Duarte et al., “hls4ml LHC jet dataset (150 particles)”, 2020. doi:10.5281/zenodo.3602260.
  36. M. Abadi et al., “Tensorflow: Large-scale machine learning on heterogeneous systems”, 2015. Software available from tensorflow.org. https://www.tensorflow.org/.
  37. F. Chollet et al., “Keras”, 2015. Software available from tensorflow.org. https://github.com/fchollet/keras.
  38. P. W. Battaglia et al., “Interaction networks for learning about objects, relations and physics”, in Advances in Neural Information Processing Systems, D. Lee et al., eds., volume 29. Curran Associates, Inc., 2016. arXiv:1612.00222.
  39. V. Nair and G. E. Hinton, “Rectified linear units improve restricted Boltzmann machines”, in Proc. of the 27th Int. Conf. on Machine Learning (ICML), p. 807. 2010.
  40. X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural networks”, in Proc. of the 14th Int. Conf. on Artificial Intelligence and Statistics (AISTATS), G. Gordon, D. Dunson, and M. Dudík, eds., volume 15, p. 315. 2011.
  41. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization”, 2017.
  42. M. Zhu and S. Gupta, “To prune, or not to prune: exploring the efficacy of pruning for model compression”, in 6th International Conference on Learning Representations, Workshop Track Proceedings. 2018. arXiv:1710.01878.
  43. C. N. Coelho et al., “Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors”, Nature Mach. Intell. 3 (2021), no. 8, 675, doi:10.1038/s42256-021-00356-5, arXiv:2006.10159.
  44. Z. Que, A. Sznajder, J. Duarte, and P. Odagiu, “l1-jet-id”, 2024. doi:10.5281/zenodo.10553804, https://github.com/fastmachinelearning/l1-jet-id.
  45. Y. LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage”, in Advances in Neural Information Processing Systems, D. S. Touretzky, ed., volume 2, p. 598. Morgan-Kaufmann, 1990.
  46. J. Frankle and M. Carbin, “The lottery ticket hypothesis: Finding sparse, trainable neural networks”, in 7th International Conference on Learning Representations. 2019. arXiv:1803.03635.
  47. A. Renda, J. Frankle, and M. Carbin, “Comparing rewinding and fine-tuning in neural network pruning”, in 8th International Conference on Learning Representations, Addis Ababa, Ethiopia, April 26, 2020. 2020. arXiv:2003.02389. https://openreview.net/forum?id=S1gSj0NKvB.
  48. H. Zhou, J. Lan, R. Liu, and J. Yosinski, “Deconstructing lottery tickets: Zeros, signs, and the supermask”, in Advances in Neural Information Processing Systems, H. Wallach et al., eds., volume 32, p. 3597. Curran Associates, Inc., 2019. arXiv:1905.01067.
  49. D. Blalock, J. J. G. Ortiz, J. Frankle, and J. Guttag, “What is the state of neural network pruning?”, in Proceedings of Machine Learning and Systems, I. Dhillon, D. Papailiopoulos, and V. Sze, eds., volume 2, p. 129. 2020. arXiv:2003.03033.
Citations (1)
View on Semantic Scholar

Summary

  • The paper introduces advanced FPGA implementations for ML-based jet classification using quantization-aware training.
  • It demonstrates architectures like deep sets and interaction networks achieving inference within 100 ns while meeting strict resource constraints.
  • The study underscores improvements in real-time trigger systems for high-luminosity LHC experiments through optimized model compression.

Introduction to FPGA-Based Jet Classification

In the field of high-energy physics, rapid and efficient data processing is critical. This is particularly true at the CERN Large Hadron Collider (LHC), where particle collision events occur every 25 ns, producing copious amounts of data. Machine learning algorithms, especially those running on field-programmable gate arrays (FPGAs), offer promising solutions for such data-intensive environments. In this light, the paper under discussion presents a comprehensive analysis of various ML-based jet flavor classification algorithms developed for FPGAs. Leveraging quantization-aware training and efficient hardware implementation, researchers propose architectures that elegantly meet the stringent resource and latency constraints of the high-luminosity LHC (HL-LHC) upgrade.

Leveraging Particle-Level Data for ML Models

The efficiency of particle detectors is paramount, particularly when upgrades promise to yield tenfold increases in data. The introduction of particle-flow (PF) reconstruction at the L1 trigger level—enabled by new tracking information—allows for novel, ML-based trigger algorithms that operate on particle-level data. The researchers focus on the classification of jets—a critical task for unraveling the secrets of high-energy physics events. They explore architectures such as multilayer perceptrons (MLPs), deep sets (DS), and interaction networks (INs), which can robustly handle unordered data inputs.

Optimizing Performance on FPGA Constraints

Understanding the practical limitations of FPGAs, including latency and resource use, the researchers meticulously fine-tune ML architectures to operate within these boundaries. The paper presents a striking balance between algorithm complexity and the ability to fit these models within the FPGA confines. Numerical precision for weights and operations is reduced, yet accuracy is maintained, a process termed quantization. Significantly, the research forecasts that even at quantization to 8 bits, complex architectures like DS and IN can run inference within an impressive 100 ns while meeting the LHL-LHC conditions.

Compression and Efficiency of Models

Compression strategies like quantization-aware training are crucial when adopting models for FPGA implementation. The paper provides an authoritative look at how this compression doesn't necessarily impair ML models' accuracy. While DS networks offer a good balance across various performance metrics, the IN models shine in cases with a larger number of constituents. However, both architectures exhibit higher resource costs and potentially greater latency, indicating a trade-off between signal efficiency and computational budget.

Conclusion and Future Implications

The paper, a testament to interdisciplinary collaboration, makes a substantial contribution to the design of future trigger systems at HL-LHC experiments. It confirms that deploying ML algorithms in real-time LHC data processing is not only feasible but could significantly enhance trigger decision quality. Innovations in model compression and optimization, alongside ongoing developments in hardware, will continue to shape this exciting intersection of machine learning and particle physics.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown