Papers
Topics
Authors
Recent
2000 character limit reached

Optimization Using Pathwise Algorithmic Derivatives of Electromagnetic Shower Simulations (2405.07944v1)

Published 13 May 2024 in physics.comp-ph

Abstract: Among the well-known methods to approximate derivatives of expectancies computed by Monte-Carlo simulations, averages of pathwise derivatives are often the easiest one to apply. Computing them via algorithmic differentiation typically does not require major manual analysis and rewriting of the code, even for very complex programs like simulations of particle-detector interactions in high-energy physics. However, the pathwise derivative estimator can be biased if there are discontinuities in the program, which may diminish its value for applications. This work integrates algorithmic differentiation into the electromagnetic shower simulation code HepEmShow based on G4HepEm, allowing us to study how well pathwise derivatives approximate derivatives of energy depositions in a sampling calorimeter with respect to parameters of the beam and geometry. We found that when multiple scattering is disabled in the simulation, means of pathwise derivatives converge quickly to their expected values, and these are close to the actual derivatives of the energy deposition. Additionally, we demonstrate the applicability of this novel gradient estimator for stochastic gradient-based optimization in a model example.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Exploration of differentiability in a proton computed tomography simulation framework. Physics in Medicine & Biology 68, 24 (dec 2023), 244002. https://doi.org/10.1088/1361-6560/ad0bdd
  2. Progress in End-to-End Optimization of Detectors for Fundamental Physics with Differentiable Programming. arXiv:2310.05673 [physics.ins-det]
  3. Forward-Mode Automatic Differentiation of Compiled Programs. http://arxiv.org/abs/2209.01895 arXiv:2209.01895 [cs].
  4. S. Agostinelli et al. 2003. Geant4—a simulation toolkit. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 506, 3 (7 2003), 250–303. https://doi.org/10.1016/S0168-9002(03)01368-8
  5. Efficient Aerodynamic Design using the Discrete Adjoint Method in SU2. In 17th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference. American Institute of Aeronautics and Astronautics, Inc., Washington, D.C. https://doi.org/10.2514/6.2016-3518 arXiv:https://arc.aiaa.org/doi/pdf/10.2514/6.2016-3518
  6. J. Allison et al. 2006. Geant4 developments and applications. IEEE Transactions on Nuclear Science 53, 1 (Feb. 2006), 270–278. https://doi.org/10.1109/TNS.2006.869826
  7. J. Allison et al. 2016. Recent developments in Geant4. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 835 (Nov. 2016), 186–225. https://doi.org/10.1016/j.nima.2016.06.125
  8. Automatic Differentiation of Programs with Discrete Randomness. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., 10435–10447. https://proceedings.neurips.cc/paper_files/paper/2022/file/43d8e5fc816c692f342493331d5e98fc-Paper-Conference.pdf
  9. Unbiased Warped-Area Sampling for Differentiable Rendering. ACM Trans. Graph. 39, 6 (2020), 245:1–245:18.
  10. Systematically Differentiating Parametric Discontinuities. ACM Trans. Graph. 40, 107 (2021), 107:1–107:17.
  11. Etalumis: bringing probabilistic programming to scientific simulators at scale. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (Denver, Colorado) (SC ’19). Association for Computing Machinery, New York, NY, USA, Article 29, 24 pages. https://doi.org/10.1145/3295500.3356180
  12. Pablo de Castro and Tommaso Dorigo. 2019. INFERNO: Inference-Aware Neural Optimisation. Computer Physics Communications 244 (2019), 170–179. https://doi.org/10.1016/j.cpc.2019.06.007
  13. Toward the end-to-end optimization of particle physics instruments with differentiable programming. Reviews in Physics 10 (2023), 100085. https://doi.org/10.1016/j.revip.2023.100085
  14. Rick Durrett. 2019. Probability: Theory and Examples. (2019). https://services.math.duke.edu/~rtd/PTE/PTE5_011119.pdf Version 5 January 11, 2019.
  15. Herbert Fischer. 1991. Automatic differentiation of the vector that solves a parametric linear system. J. Comput. Appl. Math. 35, 1 (1991), 169–184. https://doi.org/10.1016/0377-0427(91)90205-X
  16. Jean Charles Gilbert. 1992. Automatic differentiation and iterative processes. Optimization Methods and Software 1, 1 (1992), 13–21. https://doi.org/10.1080/10556789208805503
  17. Paul Glasserman. 2003. Estimating Sensitivities. Springer New York, New York, NY, 377–420. https://doi.org/10.1007/978-0-387-21617-1_7
  18. Peter W. Glynn. 1990. Likelihood ratio gradient estimation for stochastic systems. Commun. ACM 33, 10 (oct 1990), 75–84. https://doi.org/10.1145/84537.84552
  19. Generative Adversarial Nets. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.), Vol. 27. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf
  20. Andreas Griewank and Andrea Walther. 2008. Evaluating Derivatives. Society for Industrial and Applied Mathematics, Philadelphia, PA, US. https://doi.org/10.1137/1.9780898717761
  21. Adjoining Independent Computations. In Automatic Differentiation of Algorithms: From Simulation to Optimization, George Corliss, Christèle Faure, Andreas Griewank, Laurent Hascoët, and Uwe Naumann (Eds.). Springer New York, New York, NY, 299–304. https://doi.org/10.1007/978-1-4613-0075-5_35
  22. Michael Kagan and Lukas Heinrich. 2023. Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physics. arXiv:2308.16680 [stat.ML]
  23. Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1312.6114
  24. ADEV: Sound Automatic Differentiation of Expected Values of Probabilistic Programs. Proceedings of the ACM on Programming Languages 7, POPL (Jan. 2023), 121–153. https://doi.org/10.1145/3571198
  25. Differentiable Monte Carlo Ray Tracing through Edge Sampling. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37, 6 (2018), 222:1–222:11.
  26. Niels Lohmann. 2024. JSON for Modern C++. https://github.com/nlohmann/json
  27. Distributions for Compositionally Differentiating Parametric Discontinuities. Proc. ACM Program. Lang. 8, OOPSLA1, Article 126 (apr 2024), 30 pages. https://doi.org/10.1145/3649843
  28. Monte Carlo Gradient Estimation in Machine Learning. Journal of Machine Learning Research 21, 132 (2020), 1–62. http://jmlr.org/papers/v21/19-346.html
  29. Mihály Novák. 2023a. HepEmShow: a compact EM shower simulation application. https://hepemshow.readthedocs.io/en/latest/
  30. Mihály Novák. 2023b. The HepEmShow R&D Project. https://github.com/mnovak42/hepemshow
  31. G4HepEm. https://github.com/mnovak42/g4hepem
  32. Stochastic Backpropagation and Approximate Inference in Deep Generative Models. In Proceedings of the 31st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 32), Eric P. Xing and Tony Jebara (Eds.). PMLR, Bejing, China, 1278–1286.
  33. High-Performance Derivative Computations Using CoDiPack. ACM Trans. Math. Softw. 45, 4, Article 38 (dec 2019), 26 pages. https://doi.org/10.1145/3356900
  34. Gradient estimation using stochastic computation graphs. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2 (Montreal, Canada) (NIPS’15). MIT Press, Cambridge, MA, USA, 3528–3536.
  35. D. F. Shanno. 1970. Conditioning of quasi-Newton methods for function minimization. Math. Comp. 24, 111 (1970), 647–656. https://doi.org/10.1090/S0025-5718-1970-0274029-X
  36. Nathan Simpson and Lukas Heinrich. 2023. neos: End-to-End-Optimised Summary Statistics for High Energy Physics. Journal of Physics: Conference Series 2438, 1 (feb 2023), 012105. https://doi.org/10.1088/1742-6596/2438/1/012105
  37. Policy gradient methods for reinforcement learning with function approximation. In Proceedings of the 12th International Conference on Neural Information Processing Systems (Denver, CO) (NIPS’99). MIT Press, Cambridge, MA, USA, 1057–1063.
  38. Charlie Tang and Russ R Salakhutdinov. 2013. Learning Stochastic Feedforward Neural Networks. In Advances in Neural Information Processing Systems, C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (Eds.), Vol. 26. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2013/file/d81f9c1be2e08964bf9f24b15f0e4900-Paper.pdf
  39. Laszlo Urban. 2006. A model for multiple scattering in Geant4. (12 2006).
  40. Clad – Automatic Differentiation Using Clang and LLVM. Journal of Physics: Conference Series 608, 1, 012055. https://doi.org/10.1088/1742-6596/608/1/012055
  41. Automatic differentiation of dominant eigensolver and its applications in quantum physics. Phys. Rev. B 101 (Jun 2020), 245139. Issue 24. https://doi.org/10.1103/PhysRevB.101.245139
  42. Monte Carlo Estimators for Differential Light Transport. Transactions on Graphics (Proceedings of SIGGRAPH) 40, 4 (2021). https://doi.org/10.1145/3450626.3459807
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free