Operational Experience and R&D results using the Google Cloud for High Energy Physics in the ATLAS experiment (2403.15873v1)
Abstract: The ATLAS experiment at CERN relies on a worldwide distributed computing Grid infrastructure to support its physics program at the Large Hadron Collider. ATLAS has integrated cloud computing resources to complement its Grid infrastructure and conducted an R&D program on Google Cloud Platform. These initiatives leverage key features of commercial cloud providers: lightweight configuration and operation, elasticity and availability of diverse infrastructure. This paper examines the seamless integration of cloud computing services as a conventional Grid site within the ATLAS workflow management and data management systems, while also offering new setups for interactive, parallel analysis. It underscores pivotal results that enhance the on-site computing model and outlines several R&D projects that have benefited from large-scale, elastic resource provisioning models. Furthermore, this study discusses the impact of cloud-enabled R&D projects in three domains: accelerators and AI/ML, ARM CPUs and columnar data analysis techniques.
- Worldwide LHC Computing Grid, URL: http://cern.ch/lcg [accessed 2023-11-01]
- ATLAS Software and Computing HL-LHC Roadmap, CERN-LHCC-2022-005
- ARM Architecture Reference Manual for A-profile architecture, URL: https://developer.arm.com/documentation/ddi0487/latest/ [accessed 2023-08-14]
- NorduGrid ARC documents, URL: http://www.nordugrid.org/documents/ [accessed 2023-11-01]
- HTCondor-CE Overview, URL: https://osg-htc.org/docs/compute-element/htcondor-ce-overview/ [accessed 2023-11-01]
- About EGI, URL: https://www.egi.eu/about/ [accessed 2023-11-01]
- OSG Site documentation, URL: https://osg-htc.org/docs/ [accessed 2023-11-01]
- Google Cloud Platform Spot VMs, URL: https://cloud.google.com/spot-vms [accessed 2023-11-01]
- Kubernetes, URL: https://kubernetes.io/docs/home/ [accessed 2023-06-07]
- OAuth 2.0, URL: https://oauth.net/2/ [accessed 2023-11-01]
- Indigo IAM, URL: https://indigo-iam.github.io/v/current/docs/ [accessed 2023-11-01]
- Dask Gateway, URL: https://gateway.dask.org/ [accessed 2023-06-07]
- Helm, URL https://helm.sh/ [accessed 2023-12-04]
- GCP4HEP repository, URL https://github.com/gcp4hep [accessed 2023-12-04]
- Amazon Web Services Services, URL: https://aws.amazon.com [accessed 2023-06-09]
- Apache Parquet, URL: https://parquet.apache.org [accessed 2023-10-18]
- XRootD, URL: https://xrootd.slac.stanford.edu [accessed 2023-10-18]
- aiohttp, URL: https://docs.aiohttp.org [accessed 2023-10-18]
- fsspec, URL: https://github.com/fsspec/filesystem_spec [accessed 2023-10-18]
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.