Phoebus: Performance Portable GRRMHD for Relativistic Astrophysics

Abstract: We introduce the open source code PHOEBUS (phifty one ergs blows up a star) for astrophysical general relativistic radiation magnetohydrodynamic simulations. PHOEBUS is designed for, but not limited to, high energy astrophysical environments such as core-collapse supernovae, neutron star mergers, black-hole accretion disks, and similar phenomena. General relativistic magnetohydrodynamics are modeled in the Valencia formulation with conservative finite volume methods. Neutrino radiation transport is included with Monte Carlo and moment methods. PHOEBUS is built on the PARTHENON (Grete et al. 2022) performance portable adaptive mesh refinement framework, uses a GPU first development strategy, and is capable of modeling a large dynamic range in space and time. PHOEBUS utilizes KOKKOS for on-node parallelism and supports both CPU and GPU architectures. We describe the physical model employed in PHOEBUS, the numerical methods used, and demonstrate a suite of test problems to showcase its abilities. We demonstrate weak scaling to over 500 H100 GPUs.