Coalescing MPI communication in 6D Vlasov simulations: solving ghost domains in Vlasiator

Abstract: High-performance computing is used for diverse simulations, some of which parallelize over the Message Passing Interface (MPI) with ease, whilst others may have challenges related to uniform balancing of computational load and communication between simulation domains. We introduce an alternative approach to solving advection equations, specifically in an application to solving the six-dimensional Vlasov equation for modelling space plasmas. Communicating larger ghost domains around the partition assigned to each MPI task and computing on these ghost cells allows for coalescing several discrete communication calls into one. This approach needs more overall data communication and computation, but provides interesting new avenues for the balancing of computational load between MPI tasks. We discuss this trade-off, how it may assist in developing other algorithmic improvements, and how the transition to heterogeneous CPU-GPU architectures may impact its usefulness.