AMReX: Adaptive Mesh Refinement Framework

• AMReX is a software framework that provides a unified infrastructure for block-structured adaptive mesh refinement and parallel runtime management.
• It decouples low-level data structures from high-level mesh refinement and physics algorithms, supporting diverse applications from astrophysics to combustion.
• The framework integrates MPI, OpenMP, and GPU offload strategies with advanced tools like pyAMReX for in situ analysis and rapid prototyping.

Searching arXiv for relevant AMReX papers to ground the article. AMReX is a software framework for the development of block-structured mesh applications with adaptive mesh refinement (AMR). Initially developed and supported by the AMReX Co-Design Center as part of the U.S. DOE Exascale Computing Project, and formerly known as BoxLib, it provides a unified infrastructure for multiphysics applications that solve systems of partial differential equations in simple or complex geometries and that use particles and/or particle-mesh operations (Myers et al., 2024, Zhang et al., 2020, Frey et al., 2018).

1. Historical position and design scope

Block-structured adaptive mesh refinement provides the basis for the temporal and spatial discretization strategy for a number of ECP applications in accelerator design, additive manufacturing, astrophysics, combustion, cosmology, multiphase flow, and wind plant modelling (Zhang et al., 2020). In this setting, AMReX separates the basic data structures and their parallel support from the higher-level mesh-refinement and physics algorithms, so the same core runtime can support markedly different discretizations and application domains (Myers et al., 2024).

The framework is explicitly aimed at machines “from laptops to exascale architectures” and has continued to grow post-ECP, with both new functionality in the C++ core and the addition of pyAMReX as a Python binding (Myers et al., 2024, Zhang et al., 2020). A plausible implication is that AMReX should be understood less as a single solver than as an execution and data-management substrate for families of AMR applications.

2. Core abstractions and data model

At the lowest level, AMReX organizes a computational domain into a hierarchy of levels, each decomposed into non-overlapping rectangular patches. The fundamental index-space object is the Box,

$$\mathrm{Box}=\{\; i\in\mathbb{Z}^D \mid i_{\min}\le i\le i_{\max}\;\},$$

with an IndexType distinguishing cell-centered, face-centered, or node-centered data (Zhang et al., 2020). A BoxArray stores the collection of boxes for one AMR level; a DistributionMapping assigns each box to an MPI rank; and an FArrayBox stores the per-patch numerical data. The standard floating-point distributed field container is the MultiFab, a FabArray<FArrayBox> specialization that carries ghost-cell metadata and supports halo fills, coarse-fine synchronization, and inter-level interpolation (Zhang et al., 2020, Myers et al., 2024).

Structure	Role
Box	Single logically rectangular index range
BoxArray	Patch decomposition of one AMR level
DistributionMapping	Assignment of boxes to MPI ranks
FArrayBox	Fortran-like array on one box
MultiFab	Distributed collection of FArrayBox objects
MFIter	Iterator over local boxes or tiles
Array4<T>	Non-owning view for CPU/GPU kernels
ParIter	Iterator over distributed particle tiles

Ghost-zone allocation follows the standard pattern

$$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$

and a Geometry object maps index space to physical space (Zhang et al., 2020). In practice, application codes reuse these same abstractions for cell-centered hydrodynamics, face-centered fluxes, vertex-centered relativity variables, embedded-boundary data, and particle containers (Shankar et al., 2022, Sharma et al., 2024).

This data model is intentionally general. GRaM-X, for example, stores hydrodynamic conserved and primitive variables as cell-centered MultiFabs, spacetime variables as vertex-centered MultiFabs, and fluxes as face-centered MultiFabs, all within the same AMReX-based hierarchy (Shankar et al., 2022).

3. Mesh refinement, synchronization, and multilevel algorithms

AMReX uses user-supplied error estimators or tagging criteria to decide where refinement is needed. A standard gradient-based form is

$$\eta_i=\max_d \frac{|U_{i+e_d}-U_{i-e_d}|}{2\Delta x},$$

with tagging when $\eta_i>\eta_{\rm tol}$ (Zhang et al., 2020). Tagged cells are clustered into rectangular patches by the Berger-Rigoutsos algorithm; recent work also references efficient “bittree” regridding (Myers et al., 2024). Successive levels are related by an integer refinement ratio $r$, so that $\Delta x^\ell=\Delta x^{\ell-1}/r$ or equivalently $\Delta x_\ell=\Delta x_0/r^\ell$, depending on notation (Frey et al., 2018, Zhang et al., 2020).

The multilevel control structure is exposed through standard hooks such as advance(ℓ), fillPatch(ℓ), averageDown(ℓ), and reflux(ℓ) (Myers et al., 2024). Prolongation and restriction are the canonical inter-level operators:

$$u^{\ell+1}=P(u^\ell),\qquad u^\ell = \frac{1}{r^D}\sum u^{\ell+1}_{i'}.$$

For hyperbolic conservation laws, AMReX supports subcycling in time, with level-dependent time steps constrained by

$$\Delta t^\ell \le \mathrm{CFL}\,\frac{\Delta x^\ell}{\max(|\mathbf{u}|+c)}.$$

Refluxing then restores coarse-fine conservation after fine-level updates (Zhang et al., 2020).

Although Berger-Colella synchronization is the default conceptual model, application papers show that AMReX does not enforce a single coarse-fine strategy. Alamo uses a “reflux-free” strategy for strong-form mechanics, while recent high-order finite-difference work on shock-turbulence interaction derives fifth-order WENO prolongation, sixth-order central-WENO restriction, hybrid interpolation near shocks, and standard refluxing at shared face locations within an AMReX implementation (Runnels et al., 11 Mar 2025, Wang et al., 11 Nov 2025).

4. Parallel execution model and performance portability

AMReX combines MPI across nodes with OpenMP threading on CPUs and ParallelFor-style GPU offload on CUDA, HIP, or SYCL back ends (Myers et al., 2024). The owner-computes pattern is embodied in MFIter, which traverses locally owned boxes or tiles, while Array4 provides lightweight views suitable for device kernels (Zhang et al., 2020, Myers et al., 2024). Arena-based allocators manage host, pinned, managed, and device memory; GPU-aware MPI and fused pack/unpack kernels support asynchronous ghost-cell exchange and flux-register communication (Zhang et al., 2020, Myers et al., 2024).

Several optimization layers are now integral to the framework. Kernel fusion can merge work over local boxes into a single launch; on AMD MI250X, the triad memory-bandwidth kernel cited in the AMReX/pyAMReX review rises from approximately $10\%$ to approximately $$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$0 of peak bandwidth under this strategy (Myers et al., 2024). Compile-time specialization expands small sets of runtime branches into optimized kernels, and ParReduce supports mixed reductions in a single device pass (Myers et al., 2024).

Representative application studies show how these abstractions scale. WarpX reports strong scaling to $$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$1 MPI ranks with approximately $$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$2 parallel efficiency on uniform grids and weak scaling on up to $$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$3 nodes with $$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$4 efficiency when particle and field work are included (Vay et al., 2018). GRaM-X, which uses CarpetX as a driver thorn built on top of AMReX’s block-structured AMR engine, reports weak scaling efficiency of approximately $$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$5 on 2304 nodes, corresponding to 13824 NVIDIA V100 GPUs on Summit (Shankar et al., 2022). Native async I/O in AMReX offloads writes to background threads and has been reported to reduce I/O overhead by up to $$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$6 on Summit (Zhang et al., 2020).

A common simplification is to describe AMReX merely as an “AMR library.” The performance literature suggests a broader characterization: it is simultaneously a mesh hierarchy manager, a distributed memory runtime, a device-execution abstraction layer, and an interoperability layer for solvers, particles, and I/O.

5. Numerical methods and application ecosystem

AMReX supports hyperbolic, parabolic, and elliptic PDE solvers, native geometric multigrid, external hypre and PETSc interfaces, embedded-boundary methods, and particle-mesh operations (Zhang et al., 2020, Myers et al., 2024). For conservation laws, a standard finite-volume update is

$$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$7

while for particle deposition and interpolation the framework provides ParticleContainer, ParIter, and grid-particle synchronization primitives such as SumBoundary and FillBoundary (Zhang et al., 2020).

The application spectrum is correspondingly broad. In astrophysics, Castro and Maestro are built on AMReX; Castro targets highly compressible stellar explosions, while Maestro targets subsonic convective flows (Zingale et al., 2017). In plasma and accelerator modeling, WarpX combines AMReX with PICSAR kernels for exascale particle-in-cell simulations, and ImpactX is cited among the new non-ECP AMReX applications (Vay et al., 2018, Myers et al., 2024). In cosmology and reacting flow, Nyx, PeleC, and PeleLMeX are listed as major AMReX-based codes (Myers et al., 2024). In numerical relativity and GRMHD, CarpetX and GRaM-X use AMReX to support 3D AMR with vertex-, cell-, and face-centered data on GPUs (Shankar et al., 2022). Additional application papers describe AMReX-based electrostatic PIC in OPAL, phase-field and strong-form mechanics in Alamo, and high-speed reacting-flow solvers for hypersonic propulsion (Frey et al., 2018, Runnels et al., 11 Mar 2025, Sharma et al., 2024).

This diversity matters methodologically. It shows that AMReX is not restricted to finite-volume gas dynamics or to a single mesh centering convention. The same core abstractions support PIC deposition, explicit shock-capturing, multigrid elliptic solves, strong-form mechanics, high-order finite differences, tabulated equations of state, and embedded-boundary cut-cell workflows.

6. Post-ECP developments: Python, I/O, compression, and load balancing

A major post-ECP extension is pyAMReX, a Python interface built with pybind11 that exposes AMReX types and provides zero-copy GPU-resident data exchange through __array_interface__ and __cuda_array_interface__ for NumPy, CuPy, PyTorch, and Numba workflows (Myers et al., 2024). The stated goals are in situ analysis, AI/ML coupling, application steering, and rapid massively parallel prototyping. DLPack support is described as underway (Myers et al., 2024).

AMReX’s I/O path has also become a target of active research. AMRIC inserts preprocessing, compressor optimizations, and HDF5-filter extensions into the AMReX HDF5 plotfile pipeline, including a virtual data-layout change that concatenates all boxes of a field and a modified filter that receives both the global chunk size and the actual used length on each rank (Wang et al., 2023). On Summit, using Nyx and WarpX, this framework reports up to $$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$8 higher compression ratios and up to $$i_{\min}^{\rm alloc}=i_{\min}-n_{\rm grow},\qquad i_{\max}^{\rm alloc}=i_{\max}+n_{\rm grow},$$9 faster I/O than AMReX’s original compression solution (Wang et al., 2023).

Load balancing remains an open algorithmic trade-off rather than a settled design point. AMReX has long used space-filling-curve and knapsack strategies (Zhang et al., 2020), but recent work evaluates brute-force, Knapsack, standard SFC, painter’s-partition-based SFC, and a hybrid SFC+Knapsack methodology based on hardware topology (Nanda et al., 21 May 2025). That study recommends painter’s-partition-based SFC as the first alternative to the default percentage-tracking SFC when both locality and balance matter, while also noting that plain Knapsack remains the most robust as weight variation grows (Nanda et al., 21 May 2025).

These developments suggest that the framework’s current evolution is less about changing the core block-structured AMR abstraction than about extending the surrounding ecosystem: language bindings, in situ data workflows, topology-aware load balancing, and high-performance storage paths.