REvolve Framework for Optimal Checkpointing

• REvolve framework is a collection of algorithms and software abstractions that enable memory-optimal checkpointing for large-scale time-dependent adjoint computations.
• It employs dynamic programming to determine binomial checkpointing schedules and extends to multistage schemes, significantly reducing recomputation cost.
• The framework integrates with scientific libraries like Devito and PETSc to automate forward and reverse computations in PDE-driven simulations.

The term “REvolve framework” refers principally to a family of algorithms, software abstractions, and implementations for memory-optimal checkpointing in large-scale time-dependent adjoint computations—especially relevant for PDE-constrained optimization and inversion problems. The framework achieves a provably optimal trade-off between memory usage and recomputation cost by strategically storing and recomputing forward states to support reverse-mode (adjoint) sweeps with limited resources. It underpins both foundational algorithmic work and widely used scientific-computing libraries.

1. Mathematical Foundations of Checkpointing

Checkpointing addresses the critical bottleneck in adjoint or reverse-mode automatic differentiation of long time-integration problems: storing the entire trajectory of solution vectors is prohibitive, while full recomputation is computationally wasteful. Formally, for a time-marching process evolving states $u_n$ by $u_{n+1} = \mathcal{T}(u_n)$ over $n=0,\ldots,N-1$, and adjoint equations propagated backward, the discrete adjoint computation requires access to all intermediate $u_n$ during the reverse sweep.

Let $N$ denote the total number of steps, and $M$ the number of available checkpoints that can each store a state vector. The goal is to minimize the total number of recomputed forward steps during the adjoint phase, given the memory constraint $M$ (Kukreja et al., 2018, Zhang et al., 2021).

The optimal solution is obtained by dynamic programming. Denoting $T(N,M)$ as the minimal work (in terms of forward steps), the recurrence

$$T(0,M) = 0, \quad T(N,0) = \frac{N(N+1)}{2}, \quad T(N,M) = \min_{1\le j\le N} \left[ j + T(N-j,M-1) + T(j,M) \right]$$

yields the schedule with the least recomputation (Kukreja et al., 2018, Zhang et al., 2021). This binomial structure leads to "binomial checkpointing" and is provably globally optimal for the class of serial strategies.

2. Classical REvolve Algorithm and Generalizations

The original REvolve algorithm (Griewank & Walther) provides an efficient implementation of the above recurrence, exposing a user interface as a runtime “controller” that issues a sequence of actions: {advance, takeshot, restore, youturn, firsturn}. At each forward or reverse pass, the application consults this controller, which then signals whether to advance the simulation, checkpoint the current state, or restore from a stored snapshot (Zhang et al., 2021, Kukreja et al., 2018).

The classical algorithm restricts each checkpoint to storing only complete solution vectors, and produces a schedule derived analytically from binomial coefficients: $\text{Let } t \text{ satisfy } C(M + t, t) \geq N.$ Then the minimal recomputation cost is $T(N, M) = tN - C(M + t, t-1)$.

3. Extensions: Multistage Schemes and Fully Optimal Schedules

For modern time-stepping methods, such as $\ell$-stage Runge–Kutta, each step includes not only the solution but also intermediate stage vectors. Storing these stages can reduce recomputation further. A modified REvolve shifts the schedule by one to “store solution plus all stages,” yielding savings of one recomputation per backward step without increasing storage (Zhang et al., 2021).

For the most general multistage case, the CAMS algorithm offers a fully optimal dynamic-programming solution, where each checkpoint can hold either a solution or a single stage. The corresponding DP recurrences distinguish whether the last checkpoint held a solution or a stage, and are parametrized by $(m,s)$: remaining steps and remaining storage units, respectively. CAMS thus achieves minimum recomputation for any combination of solution and stage storage assumptions, filling an $O(N^2M)$ table for worst-case complexity (Zhang et al., 2021).

4. Software and API Abstractions

High-level interfaces, such as pyRevolve and integrations into domain-specific languages (e.g., Devito for seismic PDEs), expose checkpointing orchestration at the level of Python classes: Checkpoint (defining save/load/size), Operator (defining apply for forward/reverse), and Revolver (which coordinates actions and storage) (Kukreja et al., 2018). The interface supports both manual and automated workflow, with DevitoCheckpoints serializing arrays to contiguous NumPy buffers and the Revolver logic translating controller instructions into checkpoint and recomputation operations with minimal user intervention.

For C/C++ and MPI-based codes, REvolve and CAMS are provided as standalone libraries, callable from PETSc TSAdjoint, SUNDIALS, and other solver frameworks (Zhang et al., 2021). Both in-memory and out-of-core (disk- or SSD-backed) checkpointing are supported in production codes for large-scale simulation.

5. Performance, Optimality, and Scalability

REvolve and its variants deliver substantial practical savings in both wall-clock time and memory usage while obtaining mathematically guaranteed minimal recomputation. Benchmarking on Gray–Scott PDE-constrained optimization (problem size $128\times128$ and $2048\times2048$, 300 time steps, 2048 MPI ranks) demonstrates that the CAMS algorithm reduces recomputation by up to $2\times$ over classical REvolve—e.g., with $M=60$, $N=300$, about 260 forward steps saved, nearly doubling speed (Zhang et al., 2021).

Empirical studies in seismic imaging confirm that checkpointing allows solutions to problems previously infeasible due to memory constraints, and that the observed memory–runtime trade-off closely follows the predicted curve for serial checkpointing. The framework scales to leadership-class supercomputers and integrates cleanly with high-productivity scientific libraries without requiring intrusive code restructuring (Kukreja et al., 2018, Zhang et al., 2021).

6. Implementation Notes and Integration in Scientific Workflows

Implementations distinguish between global “controller” patterns and lightweight “consultant” modes. In the latter, the checkpointing logic provides actions on demand, allowing mature workflow engines (such as PETSc) to embed checkpointing decisions at defined hooks in forward and reverse loops, thereby avoiding invasive global management (Zhang et al., 2021).

The interface is non-intrusive: users register fields to be stored, supply callbacks for advancing the state or applying the adjoint, and the framework orchestrates checkpointing with minimal impact on application structure.

API exposure spans C/C++ (REvolve, CAMS), Python (pyRevolve, pkg-cams), and domain-specific language wrappers. Practical guides recommend one-time DP table initialization offline, after which schedules can be queried as needed for arbitrary time subranges.

7. Future Directions and Applications

The REvolve framework is adopted widely in large-scale PDE-constrained optimization (e.g., seismic inversion, parameter estimation for reaction–diffusion systems). Its extensions (CAMS) are integrated into major libraries (PETSc TSAdjoint). Potential future work involves further generalization to non-uniform checkpoint/storage models, inhomogeneous hardware hierarchies, and integration with asynchronous and distributed-memory execution. Broader adoption is foreseen in optimization-driven simulation, time-dependent sensitivity analysis, and machine-learning pipelines for dynamical systems (Zhang et al., 2021, Kukreja et al., 2018).