ORCHA: Heterogeneous Computing Orchestration

• ORCHA is a lightweight system for performance portability in post-exascale HPC, enabling orchestration of diverse hardware and solver variants.
• It integrates CG-Kit, Macroprocessor, and Milhoja to manage task mapping, data layout variants, and asynchronous execution with minimal legacy code modifications.
• Performance studies demonstrate up to 38% speedup using concurrent CPU/GPU mapping, highlighting effective exploitation of hardware heterogeneity.

ORCHA, short for Orchestration for Heterogeneous Computing Architectures, is a lightweight performance-portability system for post-exascale high-performance computing developed to meet the needs of large multiphysics codes such as Flash-X. It addresses a setting in which hardware heterogeneity and software heterogeneity interact: nodes mix CPUs with accelerators and potentially future specializable chiplets, while application frameworks accumulate multiple solver variants with divergent data-layout and arithmetic requirements. ORCHA is built around three stand-alone tools—CG-Kit, Macroprocessor, and Milhoja—plus an application-specific interface layer, FlashX-RecipeTools, which uses code generation and code synthesis to stitch them together for Flash-X (Lee et al., 12 Jul 2025).

1. Motivation and problem setting

ORCHA was designed against a background in which HPC systems are moving away from homogeneous, Dennard-scaled multicore CPUs toward heterogeneous nodes that may include GPUs, FPGAs, special-purpose chiplets, and deep-memory hierarchies (Lee et al., 12 Jul 2025). The central problem is not merely kernel offload. The paper identifies two simultaneous heterogeneity crises.

The first is hardware heterogeneity: different platforms require different data layouts, memory-movement patterns, and parallelization strategies on CPU versus GPU, with future chiplet-based systems expected to increase this variability further. The second is software heterogeneity: multiphysics codes accrete multiple solvers, and each solver may demand distinct data structures or algorithmic variants, leading to combinatorial code multiplication if every combination is hand-tuned.

Within that context, conventional directive-based offload is presented as insufficient for the targeted use case. Fortran-based frameworks such as Flash-X could employ OpenACC or OpenMP to offload kernels to GPUs, but that approach is described as falling short of three goals: orchestrating asynchronous data transfers and overlapping computation across devices, expressing and exploring alternative device mappings for entire solver pipelines, and unifying variants of data layouts and arithmetic without manual code duplication. Task runtimes such as Legion or StarPU are acknowledged as capable of fine-grained orchestration, but the paper states that they demand deep, risky refactoring of large legacy bases and do not offer a simple ramp-on path.

ORCHA was therefore designed to satisfy four explicit requirements: expressing variants in data structures and kernel implementations without combinatorial code multiplies; expressing “what to compute where” via a high-level mapping language; orchestrating data and computation movement efficiently at runtime; and enabling gradual adoption by isolating application-specific transformations in a thin interface layer. This positioning distinguishes ORCHA from both directive-only offload workflows and fully general task-runtime migrations.

2. Architectural decomposition

ORCHA’s architecture is modular. Its three core tools each target one of the first three requirements, and they interoperate through code-generation and code-synthesis pipelines (Lee et al., 12 Jul 2025). The system-level decomposition is as follows.

Component	Role
CG-Kit	High-level control-flow and mapping language
Macroprocessor	Variant specification for data layouts and arithmetic
Milhoja	Runtime orchestration manager
FlashX-RecipeTools	Thin Python/CMake glue layer for Flash-X

This decomposition is central to ORCHA’s method. Rather than embedding all concerns into a monolithic runtime or a directive-based source transformation pipeline, ORCHA separates mapping specification, variant management, and execution-time orchestration. The application-specific adapter layer then bridges those generic facilities to a concrete legacy code base.

A key architectural feature is that ORCHA remains compatible with gradual adoption. The paper emphasizes that application-specific transformations are isolated in a thin interface layer rather than requiring wholesale refactoring of Flash-X internals. This suggests a design philosophy oriented toward legacy multiphysics frameworks in which invasive changes are a primary adoption barrier.

3. CG-Kit: recipe-driven control flow and mapping

CG-Kit is a Python-based front end in which a user writes a recipe declaring tasks, corresponding to solver invocations, and mapping each task to a device (Lee et al., 12 Jul 2025). The paper gives an illustrative grammar:

$D=\{CPU, GPU\}$7

with <device> ∈ {CPU, GPU}.

The paper’s example for a simple four-node graph includes statements of the form:

$D=\{CPU, GPU\}$8

From such a recipe, CG-Kit parses the specification, constructs a directed acyclic graph of TaskFunctions, and applies graph-level optimizations such as latency hiding and minimizing data movement. The optimized DAG is then lowered to a parameterized source tree of code templates in Fortran or C++ that still contain placeholders for data-layout macros and interoperability wrappers. CG-Kit finally emits compilable source code by instantiating the PST.

Two aspects are technically significant. First, the mapping abstraction is expressed at the task-graph level rather than at the level of individual loop annotations. Second, optimization occurs before final code emission, so mapping decisions can influence the generated orchestration structure. This makes the recipe a first-class performance-portability artifact rather than a mere build-time configuration file.

The paper further states that, for explored Flash-X configurations, ORCHA can compare candidate mappings in milliseconds by plugging measured processing rates and bandwidths into a cost model on Perlmutter. A plausible implication is that CG-Kit functions as both an overview front end and a lightweight mapping-exploration mechanism, not simply as a static code generator.

4. Macroprocessor and Milhoja

The Macroprocessor is a domain-specific macro system that unifies data-structure and arithmetic-operation variants in languages lacking native templates, particularly Fortran (Lee et al., 12 Jul 2025). The paper presents example layout macros:

$D=\{CPU, GPU\}$9

These definitions allow the same loop-kernel source to be compiled into multiple variants without editing the loop body. Formally, if $M$ is the set of macro names and $D=\{CPU, GPU\}$ is the set of devices, variant selection is represented by

$V: M \times D \to Definition$

which picks the appropriate macro body for each name-device pair.

The paper also states that the Macroprocessor permits inheritance of macro sets and resolves conflicts via simple arbitration, enabling new layouts or loop-parallelization models to be layered atop existing ones without modifying legacy code. In technical terms, ORCHA therefore treats low-level layout and arithmetic variability as a composable metaprogramming problem rather than as hand-maintained source branching.

Milhoja, by contrast, is a C++ domain-specific runtime that repeatedly executes a fixed TaskFunction DAG over time steps. It manages allocation of thread teams on CPU or GPU streams, asynchronous host-device transfers of DataPackets, scheduling of tasks according to the CG-Kit mapping, and minimal MPI interaction by constraining each TaskFunction to be MPI-free; any MPI call must bracket a TaskFunction.

A DataPacket is defined as a flattened chunk of many mesh blocks tiled together to amortize PCIe latency. On CPU, Milhoja instead uses lightweight TileWrappers, since device transfers are unnecessary. The scheduler overlaps transfers and computation by exploiting hardware concurrency, including CUDA streams.

A common misconception would be to read Milhoja as a general-purpose MPI-integrated task runtime. The paper instead presents it as a domain-specific orchestration manager with an explicit restriction: TaskFunctions are MPI-free, and MPI must occur outside them. That restriction reduces entanglement with distributed-memory concerns while preserving asynchronous host-device orchestration.

5. FlashX-RecipeTools and the Flash-X integration path

To avoid invasive refactoring of Flash-X, the ORCHA workflow introduces FlashX-RecipeTools, described as a thin Python/CMake glue layer (Lee et al., 12 Jul 2025). Its responsibilities are explicit and sequential.

First, it parses user recipes into CG-Kit. Second, it scans Fortran solver sources for structured annotations describing each subroutine’s data requirements, scratch-space sizes, and MPI-freedom. Third, it emits a JSON TaskFunction Specification listing, for each task, the Fortran routines to call and their argument-to-mesh-tag mappings. Fourth, it invokes the CG-Kit, Macroprocessor, and Milhoja code generators to produce interoperability wrappers, TaskFunction stubs, and DataPacket or TileWrapper classes with the required stride and memory hints.

The annotation mechanism is illustrated with Fortran comments such as:

$V: M \times D \to Definition$0

FlashX-RecipeTools collects these annotations, after which the TaskFunction Generator produces C++ code of the form:

$V: M \times D \to Definition$1

The paper states that this is generated automatically once the user provides the recipe and the annotated Fortran sources. The significance of this layer is methodological: application adaptation is expressed through annotations and generated wrappers rather than through direct hand-written rewrite of solver internals. This supports the paper’s broader claim that ORCHA provides a non-intrusive ramp-on path for large Fortran bases.

6. Hardware mappings, cost model, and case study

The paper studies ORCHA through two Flash-X test problems: a Sedov blast wave and a cellular nuclear burning detonation (Lee et al., 12 Jul 2025). Let $T=\{t_1,\ldots,t_n\}$ denote the set of TaskFunctions and let $M:T \to \{CPU,GPU\}$ denote their device mapping. The total execution time is modeled as

$$t_{total}(M) = \sum_{i=1}^n \frac{D_i(M(t_i))}{P_{M(t_i)}} + \sum_{edges\ (i \to j)} \frac{B_{ij}}{BW_{M(t_i),M(t_j)}}$$

where $D_i(dev)$ is the workload or data size for task $i$ on device $dev$, $P_{dev}$ is the peak processing rate on that device, $D=\{CPU, GPU\}$0 is the data volume moved between tasks $D=\{CPU, GPU\}$1 and $D=\{CPU, GPU\}$2, and $D=\{CPU, GPU\}$3 is the bandwidth between those devices. Using measured $D=\{CPU, GPU\}$4 and $D=\{CPU, GPU\}$5 on Perlmutter, CG-Kit can compare different mappings in milliseconds.

Three concrete mappings are reported:

Mapping	Description
GPU-Centric	All Hydro+EOS tasks → GPU; all nuclear-burn tasks → CPU in serial
CPU/GPU Balanced	Split Hydro work between CPU thread teams and GPU DataPackets; then run Burn on CPU
CPU/GPU Concurrent	Duplicate Hydro inputs into both a GPU task and a CPU task so that Hydro runs on GPU and Burn runs on CPU simultaneously

For the 2D cellular test (13-isotope $D=\{CPU, GPU\}$6–chain burn), the paper reports the following per-time-step timings: CPU-only baseline with 64 MPI ranks, 73.8 s; GPU-Centric, 48.9 s; CPU/GPU Balanced, 80.6 s; and CPU/GPU Concurrent, 45.7 s. The associated changes are given as approximately 34 % speedup, 9 % slowdown, and 38 % speedup, respectively.

These data show that heterogeneous execution is not automatically advantageous in every partitioning regime. The CPU/GPU Balanced configuration is slower than the CPU-only baseline, whereas CPU/GPU Concurrent is the fastest of the reported mappings. The paper describes the concurrent mode as an asynchrony-for-fidelity trade-off, which indicates that performance benefits may be coupled to solver-level or workflow-level compromises that must be evaluated in application context.

For the Sedov case, the paper further reports that packing 80–160 blocks per DataPacket saturates the GPU and outperforms naïve OpenACC on unit-block offloads by an order of magnitude. This emphasizes that ORCHA’s performance behavior depends not only on device assignment but also on aggregation granularity and transfer amortization.

7. Scope, limitations, and future directions

ORCHA is presented as a modular toolbox for next-generation performance portability rather than as a complete autotuning system (Lee et al., 12 Jul 2025). Its principal contributions are stated explicitly: CG-Kit provides a high-level mapping language and graph optimizer; Macroprocessor unifies data-layout and arithmetic variants in legacy code; Milhoja provides lightweight, task-based orchestration without MPI entanglement; and application adapters such as FlashX-RecipeTools provide a non-intrusive ramp-on for large Fortran bases.

The paper also states several limitations. Kernel-level autotuning, including tile sizes and vector widths, is not yet automated. Runtime-level heuristics for task placement could be made more adaptive. Chiplet-style on-package accelerators will require new bandwidth models and partitioning strategies. These limitations define the boundary of ORCHA’s current scope: it supports mapping exploration and orchestration, but it does not yet automate all lower-level code-shape choices or future heterogeneous packaging models.

Future work is described as integrating autotuned code generation, adaptive scheduling policies, and chiplet-aware cost models while preserving the recipe-driven workflow. This suggests that the system is intended to evolve by extending its existing abstractions rather than replacing them. In that sense, ORCHA’s central idea is not any single runtime or macro facility, but the combination of recipe-level orchestration, variant-aware code generation, and thin adaptation layers for large legacy multiphysics applications.