Exploring Performance-Productivity Trade-offs in AMT Runtimes: A Task Bench Study of Itoyori, ItoyoriFBC, HPX, and MPI

Abstract: Asynchronous Many-Task (AMT) runtimes offer a productive alternative to the Message Passing Interface (MPI). However, the diverse AMT landscape makes fair comparisons challenging. Task Bench, proposed by Slaughter et al., addresses this challenge through a parameterized framework for evaluating parallel programming systems. This work integrates two recent cluster AMTs, Itoyori and ItoyoriFBC, into Task Bench for comprehensive evaluation against MPI and HPX. Itoyori employs a Partitioned Global Address Space (PGAS) model with RDMA-based work stealing, while ItoyoriFBC extends it with futurebased synchronization. We evaluate these systems in terms of both performance and programmer productivity. Performance is assessed across various configurations, including compute-bound kernels, weak scaling, and both imbalanced and communication-intensive patterns. Performance is quantified using application efficiency, i.e., the percentage of maximum performance achieved, and the Minimum Effective Task Granularity (METG), i.e., the smallest task duration before runtime overheads dominate. Programmer productivity is quantified using Lines of Code (LOC) and the Number of Library Constructs (NLC). Our results reveal distinct trade-offs. MPI achieves the highest efficiency for regular, communication-light workloads but requires verbose, lowlevel code. HPX maintains stable efficiency under load imbalance across varying node counts, yet ranks last in productivity metrics, demonstrating that AMTs do not inherently guarantee improved productivity over MPI. Itoyori achieves the highest efficiency in communication-intensive configurations while leading in programmer productivity. ItoyoriFBC exhibits slightly lower efficiency than Itoyori, though its future-based synchronization offers potential for expressing irregular workloads.

• The paper integrates and evaluates two AMT runtimes— Itoyori and ItoyoriFBC —against MPI and HPX, revealing that no single system is consistently optimal: MPI excel on regular tasks, HPX on imbalances, Itoyori on communication intensities.
• Itoyori and ItoyoriFBC demonstrate significantly lower lines of code (LOC) and number of library constructs (NLC) compared to MPI, indicating higher productivity, while HPX, despite expectations, ranked lowest in productivity.
• The research also shows that MPI and HPX maintain efficiency at low task widths, while Itoyori requires sufficient parallelism to amortize overhead, establishing crucial preconditions for adopting Itoyori,

Overview

This paper presents the first integration of two cluster-level Asynchronous Many-Task (AMT) runtimes, Itoyori and ItoyoriFBC, into Task Bench (2601.14608), and evaluates them against MPI and HPX on both performance and programmer productivity. Itoyori combines a Partitioned Global Address Space (PGAS) model with nested fork-join (NFJ) parallelism and RDMA-based random work stealing (RDWS); ItoyoriFBC replaces NFJ with Future-Based Cooperation (FBC), allowing task dependencies to be expressed as arbitrary DAGs. The evaluation uses application efficiency and Minimum Effective Task Granularity (METG) as performance metrics, and Lines of Code (LOC) together with Number of Library Constructs (NLC) as productivity metrics. The central finding is that no single system dominates: MPI wins on regular workloads, HPX on load imbalance, and Itoyori on communication-intensive patterns while simultaneously leading in productivity.

Benchmark methodology

Task Bench generates synthetic task graphs whose structure is controlled by parameters such as width (parallelism), steps (depth), and dependency pattern. The study uses three graph types — stencil (nearest-neighbor), spread (long-range), and all_to_all (dense) — combined with compute_bound kernels (fixed FLOPs) and a load_imbalance kernel with randomized iteration counts. Experiments ran on Goethe-NHR nodes with dual Intel Xeon Gold 6148 processors (40 cores per node) connected via InfiniBand, using Open MPI 5.0.5, HPX v1.11.0, Itoyori v0.0.2, and results averaged over five runs.

The productivity comparison is stark:

Metric	MPI	HPX	Itoyori	ItoyoriFBC
LOC	137	224	77	115
NLC	11	23	14	11

Itoyori's checkout/checkin PGAS API reduces code size by nearly 50% relative to MPI. Notably, HPX ranks last on both metrics — a result that directly contradicts the common assumption that AMT runtimes inherently improve productivity over MPI. The authors concede this may partly reflect the effort of adapting an MPI-centric Task Bench reference architecture to HPX rather than an intrinsic property of the programming model.

Task granularity and METG

At low graph widths, both Itoyori variants suffer from RDWS latency: failed steal attempts cost approximately $3\,\mu s$ and dominate fine-grained workloads. Increasing the width to 16 tasks per core amortizes this overhead substantially, raising Itoyori's efficiency from 25% to 76% at $2^{14}$ iterations. MPI and HPX maintain higher efficiency at low widths due to static partitioning and intra-node sharing, respectively. The METG(50%) metric confirms that Itoyori requires coarser task granularity than MPI or HPX, though the gap narrows as available parallelism grows. This establishes a concrete precondition for adopting Itoyori: sufficient parallelism must exist to mask scheduling overheads.

Weak scaling and load imbalance

Under weak scaling from 1 to 16 nodes with a stencil pattern, MPI and HPX scale well — METG(50%) remains stable for HPX and increases only marginally for MPI — because localized communication resolves most dependencies intra-node. Itoyori and ItoyoriFBC decline across nodes: randomized task distribution causes frequent remote-node accesses even for stencil-local patterns, incurring latencies absent under MPI's static mapping.

With the load_imbalance kernel, MPI degrades roughly linearly with imbalance, dropping to about 85% efficiency at the highest factor, yet remains competitive because of its low baseline overhead. HPX, Itoyori, and ItoyoriFBC all mitigate imbalance through work stealing and maintain superior efficiency at high imbalance factors. The implication is that dynamic load balancing pays off only when irregularity is significant enough to outweigh its overhead.

Communication-intensive patterns

The spread benchmark exposes topology sensitivity: Itoyori sustains over 89% efficiency with 40 dependencies per task, while HPX drops sharply when scaling from one to two nodes (87% to 65%), which the authors attribute to static partitioning among nodes. In the all_to_all worst case, Itoyori outperforms all other systems by a wide margin. The advantage is architectural: the PGAS API aggregates remote memory accesses for multiple dependencies into single contiguous RDMA operations, drastically reducing message injection rate, whereas MPI and HPX must serialize each dependency as an individual message. ItoyoriFBC forfeits this optimization because its current implementation re-wraps futures per dependency, preventing contiguous access — a concrete implementation limitation rather than a fundamental constraint of FBC. Experiments beyond 8 nodes at width 16 failed due to memory exhaustion imposed by the dense dependency structure, bounding the scalability claims for this pattern.

Limitations and open questions

Several caveats qualify the results. All implementations enforce a bulk-synchronous barrier between time steps; ItoyoriFBC could theoretically overlap successive steps without barriers, so its measured efficiency understates what a fully asynchronous FBC implementation might achieve. The HPX productivity numbers are confounded by the adaptation of an MPI-centric code structure, and HPX experiments were capped at 32 tasks per core by MPI tag-space constraints ($2^{22}$ tags) in its inter-node layer. The all_to_all scalability data stops at 8 nodes because of memory limits. The paper also leaves open whether future re-wrapping in ItoyoriFBC can be eliminated, and whether Task Bench can be extended to evaluate fault tolerance and dynamic resource management, capabilities increasingly expected of AMT runtimes but outside this study's scope.

Conclusion

This study provides the first controlled Task Bench comparison of Itoyori and ItoyoriFBC against MPI and HPX, quantifying trade-offs along four axes: granularity sensitivity, weak scaling, imbalance resilience, and communication intensity. Its principal contributions are empirical evidence that PGAS aggregation yields decisive advantages in dense communication patterns, that these advantages coexist with the best productivity metrics in the study, and that AMT adoption does not automatically improve programmer productivity, as HPX's last-place ranking demonstrates. The results support a differentiated recommendation: MPI for deterministic, fine-grained workloads; HPX for dynamically imbalanced applications; and Itoyori where global-address-space abstractions match the communication structure, provided task granularity suffices to amortize work-stealing costs.