HPCSim: High-Performance Simulation

• HPCSim is a family of high-performance simulation systems defined by domain-specific fidelity and flexible parallel architectures for tackling large-scale computational challenges.
• It employs recurrent patterns such as discrete-event execution, hybrid process-thread-accelerator decomposition, and explicit control of communication to address diverse simulation needs.
• Performance evaluations show significant speedups, minimal memory footprints, and robust scalability across applications like workload management, plasma simulation, and quantum circuit modeling.

In the literature covered here, “HPCSim” denotes a family of high-performance simulation environments and workflows rather than a single canonical software artifact. The label appears in workload-management simulation for HPC clusters, GPU-accelerated Monte Carlo valuation in computational finance, resilient multi-GPU Particle-in-Cell plus Monte Carlo plasma simulation, hybrid-parallel PDAE simulation for power electronic equipment, cycle-approximate GPU architectural simulation, heterogeneous quantum-circuit simulation, and multiscale biomedical coupling frameworks (Galleguillos et al., 2018, Cvetanoska et al., 2012, Williams et al., 26 Jun 2026, Shi et al., 17 Jan 2025, Lin et al., 21 May 2026, Suh et al., 2024, Groen et al., 2012). What unifies these uses is not a shared code base, but a recurring objective: preserving domain-specific model fidelity while exposing enough parallel structure, instrumentation, and data movement control to make large simulations tractable on contemporary HPC platforms.

1. Terminological scope and principal usages

Across the cited work, “HPCSim” functions as a contextual descriptor for simulation systems that are explicitly engineered for high-performance execution, scalability studies, or HPC-oriented design space exploration. In some cases it names a concrete simulator, as with the ACALSim-based GPU simulator called HPCSim; in others it describes a simulation methodology or workflow, as in power electronics, plasma PIC/MC, or biomedical multiscale coupling (Lin et al., 21 May 2026, Shi et al., 17 Jan 2025, Williams et al., 26 Jun 2026, Groen et al., 2012).

A related ambiguity arises around tools that are not themselves named “HPCSim” but are presented as appropriate answers to “HPCSim” needs. AccaSim is one such case: it is described as “an open-source, Python library for simulating the workload management system (WMS) of High Performance Computing (HPC) clusters,” with explicit emphasis on job dispatching research rather than network or storage simulation (Galleguillos et al., 2018). EvalNet is another: it is framed as an evaluation layer that feeds or calibrates packet-level simulators such as htsim and OMNeT++/INET for extreme-scale interconnect studies (Besta et al., 2021).

Use in the literature	Domain	Distinguishing focus
AccaSim (Galleguillos et al., 2018)	HPC workload management	Queues, scheduling, allocation, backfilling
GPU Monte Carlo “HPCSim” (Cvetanoska et al., 2012)	American option pricing	QMC, CUDA kernels, backward induction
BIT1-based HPCSim (Williams et al., 26 Jun 2026)	Plasma PIC/MC	Resilient multi-GPU execution, PLB, checkpoint/restart
Hybrid-Parallel Collaborative simulation (Shi et al., 17 Jan 2025)	Power electronics	PDAE coupling of device physics and circuit dynamics
HPCSim atop ACALSim (Lin et al., 21 May 2026)	GPU architecture	Cycle-approximate A100-class design exploration
NWQ-Sim as HPC-based quantum simulation (Suh et al., 2024)	Quantum circuits	State vector and density matrix simulation on HPC
MMSF/MUSCLE workflows (Groen et al., 2012)	Multiscale biomedicine	Coupling existing single-scale codes across resources

This distribution suggests that HPCSim is best read as a domain-bound term whose precise meaning is fixed by the simulation object, execution substrate, and coupling strategy specified in each paper.

2. Recurrent architectural patterns

Despite the heterogeneity of application areas, several architectural patterns recur. One is discrete-event execution. AccaSim advances time through real job events—submission $T_{sb}$, start $T_{st}$, and completion $T_c$—and models the job lifecycle as “loaded,” “queued,” “running,” and “completed” (Galleguillos et al., 2018). The ACALSim-based GPU HPCSim also uses event queues tagged by cycle, but augments this with fast-forwarding to the minimum next-event tick and a two-phase execution discipline in which Phase 1 computes next-state updates in parallel and Phase 2 commits them atomically, swaps communication buffers, and performs arbitration (Lin et al., 21 May 2026).

A second pattern is hybrid decomposition across processes, threads, and accelerators. The multi-GPU BIT1 framework organizes 1D domain decomposition across MPI ranks with OpenMP-parallel work inside each rank, and realizes GPU acceleration through OpenMP target offloading on both Nvidia H100 and AMD MI250X systems (Williams et al., 26 Jun 2026). The power-electronics framework likewise combines distributed-memory process assignment across loosely coupled modules or devices with intra-process threading for tightly coupled PDE solves, explicitly describing this as “hybrid-process-thread parallelization on shared and distributed memory” (Shi et al., 17 Jan 2025). NWQ-Sim separates concerns differently: its state-vector mode follows a PGAS-style decomposition, while its density-matrix mode follows a BSP-style execution model layered over multi-node CPU/GPU systems (Suh et al., 2024).

A third pattern is explicit control over communication semantics. In ACALSim, backpressure propagates through SimPort’s blocking push semantics rather than explicit credit logic, and zero-copy communication is enabled by a shared-memory data model with SimChannel ping/pong buffers (Lin et al., 21 May 2026). In the biomedical framework, MUSCLE converts an MML/xMML specification into an executable coupling recipe, while the MUSCLE Transport Overlay and MPWide handle local, LAN, and WAN communication across security domains and supercomputer sites (Groen et al., 2012). In plasma PIC/MC, openPMD and ADIOS2 provide a standardized data interface for checkpoint/restart and in-situ streaming, so simulation, persistence, and analysis are treated as a single workflow rather than disjoint stages (Williams et al., 26 Jun 2026).

A fourth pattern is explicit heterogeneity at the model level. AccaSim’s ResourceManager builds clusters from a JSON description of node groups, per-node resource inventories, and counts, with support for heterogeneous nodes and accelerators (Galleguillos et al., 2018). The ACALSim GPU simulator instantiates 108 SMs, 32 L2 slices, 10 HBM2 controllers, and a crossbar interconnect via a compact JSON configuration (Lin et al., 21 May 2026). H3PC addresses heterogeneity differently, through non-conforming quadrilateral or hexahedral elements, parallel adaptive mesh refinement, and real-gas thermochemistry provided by Mutation.jl and Mutation++ (Peyvan et al., 12 Nov 2025).

3. Mathematical and algorithmic regimes

The algorithmic content grouped under HPCSim is notably diverse. In financial computing, the option-pricing formulation is an optimal-stopping problem under Black–Scholes risk-neutral dynamics, with the underlying following geometric Brownian motion and the valuation expressed by Bellman recursion on a discrete exercise grid (Cvetanoska et al., 2012). The implementation reported there uses Quasi-Monte Carlo rather than standard Monte Carlo, transforms quasi-random uniforms to standard normals through the Moro inverse cumulative normal transformation, and uses a backward induction scheme with a Black–Scholes European value as a local exercise-boundary proxy instead of Longstaff–Schwartz regression (Cvetanoska et al., 2012).

In workload-management simulation, the key objects are queues, dispatching rules, and performance metrics. AccaSim separates scheduling from allocation through SchedulerBase.schedule() and AllocatorBase.allocate(), includes FIFO, SJF, LJF, and EASY Backfilling with FIFO priority, and reports canonical metrics such as slowdown

$s = \frac{W + R}{R},$

bounded slowdown

$$s_b = \max\!\left(1,\frac{W+R}{\max(R,\tau)}\right),$$

and turnaround time $T = W + R$ (Galleguillos et al., 2018). The resulting HPCSim regime is discrete-event and policy-centric rather than physics-centric.

Plasma and particle methods define a different regime. BIT1 retains the canonical PIC cycle—particle push, charge/current deposition, electrostatic field solve, field interpolation, optional density smoothing, and Monte Carlo collisions plus wall interactions—and uses collision probabilities of the form

$P_c = 1 - \exp(-\nu(v)\Delta t)$

with $\nu(v)=n\sigma(v)v$ (Williams et al., 26 Jun 2026). The framework emphasizes natural sorting, portable GPU offload, checkpoint-triggered particle load balancing, and resilience. The closely related openPMD-SST work retains the 1D3V electrostatic PIC/MC formulation but focuses on OpenMP task-based mover parallelism and in-memory streaming for diagnostics and checkpointing (Williams et al., 3 Dec 2025).

Power-electronics HPCSim is centered on a coupled PDAE system,

$M(y)\frac{dy}{dt}=f(y,t), \qquad g(y,t)=0,$

where $y=[\psi,n,p,x_c]$ stacks semiconductor PDE states and circuit variables (Shi et al., 17 Jan 2025). The core numerical device is a Gauss–Seidel dynamic iteration that alternates between PDE solves for device internals and DAE solves for circuit dynamics, with coupling mediated by terminal voltages, currents, and a Norton-equivalent linearization. The equivalent conductance

$T_{st}$0

is central to the circuit Jacobian augmentation (Shi et al., 17 Jan 2025).

Quantum-oriented HPCSim uses yet another mathematical regime. NWQ-Sim distinguishes ideal state-vector simulation from density-matrix simulation with device noise channels, with memory scaling $T_{st}$1 bytes for complex double precision state vectors and $T_{st}$2 bytes for density matrices (Suh et al., 2024). The comparison study based on Quantum Volume benchmarks keeps to noiseless state-vector simulation and evaluates time to execute a square random circuit of width $T_{st}$3 and depth $T_{st}$4, using heavy-output probability only as a correctness check rather than a pass/fail threshold (Niekerk et al., 2024).

Several HPCSim systems are also fundamentally PDE solvers. H3PC solves multicomponent Euler and Navier–Stokes equations with chemistry using entropy/energy-stable DGSEM, split-form flux differencing, positivity preservation, and subcell finite-volume blending (Peyvan et al., 12 Nov 2025). The neuromuscular framework couples 3D finite elasticity, a 1D monodomain equation, and 0D chemical kinetics through operator splitting, upgrading from first-order Godunov splitting to second-order Strang splitting with Heun and Crank–Nicolson components (Bradley et al., 2018). The HPS Helmholtz implementation targets the variable-coefficient impedance problem with hierarchical Poincaré–Steklov operators and an impedance-to-impedance formulation on corner-free Chebyshev leaf grids (Beams et al., 2018).

4. Tooling, I/O, monitoring, and reproducibility

HPCSim systems are typically accompanied by extensive tooling for experiment control and observability. AccaSim includes an Experiment class that auto-generates scheduler/allocator combinations, a PlotFactory for slowdown, queue-size, and CPU-time plots, a system-status CLI, a utilization GUI, and a WorkloadGenerator that synthesizes traces from real datasets using a modified Slot Weight Method with 48 half-hour slots per day (Galleguillos et al., 2018). This makes experimentation itself a first-class part of the simulator design.

In plasma simulation, tooling is tied directly to resilience and in-situ analytics. The BIT1 multi-GPU framework standardizes checkpoint/restart through openPMD records and ADIOS2 engines, using BP4 for file-based checkpoints and SST for in-memory streaming to analysis and visualization processes (Williams et al., 26 Jun 2026). The related openPMD-SST work makes the same pipeline explicit: simulation ranks write an openPMD iteration, SST stages buffers in memory, openpmd-pipe can convert sst streams to bp4, and a Python consumer can render plasma profiles, per-rank particle loads, and species-count histories without interrupting execution (Williams et al., 3 Dec 2025).

Profiling and tracing are equally central. The BIT1 paper reports Nvidia Nsight Systems and AMD ROC-Profiler with Perfetto to analyze kernel timelines, overlap, and redundant mappings (Williams et al., 26 Jun 2026). The openPMD-SST paper combines gprof, perf, IPM, and Darshan to attribute time to sorting, mover, MPI, and I/O paths (Williams et al., 3 Dec 2025). The ACALSim paper exposes phase-level profiling, scheduling overheads, and parallelism degree so that ThreadManager variants can be selected or redesigned based on observed contention (Lin et al., 21 May 2026).

Some HPCSim workflows specialize in route generation and aggregate statistics rather than field or particle data. EvalNet constructs routing tables and precomputed route sequences for htsim or XML routing configurations for OMNeT++/INET, then aggregates flow completion times, throughput, and utilization data, often via SQLite-backed post-processing (Besta et al., 2021). In multiscale biomedicine, reproducibility instead comes from declarative orchestration: xMML stores the multiscale model and coupling semantics, QosCosGrid reserves and co-schedules resources, and the Application Hosting Environment standardizes deployment and authentication across sites (Groen et al., 2012). The neuromuscular framework adds a parallel binary output format with optional ZFP compression and a MegaMol-based visualization environment for distributed rendering and interactive analysis of large outputs (Bradley et al., 2018).

5. Reported performance and scaling evidence

The performance evidence associated with HPCSim is as heterogeneous as its mathematical content. Some systems are evaluated against competing simulators, some against commercial software, some against hardware, and some against operational constraints such as coupling overhead or I/O latency.

System	Reported scale or performance	Interpretation
AccaSim (Galleguillos et al., 2018)	MetaCentrum, 5,731,100 jobs: total time ≈ 06:23, memory avg ≈ 19 MB, max ≈ 19 MB	Very low memory use on million-job traces
CUDA American-option HPCSim (Cvetanoska et al., 2012)	At 1,000,000 paths, CPU >6500× slower than GPU; GPU can simulate more than one million paths in approximately 0.15 microseconds, “as reported”	Extreme reported GPU speedup; throughput claim reproduced verbatim
BIT1 multi-GPU PIC/MC (Williams et al., 26 Jun 2026)	On Frontier up to 800 GPUs: strong scaling speedup 9.13× and weak-scaling PE 88.0% with SST + PLB	Resilience and scalable I/O improve scaling under checkpointing
Hybrid PDAE power-electronics HPCSim (Shi et al., 17 Jan 2025)	Converter optimization with 160 devices: >5 days reduced to <2 hours, up to 60× speedup; full PDAE with ~5 million DOFs per time step and ~2,600 time steps	Full-converter carrier-level studies become tractable
ACALSim-based GPU HPCSim (Lin et al., 21 May 2026)	14.3× speedup over SST at GEMM $T_{st}$5; 41% lower memory footprint; LLaMA-7B and LLaMA-13B layers in 17.7 min and 30.4 min	Framework overhead, not timing-core differences, is the comparison target
Biomedical multiscale framework (Groen et al., 2012)	Coupling overhead typically 1–10%; HemeLB–pyNS overhead 1.2% at 512 cores and ~4–5% at 2048 cores	WAN-coupled multiscale execution can remain a small fraction of runtime
Neuromuscular OpenCMISS workflow (Bradley et al., 2018)	Overall runtime improved by up to 2.6; good scalability on up to 768 cores	Algorithmic and partitioning changes dominate gains
Shared-memory HPS Helmholtz solver (Beams et al., 2018)	For $T_{st}$6 unknowns, build time drops from ~10 minutes to ~36 seconds, a 17.5× speedup	Strong node-level acceleration for direct high-order Helmholtz solves

The interpretation of these figures is domain-specific. AccaSim’s salient result is low memory footprint under large trace workloads, not floating-point throughput (Galleguillos et al., 2018). The ACALSim GPU HPCSim isolates framework overhead by reusing identical timing cores in both the ACALSim and SST versions (Lin et al., 21 May 2026). In the power-electronics work, the key claim is not just speedup, but preserving “carrier-level accuracy” while enabling experiments over 160 PDE-modeled devices (Shi et al., 17 Jan 2025). In the biomedical case, the critical metric is coupling overhead rather than raw solver speed (Groen et al., 2012).

6. Limits, boundaries, and common misconceptions

A common misconception is that HPCSim denotes a single standardized simulator. The surveyed literature does not support that reading. Instead, the term attaches to domain-specific systems with sharply different fidelity targets, numerical kernels, and execution models. This suggests that any reference to HPCSim should be interpreted through its immediate disciplinary context rather than through a presumed shared software lineage.

The limitations reported by the papers are correspondingly local. AccaSim focuses on WMS behavior and does not model interconnect/network topology, contention, communication latencies, or storage/I/O behavior; power, energy, and failures are absent by default unless integrated through AdditionalData (Galleguillos et al., 2018). The American-option GPU implementation avoids Longstaff–Schwartz regression by using a continuation proxy and explicitly states that the resulting algorithm yields an upper bound for American call values; it also provides no confidence intervals or benchmark error comparisons (Cvetanoska et al., 2012). The BIT1 plasma framework is presently a 1D3V electrostatic code base; PLB is checkpoint- or start-triggered rather than continuously dynamic, and fault tolerance still relies on checkpoint/restart rather than process replication or kernel-level checkpointing (Williams et al., 26 Jun 2026).

The same pattern holds elsewhere. The hybrid power-electronics framework notes coupling stiffness, memory-bound Jacobian assembly, and nested synchronization costs in the Gauss–Seidel iteration (Shi et al., 17 Jan 2025). The ACALSim GPU HPCSim is explicitly “cycle-approximate,” and timing-model accuracy remains the responsibility of simulator developers rather than the framework itself (Lin et al., 21 May 2026). EvalNet’s htsim path cannot faithfully model adaptive per-switch routing because htsim lacks per-switch state, and precomputing many routes becomes a memory bottleneck in high path-diversity topologies (Besta et al., 2021). NWQ-Sim is constrained by the standard exponential ceilings of state-vector and density-matrix simulation, while the single-node Quantum Volume comparison reports practical state-vector limits around 32–33 qubits on the tested hardware (Suh et al., 2024, Niekerk et al., 2024).

A plausible implication is that HPCSim is best understood not as a named product category but as a design stance. In the works surveyed here, that stance combines explicit fidelity choices, parallel decomposition matched to hardware, careful treatment of data movement and I/O, and built-in instrumentation for scaling analysis. The concrete realization may be a job-dispatch simulator, a PIC/MC code, a PDAE co-simulation environment, a GPU architecture simulator, or a quantum-circuit engine, but the governing question remains the same: how to retain the structure that matters scientifically while making the computation operational at scale.