GPU-Native Compilation Techniques

• GPU-native compilation is the process of converting high-level program representations directly into GPU-executable code that fully leverages parallelism and memory hierarchy.
• It employs multi-layered pipelines with custom intermediate representations and auto-tuning methods to optimize energy, load balancing, and overall runtime efficiency.
• Applications in scientific simulations, graph processing, and quantum chemistry demonstrate significant speedups and performance improvements using GPU-native techniques.

GPU-native compilation is the process of transforming high-level program representations or domain-specific languages directly into executable code that runs natively on GPU hardware—without intermediate CPU-oriented stages or reliance on external source modification. This compilation paradigm encompasses pipelines from JIT and AOT compilation, energy-aware kernel generation, and functional, imperative, or scheduling-driven IR transformations. GPU-native compilation aims to fully exploit the device’s parallelism, memory hierarchy, and instruction set, ensuring high performance, portability, and programmable flexibility across heterogeneous GPU architectures.

1. End-to-End GPU-Native Compilation Pipelines

GPU-native compilation frameworks feature multi-layered compilation pipelines covering diverse programming ecosystems and targets.

• MC/DC Monte Carlo Transport: MC/DC implements a Python → Numba → vendor CUDA/HIP toolchain stack. Python transport kernels are decorated and JITed by Numba, which emits LLVM IR. The Harmonize layer connects to NVCC (NVIDIA) or hipcc (AMD/ROCm), controlling device and host code generation. For NVIDIA, Numba’s CUDA IR is converted to PTX and compiled to a .so via NVCC relocation/linking steps. For AMD, the process involves LLVM IR manipulation, bundling, and HIP backends. Custom extensions include Numba-HIP patches for device intrinsics, but vendor compilers drive lowering to final code (Morgan et al., 9 Jan 2025).
• Strategy-Preserving Functional Compilation: DPIA enables high-level, data-parallel functional programs to be compiled to OpenCL or CUDA via type-safe, race-free rewriting. The process applies multi-stage translation: (i) acceptor- and continuation-passing, (ii) lowering functional combinators (map, reduce) into imperative (parfor, for), and (iii) substitution of data layout and indexing, resulting in verified imperative kernels with preserved semantics (Atkey et al., 2017).
• Automatic Kernel Generation in DSLs: Frameworks such as GraphIt/GG, IrGL, and SGAP start from a high-level, algorithm-oriented DSL plus a GPU scheduling language. They lower programs to IRs, analyze dependencies and parallelism, and generate device kernels with automatically applied load balancing, atomics, kernel fusion, or segmented reductions, yielding competitive or superior performance to hand-written CUDA (Brahmakshatriya et al., 2020, Pai et al., 2016, Zhang et al., 2022).
• JIT-Driven Domain Kernels: Quantum chemistry codes employ runtime code generators emitting CUDA with problem-specific specializations (basis size, angular momentum, precision). NVRTC compiles these kernels at runtime, yielding highly specialized, cache-efficient kernels with speedups over monolithic, AOT-compiled baselines (Wu et al., 13 Jul 2025).

The following table summarizes several diverse GPU-native compilation pipelines:

Framework	Input Language	Compilation Layers	Final Device Code
MC/DC	Python	Numba → Harmonize → NVCC/HIPCC	PTX / GCN/HSA kernel
DPIA	Functional (DPIA)	Type-system → imperative IR → OpenCL/CUDA	OpenCL kernel
GraphIt/GG	DSL + Schedule	SSA IR → Analysis → CUDA Codegen	CUDA kernel
Quantum Chemistry (xQC)	Python/CUDA	Template C++/Python → NVRTC (JIT)	PTX kernel

2. Intermediate Representation and Scheduling

GPU-native compilation relies on explicit IRs and scheduling abstractions that bridge the gap between high-level algorithms and hardware-oriented kernels.

• IrGL: An IR for irregular, data-driven programs, supporting ForAll, Atomic, and Exclusive constructs, systematically lowered via the Galois GPU compiler into CUDA kernels with safe lock protocols, occupancy tuning, worklist analysis, and cooperative conversion passes (Pai et al., 2016).
• GraphIt/GG: GraphIt introduces an SSA IR and a GPU scheduling language supporting SimpleGPUSchedule/HybridGPUSchedule, exposing parameters for load balancing (ETWC, EdgeBlocking), kernel fusion, and direction optimization. These are applied as IR transformations during lowering (Brahmakshatriya et al., 2020).
• SGAP’s Segment Group: Extends existing sparse compilers (e.g., TACO) with atomic-parallelism and segment group abstractions, mapping IR reductions onto hardware shuffle and reduction primitives. Schedules parameterize reduction width, strategy, and synchronization granularity, permitted as first-class IR constructs (Zhang et al., 2022).
• DPiA: Functional-level strategies (map/reduce) are preserved in the imperative IR through a formally-verified, strategy-preserving lowering that exposes parallel loops, storage allocation, and memory hierarchy in the generated code (Atkey et al., 2017).

These IRs serve as the locus for performance tuning, correctness preservation, and adaptability to hardware features without requiring low-level manual intervention.

3. Runtime Management, Memory, and Synchronization

GPU-native frameworks ensure that device code is efficiently managed, memory is utilized for coalescing and caching, and synchronization and transfers are abstracted or minimized.

• MC/DC: Particle data resides in two global-memory banks, which are swapped after each event kernel launch. Synchronization between steps uses device-wide barriers (cudaDeviceSynchronize). On memory-sharing APU hardware (MI300A), allocation and transfer steps are no-ops, as device and host share memory (Morgan et al., 9 Jan 2025).
• GraphIt/GG: Device code allocates memory for graph frontiers, uses shared memory buffers for dynamic work queues (ETWC), and applies data-locality transformations (EdgeBlocking) to minimize cache misses and random global-memory traffic. Atomics are inserted automatically for safe parallel reduction (Brahmakshatriya et al., 2020).
• OpenMP 5.1 GPU Runtime: Allocation, atomic updates, and synchronization are mapped to the underlying device memory and barrier instructions (e.g., loader_uninitialized, atomic compare/capture, __syncthreads) via extensions for portable correctness and performance parity with CUDA/HIP (Tian et al., 2021).
• Legacy Code (GPU First): Automatic migration of program memory spaces from CPU to GPU, including stack, globals, and heap (partial libc on device), is performed transparently during GPU-First compilation. Host-device synchronization for system calls uses managed memory buffers and RPC rendezvous (Tian et al., 2023).

4. Search-Based and Stochastic Kernel Optimization

Advanced GPU-native compilation incorporates automated search, scheduling mutation, and runtime tuning mechanisms:

• Energy-Aware Search-Based Compilation: The framework in (Zhang et al., 28 Nov 2024) formulates candidate CUDA/OpenCL kernel schedules in terms of tiling, mapping, vectorization, and memory annotation. A genetic search filters for low latency, then minimizes energy via a dynamic XGBoost model. The energy model is updated on-the-fly via a small set of on-device NVML measurements per generation, reducing measurement cost and accelerating convergence.
• SIP (Stochastic Instruction Perturbation): SIP operates at the SASS (native GPU ISA) level, defining a search space over memory instruction schedules. Using stochastic simulated annealing, SIP prunes this space with data-dependency analysis and acceptance criteria tied to runtime reductions. Practical speedups of 6–12% over prior hand-tuned kernels are reported; optimized SASS is deployed as a post-link plugin (He et al., 25 Mar 2024).
• Auto-Tuning and Segment Group: Sgap’s segment group approach in SpMM provides a tunable parameter r (synchronization group size) and reduction strategies, auto-tuned for each tensor workload. Fine-grained adaptation (e.g., number of writers per reduction) eliminates warp-waste and improves load balancing (Zhang et al., 2022).

These methods complement static scheduling by embracing both search and stochasticity to further close the gap to device capacity.

5. Portability, Extensibility, and Language Integration

GPU-native compilation frameworks increasingly emphasize portability, extensibility, and integration with host ecosystems:

• OpenMP 5.1 Device Runtime: Nearly the entire GPU runtime is written in portable OpenMP, with a handful of vendor intrinsics. Device code is built by LLVM/Clang, bundled with host-side objects, and works across NVIDIA and AMD hardware without vendor SDKs. Portability is achieved via #pragma omp declare variant and a minimal set of builtins (e.g., atomicInc) (Tian et al., 2021).
• Cross-Front-End Compilation (GPU First, RLLVMCompile): Legacy C/C++ codes and R programs are migrated to GPUs without source changes. This is accomplished by wrapping, IR analysis, device-oriented linking, or direct AST→LLVM IR→PTX/AOT codegen stacks, thus accelerating complex workflow migration and experimentation (Tian et al., 2023, Lang, 2014).
• DSL and Functional Integration: High-level DSLs (GraphIt, Triton, DPIA) are compiled to GPU-native code with schedule-defining languages or annotations. The Triton ML stack provides multi-level lowering (workgroup, warp, intrinsic) with user-definable tiling and warplevel APIs, matching the logical/physical hierarchy of GPU hardware (Wang et al., 19 Mar 2025).
• Probabilistic and Neural Compilation: Theoretical work formalizes parallelized, fully GPU-native edit–compile–test cycles, including neural sequence-to-bytecode models with probabilistic verification, resulting in 10–100× speedup over CPU workflows and new trade-offs between determinism and parallelism (Metinov et al., 12 Dec 2025).

6. Performance Metrics, Evaluation, and Trade-Offs

Empirical results across frameworks demonstrate the impact of GPU-native compilation:

• MC/DC: Achieves up to 15× speedup for multi-group neutron transport and up to 5× for continuous-energy transport versus 112-core CPU, with variations across GPU hardware reflecting device memory bandwidth and kernel divergence (Morgan et al., 9 Jan 2025).
• Energy-Aware Kernels: The search-based approach reduces energy by up to 21.7% with ≤5% latency penalty, outperforming cuBLAS in several MM/MV/CONV benchmarks on A100 and RTX 4090 (Zhang et al., 28 Nov 2024).
• Functional DPIA: Semantics- and strategy-preserving translation generates OpenCL code with ≤5% slowdown relative to ad-hoc and frequently matches or outperforms vendor libraries on various BLAS ops (Atkey et al., 2017).
• Quantum Chemistry (xQC): JIT-compiled, problem-specific kernels deliver 2–4× speedup in FP64 and up to 16× in FP32 relative to baseline; JIT specialization results in compact, highly-tuned code (Wu et al., 13 Jul 2025).
• Graph Algorithms: GG achieves up to 5.11× speedup in graph kernels versus established frameworks, with ETWC and EdgeBlocking expanding the space of optimizations available via scheduling DSL (Brahmakshatriya et al., 2020).
• SIP Autotuning: SASS-level stochastic autotuning offers additional 6–12% throughput despite being evaluated on already hand-tuned Triton kernels (He et al., 25 Mar 2024).

Significant trade-offs include (i) search and auto-tuning time vs. kernel efficiency, (ii) monolithic vs. decomposed (fine-grained) kernel design with thread divergence, (iii) JIT specialization overhead vs. runtime flexibility, and (iv) probabilistic correctness in neural pipelines vs. strict determinism in traditional compilers.

7. Future Directions and Theoretical Implications

Theoretical models predict substantial gains for GPU-native workflows:

• GPU-Controlled Iteration: Collapsing CPU–GPU boundaries and hosting compilation/verification loops inside GPU memory enable rapid code exploration, reducing per-iteration latency to milliseconds and overall energy by orders of magnitude (Metinov et al., 12 Dec 2025).
• Hybrid and Probabilistic Verification: Combining traditional compilation and neural translation in hybrid GPU-native compilers allows guaranteed-correct fallback for hard problems and fast approximate coding for simpler patterns, with quantifiable probability of success and scalable parallel code evaluation (Metinov et al., 12 Dec 2025).
• Portability and Standardization: The path towards shipping OpenMP offloading in generic distributions, adopting device-specific DSLs as first-class scheduling/optimization languages, and standardizing extensions in mainstream compilers is enabled by these advances (Tian et al., 2021, Wang et al., 19 Mar 2025).
• Analog/Nuromorphic Hardware: The theoretical framework generalizes to future analog computing substrates, where compilation and iteration are measured in physical reconfiguration and energy/power units, with probabilistic testing dominating correctness guarantees (Metinov et al., 12 Dec 2025).

The GPU-native compilation paradigm, spanning IR innovation, device-oriented tuning, and theoretical modeling, underpins a new generation of high-performance, portable, and extensible compute frameworks for both scientific and AI workloads.