ACCeLLiuM: Automated OpenACC Pragma Generation

• ACCeLLiuM is an open-source resource bundle for automated OpenACC pragma generation in data-parallel loops of C/C++ code, addressing a niche in GPU programming.
• It comprises a supervised fine-tuning dataset of 4,033 pragma-loop pairs, two LLMs fine-tuned via QLoRA, and an open-source pipeline for reproducible benchmarking.
• Empirical results show significant gains in directive accuracy and clause synthesis, underscoring the impact of domain-specific fine-tuning on OpenACC tasks.

Searching arXiv for the ACCeLLiuM paper and closely related OpenACC pragma-generation context. ACCeLLiuM is an open-source resource bundle for automated OpenACC pragma generation for data-parallel loops in C/C++ code. It was introduced to address a gap in the literature: prior automation efforts focused mostly on OpenMP, while OpenACC pragma generation had lacked a public dataset and specialized LLMs. The system is explicitly scoped to the second stage of GPU-offloading assistance: given a loop already known to be parallelizable, generate the correct OpenACC pragma, including the directive and its clauses. The released artifacts comprise a supervised fine-tuning dataset, two open-weights LLMs fine-tuned for OpenACC, and an open-source pipeline for dataset creation, fine-tuning, and evaluation (Jhaveri et al., 20 Sep 2025).

1. Problem Setting and Technical Scope

The motivating problem is the increasing difficulty of effective GPU programming as GPU hardware and parallel programming frameworks become more complex. Directive-based models such as OpenACC abstract away low-level implementation details, but the construction of effective pragmas still requires expertise in both parallel execution and data movement. ACCeLLiuM targets this bottleneck directly by learning to synthesize OpenACC directives for data-parallel loops in serially written C/C++ programs (Jhaveri et al., 20 Sep 2025).

The scope is deliberately constrained. ACCeLLiuM does not identify parallelizable loops; instead, it assumes that a loop has already been recognized as a parallelization candidate. Its task is to choose the appropriate OpenACC directive and clauses for that loop. This distinction is central to the benchmark design, because it isolates pragma generation from loop-discovery and whole-program parallelization analysis.

The paper also frames the task as OpenACC-specific rather than a generic code-generation problem. That specificity matters because clause selection depends on domain features such as data residence, reduction structure, loop nesting, and concurrency control. The reported error patterns indicate that this domain specialization is not recoverable from base-model pretraining alone.

2. Composition of the Resource Bundle

ACCeLLiuM contains three primary components: the ACCeLLiuM SFT dataset, two supervised fine-tuned LLMs, and an open-source experimental pipeline (Jhaveri et al., 20 Sep 2025).

Component	Contents	Scale
ACCeLLiuM SFT dataset	OpenACC pragma-loop pairs from public GitHub C/C++ repositories	4,033 pairs
Fine-tuned models	Llama 3.1 70B; CodeLlama 34B	2 models
Pipeline	Dataset creation, fine-tuning, evaluation	Open-source

The dataset contains 4,033 OpenACC pragma-loop pairs mined from public GitHub C/C++ repositories. It is split into 3,223 training pairs and 810 test pairs. The two fine-tuned models are Llama 3.1 70B and CodeLlama 34B, both adapted with supervised fine-tuning for OpenACC pragma generation. The open-source pipeline covers the full workflow from mining and filtering examples to training and benchmarking.

This packaging is significant because the paper presents ACCeLLiuM not only as a model release but also as a reproducible benchmark. The release of code, models, and dataset is intended to support subsequent work on LLM-assisted GPU offloading.

3. Dataset Construction and Task Formulation

The dataset was constructed from public C/C++ source files retrieved through the GitHub Code Search API. The search targeted files containing #pragma acc loop and #pragma acc parallel loop. Source files were parsed into ASTs with tree-sitter; pragmas and the immediately following for loop were located, and pragma whitespace was normalized (Jhaveri et al., 20 Sep 2025).

The extraction and filtering process is central to the benchmark definition. The raw pipeline produced 25,656 pragma-loop pairs initially. After filtering out empty or infinite loops, loops with incompatible control flow such as break, goto, continue, and return, and noisy compiler-test examples from GCC/LLVM suites, the corpus was reduced to 10,503 valid pragma-loop pairs. Duplicates were then removed based on exact loop-body matches, yielding 4,033 unique examples. The final split is 3,223 training examples and 810 testing examples.

The supervised task is formulated as single-line pragma synthesis. The input is a code snippet with a <TARGET_PRAGMA_LOCATION> placeholder placed before the loop. The model must output exactly one line: the best OpenACC pragma directive for that loop. The representation used for training is a JSONL chat format.

Two examples illustrate the intended output space:

#pragma acc parallel loop present(mat[0: size*size]) reduction(+:sum)

This example uses parallel loop as the directive, present(...) for memory already on device, and reduction(+:sum) for accumulation across iterations.

#pragma acc parallel loop collapse(2) copyin(a) copyout(b)

This example encodes nested-loop parallelism via collapse(2), input transfer via copyin(a), and output transfer via copyout(b).

These examples clarify that the task is not limited to directive-type classification. It includes clause selection, clause arguments, and syntactic realization.

4. Supervised Fine-Tuning Procedure

The paper uses supervised fine-tuning with QLoRA via Unsloth. In this setup, the base models are quantized and only adapter weights are trained. Training was performed on 1× NVIDIA H100 80GB for 3 epochs in bf16 using AdamW with learning rate 6e-5, warm-up ratio 0.1, and a cosine schedule (Jhaveri et al., 20 Sep 2025).

The use of QLoRA is presented as a practical mechanism for adapting large open-weights models to a narrow compiler-adjacent code-generation task under limited hardware resources. The methodological point is not architectural novelty in the LLMs themselves, but task specialization through lightweight supervised adaptation.

The two released fine-tuned models are based on Llama 3.1 70B and CodeLlama 34B. Both are trained on the ACCeLLiuM SFT dataset to map loop-local code context to a single OpenACC pragma. The benchmark therefore measures how well supervised adaptation can inject OpenACC-specific expertise into otherwise general-purpose code models.

5. Evaluation Protocol and Metrics

Evaluation is performed on the 810 held-out test examples and measures both semantic correctness and syntactic validity (Jhaveri et al., 20 Sep 2025). Semantic evaluation uses four metrics: exact match accuracy, Levenshtein similarity, directive-type match, and clause-wise Jaccard similarity. Exact match requires identity with the reference pragma, including directive, clauses, clause order, and clause variables or arguments. Directive-type match checks whether the main directive class is correct. Clause-wise Jaccard similarity measures overlap in clause sets while ignoring order.

Syntactic validity is evaluated through minimal compilable units. For each test example, the evaluation constructs one MCU with no pragma, one with the reference pragma, and one with the generated pragma. Compilation is checked with an OpenACC-capable compiler, nvc with -acc. An important qualification is that only 762 of the 810 test loops were compilable even before adding pragmas; the remaining cases were excluded from syntax-check evaluation.

The paper explicitly distinguishes exactness from functional usefulness. Clause ordering is one source of divergence:

#pragma acc parallel loop present(val[0:gs0]) reduction(+ : sum) #pragma acc parallel loop reduction(+ : sum) present(val[0:gs0])

These are treated as functionally identical by the compiler, although exact-string metrics count them as different. The paper also notes that a generated pragma may include additional clauses that are not present in the reference yet still provide useful control over execution or data movement:

#pragma acc parallel loop copyin(a) present(b) #pragma acc parallel loop present(b) copyin(a) reduction(+:c)

This evaluation design underlies one of the paper’s main interpretive claims: exact-string matching is too strict to capture compiler-equivalent or practically useful outputs.

6. Empirical Results, Error Modes, and Interpretation

The base models perform poorly without task-specific adaptation. Llama 3.1 base has essentially 0 exact match accuracy, and CodeLlama base is also extremely weak, with only about 0.01 exact match accuracy as reported in the text. Both struggle to generate valid OpenACC pragmas consistently (Jhaveri et al., 20 Sep 2025).

After supervised fine-tuning, performance improves substantially. Fine-tuned Llama 3.1 reaches 43% exact match accuracy, 89% correct directive type, 0.63 clause-wise Jaccard similarity, and 0.77 Levenshtein similarity. Fine-tuned CodeLlama reaches 50.4% exact match accuracy in the abstract, about 50% in the results text, 87.3% correct directive type in the abstract, 87% in the results text, 0.69 Jaccard similarity, and 0.79 Levenshtein similarity. On the 762 compilable MCUs, fine-tuned Llama 3.1 compiles successfully in 83.3% of cases, fine-tuned CodeLlama in 80.9%, and the human-written reference pragmas in 88.5%.

For directive-type prediction, the paper reports precision, recall, and F1. Llama 3.1 fine-tuned obtains 55.24 / 48.80 / 49.45, while CodeLlama fine-tuned obtains 45.20 / 33.80 / 37.47. These scores indicate that strong directive-type matching does not eliminate residual errors in clause generation.

The principal error patterns are also identified. First, the models produce incorrect or partially correct clauses, especially confusion between data clauses such as present and copyin. Second, clause lists may omit variables, such as dropping one array from present(b,c). Third, the model may choose the wrong directive, for example generating kernels instead of parallel loop. Fourth, clause reordering frequently causes exact-match failures even when the output remains functionally acceptable.

The paper attributes many clause errors to missing full-program context. Because the model sees only the loop itself, it lacks surrounding information that would help determine whether data is already present on the GPU. This explanation is consistent with the observed difficulty of choosing between data-movement clauses and data-residency clauses.

A common misconception addressed by the benchmark is that non-exact outputs are necessarily useless. The reported near-miss behavior contradicts that view: outputs often use the correct directive, contain the right clauses in a different order, add useful optimization clauses, or differ only in formatting or argument details. The paper therefore treats exact match as one metric among several rather than as a complete proxy for utility.

7. Limitations, Reproducibility, and Research Significance

The limitations are explicit. ACCeLLiuM handles only single, pre-identified data-parallel loops; it performs static evaluation rather than end-to-end runtime performance assessment; the absence of full-program context likely causes many clause errors; and some human-written reference pragmas in the dataset do not compile perfectly, indicating residual label noise (Jhaveri et al., 20 Sep 2025).

These constraints define the proper interpretation of the benchmark. ACCeLLiuM is not a full automatic parallelization system, nor is it a performance autotuner. It is a narrowly defined pragma-generation benchmark and model suite. A plausible implication is that extension to whole-program OpenACC assistance would require integration with loop-identification methods, broader program context, and runtime-aware evaluation, but those steps are outside the released system.

The broader significance of ACCeLLiuM lies in three findings stated by the paper. First, base LLMs are not sufficient for expert OpenACC pragma generation. Second, domain-specific supervised fine-tuning on real code produces strong gains. Third, LLMs can learn practically useful parallelization and data-movement patterns from real-world pragma-loop pairs. Within that framing, ACCeLLiuM functions as both a benchmark and a concrete demonstration that automated OpenACC pragma generation is tractable when posed as a specialized supervised learning problem.