Iceberg: Synthetic Augmentation for HLS Modeling

• Iceberg is a synthetic data augmentation framework that improves HLS modeling by generating diverse, HLS-compatible kernels and exploring broad design configurations.
• It uses an LLM for generating program variations and a GNN ensemble with MC-Dropout to produce weak labels, thus reducing reliance on costly HLS compilations.
• Empirical results demonstrate significant reductions in mean squared error and enhanced offline design-space exploration compared to established HLS modeling methods.

Searching arXiv for the specified paper and closely related HLS modeling work. Iceberg is a synthetic data augmentation framework for High-Level Synthesis (HLS) surrogate modeling and, by extension, the synthetic dataset produced by that framework. In the problem setting addressed by the work, HLS compiles C/C++-like kernels into hardware designs while design configurations such as loop unrolling, pipelining, and tiling determine quality-of-result (QoR) metrics including latency, throughput, and resource usage. Iceberg is designed to reduce the generalizability gap of deep learning models for HLS modeling by expanding both the program dimension $P$ and the design-configuration dimension $T$, then pretraining an in-context prediction model on actual and weakly labeled synthetic data before few-shot adaptation to real applications (Ding et al., 14 Jul 2025).

1. Problem setting and motivation

HLS modeling aims to predict QoR from a program and a pragma configuration, thereby replacing repeated HLS tool executions that can take minutes to hours. In the notation used in the work, the program dimension is denoted by $P$, representing different kernels, and the design configuration dimension is denoted by $T$, representing different pragma combinations for a given program. Existing machine-learning models for HLS, including HARP and Hierarchical-MoE, are described as struggling to generalize to unseen programs $P$ and unseen design configurations $T$, because existing datasets do not cover sufficient diversity along either axis (Ding et al., 14 Jul 2025).

The framework targets two bottlenecks. First, public C/C++ code is abundant but often not syntactically or semantically compatible with HLS toolchains, or is unsuitable for acceleration. Second, dense exploration of pragma combinations is expensive because each label ordinarily requires a full HLS compile. Iceberg addresses these constraints by generating synthetic yet HLS-compatible programs with a LLM and by producing weak labels for unseen configurations using learned surrogate models rather than re-running HLS. This suggests that pretraining on a broad synthetic corpus can induce a stronger prior for downstream adaptation than supervised learning on small, application-specific HLS datasets alone (Ding et al., 14 Jul 2025).

2. Dual meaning of Iceberg: framework and dataset

Iceberg denotes both a method and a dataset. As a method, it is a synthetic data augmentation framework organized around two generation axes. Along the program dimension $P$, it uses an LLM to generate diverse HLS-compatible kernels. Along the design-configuration dimension $T$, it samples unseen pragma settings for each program and labels them with weak labels generated by a GNN ensemble with MC-Dropout. These actual and weak labels are then used to pretrain an in-context meta-model called G-TNP (Ding et al., 14 Jul 2025).

As a dataset, Iceberg includes a large corpus of synthetic programs plus associated labels. The paper reports that approximately 4400 programs were generated, about 3401 were synthesizable, and 214 of these received actual HLS labels because of compute budget constraints. For those 214 programs, AutoDSE and Vitis HLS were used to collect 14,840 labeled design configurations. This actual-labeled subset constitutes the core Iceberg dataset, and weakly labeled extensions over additional configurations form the hybrid training set used in the strongest model variants (Ding et al., 14 Jul 2025).

3. Program-space expansion with LLM-generated HLS kernels

For the program axis, Iceberg uses an LLM with prompt engineering to generate C programs that omit main, expose a single entry function top, use constant loop bounds, and satisfy the syntax and semantic requirements of downstream tools such as Vitis HLS and Merlin. The prompts request diversity in loop structure, function structure, memory footprints, and application domains, including scientific kernels, cryptography, and signal processing (Ding et al., 14 Jul 2025).

A notable component is iterative generation. Previously generated programs from a domain are shown to the model, which is then prompted to generate novel variants significantly different from those examples. The stated purpose is to reduce redundancy, such as repeated generation of the same canonical kernel for a domain. Generated programs are then validated, and a subset undergoes AutoDSE plus HLS to obtain actual labels. This actual-label step is expensive but anchors the synthetic corpus to the real QoR function used by the downstream toolchain (Ding et al., 14 Jul 2025).

The training corpus is described as deliberately structurally and numerically diverse. This design is intended to reduce domain shift when adapting to HLSyn programs and to real-world applications such as ML kernels, video processing, and 3D rendering. A plausible implication is that program diversity in Iceberg functions analogously to large-scale pretraining corpora in other ML domains, but here the diversity is engineered to remain compatible with HLS synthesis constraints (Ding et al., 14 Jul 2025).

4. Weak labels, synthetic functions, and the G-TNP architecture

The second axis of expansion densifies the configuration space $T$ without paying the full HLS cost. Let $\mathcal{A}$ denote the actual-labeled set: $T$0 where $T$1 is the true QoR from HLS. GNN surrogates are trained on $T$2 with mean-squared error: $T$3 After training, ensemble members with random dropout masks act as synthetic functions that assign approximate QoR values to unseen configurations $T$4. For each sampled program $T$5 and synthetic function $T$6, the method constructs

$T$7

In the reported experiments, $T$8 such program-function task pairs were sampled and $T$9 configurations were generated for each (Ding et al., 14 Jul 2025).

These weak labels are noisy by construction, but the paper emphasizes that they are generated by learned functions similar to the real HLS function, rather than by unrelated priors such as random Gaussian processes. The hybrid training variants denoted Ice-H use a 50:50 ratio of weak to actual labels during training. The authors report that weak labels from a single synthetic function are insufficient and that diversity across multiple synthetic functions $P$0 is crucial for generalization (Ding et al., 14 Jul 2025).

The pretrained predictor is G-TNP, a GNN encoder plus Transformer Neural Process. Because an HLS kernel with $P$1 loops can induce $P$2 pragma parameters, raw configuration vectors are variable-length across programs. A HARP-like GNN therefore encodes each program-configuration pair into a fixed-dimensional representation $P$3. The TNP then models a conditional predictive distribution over target labels given context embeddings and labels: $P$4 The Transformer encoder uses 6 layers, 128-dimensional hidden states, 8 attention heads, and no positional embeddings, which makes the model permutation invariant over context and target sets. At test time, a few labeled configurations from a new application are supplied as context, enabling in-context adaptation without parameter updates, with optional fine-tuning for further improvement (Ding et al., 14 Jul 2025).

5. Training pipeline and adaptation protocol

The full pipeline consists of six stages. Synthetic programs are first generated and validated. A subset then receives actual labels via AutoDSE and HLS runs. GNN surrogates are trained on those actual labels. An ensemble plus dropout produces weak labels for additional sampled configurations. Actual and weak labels are combined into a hybrid dataset. Finally, G-TNP is pretrained on that hybrid dataset and then adapted to new real applications using few-shot context points and, optionally, fine-tuning (Ding et al., 14 Jul 2025).

At adaptation time, around 50 sampled design configurations per test program are synthesized through Vitis HLS to obtain true QoR. These few-shot samples serve as context for in-context inference. Optional fine-tuning variants, Ice-A-FT and Ice-H-FT, perform 200 MSE-optimization steps on those few-shot samples. The paper distinguishes in-context-only variants, Ice-A and Ice-H, from in-context-plus-fine-tuning variants, Ice-A-FT and Ice-H-FT (Ding et al., 14 Jul 2025).

The meta-learning view is explicit. Each pair $P$5 defines a task over configuration embeddings and labels, and G-TNP is trained over many such tasks by random context-target splits. This is analogous to MAML-style meta-learning in its objective of rapid adaptation, but implemented through amortized inference over context sets rather than gradient-based inner loops. This suggests that the method is designed not merely to regularize a supervised predictor, but to learn a family of HLS response functions across programs and synthetic label generators (Ding et al., 14 Jul 2025).

6. Empirical performance, baselines, and ablations

The paper reports substantial gains in few-shot HLS modeling accuracy and offline design-space exploration. On six real-world applications, Iceberg improves the geometric mean modeling accuracy by $P$6, operationalized as a large reduction in geometric mean MSE relative to Hierarchical-MoE. Reported geometric mean MSE values on six real programs are approximately 1.26 for HARP, 0.92 for H-MoE, 0.17 for Ice-A, and 0.13 for Ice-H. This corresponds to roughly a sevenfold geomean MSE reduction from H-MoE to Ice-H (Ding et al., 14 Jul 2025).

For HLSyn test programs, Iceberg pretraining with fine-tuning yields about 31–38% geometric mean MSE reduction relative to H-MoE. In offline DSE evaluation, where models rank presynthesized candidates and best@1 is measured using actual latencies, Iceberg-based models show up to $P$7 and $P$8 better performance on two test datasets. The paper interprets “$P$9 better” as identifying a design whose latency is about $T$0 lower than the one selected by the baseline under the same offline protocol (Ding et al., 14 Jul 2025).

The principal supervised baselines are HARP and Hierarchical-MoE. Ablation studies isolate several effects. Pretraining HARP on the Iceberg dataset before fine-tuning on HLSyn reduces test MSE from 0.21 to 0.04, an improvement of about 80%. On HLSyn-only pretraining, adding weak labels reduces MSE versus Ice-A by about 55%, mitigating overfitting caused by sparse configuration coverage. On the full Iceberg pretraining corpus, Ice-H still consistently outperforms Ice-A, though the gap is smaller because actual-label diversity is already greater. Another ablation compares GNN-based synthetic functions with random Gaussian processes and finds the GP-based weak labels lead to poor performance and overfitting, reinforcing the importance of function-class alignment between the synthetic label generator and the real HLS response (Ding et al., 14 Jul 2025).

7. Limitations, assumptions, and significance

Several limitations are acknowledged. The approach depends on LLM-generated code quality; synthetic programs must be syntactically correct, synthesizable, and meaningful for HLS. Weak-label quality is also a limiting factor: the reported GNN ensemble has MSE of about 0.047 on HLSyn but about 0.321 on Iceberg, implying that weak labels are noisier precisely on the more diverse synthetic corpus. When weak labels are too inaccurate, the gains of Ice-H over Ice-A diminish (Ding et al., 14 Jul 2025).

The framework is specific to a toolchain centered on Vitis HLS 2023.2, Merlin, and pragma spaces involving PARALLEL, PIPELINE, and TILE. DSE evaluation is offline rather than sequential online optimization. Training a GNN ensemble to generate synthetic functions is itself computationally expensive, even if cheaper than exhaustive HLS compilation. The work also notes that G-TNP can severely overfit on small training datasets without synthetic augmentation, and that improperly chosen synthetic functions can be harmful (Ding et al., 14 Jul 2025).

Within ML for EDA and ML for systems, Iceberg represents an overview of three ideas: synthetic program generation, weak labeling through related surrogate functions, and in-context meta-learning. The paper explicitly suggests extensions to other EDA problems, including floorplanning, placement and routing, timing and power prediction, and design-rule checking. A plausible implication is that Iceberg’s central contribution is not only a particular HLS dataset, but a template for generating pretraining corpora when labels are expensive and distribution shift is severe (Ding et al., 14 Jul 2025).