MailoHLS: Multi-Adapter Structure-Aware Learning for Pareto-Driven HLS Pragma Optimization

Abstract: High-Level Synthesis (HLS) enables rapid development of FPGA accelerators, yet achieving high-quality results (QoR) remains challenging due to the large and irregular design space induced by compiler directives (a.k.a pragmas). Selecting effective configurations requires reasoning over complex interactions between program structure, memory behavior, and often conflicting objectives such as latency and resource utilization. Prior model-driven approaches exhibit limited generalization across kernels and fail to capture higher-level optimization intent. Recently, LLMs capture code semantics and high-level intent, but their sequential representations hinder modeling of structural dependencies and global trade-offs, leading to suboptimal HLS designs. We present MailoHLS, a hybrid framework that combines LLM-based semantic reasoning with GNN-based structural modeling for objective-aware directive optimization. By integrating structural embeddings via cross-attention and leveraging PEFT with objective-conditioned LoRA adapters and Pareto-driven optimization, MailoHLS enables joint reasoning over code semantics, structure, and design trade-offs. Across seen and unseen kernels, MailoHLS achieves up to 12.42x and 8.4x speedup (9.48x and 4.97x geometric mean) for latency optimization, consistently producing near-Pareto-optimal designs. On fully unseen applications, it reaches up to 10.2x speedup (6.58x geometric mean), outperforming high-end LLMs and prior approaches while narrowing the gap to the Pareto frontier.

• The paper introduces MailoHLS, a hybrid framework that integrates LLM-based semantic representations with GNN structural embeddings for directive-level HLS pragma optimization.
• It employs cross-attention fusion and LoRA-based adapters to balance latency and resource usage, demonstrating strong QoR improvements in both seen and unseen kernels.
• Extensive experiments show that MailoHLS outperforms state-of-the-art methods by achieving up to 10.2× speedup while ensuring valid hardware constraints.

Multi-Adapter Hybrid Structural-Semantic Modeling for Pareto-Optimal HLS Pragma Optimization

Introduction

High-Level Synthesis (HLS) is the dominant approach for mapping high-level code (C/C++) to FPGA hardware, but optimal pragma selection for Quality-of-Results (QoR) objectives like latency and resource utilization remains an intractable manual or computation-heavy process. The pragma design space is combinatorially large, with directive dependencies that are highly non-local and program-structure-sensitive. Prior DSE and model-driven approaches suffer from limited generalization, while LLMs capture intent and code semantics but are inherently structure-agnostic and often violate non-functional hardware constraints. The paper introduces MailoHLS, a unified framework for HLS pragma optimization that fuses LLM-based semantic representations with GNN-derived embeddings of program structure via cross-attention, using PEFT with LoRA for efficient multi-objective adapter specialization. MailoHLS reframes HLS optimization into a directive-level structured prediction problem, achieving strong QoR across seen and unseen kernels, with robust objective-conditioned generalization.

Structural and Semantic Reasoning in HLS Optimization

MailoHLS is motivated by the observation that HLS performance optimization is fundamentally a structural reasoning task (Figure 1). Pragmas are not independent: for example, loop unrolling is only performance-optimal when matched with sufficient memory partitioning, else memory bandwidth bottlenecks degrade throughput. Existing GNN-based surrogates encode explicit program structure and dependencies, but lack code semantic abstraction and intent modeling. LLMs provide strong code modeling abilities, but without explicit structure fail to capture hardware-side constraints, resulting in invalid or inefficient pragma assignment, particularly as kernel complexity grows (Figures 2, 4).

Figure 1: GNNs encode control/data dependencies and program structure via iterative message passing.

Structural reasoning is complemented with semantic embeddings: intents (e.g., minimize latency, minimize BRAM/LUT/DSP area) and higher-level semantic cues in control/data flow must be incorporated for objective-aware trade-offs.

Figure 2: GEMV kernel—pragmas (colored nodes) interact structurally and must be coupled to loop topology and memory accesses for valid partitioning and unrolling.

MailoHLS constructs ProGraML-like hierarchical graphs from LLVM IR, explicitly modeling action points for pragma insertion, array accesses, and loop hierarchy (Figure 3, Figure 4). Auxiliary nodes compress long-range dependencies, critical for resource feasibility and parallelism analysis.

Figure 3: MailoHLS architecture—GNN encodes structural graph, LLM encodes code semantics, cross-attention fuses both with per-objective LoRA adapters.

Figure 4: GEMV kernel’s graph representation, showing how structural and optimization nodes are connected.

Hybrid Model Architecture

MailoHLS consists of three main submodules:

1. Directive Placeholder Insertion: Code is tokenized and annotated with placeholders marking valid action points for HLS directives.
2. Structural Embedding via GNN: A separation-trained GNN encodes IR-derived graphs, learning node embeddings that capture resource pressure, parallelism, and memory access patterns. These intermediate representations are exported as structural memory aligned with the token level.
3. Semantic Encoding + Structural Fusion: A decoder-only LLM (e.g., DeepSeekCoder) encodes semantic context and performs directive value prediction. Structural embeddings are fused at selected transformer layers via token-structure cross-attention, gated for relevance, and only for tokens aligned to pragma placeholders. Multiple LoRA adapters enable specialization for latency/resource/balanced objectives.

   Figure 5: LLM backbone training pipeline with SFT, DPO, and LoRA adapters for objective-conditioned fine-tuning.

   

   Figure 6: MailoHLS performance vs. LLM backbone size—QoR improvement saturates, confirming fusion (not just backbone scale) is essential.

Training Paradigm and Objective-Aware Adapters

Training leverages the GN$\Omega$SIS dataset of 219K synthesized design points. For each objective:

• Stage 1: SFT (supervised fine-tuning) on LoRA; train for directive assignment from labeled, evaluated design points.
• Stage 2: Freeze LoRA, enable cross-attention and train only structural fusion (no value head), aligning GNN and LLM representations.
• Stage 3: DPO (Direct Preference Optimization), use Pareto-ranked pairs to bias towards target objective, optimizing only cross-attention and gating.

This pipeline disentangles code meaning, structural logic, and ultimate trade-off requirements, yielding adapters with explicit control over latency-resource-balance regimes.

Evaluation and Ablation

MailoHLS demonstrates:

• Seen Kernels: Geometric mean speedup of $\mathbf{9.48\times}$ for latency (vs. $12.42\times$ Pareto ref.), with tight resource usage ($9.35\%$ vs. $9.75\%$); balanced and area objectives also closely match Pareto/knee points.
• Unseen Kernel Families: Robust generalization—latency-optimized adapter achieves $4.97\times$ speedup (with substantially lower resource use than reference designs), balanced adapter retains objective sensitivity.
• Fully Unseen Applications: Up to $10.2\times$ speedup ($6.58\times$ geometric mean), outperforming high-end LLMs and prior model/data-driven DSE baselines, which often generate invalid or non-feasible designs.

Figures 8 and 9 (Latency-Optimized Adapters) show MailoHLS objectives: tight Pareto tracking for seen and unseen domains.

Figure 7: Latency-optimized adapter—MailoHLS approaches Pareto region with explicit resource/speedup trade-off.

Figure 8: Balanced adapter yields competitive speedup with moderate resource cost, outperforming area-only policies in robustness.

Ablation on the complex Kalman filter kernel indicates: SFT alone yields subpar QoR and often violates resource constraints; adding GNN-based cross-attention achieves the bulk of feasible speedup gains; DPO finalizes objective alignment (Figure 9).

Figure 9: Training ablation on Kalman filter—main QoR improvement is from structural fusion; DPO sharpens objective-alignment.

Comparisons and Implications

Against prior approaches (e.g., CollectiveHLS, LIFT, and recent instruction-prompted LLM methods):

• General-purpose LLMs—incapable of structurally valid and resource-feasible pragma assignment in complex kernels.
• Data-driven DSE—can hit high single-point speedup but often violate area or synthesis constraints.
• LIFT—structurally aware, but lacks MailoHLS' explicit objective adapters and strong per-objective specialization.
• Only MailoHLS generates three explicit operating points per kernel, delivering not just high throughput, but structured, interpretable control of the latency-resource trade-off.

MailoHLS shows that the fusion of structural/semantic models is essential—simply scaling LLMs or “instructing” them with better prompts does not yield reliable, objective-aware, synthesizable solutions in hardware DSE contexts.

Conclusion

MailoHLS defines a new state-of-the-art paradigm for HLS pragma optimization: hybrid GNN-LLM architectures with explicit directive-level prediction, cross-attention fusion, and LoRA-based multi-objective specialization. It reconciles semantic understanding with non-local structural constraints, enables robust Pareto-efficient design synthesis across kernel domains, and provides objective-conditioned adapters that can be selected at inference time. Future research can explore extended objective sets (e.g., power/timing), generalization to full SoC synthesis, or more efficient architecture-agnostic embedding schemes.

MailoHLS demonstrates that systematic multi-modal fusion and adapter-based specialization are necessary and sufficient for advancing the reliability and interpretability of AI-driven HLS design automation. (2606.07246)