Courier-FPGA: HLPS for FPGA Accelerators

• Courier-FPGA is a high-level physical synthesis framework that captures hierarchical module structure and enables systematic pipeline insertion.
• It integrates HLS, handcrafted RTL, and vendor IP, supporting flexible and efficient design optimization for multi-die FPGAs.
• Empirical benchmarks show up to 62% improved operating frequency with minimal resource overhead, ensuring scalable FPGA deployments.

RapidStream IR (RIR) is a high-level physical synthesis (HLPS) infrastructure designed to streamline the development, optimization, and deployment of large-scale FPGA accelerators. RIR formally captures hierarchical module structure, interconnection protocols, pipelining, and spatial layout abstractions, enabling efficient floorplanning and frequency optimization for designs incorporating code from HLS-generated RTL, handcrafted RTL, and vendor IP on modern, multi-die FPGA devices. By providing a flexible intermediate representation and a composable pass-based transformation framework, RIR automates previously manual, labor-intensive steps such as coarse-grained partitioning, pipelining insertion, and multi-source integration—achieving substantial improvements in operating frequency and enabling portability across new device families (Lau et al., 2024).

1. Motivation and Technical Challenges

Traditional high-level synthesis (HLS) exposes algorithmic structure but omits detailed, placement- and routing-specific control, particularly when targeting multi-die FPGAs (e.g., AMD Alveo U55C, Versal VP1552). This abstraction mismatch often results in unmodeled inter-die latency and local routing congestion, degrading the achievable frequency far below hardware limits. Manual solutions—such as floorplanning, hand-inserted pipeline stages, or custom RTL wrappers—are brittle, do not generalize across device targets, and dramatically reduce productivity.

RIR addresses three key deficiencies in prior HLPS and HLS-driven tools:

• Coarse-grained Hierarchical Partitioning and Pipelinability: RIR explicitly models which module groups reside in which physical regions and supports systematic pipeline insertion (either feedforward or handshake) at any level of the module hierarchy.
• Hybrid Source Integration: RIR unifies the treatment of HLS outputs, RTL, and IP blocks, admitting system-level analysis and optimization across heterogeneous sources.
• Platform Portability and Extensibility: A virtual-device abstraction and pass infrastructure decouple synthesis logic from vendor-specific floorplanning constructs and allow extensibility for custom optimization and analysis tools.

2. Intermediate Representation Formalism

RIR defines a design $D$ as the tuple

$D = (M,\,E,\,I,\,V)$

where:

• $M = \{m_1, \dots, m_n\}$ is the set of modules, each $m$ defined as $m = (P_m,\,C_m,\,I_m,\,S_m,\,A_m)$:
  • $P_m$: Ports (direction and bit-width).
  • $C_m$: Internal connections (wires).
  • $I_m$: Interface annotations (e.g., handshake or feedforward for pipelining).
  • $S_m$: Optional spatial/slot assignments.
  • $A_m$: Metadata (e.g., resource estimates, constraints).
• $D = (M,\,E,\,I,\,V)$0: Hierarchical edges $D = (M,\,E,\,I,\,V)$1 indicating submodule relationships.
• $D = (M,\,E,\,I,\,V)$2: Channel protocols—feedforward, $D = (M,\,E,\,I,\,V)$3, and handshake, $D = (M,\,E,\,I,\,V)$4.
• $D = (M,\,E,\,I,\,V)$5: Virtual-device description—grid of slots, inter-slot wire capacities, resource budgets.

Leaf modules encapsulate opaque RTL/IP/netlist bodies. Grouped modules declare module structure, ports, and pipelinability without internal logic. By annotating grouped modules at any hierarchy, RIR supports arbitrary-granularity pipelining.

YAML-like IR snippet:

$D = (M,\,E,\,I,\,V)$7

3. Pass-Based Transformation and Optimization Pipeline

RIR employs a suite of composable transformation passes to refine its IR toward an optimized, physical netlist. The main passes are:

1. Hierarchy Rebuild: Converts large leaf modules into grouped modules with auxiliary leaves, exposing internal wiring for subsequent optimization.
2. Interface Inference: Propagates interface annotations up/down the module hierarchy.
3. Partitioning: Splits grouped modules into smaller submodules via netlist analysis.
4. Passthrough: Collapses structure-only auxiliary modules, reducing IR complexity.
5. Flattening & Grouping: Temporarily flattens hierarchy for global optimization (e.g., via ILP), then regroups pipelinable modules as needed.
6. Wrapping + Pipeline Insertion: Introduces helper modules (flip-flops or small FIFOs) to pipeline wires crossing slot/die boundaries.

The optimization core centers on a slot-based floorplanning ILP that assigns modules to slots under resource and interconnect constraints, minimizing cross-slot latency: $D = (M,\,E,\,I,\,V)$6 Subject to slot assignment, resource, and maximum cross-slot wire constraints.

4. Integration of Heterogeneous Sources and Device Abstraction

RIR supports mixed-source designs via importers:

• Verilog/VHDL: Extracts module signatures and preserves logic bodies, using tools such as Slang.
• HLS: Ingests Vitis/Catapult/Intel HLS reports or user-defined pragmas to mark pipelinable bundles.
• Xilinx IP: Retrieves interface data from the vendor's IP catalog.

Device abstraction in RIR is managed through a Python API, mapping physical regions to slot indices and assigning resource/wire budgets. This enables the same IR and passes to operate independent of physical device specifics. Exported constraint scripts and wrapper code enable integration with EDA tools and vendor flows, preserving user-authored RTL/IP.

5. Extensibility, Instrumentation, and Case Studies

The pass infrastructure admits custom passes operating on clearly separated aspects: module graphs, interface annotations, and spatial metadata. Examples include:

• Custom Network-on-Chip (NoC) Routing: Inserting routing logic by a straightforward pass modification.
• Floorplan Exploration: Varying slot resource budgets and solving the ILP to analyze Pareto trade-offs in wirelength versus load imbalance.
• Parallel Synthesis Backend: Marking slots as black boxes, synthesizing them in parallel (achieving up to 2.7× wall-clock improvement in large systolic arrays), and re-integrating slot-level netlists.
• IR-Level Debugging: Auto-inserting probe modules on handshake interfaces for performance counter instrumentation or debug builds.

6. Empirical Evaluation and Performance Outcomes

Comprehensive benchmarking targets AMD FPGAs (e.g., U250, U280, VP1552) using Vivado 2023.2. Tested applications include ConvNets, LLM (LLaMA2) accelerators, genomics accelerators (Minimap2), and K-NN, spanning hierarchical depths and source mixes.

Benchmark	Application	Device	Hier. Lvl	Source	Freq_Orig (MHz)	Freq_RIR (MHz)	Relative Δ%
CNN 13×4	ConvNet	U250	1	HLS	233	335	+44%
LLaMA2	LLM accel	U280	4	HLS/RTL/IP	150	243	+62%
Minimap2	Genomics	VP1552	3	HLS	264	285	+8%
KNN	K-NN accel	U280	2	HLS/RTL	292	292	0%

Mean improvement ranges from 7% to 62% in operating frequency across workloads, with negligible cost in LUT/FF/BRAM/DSP utilization (≤1% resource difference), matching or exceeding prior flows including AutoBridge and manual partitioning, while accommodating new devices and mixed-language, deeply hierarchical designs.

7. Prospective Directions

Future development trajectories for RIR include:

• Automated Network-on-Chip Synthesis: Automatically inferring on-chip interconnect topology from interface annotations and integrating router chains between handshake endpoints.
• Parallel Placement and Routing: Utilizing RIR slot assignments to enable parallel execution of placement/routing steps or linking to custom toolchains (e.g., via RapidWright).
• Enhanced On-Board Instrumentation: Systematic emission of counters, flow monitors, and dynamic throttling for in-system performance profiling and fuzz-driven validation.

RIR’s combination of formal representations, slot-based abstraction, and reusable transformation passes establishes it as an adaptable HLPS infrastructure for high-performance, portable, and extensible FPGA accelerator design (Lau et al., 2024).