Compiler-in-the-Loop

• Compiler-in-the-Loop is a methodology where the compiler actively participates in iterative design, providing immediate feedback and correctness checks during program transformations.
• It facilitates a clear separation between temporal definition and spatial mapping, enabling detailed control over loop transformations and hardware synthesis.
• This approach streamlines the development process by significantly reducing verification time and engineering effort, achieving performance similar to hand-tuned designs.

Compiler-in-the-Loop refers to a methodology and paradigm in which the compiler is made an active, iterative, and programmatically steerable partner in the software and hardware development process. Contrasted with traditional one-shot, compiler-driven workflows, a compiler-in-the-loop system exposes precise control and rapid feedback for program transformations, facilitating both correctness and high performance. It is characterized by a development loop where the user incrementally specifies, maps, and validates high-level constructs, with the compiler providing immediate structural, legality, and semantic equivalence checks, and output artifacts ready for evaluation or deployment. This approach is central to the T2S (Temporal to Spatial) programming methodology for spatial accelerators, where programmatic control of the loop and data transformations leads directly to generative and verifiable hardware design (Rong, 2017).

1. Split Programming Model: Temporal Definition and Spatial Mapping

A T2S program is always structured as the combination of two strictly separated components:

1. Temporal Definition (functional specification):
   • Expresses what to compute, as an unoptimized set of loop nests or matrix recurrences.
   • Adopts a declarative, dataflow style nearly identical to Halide.
   • Exemplified by matrix operations such as

   $C(i,j) = 0;\quad C(i,j) += A(i,k) * B(k,j)$

• Bounds and variable ranges are declared explicitly, ensuring clarity and analyzability.

1. Spatial Mapping (schedule):
   • Expresses how the temporal computation is mapped onto resources such as spatial pipelines, systolic arrays, caches, and relays.
   • Composed of loop-nest and channel directives, e.g. tiling (tile), loop unrolling (unroll), buffer insertion (buffer), producer/consumer isolation (isolate_producer_chain, isolate_consumer_chain), and data relays (relay).
   • These transformations are programmatically specified, making explicit the spatial layout and communication between components.

This strict partitioning allows the programmer to focus first on semantics and then on optimization, eliminating interleaving of correctness and performance considerations.

2. Workflow from Directives to Compiler Execution

The compilation pipeline, as realized in a T2S system, is driven directly by the schedule supplied by the user—enabling what the paper terms "precise control of a compiler." The key steps are:

1. IR Construction: The compiler builds an initial intermediate representation (IR) reflecting the unoptimized, purely temporal loop nest.
2. Transformation Application: Each loop transformation (split, tile, reorder, unroll, etc.) is injected into the IR per the spatial mapping.
3. Kernel Isolation and Channel Insertion: Upon encountering isolate_producer_chain or isolate_consumer_chain, the IR is split into separate functions. FIFO channels (register- or memory-based) are inserted as communication links, and relevant loop nests are cloned into producer/consumer kernels.
4. Further Specialization: Each subfunction may be further optimized—unrolling, insertion of buffers and relays to build systolic arrays or software caches.
5. Static Loop-nest Equivalence Checking: Before final code generation, the system checks, at the IR level, semantic equivalence between the transformed program and the original temporal specification.
6. Low-level and Backend Optimization: Final synthesis to hardware (RTL or HLS output) or further software lowering.

Typical T2S scheduling directives and their effect are as follows:

Directive	Purpose	Effect on IR/Hardware
tile(i,j,ii,jj,II,JJ)	Loop tiling for blocking	Splits loops into tile+intra-tile
unroll(ii,jj)	Creates 2-D array of processing elements	Enables PE parallelism
buffer(A,j,REGISTER)	Builds register/BRAM-based line buffers	On-chip data reuse
relay(A,<di,dj, ...>)	Data forwarding in systolic arrays	Peer-to-peer communication

By composing these, the programmer can systematically articulate complex architectural mappings such as multi-level caching, ND systolic arrays, and host-device serialization.

3. Static Verification and Semantic Equivalence

Because all transformation directives operate at the level of loop nests and affine index domains, T2S is able to perform automatic, static verification of semantic equivalence for the transformed IR:

• The compiler maintains iteration domains $$D = \{(i,j,k)\mid 0 \leq i < I, 0 \leq j < J, 0 \leq k < K\}$$ for each statement.
• Each affine transformation specifies a mapping (bijective or otherwise legality-checked) from original to transformed indices.
• Data dependence graphs are used to check that unrolling, reordering, buffering, or relaying do not violate original ordering.
• Before code generation, the system formally proves that each iteration in the original domain maps uniquely and that source dependencies are preserved.
• If runtime parameters (e.g., loop bounds divisible by tile size) are violated, the compiler emits runtime checks.

If all preconditions are met, the generated bitstream or object is "correct by construction" as confirmed by the static proof built into the compiler.

4. Iterative Compiler-in-the-Loop Development Cycle

The "compiler-in-the-loop" paradigm in T2S refers precisely to a rapid, feedback-driven, iterative workflow:

1. Temporal specification is authored in minutes.
2. Initial spatial mapping is drafted in hours.
3. Immediate feedback from the compiler surfaces dependence violations, buffer allocation insufficiencies, and any unsatisfied constraints (e.g., divisibility of tile sizes).
4. Rapid refinement: The user iteratively refines parameters (tiling/unrolling factors, buffer insertion points), with each compile–feedback–refine cycle completing in 30 seconds to 2 minutes.
5. Design space exploration: Many architectural mappings can be explored in a single working day, with compiler analysis and legal hardware/image synthesis delivered after each iteration.

This model mirrors the compile–test–tune loop familiar from frameworks like Halide (for CPUs/GPUs), but is recast for spatial hardware targets, where costly full synthesis cycles historically made such iteration infeasible (Rong, 2017).

5. Empirical Evaluation and Productivity Impact

Quantitative evaluation demonstrates dramatic gains in engineering productivity and maintains high performance. Four canonical workloads are summarized as follows:

Workload	Prior Impl.	Perf.	Eng. Effort	T2S Perf.	T2S Effort
SGEMM	FPGA HLS (Altera)	500 GFLOPS	18 months	500 GFLOPS	~5 hours
Conv+ReLU (VGG-16)	FPGA RTL	250 GFLOPS	6 months	250 GFLOPS	~3 hours
SpMV	FPGA HLS	40 GOPS	3 months	40 GOPS	~2 hours
Merge Sort	FPGA RTL	150 MKeys/s	2 months	150 MKeys/s	~2 hours

Key observations:

• Generated designs achieve throughput within 1–2% of hand-tuned, engineering-intensive approaches.
• Coding and verification effort is reduced by 2–3 orders of magnitude (months to hours).
• Portability is enhanced: a single T2S specification can be retargeted across back-ends (FPGA, CGRA, ASIC) with minimal changes.

Separation of correctness (temporal) and optimization (spatial) concerns, together with built-in static verification and rapid feedback, yields a next-generation spatial programming model that is programmer-directed but compiler-verified and performance-portable.

6. Implications and Synthesis

Compiler-in-the-loop as instantiated by T2S signals a fundamental methodological advance for spatial computing. By providing a high-level, explicitly programmatic interface to both the functional and mapping layers, and by embedding formal verification and rapid iteration into the core workflow, the model achieves:

• High-performance mapping of dataflow algorithms to spatial architectures.
• Orders-of-magnitude gains in developer productivity and reduced verification burden.
• Static, sound-by-construction correctness guarantees integrated directly into the transformation pipeline.
• A workflow in which the compiler serves not as a black-box but as an "active partner," guiding, checking, and instantiating user-driven architectural design.

T2S stands as a prototype of the broader compiler-in-the-loop philosophy, providing a template for converging correctness, productivity, and performance-centric development in spatial domains (Rong, 2017).