Host-Target Protocol (HTP) in FPGA Emulation
- Host-Target Protocol (HTP) is a specialized command protocol enabling Linux syscall emulation on FPGA targets through a low-bandwidth UART link.
- It minimizes communication overhead by using coarse-grained commands like PageSet and PageCopy, reducing UART traffic by >95% compared to direct debug interfaces.
- HTP supports multicore operation, efficiently managing per-core events and achieving >96% performance accuracy in emulated syscall execution.
The Host–Target Protocol (HTP) is the command and transport protocol used in the FASE framework to connect a Linux host runtime to a processor target implemented on FPGA, enabling host-side syscall emulation and end-to-end performance validation without integrating a full SoC or target operating system on the FPGA (Meng et al., 10 Sep 2025). In this setting, the host is a Linux workstation running the FASE runtime, while the target consists of CPU cores, a memory system, and a small FASE controller on FPGA. HTP is carried over a low-bandwidth UART link and provides a compact, operation-level interface for controlling instruction streams, inspecting and modifying architectural state, moving data at page granularity, and reading performance counters. Its central design problem is to preserve performance fidelity despite a physical communication channel that is low bandwidth and high latency, while relying only on a minimal, microarchitecture-agnostic CPU debug interface (Meng et al., 10 Sep 2025).
1. Definition and system role
Within FASE, HTP is the protocol interpreted by the FASE Controller on FPGA and used by the host runtime to operate a processor design directly, while Linux-style syscalls are emulated remotely on the host (Meng et al., 10 Sep 2025). This arrangement implements what the paper characterizes as “CPU on FPGA, OS on host,” with HTP serving as the operational boundary between the two sides.
The protocol exists for three stated reasons. First, FPGA–host links are low bandwidth and high latency; the implementation uses a UART at up to 921,600 bps, so naïve remote debugging would impose prohibitive overhead. Second, arbitrary FPGA designs do not expose a unified hardware interface, so FASE restricts hardware requirements to a minimal CPU interface and lifts higher-level functionality into HTP. Third, syscalls must be offloaded to the host because the target does not run an OS; when user code traps, the event is intercepted and handled via host-directed HTP operations (Meng et al., 10 Sep 2025).
A plausible implication is that HTP should not be understood as a generic debug protocol. Its semantics are specialized for syscall emulation, thread management, virtual memory manipulation, and performance validation under stringent transport constraints. This distinguishes it from host-target mechanisms designed for simulation-only environments, where host access to target state is not limited by a slow physical link.
2. Design objectives and constraints
HTP is explicitly optimized to minimize cross-device traffic over UART. Its commands are semantically coarse-grained: page-set, page-copy, page-RW, MMU setup, and hardware-assisted futex handling are all defined so that large effects can be produced with short messages. The reported consequence is that HTP reduces UART traffic by >95% relative to directly driving the CPU debug interface from the host, and that traffic for page-level operations can drop to <1% of the naïve approach (Meng et al., 10 Sep 2025).
The protocol must nevertheless expose sufficient functionality for correct Linux-style syscall emulation. The FASE runtime uses HTP to implement process-like threads, Linux virtual memory including mmap/munmap/brk, page tables, copy-on-write, lazy initialization, futex and other synchronization primitives, and file-descriptor-based I/O syscalls (Meng et al., 10 Sep 2025). This means that HTP is not merely a state-inspection channel; it is the sole mechanism by which the host can realize process management and memory semantics on a target lacking a resident OS.
HTP also has first-class multicore support. Requests identify CPUs, Next reports which CPU trapped, and the protocol supports per-core MMU/TLB state, HFutex masks, and user-time counters (Meng et al., 10 Sep 2025). At the same time, it is designed to remain microarchitecture-agnostic: only three core-side bundles are required from the CPU—Priv, Reg, and Inject—and HTP abstracts over them so that the host sends high-level operations rather than manipulating implementation-specific signals.
This design suggests a deliberate separation of concerns. The host reasons in terms of threads, syscalls, page tables, and counters, while the controller translates those requests into low-level register accesses and injected instructions. HTP is thus both a transport protocol and an abstraction boundary between system-level emulation and minimal hardware control.
3. Architectural realization in FASE
On the target side, HTP terminates at the FASE Controller, which is connected to UART RX/TX, parses messages, maintains per-core interrupt state and an event queue, issues register and memory accesses, performs injected instruction sequences, carries out page-level memory operations, and implements HFutex filtering (Meng et al., 10 Sep 2025). The target CPU cores and memory system run the user application, while the controller mediates all host-directed interventions.
The required CPU interface is intentionally small. Priv exposes the current privilege level. The Reg bundle supports general-purpose register read/write through a valid-ready interface. The Inject bundle supports non-branch instruction injection and includes StopFetch, Inject, InjectInst, and InjectBusy signals (Meng et al., 10 Sep 2025). HTP is not visible directly to the core. Instead, the controller interprets each HTP command and realizes it as a sequence of register operations or instruction injections.
For example, a Register read causes the controller to assert the relevant register-access signals and complete the valid-ready exchange. Memory access and page operations are realized by sequences of injected loads and stores while StopFetch suppresses normal fetch. Control-flow operations such as Redirect use register writes to set PC and stack state, internal controller updates to clear interrupt conditions, and de-assertion of StopFetch to resume execution (Meng et al., 10 Sep 2025).
There is no explicit layered protocol stack. The paper describes HTP as a flat set of RPC-like commands with typed arguments, interpreted by a finite-state machine in the controller. On FPGA, the implementation includes an RX/TX controller, request buffer, FIFO page-data buffer for PageRW, per-core interrupt status, per-core HFutex Mask Cache, and an event queue (Meng et al., 10 Sep 2025). This architecture is designed so that controller-side computation is cheap relative to UART transfer time; accordingly, work is pushed onto the FPGA controller wherever possible.
4. Command set and semantics
The paper does not specify byte-level packet formats, but it defines the logical command set of HTP clearly (Meng et al., 10 Sep 2025). These commands can be organized by function.
| Command class | Commands | Role |
|---|---|---|
| Instruction stream management | Redirect, Next, MMU/TLB, SyncI, HFutex |
Resume execution, capture traps, manage address spaces, synchronize instruction visibility, and configure hardware-assisted futex filtering |
| Word-level data access | Register, Memory |
Read/write architectural registers and aligned physical memory |
| Page-level data access | PageSet, PageCopy, PageRW |
Zero/fill pages, replicate pages, and stream whole pages across UART |
| Performance counters | Tick, UTick |
Return global ticks since reset and per-CPU unprivileged ticks since reset |
Redirect resumes a CPU at a specified address, typically after syscall handling or signal delivery. It clears the interrupted status for that CPU, writes the return PC and other required registers, and de-asserts StopFetch (Meng et al., 10 Sep 2025). Next waits for and retrieves the next trap or exception from any CPU; the controller monitors Priv, marks a CPU interrupted when it enters privileged mode, asserts StopFetch, queues the CPU ID, and returns that ID to the host when the event is consumed (Meng et al., 10 Sep 2025).
MMU/TLB sets per-core virtual memory context and flushes TLBs, such as by writing SATP for RISC-V. SyncI ensures instruction/data stream consistency after code loading or page-table updates. HFutex configures a per-core list of futex addresses for which local handling is allowed; when a futex wake syscall targets a masked address, the controller may respond locally without host involvement (Meng et al., 10 Sep 2025).
At finer granularity, Register reads or writes general-purpose registers for context save/restore and syscall decoding, while Memory reads or writes aligned physical memory for page tables, metadata, or small transfers. At coarser granularity, PageSet fills one or more pages with a constant word, PageCopy copies one or more physical pages, and PageRW streams a whole page through the UART using a FIFO-based page buffer (Meng et al., 10 Sep 2025).
The semantics also imply that each request includes a command opcode, a CPU ID for per-core operations, and operation-specific arguments such as addresses, counts, and flags. Syscalls themselves are not HTP messages. Instead, the host infers the syscall number and arguments by using Register after Next reports a trap (Meng et al., 10 Sep 2025). This preserves a clean separation between protocol-level state transfer and host-level syscall semantics.
5. Protocol operation in syscall emulation
A typical syscall flow in FASE illustrates HTP’s operational model (Meng et al., 10 Sep 2025). User code running on a Rocket core executes an ecall or similar instruction and enters privileged mode. The controller detects the privilege change via Priv, marks the corresponding CPU as interrupted, asserts StopFetch, and enqueues the CPU ID. The host main loop blocks in Next until an event is available, then receives the CPU ID.
The host runtime next uses Register to inspect architectural state: syscall number, syscall arguments, and PC. Using the syscall ABI and its own tables, it determines which Linux syscall was requested. Purely logical operations such as getpid may be completed on the host without touching target memory, but syscalls involving buffers, mapping changes, futexes, or thread control require HTP operations such as Memory, PageRW, PageSet, PageCopy, MMU/TLB, SyncI, and Redirect (Meng et al., 10 Sep 2025).
The runtime may then make a scheduling decision. It can return immediately to the same thread, block the caller and context-switch to another runnable thread on the same core, or redirect execution to a signal trampoline. Thread contexts are stored in host memory. When resuming execution, the runtime writes back GPRs and PC using Register, issues MMU/TLB and SyncI if the address space or relevant memory mappings changed, and finally sends Redirect (Meng et al., 10 Sep 2025).
This control loop shows that HTP is the complete actuation interface for remote syscall emulation. Thread scheduling, synchronization, signal handling, I/O bypass, virtual memory management, page-fault handling, and performance measurement all reduce to sequences of HTP commands. A plausible implication is that the accuracy of end-to-end performance validation depends directly on how efficiently this command stream can express system-level effects over the UART bottleneck.
6. Traffic minimization and multicore behavior
HTP’s most distinctive property is its effort to amortize host-target latency by raising the abstraction level of protocol operations. PageSet and PageCopy are central examples: zero-initialization and copy-on-write replication can be executed entirely on the FPGA controller, so the host sends only compact descriptors instead of page payloads (Meng et al., 10 Sep 2025). PageRW is reserved for cases where data genuinely must cross the host-target boundary, such as ELF loading or file-backed buffer transfer.
Word-level access is deliberately used for metadata rather than bulk transfer. Memory and Register handle page tables, syscall arguments, return values, and special cases, while the controller locally executes repeated injected instruction sequences as needed (Meng et al., 10 Sep 2025). The protocol also minimizes unsolicited communication: only significant events, specifically traps to privileged mode, are pushed into the event queue for host consumption via Next.
HFutex is an additional optimization for synchronization-heavy workloads. The runtime supplies lists of futex addresses for which local wake handling is allowed, and the controller caches them in per-core HFutex Mask Caches. If a futex wake targets a masked address, the controller can immediately return success without any UART transaction or host-side syscall handling (Meng et al., 10 Sep 2025). This is particularly effective when wake calls are often spurious.
For multicore operation, nearly every request includes a CPU ID. The controller routes per-core operations to the appropriate Priv, Reg, and Inject bundles. Exceptions are placed into a FIFO event queue, and Next pops them in order of occurrence, providing deterministic trap-handling order for the runtime. On any given core, state-manipulating commands such as Register, Memory, and Redirect are sequenced by the controller FSM and execute in order (Meng et al., 10 Sep 2025). The runtime itself is effectively single-threaded, apart from an auxiliary thread for blocking host syscalls, so HTP accesses are serialized on the host side.
This design suggests that HTP achieves correctness through controlled serialization and event ordering, rather than through a complex concurrent transport layer. That choice is consistent with the protocol’s target use case: validation fidelity matters more than maximizing host-side parallelism.
7. Quantitative performance, comparison, and limitations
The evaluation in FASE ties several results directly to HTP’s design (Meng et al., 10 Sep 2025). Relative to directly driving the CPU debug interface from the host, HTP reduces UART communication by >95%, and page-level operations reduce traffic to <1% of the direct method. In single-thread CoreMark, FASE introduces <1% performance error and achieves >2000× higher efficiency than Proxy Kernel due to FPGA acceleration. Against a full LiteX SoC baseline running Linux, FASE reports >96% performance validation accuracy for most single-thread workloads and >91.5% for most multi-thread workloads, while user-mode CPU time shows >95.9% accuracy on most workloads (Meng et al., 10 Sep 2025).
The paper defines relative error as
where is time in the full-system LiteX baseline and is time measured with FASE using Tick or UTick via HTP (Meng et al., 10 Sep 2025). Reported errors correlate strongly with UART traffic, including the number of syscalls, page faults, and futex operations. Baud-rate sensitivity experiments show that increasing UART speed generally reduces error with diminishing returns, indicating that HTP’s physical transport remains the dominant bottleneck.
HFutex yields an additional reduction in futex-related UART traffic of 3–15%, depending on workload and thread count. In some cases, approximately 30% of futex wake calls are handled locally by the controller (Meng et al., 10 Sep 2025). This confirms that controller-side semantic offload is not incidental but integral to HTP’s performance properties.
The protocol’s novelty becomes clearer in comparison with prior host-target schemes. Proxy Kernel with HTIF relies on a privileged mini-OS on the target and typically requires full SoC simulation; it is usually single-core and single-thread. Traditional syscall-emulation simulators such as gem5 SE or zsim do not need a protocol like HTP because the target exists entirely in software and the host can access simulated memory directly (Meng et al., 10 Sep 2025). HTP is novel in bringing syscall-emulation-style execution to an actual FPGA implementation where host-target communication is a real physical constraint.
The paper also makes clear where the limitations lie. Accuracy is bounded primarily by the UART physical layer, even with an optimized protocol. HTP is intended for CPU/memory-level validation and does not model real peripheral timing on FPGA; I/O-intensive workloads are serviced on the host and therefore do not capture device-level behavior. The current design targets a single FPGA, so scaling to multi-FPGA or distributed co-simulation would require changes in addressing, topology, and flow control (Meng et al., 10 Sep 2025).
A plausible implication is that HTP is best viewed as a domain-specific RPC substrate for FPGA-host co-execution under severe communication constraints. Its current command set and controller semantics are sufficient to make syscall emulation practical and accurate for early-stage performance validation, but its generalization to richer devices, faster interconnects, or distributed targets would require a corresponding evolution of both the transport and the semantic contract between host runtime and target controller.
