Real-Time Graph-based Point Cloud Networks on FPGAs via Stall-Free Deep Pipelining

Abstract: Graph-based Point Cloud Networks (PCNs) are powerful tools for processing sparse sensor data with irregular geometries, as found in high-energy physics detectors. However, deploying models in such environments remains challenging due to stringent real-time requirements for both latency, and throughput. In this work, we present a deeply pipelined dataflow architecture for executing graph-based PCNs on FPGAs. Our method supports efficient processing of dynamic, sparse point clouds while meeting hard real-time constraints. We introduce specialized processing elements for core graph operations, such as GraVNet convolution and condensation point clustering, and demonstrate our design on the AMD Versal VCK190. Compared to a GPU baseline, our FPGA implementation achieves up to 5.25x speedup in throughput while maintaining latencies below 10 {\mu}s, satisfying the demands of real-time trigger systems in particle physics experiments. An open-source reference implementation is provided.