AURORA: A High Performance DAQ Framework for Next-Generation Rare-Event Search Experiments

Abstract: The upcoming PandaX-xT experiment will deploy over 3,000 readout channels operating at a 500 MSa/s sampling rate, generating a sustained data bandwidth up to 1.6 GB/s. To meet this demanding requirement, we present AURORA, a high-performance, distributed data acquisition (DAQ) framework designed for scalability, low latency, and efficient resource utilization. Built on a modular architecture and leveraging modern I/O and networking technologies, including multi-level buffering, deferred and asynchronous processing, AURORA achieves a projected throughput of over 3 GB/s on the aggregation node in benchmark tests. While developed to support PandaX-xT, the framework is experiment-agnostic and readily adaptable to other large-scale particle and nuclear physics experiments.

• The paper presents a distributed, modular DAQ framework achieving sustained data rates of 1–1.6 GB/s, validated through extensive benchmarking.
• AURORA integrates multi-level buffering and asynchronous control to ensure robust, low-latency data acquisition for complex detector arrays.
• The system's scalability and error-free long-term performance mark it as a benchmark for future high-rate, rare-event search experiments.

AURORA: High-Performance DAQ for Next-Generation Rare-Event Searches

Introduction

The AURORA framework addresses the critical demands in large-scale rare-event search experiments, specifically those leveraging dual-phase xenon time-projection chambers such as PandaX-xT. The framework is motivated by limitations observed in previous DAQ architectures when scaling to thousands of readout channels and gigabyte-per-second data throughput. AURORA is designed as a distributed, experiment-agnostic system emphasizing modularity, low latency, robust data integrity, and horizontal scalability. Its architecture and implementation set a reference for future particle and nuclear physics experiments with similar stringent requirements.

System Requirements and DAQ Architecture

AURORA’s design responds to a multidimensional requirement set—sustained high throughput (1–1.6 GB/s with peaks above), robust end-to-end data integrity, reliable operation in multi-day campaigns, and seamless integration across multiple experimental subsystems. The proposed distributed architecture decouples data readout, aggregation, and persistent storage, using modular DAQ servers and an aggregation (collector) server as the main building blocks. Each DAQ server (running daq_reader) interfaces optically with multiple digitizers, while the collector server orchestrates run control, data merging, and logging. This design supports logical segregation of data streams (e.g., TPC and veto) with full timestamp correlation for later joint analyses.

Figure 1: Schematic overview of the distributed DAQ architecture, showing data flow and integration with configuration, monitoring, and message systems.

Data Flow and Multi-Level Buffering

Central to AURORA’s performance is its multi-level buffering and asynchronous processing in both daq_reader and collector. Data is transmitted as contiguous blocks (self-describing with timestamps and channel info), enabling time-based partitioning at the collector. The collector’s BufferManager implements a fixed-length, time-partitioned buffer list for data arriving from distributed sources. On the reception of new data blocks, BufferManager aligns them into their corresponding time windows, utilizing accompanying lightweight metadata for efficient downstream chronologically ordered output.

Figure 2: The structure of raw data blocks containing headers and payload, used throughout the DAQ chain.

Figure 3: Flow and state synchronization of the acquisition cycle, coordinated by control commands across all DAQ nodes.

These buffering strategies are coupled with explicit state machines for both daq_reader and collector, governed by reliable, asynchronous control messages. The system reacts predictably to control transitions and is resilient to race conditions even in burst or failure scenarios.

Modular Program Components

The daq_reader software abstracts physical digitizer control, data acquisition, buffering, and all network interactions as C++ classes, utilizing modern asynchronous I/O abstractions based on the Asio library. Digitizer configuration is centrally managed via a PostgreSQL server and delivered to each node at run start, ensuring configuration coherence across the distributed system.

Figure 4: Internals of daq_reader, illustrating the decoupling of digitizer threads and network I/O via asynchronous ring buffering.

The collector server is similarly structured, with separation of data streams, buffer management, and file output. Each logical stream instance manages its own buffer pipeline, enabling independent processing and output.

Figure 5: Collector program architecture with stream, buffering, and file output modules for high concurrency and extensibility.

Special care is taken in the implementation of buffer recycling and data sorting, which is synchronized with hardware-clock feedback to mitigate clock drift effects.

Figure 6: Distribution and recycling of data blocks into their associated timed buffers, optimizing for arrival latency and temporal ordering.

Integrated Control, Monitoring, and Messaging

AURORA exposes a RESTful HTTP interface for seamless, scriptable run control, enabling integration with higher-level run management or graphical UIs. All experiment metadata, configuration, and data file records are managed in PostgreSQL. In addition, operational status, error conditions, and performance metrics are streamed to InfluxDB, enabling live experiment monitoring. File generation events are announced via Kafka, triggering downstream data transfer and processing jobs.

Performance Benchmarks and Validation

Extensive benchmarking, utilizing both synthetic and live experimental data, confirms all internal software stages surpass the critical 1.6 GB/s sustained throughput target. Notably, the aggregate memory-copy bandwidth of the copy-to-buffer and bufferreader stages reaches 15–25 GB/s and over 3 GB/s, respectively, ruling out internal processing as a bottleneck for anticipated workloads. The final disk write stage, using high-end NVMe hardware and XFS filesystem, achieves a stable 5.47 GB/s. Stress tests on the PandaX-20T prototype with 176 digitizers at 1.4 GB/s demonstrated sustained error-free operation over 58 hours, which is a marked achievement over the 24-hour stability goal.

Live campaign experience during the PandaX-4T calibration phase validated robustness, with daily average rates of 800–900 MB/s, confirming the system's suitability for physics runs involving intensive calibration or elevated background.

Scalability and Adaptability

The decoupled nature of AURORA's DAQ nodes and collectors naturally enables horizontal scaling. Should readout requirements increase, additional DAQ/collector pairs can be deployed. The system design ensures that all files are strictly time-ordered and indexed, so offline merging is straightforward, and there's no loss of analysis integrity.

Moreover, the streaming buffer output can, with minimal extension, deliver real-time data to online analysis or trigger systems for low-latency event identification (e.g., supernova neutrino bursts), supporting emerging multi-messenger alert infrastructure requirements.

AURORA's architecture is primarily experiment-agnostic. Adaptation to other use cases requires only minor changes in the data parsing/metadata extraction layer, as the buffer and output logic assume only ordered, timestamped blocks.

Conclusion

AURORA provides a technically rigorous, scalable, and robust DAQ architecture, validated in high-background, high-rate experimental environments. It demonstrates clear separation of concerns, robust multi-level buffering, and a practical asynchronous architecture tailored to the demanding conditions of large-scale rare-event search experiments. The framework is highly extensible for future upgrades and diverse experiments, positioning it as a practical reference for high-rate, distributed DAQ system design in particle and nuclear physics.

The implications are direct: reliable DAQ at scale enables next-generation dark matter searches, low-threshold rare-event detection, and supports both real-time and offline physics analysis. Future directions involve extending online streaming analysis, increasing automation, and generalizing to larger, more complex detector arrays across the field.