Pushing compute and AI onto detector silicon

Abstract: In order to take full advantage of the U.S. Department of Energy's billion-dollar investments into the next-generation research infrastructure (e.g., exascale, light sources, colliders), advances are required not only in detector technology but also in computing and specifically AI. Let us consider an example from X-ray science. Nanoscale X-ray imaging is a crucial tool to enable a wide range of scientific explorations from materials science and biology to mechanical and civil engineering. The next-generation light sources will increase the X-ray beam brightness and coherent flux by 100 to 1,000 times. In order to image larger samples, the continuous frame rate of pixel array detectors must be increased, approaching 1 MHz, which requires several Tbps (aggregated) to transfer pixel data out to a data acquisition system. Using 65-nm CMOS technology, an optimistic raw data rate off such a chip is about 100-200 Gbps. However, a continuous 1 MHz detector with only $256 \times 256$ pixels at 16-bit resolution, for example, will require 1,000 Gbps (i.e., 1 Tbps) bandwidth off the chip! It is impractical to have multiple high-speed transceivers running in parallel to provide such bandwidth and represents the first data bottleneck. New approaches are necessary to reduce the data size by performing data compression or AI-based feature extraction directly inside a detector silicon chip in a streaming manner before sending it off-chip.

• The paper introduces embedding AI and computational functions directly on detector silicon to mitigate data bottlenecks in high-speed applications.
• It proposes on-chip data compression and feature extraction using advanced ASIC designs to lower off-chip transfer bandwidth.
• The integration leverages collaborative co-design and next-gen CMOS processes to enable MHz frame rates and boost scientific measurement efficiency.

Overview

The whitepaper titled "Pushing compute and AI onto detector silicon" explores the integration of AI and advanced computing directly onto pixel detector silicon, particularly in the context of the Department of Energy's next-generation scientific infrastructure investments. This paper advocates for embedding computational and AI capabilities within the detector ASICs (Application-Specific Integrated Circuits), thus addressing significant data bottlenecks caused by the increasing demands of data acquisition systems.

Challenges and Technological Needs

One primary challenge highlighted is the data bottleneck encountered with the traditional design of pixel detectors used in fields such as X-ray imaging, astronomy, and materials science. The transition to more sophisticated light sources is projected to increase X-ray beam brightness by a factor of 100 to 1,000, necessitating the enhancement of pixel detectors' frame rates up to 1 MHz. However, the raw data acquisition rates significantly exceed the capabilities of existing high-speed transceivers. For instance, using contemporary 65-nm CMOS technology, pixel data from a $256 \times 256$ pixel detector running at 1 MHz would demand 1 Tbps bandwidth, far beyond the practical throughput of 100-200 Gbps achievable off-chip.

To mitigate this, the paper argues for integrating digital functionalities such as data compression or AI-driven feature extraction directly onto the detector ASIC. This approach promises to elevate data throughput by performing operations in-situ, reducing the volume of data needing off-chip transfer, which subsequently enhances measurement speed and experiment focus.

Opportunities for Integration

The advancement of ASIC technology and the availability of fabless semiconductor models and advanced foundries enable the integration of computing capabilities directly on detectors. This shift allows for high-level digital data processing on the ASICs, reducing relayed data volumes and bandwidth constraints. Projects like the EU-XFEL's AGIPD detectors already utilize MHz frame rates through limited on-chip storage; however, a full transition to digital designs could further accelerate data transfer capabilities, utilizing sub-100 nm CMOS processes.

The whitepaper emphasized the need for collaborative efforts in co-design among ASIC developers, computer scientists, and application-domain scientists to exploit these potential opportunities. Addressing legal and protective constraints like NDAs from chip foundries, along with the high entry barriers of Electronic Design Automation (EDA) tools, is critical to fostering a successful research infrastructure within DOE labs.

Timeliness and Strategic Importance

High-performance facilities worldwide, exemplified by ALS-U, APS-U, CHESS-U, and other new-generation facilities, are investing heavily to exploit unprecedented brightness levels facilitated by new accelerators. However, maximal potential usage of these improvements is contingent upon detector advancements that can accommodate AI and computational functionality directly on the chips. As scientific processes increasingly leverage AI for data analysis, accelerating detector operations with AI capabilities becomes imperative.

The integration aligns with the trend of shifting computational loads from numerical processing to AI-centric workloads. Thus, embedding ultrafast AI inference architectures in ASICs not only enhances detector capabilities but also offers benefits for future high-performance computational infrastructures.

Conclusion

The document effectively presents the case for incorporating AI and computational technology onto detector silicon chips in scientific imaging. By doing so, substantial improvements in data throughput and reduction in bottlenecks can be realized, increasing the temporal resolution and effectiveness of scientific measurements. The integration of AI on ASICs forms a crucial component of broader advancements in high-performance AI-driven data processing, promising significant contributions to scientific innovation and discovery. The whitepaper underscores that strategic and coordinated efforts could yield substantial progress in leveraging ASIC capabilities across diverse scientific domains.