Integrated Detector Control and Calibration Processing at the European XFEL

Abstract: The European X-ray Free Electron Laser is a high-intensity X-ray light source currently being constructed in the area of Hamburg, that will provide spatially coherent X-rays in the energy range between $0.25\,\mathrm{keV}$ and $25\,\mathrm{keV}$. The machine will deliver $10\,\mathrm{trains/s}$, consisting of up to $2700\,\mathrm{pulses}$, with a $4.5\,\mathrm{MHz}$ repetition rate. The LPD, DSSC and AGIPD detectors are being developed to provide high dynamic-range Mpixel imaging capabilities at the mentioned repetition rates. A consequence of these detector characteristics is that they generate raw data volumes of up to $15\,\mathrm{Gbyte/s}$. In addition the detector's on-sensor memory-cell and multi-/non-linear gain architectures pose unique challenges in data correction and calibration, requiring online access to operating conditions and control settings. We present how these challenges are addressed within XFEL's control and analysis framework Karabo, which integrates access to hardware conditions, acquisition settings (also using macros) and distributed computing. Implementation of control and calibration software is mainly in Python, using self-optimizing (py) CUDA code, numpy and iPython parallels to achieve near-real time performance for calibration application.