---
title: CHIMERA Pipeline for CTA Data Analysis
url: https://www.emergentmind.com/topics/chimera-pipeline
type: topic
---

# CHIMERA Pipeline for CTA Data Analysis

In the context of the Cherenkov Telescope Array, the CHIMERA Pipeline denotes a modern, open-source, modular event-reconstruction and instrument-response-generation chain for imaging atmospheric Cherenkov telescope data. It is built around **ctapipe** for low-level calibration and event reconstruction, **pyirf** for instrument response functions and sensitivity estimation, and **protopipe** for orchestration, machine learning, and production-oriented workflow control. Operationally, the pipeline transforms raw or pre-calibrated telescope waveforms into reconstructed event lists and Gamma-Astro-Data-Formats-compliant response products that can be consumed by higher-level science tools to derive spectra, light curves, and sky maps [2110.11097].

## 1. Definition, scope, and terminology

Within CTA software discourse, the pipeline is the event-reconstruction backbone that converts telescope-level records into analysis-ready data products. Its primary tasks are reconstructing the physical properties of each recorded shower and providing the corresponding instrument response functions. In that sense, it is the core chain from raw telescope data to higher-level gamma-ray analysis [2110.11097].

A terminological subtlety is important. The paper "Prototype Open Event Reconstruction Pipeline for the Cherenkov Telescope Array" does not use the term “CHIMERA” explicitly; rather, the phrase is used in some contexts as a generic label for the CTA-style open event-reconstruction and IRF-generation chain built from ctapipe, pyirf, and protopipe [2110.11097]. This suggests that, in CTA usage, “CHIMERA Pipeline” functions as an umbrella designation for the architecture and workflow rather than the name of a single standalone package.

The scope is specifically that of an **imaging atmospheric Cherenkov telescope** analysis chain. CTA is the next-generation gamma-ray observatory currently under construction, with a northern site in La Palma, Spain, and a southern site in Paranal, Chile, and it is explicitly planned as the first open gamma-ray observatory [2110.11097]. That institutional design directly shapes the pipeline’s modularity, open-source distribution, and use of standardized formats.

## 2. Layered architecture and data products

The pipeline is organized as a layered transformation from telescope waveforms to science-ready artifacts. Conceptually, the chain proceeds from low-level camera signals through image construction and stereoscopic shower reconstruction to event selection and response generation [2110.11097].

| Stage | Data level | Principal output |
|---|---|---|
| Raw telescope data | DL0/DL1a | Time series per pixel and telescope |
| Waveform calibration and extraction | DL1 | Integrated charge and peak time per pixel |
| Image cleaning and parameterization | DL1b | Cleaned images and image parameters |
| Event reconstruction | DL2 | Direction, core position, energy, particle classification |
| Event selection and IRF computation | pyirf stage | Effective area, PSF, energy dispersion, background rates |

At the software level, the division of labor is explicit. **ctapipe** implements the “classical” four-step IACT analysis: image extraction, image cleaning, image parameterization, and event property reconstruction. It also provides a data model based on Containers, HDF5 I/O, plugin EventSource interfaces for different formats, configuration, and visualization tools. **pyirf** operates on labeled reconstructed event lists to compute effective area, point spread function, energy dispersion or migration matrices, background rates, and sensitivity, and exports results in FITS compliant with Gamma-Astro-Data-Formats. **protopipe** sits on top of both, wrapping ctapipe algorithms into executable tools and adding machine learning, YAML configuration, workflow management, and grid execution via DIRAC and CTADIRAC [2110.11097].

The architecture is deliberately open, modular, and extensible. ctapipe exposes algorithms via Components and plugin interfaces; pyirf acts on generic reconstructed event tables; protopipe composes separate tools for training, reconstruction, and IRF production. This modularity is central to CTA’s open-observatory model, because it permits external users to introduce alternative methods while preserving standard data models and IRF formats [2110.11097].

## 3. Event reconstruction chain

The first transformation stage is waveform calibration and extraction. For each camera pixel \(i\), the input is a waveform \(w_i(t)\) in ADC counts versus sample index. The goal is to derive an integrated charge \(Q_i\) in photo-electrons and a peak time \(t_{i,\text{peak}}\). ctapipe supports simple peak finding as well as multi-pixel or shower-time models that refine integration windows from neighboring-pixel information or expected shower time evolution [2110.11097].

The calibration chain includes pedestal subtraction, gain calibration, and timing alignment. In schematic form,
\[
Q_i \approx \frac{1}{g_i} \sum_{t \in \text{window}} [\,w_i(t)-p_i\,],
\]
with \(p_i\) the pedestal and \(g_i\) the gain. These steps take the data from pre-calibrated raw form to DL1 products suitable for imaging and stereoscopic reconstruction [2110.11097].

Image cleaning removes night-sky-background and electronics noise. ctapipe implements several cleaning algorithms, including threshold-based schemes with a high threshold for “core” pixels and a lower threshold for neighboring “boundary” pixels, optionally combined with timing cuts. The output is a cleaned image in which selected signal pixels are preserved and other pixels are zeroed or flagged [2110.11097].

Image parameterization is centered on **Hillas parameters**. With cleaned charges \(Q_i\) at camera coordinates \((x_i,y_i)\), the total signal and centroid are
\[
S=\sum_i Q_i,\qquad
\bar{x}=\frac{\sum_i Q_i x_i}{S},\qquad
\bar{y}=\frac{\sum_i Q_i y_i}{S}.
\]
Second moments then yield the ellipse length, width, and orientation, together with additional descriptors such as miss, distance, asymmetry, skewness, containment parameters, and the number of isolated pixel groups [2110.11097]. These features are the standard low-dimensional representation on which downstream reconstruction and classification operate.

At DL2, stereoscopic reconstruction combines multiple telescope images. In the moments-based path, each Hillas ellipse major axis defines a line in the camera that maps to a line on the sky; intersecting such lines across telescopes yields the shower direction, and projecting the shower axis to the array plane gives the impact point or core position. ctapipe also supports the **ImPACT** algorithm, a likelihood-based template-fitting method using Monte Carlo-generated image templates over energy, impact parameter, direction, and related variables. ImPACT is computationally heavier but can outperform simple moment-based reconstruction [2110.11097].

Energy reconstruction and gamma–hadron separation are handled in protopipe through a machine-learning module. The paper explicitly lists **AdaBoostRegressor** and **RandomForestRegressor** for energy regression and **RandomForestClassifier** for particle classification, trained on simulated gamma and proton events. The classifier output is a **gammaness score**, which later enters cut optimization and sensitivity estimation [2110.11097].

## 4. Instrument response functions and pyirf

The pipeline’s second major product class is the instrument response function. In the formalism used by the pipeline, the expected distribution of reconstructed events \(e(\hat{\alpha},\hat{\delta},\hat{E},t)\) is related to the true source intensity \(I(\alpha,\delta,E,t)\) through an instrumental response kernel \(R\) and a background term \(b\):
\[
e(\hat{\alpha},\hat{\delta},\hat{E},t)
=
\int
R(\hat{\alpha},\hat{\delta},\hat{E}\mid \alpha,\delta,E,t)\,
I(\alpha,\delta,E,t)\,
\mathrm{d}\Omega\,\mathrm{d}E
+
b(\hat{\alpha},\hat{\delta},\hat{E}).
\]
The classical IACT factorization writes
\[
R
=
A_{\text{eff}}
\cdot
\operatorname{PSF}
\cdot
D(\hat{E}\mid E),
\]
with \(A_{\text{eff}}\) the effective area, PSF the point spread function, and \(D\) the energy-dispersion kernel or migration matrix [2110.11097].

pyirf computes these quantities from labeled reconstructed event lists. Inputs are Monte Carlo simulations that provide true event properties, together with reconstructed outputs from ctapipe or protopipe, including \(\hat{E}\), reconstructed direction, and gammaness. For background IRFs, proton and electron simulations are also used. pyirf then optimizes analysis cuts, notably on gammaness and angular separation, for a target science case [2110.11097].

The resulting IRFs are binned in true energy, reconstructed energy, and offset angle, and can also depend on zenith, azimuth, time, or related observational parameters. pyirf exports them in GADF FITS files with standard HDUs such as `AEFF`, `EDISP` or `MIGRA`, `PSF`, and `BKG`. These products, together with reconstructed event lists, are the interface to higher-level packages such as ctools and gammapy [2110.11097].

This IRF layer is not ancillary; it is constitutive of the pipeline. The event list alone is insufficient for physics products, because spectra, light curves, and maps depend on the effective area, directional response, energy migration, and irreducible background model associated with the event-selection configuration [2110.11097].

## 5. protopipe, production workflow, and open-observatory operation

protopipe provides the production wrapper around ctapipe and pyirf. Its workflow is organized into four tools: a ctapipe-based preparation step for training data, a protopipe.mva step for model training, a ctapipe plus machine-learning reconstruction step for analysis events, and a pyirf-based step for cut optimization and IRF production. Each tool is a standalone Python executable configured through YAML and integrated with ctapipe’s traitlets-based configuration system [2110.11097].

The machine-learning layer uses scikit-learn interfaces, including `GridSearchCV`, and relies on NumPy, pandas, and joblib for data handling and I/O. Models and intermediate event tables are serialized as gzip-compressed pickled objects. The workflow is designed for large Monte Carlo productions and supports execution through DIRAC and CTADIRAC, which is essential for CTA-scale datasets spanning many gamma, proton, and electron events, multiple array configurations, and varied observing conditions [2110.11097].

The open-observatory character of CTA is directly reflected in the software stack. ctapipe, pyirf, and protopipe are open-source, hosted on GitHub, distributed via PyPI and conda-forge, and built on standard scientific Python libraries such as Astropy, NumPy, and SciPy. The EventSource plugin system permits experiment-specific readers, including `ctapipe_io_lst` for LST-1 and MAGIC combined analysis. Standardization through GADF enables direct interoperability with external analysis tools [2110.11097].

protopipe is also positioned historically. It is being developed with the explicit goal of superseding legacy CTA reference analyses based on VERITAS EventDisplay and MAGIC MARS. The strategy is not replacement by monolith, but migration toward a CTA-native, open, modular pipeline grounded in standard data models and reproducible configuration [2110.11097].

## 6. Context, validation, and recurrent ambiguities

The principal evidence base for the CHIMERA Pipeline in the CTA sense is the prototype open event-reconstruction stack described in "Prototype Open Event Reconstruction Pipeline for the Cherenkov Telescope Array" [2110.11097]. The paper reports successful use of ctapipe and pyirf on simulated CTA data and notes that ctapipe has also been successfully applied to data from CTA prototype telescopes such as LST-1. First performance studies on prototype-telescope data are reported elsewhere, indicating that the pipeline is not restricted to synthetic validation [2110.11097].

A recurrent misconception is to treat “CHIMERA Pipeline” as the name of a single official CTA package. The software reality is layered: ctapipe provides reconstruction primitives, pyirf provides IRF generation, and protopipe provides orchestration and machine learning. The “CHIMERA” designation in this context is therefore better understood as a contextual label for the combined chain rather than a repository-level project name [2110.11097].

Another source of confusion is nomenclature outside gamma-ray astronomy. In the arXiv literature, “Chimera” also denotes unrelated systems, including a bidirectional pipeline-parallel training scheme for large neural networks [2107.06925], a compact synthetic dataset for cross-domain reasoning in large language models [2603.00889], a zero-shot image morphing framework [2512.07155], and an AI-MCU for transformer inference at the edge [2606.02358]. In CTA usage, however, the term refers specifically to the open-source IACT event-reconstruction and IRF-production chain centered on ctapipe, pyirf, and protopipe [2110.11097].

Taken in that precise sense, the CHIMERA Pipeline is the architectural blueprint for CTA’s raw-to-science processing: waveform extraction, image cleaning, Hillas-based and template-based reconstruction, machine-learning energy and particle inference, cut optimization, and standardized IRF export. Its defining characteristics are not a single algorithmic novelty but the combination of open-source implementation, layered modularity, standardized interfaces, and production-oriented orchestration required by the first open gamma-ray observatory [2110.11097].

Source: https://www.emergentmind.com/topics/chimera-pipeline