PyKOALA: Python IFS Data Reduction Framework

• PyKOALA is a Python library for IFS data reduction that standardizes the transformation of post-extraction measurements into calibrated spectra and datacubes.
• It employs a uniform data abstraction and modular correction classes to address atmospheric, instrumental, and astrometric challenges across multiple instruments.
• The framework enhances quality assurance with diagnostics such as wavelength offset maps and light-profile analyses, ensuring robust cube reconstruction.

Searching arXiv for papers on PyKOALA and closely related IFS reduction context. arxiv_search(query="PyKOALA integral field spectroscopy data reduction", max_results=10, sort_by="relevance") Searching again with broader terms around KOALA, AAOmega, and IFS data reduction. arxiv_search(query="KOALA AAOmega integral field spectroscopy reduction Python", max_results=10, sort_by="relevance") PyKOALA is a Python library for integral field spectroscopy (IFS) data reduction that provides a robust, flexible, and instrument-agnostic framework for transforming post-tracing/extraction measurements into calibrated spectra and datacubes. It began as a bespoke tool for KOALA+AAOmega at the Anglo-Australian Telescope and evolved into a general framework organized around uniform data abstractions, modular correction classes, variance tracking, and configurable reconstruction from Row-Stacked Spectra (RSS) to science-ready cubes (Corcho-Caballero et al., 24 Jul 2025).

1. Definition and scope

PyKOALA is designed to make IFS data reduction robust, flexible, and accessible while remaining instrument-agnostic. Its stated focus is the stage that begins after instrument-specific tracing and extraction, when wavelength-calibrated spectra and variances are already available in RSS form, and continues through calibration, correction, and cube reconstruction. In this regime, it standardizes how practitioners move from instrument-specific products to calibrated spectra and datacubes, with variance tracking and modular corrections (Corcho-Caballero et al., 24 Jul 2025).

The motivating problem is the complexity of IFS reduction chains. These chains involve calibration of wavelength and flux, removal of atmospheric and instrumental signatures, and reconstruction of datacubes. KOALA+AAOmega exemplifies this complexity because multiple gratings, central wavelengths, and a wide field-of-view produce variable spectral formats and fiber geometries across observations. PyKOALA addresses three recurring requirements: transforming raw instrument measurements after tracing/extraction into calibrated, science-ready spectra and cubes; managing instrument variability in fiber layouts, spectral formats, and throughput; and simplifying pipelines through a modular, configurable approach rather than monolithic, instrument-bound code.

This separation of concerns implies a specific architectural stance. Instrument-dependent extraction remains upstream, while PyKOALA concentrates on a common post-extraction layer. This suggests a deliberate attempt to maximize reuse across IFS instruments without duplicating the responsibilities of observatory-specific reduction systems.

2. Core abstractions and software architecture

The library is organized around a uniform data abstraction called DataContainer, which provides the main abstraction for observational data and ensures a uniform interface to arrays and metadata. Two concrete container types are emphasized: RSS, which stores spectra per fiber after tracing and extraction together with flux, wavelength, and variance arrays, and Cube, a three-dimensional grid sampled along one spectral axis and two spatial axes and produced by interpolating one or multiple RSS exposures.

Component	Role
DataContainer	Uniform interface to arrays and metadata
RSS	Spectra per fiber with flux, wavelength, and variance
Cube	3D product reconstructed from one or multiple RSS exposures
Correction classes	Atmospheric extinction, sky emission, telluric absorption, astrometric offsets, wavelength offsets, flux calibration
Wrapper functions	Read instrument-specific formats into PyKOALA’s standard structures

Correction classes encapsulate distinct processing steps and can be chained to build customized workflows. The paper enumerates atmospheric extinction, sky emission, telluric absorption, astrometric offsets, wavelength offset corrections, and flux calibration. Reconstruction is similarly modular: flexible kernel choices, including inverse-distance weighted schemes and drizzling, allow users to tailor cube construction to data quality and science goals.

Instrument support is intentionally lightweight. PyKOALA uses input adapters, described as wrapper functions, that read instrument-specific file formats and populate the standard data structures with spectra, wavelength, and variance. For KOALA+AAOmega, the library reads RSS FITS products produced by the AAO 2dfdr pipeline and extracts the required arrays. For other instruments, users provide a function that returns the same minimal set of arrays, enabling use of the same reduction and cubing machinery. The paper states that Hector and WEAVE data have been tested.

Geometry is propagated through RSS metadata rather than reconstructed from first principles inside PyKOALA. Fiber-to-spaxel mapping is handled during cube construction through astrometric offset corrections and interpolation kernels. WCS is implicit in the cube grid definition, and alignment is handled through astrometry corrections; deeper Astropy/WCS integration is described as future work.

3. Reduction chain and numerical methods

PyKOALA focuses on post-extraction processing. Bias correction, dark correction, bad-pixel or cosmic-ray handling, and fiber tracing or extraction are generally performed by instrument-specific pipelines such as 2dfdr. PyKOALA therefore expects wavelength-calibrated spectra and variances in RSS form and then applies wavelength refinement, atmospheric and sky corrections, flux calibration, astrometric alignment, and cube reconstruction with variance propagation (Corcho-Caballero et al., 24 Jul 2025).

Wavelength offset refinement is implemented through fiber-to-fiber wavelength offset corrections and line-spread-function estimation via cross-correlation with a reference spectrum, such as a solar spectrum in twilight frames. The library supports both weighted-average and maximum-likelihood strategies. Although the proceeding does not provide explicit formulas for the fitting routine, it situates the refinement relative to the standard polynomial wavelength model

$\lambda(x)=\sum_{k=0}^{n} a_k x^k,$

with residuals minimized over arc or reference lines. In PyKOALA, the offset correction refines such a solution per fiber by maximizing the cross-correlation metric and can report sub-pixel accuracy.

Atmospheric extinction, sky subtraction, and telluric correction are implemented as Correction classes. The proceeding describes standard IFS practice for sky estimation, including use of dedicated sky fibers or robust combination across low-signal fibers, often with variance-aware weights. Representative formulations are

$$S_\lambda = \mathrm{median}_j\left\{ D_{j,\lambda} \right\}$$

and

$$S_\lambda = \frac{\sum_j w_{j,\lambda}\, D_{j,\lambda}}{\sum_j w_{j,\lambda}},$$

where $w_{j,\lambda} \propto 1/\sigma_{j,\lambda}^2$. Telluric correction is described as division by a normalized atmospheric transmission curve, and extinction correction uses site extinction curves and airmass.

Flux calibration is implemented through a standard response function derived from spectrophotometric standards. The representative relationship given is

$$F_\lambda = \frac{C_\lambda}{t\, T(\lambda)} \times 10^{-0.4\,(ZP_\lambda + A_\lambda X)},$$

where $C_\lambda$ are counts, $t$ is exposure time, $T(\lambda)$ is total throughput, $ZP_\lambda$ is a wavelength-dependent zero point, $A_\lambda$ is atmospheric extinction, and $$S_\lambda = \mathrm{median}_j\left\{ D_{j,\lambda} \right\}$$0 is airmass.

Differential Atmospheric Refraction is handled through an astrometry offset Correction:

$$S_\lambda = \mathrm{median}_j\left\{ D_{j,\lambda} \right\}$$1

with zenith distance $$S_\lambda = \mathrm{median}_j\left\{ D_{j,\lambda} \right\}$$2 and refractive index $$S_\lambda = \mathrm{median}_j\left\{ D_{j,\lambda} \right\}$$3. The astrometric correction aligns wavelengths across fibers and exposures before cubing.

Variance tracking is a first-class design feature. Variances supplied in the RSS are propagated through corrections and reconstruction. For linear operations with weights $$S_\lambda = \mathrm{median}_j\left\{ D_{j,\lambda} \right\}$$4, the stated propagation rule is

$$S_\lambda = \mathrm{median}_j\left\{ D_{j,\lambda} \right\}$$5

Multiplicative corrections, such as extinction or telluric factors, scale variances accordingly.

Cube reconstruction supports inverse-distance weighted interpolation and drizzling, with Gaussian and parabolic kernels also illustrated. Representative kernel definitions include

$$S_\lambda = \mathrm{median}_j\left\{ D_{j,\lambda} \right\}$$6

for inverse-distance weighting, and

$$S_\lambda = \mathrm{median}_j\left\{ D_{j,\lambda} \right\}$$7

for a Gaussian kernel. Drizzling redistributes flux onto a grid via drops with controllable footprint size, following the method of Fruchter and Hook. The proceeding explicitly shows that kernel choice affects reconstructed slices and light-profile diagnostics. This suggests that PyKOALA treats reconstruction as a configurable scientific choice rather than as a fixed implementation detail.

4. Instrument-agnostic operation and the KOALA+AAOmega case

PyKOALA’s instrument independence is based on a narrow interface: an instrument wrapper returns arrays for fiber spectra, wavelength, and variance, and the rest of the pipeline operates on standardized objects. In this sense, the package is not tied to KOALA+AAOmega even though that system motivated its development. The paper reports testing beyond KOALA with Hector and WEAVE data, which is presented as evidence of generalizability (Corcho-Caballero et al., 24 Jul 2025).

The KOALA+AAOmega application remains the canonical demonstration. KOALA is described as a wide-field, 1000-element IFU feeding the double-beam AAOmega spectrograph. Its combination of gratings, central wavelengths, and field configuration necessitates refined wavelength alignment and careful cube construction. PyKOALA reads 2dfdr-generated RSS for KOALA+AAOmega, applies fiber-to-fiber wavelength offset corrections via cross-correlation, estimates the LSF per fiber, performs atmospheric, sky, telluric, and flux calibrations, and reconstructs cubes using user-selected kernels.

The validation examples emphasize diagnostics rather than a single scalar benchmark. Twilight RSS frames are used to map wavelength offsets and LSF standard deviations across fibers under weighted-average and maximum-likelihood strategies. Multi-exposure KOALA fields show that reconstructed cubes differ slightly by kernel, and the library provides collapsed light profiles to quantify these reconstruction choices. The proceeding does not tabulate specific S/N, residuals, or absolute flux metrics, but it reports effective sky subtraction, accurate flux calibration, and sub-pixel wavelength calibration.

This validation profile is characteristic of reduction software intended for heterogeneous observing modes. Rather than presenting a single “best” pipeline, PyKOALA exposes method choices and accompanying quality-assurance products so that adequacy can be judged per project.

5. Data products, metadata, and quality assurance

PyKOALA produces a sequence of intermediate and final products. These include RSS containers with flux, wavelength, and variance for each fiber; corrected RSS after extinction, sky, telluric, flux, and wavelength-offset refinements; datacubes with one spectral and two spatial axes; and variance maps carried alongside flux at all stages. The proceeding also notes quality-assurance maps such as wavelength-offset and LSF products (Corcho-Caballero et al., 24 Jul 2025).

Metadata required for reduction, including airmass, exposure time, and bandpass information, are stored in the DataContainer and consumed by Correction classes. FITS is the de facto source format because the package commonly ingests observatory pipeline products, but PyKOALA’s internal objects are Python/Numpy-based. The package can export cubes for subsequent analysis. WCS handling remains implicit via cube-grid definition and astrometric correction, with closer Astropy WCS integration planned.

Quality assurance is built around interpretable diagnostics rather than fixed acceptance thresholds. The paper identifies wavelength-offset maps and LSF standard-deviation maps per fiber from cross-correlation fits, together with cube-reconstruction diagnostics such as collapsed light profiles and kernel-comparison views. Users can compare, for example, weighted-average and maximum-likelihood wavelength alignment strategies and can select reconstruction kernels suited to a specific science case. Acceptance thresholds are described as project-tailored rather than globally prescribed.

The interaction model is a Python API built around DataContainer-derived RSS and Cube classes and sequential application of Correction classes. The documentation site, https://pykoala.readthedocs.io/, is identified as the location for tutorials and API references.

6. Limitations, comparative position, and nomenclature

PyKOALA is not presented as a full raw-frame reduction system. Its current focus is explicitly post-extraction workflows starting from wavelength-calibrated RSS. Full processing of raw frames, including bias or dark correction and tracing or extraction, remains instrument-pipeline territory such as 2dfdr. PSF modeling, fiber cross-talk deconvolution, and fully automated WCS solutions are not described in the current proceeding, and deeper Astropy integration together with broader instrument support are identified as future directions (Corcho-Caballero et al., 24 Jul 2025).

Relative to other IFS reduction software, the proceeding positions PyKOALA against both comprehensive instrument-specific pipelines and narrower analysis tools. The MUSE pipeline, as summarized in the paper, is a comprehensive, instrument-specific solution tightly coupled to MUSE hardware and data formats and executes the full chain from raw frames to cubes with sophisticated calibration and QA. PyKOALA’s distinct role is a lightweight, instrument-agnostic Python framework that starts from standardized RSS and emphasizes modular corrections and cube building. Relative to tools such as P3D or survey-specific cube combination codes such as CubEx, its emphasis is a flexible, correction-centric workflow and simple integration with existing instrument pipelines.

A recurrent source of ambiguity is the acronymic overlap with unrelated machine-learning literature. On arXiv, “KOALA” also denotes “KOALA: A Kalman Optimization Algorithm with Loss Adaptivity” (Davtyan et al., 2021), and “COALA” denotes “COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation” (Parkina et al., 10 Jul 2025). Another unrelated use of KOALA appears in federated learning for resource-constrained IoT clients (Chen et al., 2024). In the IFS context, however, PyKOALA denotes the Python library for post-extraction spectroscopic reduction and datacube reconstruction described above.

The package is therefore best understood as a reduction-layer framework: upstream extraction remains instrument-specific, downstream science analysis remains external, and PyKOALA occupies the calibration, correction, and cubing interval between them. This suggests its principal significance lies not in replacing observatory pipelines, but in standardizing the technically difficult middle stages of IFS reduction across multiple instruments.