- The paper presents a regression-based method that uses linear regression on pixel flux differences to effectively remove variable atmospheric OH emission.
- It validates the approach on long-slit K-band spectra of NGC 7538, preserving faint nebular lines and extended spatial structures even in rapidly changing conditions.
- The technique leverages stable continuum and OH line ratios to minimize artifacts, offering a robust framework for improved sky subtraction in NIR observations.
Sky Background Correction in Near-IR Spectroscopic Observations of Extended Objects Using AstroNIRCam at CMO
Introduction
This work presents a comprehensive method for accurate sky background correction in near-infrared (NIR) spectroscopic observations of extended astronomical sources, specifically applied at the Caucasus Mountain Observatory (CMO) using the AstroNIRCam instrument. The CMO, featuring a 2.5-meter Ritchey-Chrétien telescope equipped with a HAWAII-2RG NIR detector, uniquely facilitates wide-field NIR spectroscopy (1−2.5 μm) in Russia. Addressing sky background subtraction is crucial due to temporal variability in bright atmospheric hydroxyl (OH) emission, which strongly contaminates the K-band and rapidly varies on timescales shorter than standard exposure durations.
Observational Challenges at CMO
AstroNIRCam observations are impacted by a relatively high and variable precipitable water vapor (PWV, median 7.7 mm), leading to rapid atmospheric line fluctuations over typical exposures (several minutes). For sources ≳3−4 arcmin in size, direct polynomial or local sky subtraction—effective for compact or point sources—becomes invalid. Instead, dedicated sky frames are necessary. However, line intensity variations between object and sky observations introduce significant residuals, especially for faint astrophysical emission lines blended with or masked by the strong and time-variable atmospheric lines.
Data Acquisition
The methodology is validated on long-slit K-band (2.04–2.35~μm, R∼1200) spectra of the star-forming region NGC 7538. Two slit positions (268~arcsec × 0.9~arcsec) were employed, both in the core and periphery of the nebula, with alternating object and nearby sky exposures. Additional calibration with standard stars observed at similar airmass enabled precise tracking of temporal and spatial variations in both line and continuum components.

Figure 1: Brγ image of NGC~7538 showing position of two long-slit observations, indicating regions used for sky and object emission line analysis.
Characterization of Sky Line and Continuum Variability
Comprehensive analysis of the temporal behavior of both atmospheric OH emission lines and the underlying NIR continuum demonstrates:
- Bright OH0 lines (e.g., at 2.0413, 2.1802, and 2.2126~OH1m) fluctuate by OH2–OH3 over sets of consecutive frames (timescales of OH4–OH5 minutes).
- The ratios among different OH6 lines, and the continuum levels across the band, are notably more stable, with variations OH7.
- The spectral shape of the continuum can vary due to instrumental and environmental factors (e.g., telescope temperature drift causing changes in continuum slope), though these are subdominant for nebular emission line science.
Such trends are visualized in the flux time-series and flux-ratio diagnostics.

Figure 2: Exposure-normalized 2D spectrograms for slit~1: (a) object, (b) sky, (c) difference between corrected and uncorrected sky, (d) object minus corrected sky.

Figure 3: Same set as Figure 2 but for slit~2.
Regression-Based Sky Background Correction Method
The proposed correction pipeline leverages the empirical stability of atmospheric line ratios and continuum levels:
- For each observing sequence (object, sky, standard), median spectra are extracted from regions minimally contaminated by intrinsic astrophysical emission.
- The continuum in the sky spectrum is fitted and subtracted, isolating variable atmospheric line emission.
- For each pixel, linear regression is performed between the object-sky and sky-continuum flux differences; crucially, regression is computed both over all data points and for the lower envelope—mitigating the confounding influence of possible nebular line contamination.
- The correction factor (OH8) from this regression quantifies how sky line amplitude should be scaled to match object observations.
- The sky spectrogram is thus rescaled: OH9, and subtracted from the object frame.
This procedure, validated through comparison with standard star fields, minimizes residuals to the K0 level outside astrophysical line positions even under suboptimal weather.
With the corrected subtraction, weak nebular lines—such as HK1 2.0418~K2m, He~I 2.0431~K3m, and BrK4 2.1661~K5m—are reliably distinguished against the atmospheric foreground. Comparing naive and regression-corrected sky subtraction, the latter method systematically provides lower background artifacts and higher fidelity for weak lines embedded in the OH forest.
Crucially, the method achieves:
- Residual continuum difference between object and sky spectra at the K6–K7 level;
- Effective suppression of strong atmospheric line artifacts;
- Preservation of extended structure across the spatial axis of the slit, critical for mapping physical properties in the NGC 7538 region.
Practical and Theoretical Implications
This methodology addresses a longstanding practical barrier in ground-based NIR spectroscopy of large angular objects—namely, the inadequacy of traditional sky subtraction due to temporal variability of atmospheric line emission on exposure-limited timescales. Application to the NGC 7538 field enables measurements of fundamental ISM parameters, including molecular hydrogen temperature, column density, and possibly ortho/para ratios. The flexibility and robustness of the regression approach make it broadly transferable to similar facilities with comparable instrument/telescope/environmental constraints.
Theoretically, this framework could be adapted for inclusion in automated reduction pipelines, generalized for multidimensional datasets, and inform future instrument design (e.g., fast readout modes, simultaneous object/sky slitlets). The approach may further support physical studies sensitive to subtle spectral diagnostics, such as stellar feedback, turbulence, and chemical evolution in star-forming regions extending over arcminute scales.
Conclusion
The regression-based sky background subtraction technique presented effectively mitigates the impact of variable atmospheric K8 emission in ground-based K9-band spectroscopic studies of spatially extended sources when slit lengths are insufficient to sample uncontaminated sky. This advancement enables high-fidelity extraction of astrophysical line fluxes and robust ISM parameter derivation at the CMO, setting a precedent for NIR spectroscopy at sites with challenging atmospheric stability.