Mitigation of Incoherent Spectral Lines via Adaptive Coherence Analysis for Continuous Gravitational-Wave Searches

Abstract: The sensitivity of continuous gravitational-wave searches is strictly limited by non-Gaussian spectral artefacts that accumulate coherent power over long observation baselines. In this paper, we present an unsupervised mitigation framework based on adaptive network coherence analysis. Unlike traditional veto methods that discard entire frequency bands, our pipeline selectively suppresses local artefacts while preserving global potentially astrophysical signals. We validate the method using Advanced LIGO O3 data, analysing the cleaning performance across integration times of 1, 3, and 5 days. For the 5-day dataset, the pipeline identifies and mitigates 89\% and 77\% of the total spectral lines in the Hanford and Livingston detectors, respectively, while effectively preserving the coherent population consistent with astrophysical morphologies. This is achieved while modifying less than 7\% of the analysis bandwidth spanning 20~Hz to 2000~Hz. Rigorous statistical verification demonstrates that the mitigation effectively suppresses the non-Gaussian tail of the noise distribution while strictly preserving the statistical integrity of coherent signal candidates. By recovering detector sensitivity in parameter spaces previously contaminated by the spectral forest, this framework provides a robust preprocessing strategy for all-sky searches.

• The paper introduces an unsupervised adaptive network coherence analysis that differentiates coherent astrophysical signals from incoherent spectral artefacts.
• It employs a multi-stage pipeline—including robust baseline estimation, morphological line detection, and adaptive matching—to mitigate 77-89% of spectral artefacts in LIGO O3 data.
• The method preserves phase continuity by rescaling SFT coefficients, ensuring minimal signal disruption in continuous gravitational-wave candidate analysis.

Adaptive Network Coherence Analysis for Mitigating Incoherent Spectral Lines in Continuous Gravitational-Wave Searches

Introduction

Continuous gravitational-wave (CW) searches with ground-based detectors such as Advanced LIGO, Virgo, and KAGRA are fundamentally limited by non-Gaussian, narrow-band spectral artefacts—commonly referred to as the “spectral forest”—superposed on the Gaussian detector noise. These artefacts arise from a complex variety of instrumental and environmental sources including power line harmonics, mechanical resonances, calibration features, and scattered light. They severely degrade sensitivity, inducing numerous false positives in matched-filter searches and contaminating parameter space in both coherent and semi-coherent analysis pipelines. Conventional mitigation techniques, such as frequency masking or dependence on known line lists, compromise both the breadth and safety of the search, frequently discarding valuable signal-carrying data or requiring laborious manual intervention.

This work proposes and validates an automated, unsupervised mitigation strategy based on adaptive network coherence analysis. Exploiting the physical principle that true CW signals—whether astrophysical or persistent environmental—must be coherent across detectors, the framework systematically suppresses local instrumental artefacts while preserving signals that exhibit network coherence. The methodology’s viability is demonstrated on Advanced LIGO O3 data using Hanford (H1) and Livingston (L1), with strong empirical performance and rigorous statistical validation.

Pipeline and Methodology

The pipeline comprises four principal stages:

1. Robust Baseline Estimation: The power spectral density (PSD) is constructed using SFTs (1,800 s segments, unsmoothed), and a multi-step iterative algorithm fits the underlying non-Gaussian noise floor. A running-median filter yields an initial, outlier-tolerant estimate, prominent peaks (≥3× local background, ±5 bins) are masked, and a degree-10 Chebyshev polynomial is iteratively fit in log-frequency–log-amplitude space for maximal numerical stability and convergence.
2. Morphological Line Detection: Peaks above the baseline are identified if they exceed the local background by a factor of 2.5 and demonstrate physical prominence (10% above surroundings). Boundaries are determined by a decay to within 5% above baseline; adjacent features are clustered logarithmically to avoid artificial fragmentation.
3. Adaptive Network Coherence Analysis: Detected lines are cross-referenced between detectors and classified by bandwidth: very narrow ($W < 5$ mHz), narrow, medium, and wide, with thresholds up to $W ≥ 10$ Hz. Matching tolerances on frequency and amplitude are adaptively assigned—ultra-precise for narrow lines (e.g., absolute 0.28 mHz tolerance) and broader for wide-band artefacts. Coherent lines (detected in both H1 and L1) are presumed astrophysical (or permanent environmental) and preserved. Incoherent lines (detected in only one detector) are marked for suppression.

   Figure 1: The pipeline stages—baseline fit and line identification for H1 and L1 (top/middle), and adaptive matching for coherent lines (bottom)—applied to Advanced LIGO O3 data.

4. Spectral Artefact Mitigation: Instead of zeroing masked bands or injecting synthetic Gaussian noise, the method rescales the SFT coefficients in incoherent line regions to the corresponding fitted baseline, maintaining phase coherence and eliminating the risk of producing artificial discontinuities. Coherent lines remain unmodified.

Results

Spectral Cleaning Efficiency

Empirical evaluation utilized O3 SFTs, partitioned into 1-, 3-, and 5-day subsets, mimicking real-world search segment lengths and spanning the evolving detector state. The adaptive pipeline demonstrates that, for 5-day accumulations, 89% (H1) and 77% (L1) of identified spectral lines are mitigated, while only 7% (or less) of the total analysis bandwidth (20–2000 Hz) is altered. As integration time grows, non-stationary features naturally average out, resulting in further reduction in artefact counts and bandwidth loss.

Morphological and Statistical Validation

Analysis of the detected feature populations based on frequency and width establishes a clear dichotomy: retained lines are morphologically sharp, discrete, and typically associated with globally coherent phenomena such as power mains, violin modes, or harmonics; suppressed artefacts are broader and more randomly distributed, consistent with stochastic local disturbances.

Q-Q analysis of the $F$-statistic quantifies the pipeline’s impact on candidate distributions. The tail slope for incoherent bins is suppressed to $s = 0.69$, effectively damping non-Gaussian outliers, while coherent bins are strictly preserved (tail ratio $r \approx 1$ within confidence intervals), attesting to negligible impact on real CW candidates. The method operates conservatively, ensuring that astrophysical signal power is invariant post-mitigation.

Comparison against the O3 line list (LIGO-T2100200) reveals that the algorithm recovers a substantial, though not complete, fraction of catalogued artefacts—differences are attributable to averaging timescales and resolution effects. The approach identifies many artefacts unlisted in official catalogs, suggesting utility for automated discovery.

Theoretical and Practical Implications

This framework provides several advances over prior art, including:

• Physical Coherence Exploitation: By privileging network coherence, the approach generalizes to arbitrary detector networks (including Virgo/KAGRA, with straightforward scaling).
• Automated and Unsupervised Action: No manual intervention, catalogue maintenance, or model retraining is needed.
• Phase Preservation and Minimal Data Disruption: By rescaling power—not deleting bins or injecting noise—the pipeline maintains phase continuity, crucial for sub-threshold and extended search methodologies.
• Compatibility and Complementarity: The method can serve as a robust preprocessing stage for deep learning or HMM-based pipelines, which otherwise require substantial human-in-the-loop vetos or struggle with out-of-distribution artefacts.
• Safe and Conservative Vetoing: Empirical Q-Q consistency and candidate preservation support deployment even in high-confidence search modes.

Limitations, such as lack of direct simulated signal injection and the primary focus on frequency-only $F$-statistic projections, are acknowledged. Extension to short-coherence time methods (including transient HMM tracking) and fully multidimensional parameter spaces would consolidate the pipeline’s generality. A “mock data challenge”, featuring controlled signal injection and cross-method comparisons, remains a compelling future direction.

Conclusion

The adaptive coherence analysis framework for spectral artefact mitigation achieves high effectiveness and selectivity in suppressing the spectral forest contaminating GW data. It does so while rigorously preserving the statistical integrity and candidate distribution for CW searches, outperforming manual mask-based or purely data-driven methods. The empirical and morphological validation establishes robustness, and the theoretical basis (physical coherence criteria) ensures transparency and maximal safety for astrophysical discovery. Future work on short-duration spectral artefact tracking, integration with neural denoisers, and exhaustive safety testing will render the pipeline a critical tool in the expanding search for persistent GW sources.