VIPCALs: Automated VLBI Calibration Pipeline

• VIPCALs is a fully automated VLBI calibration pipeline that converts raw visibilities into calibrated data without human intervention.
• It automates critical steps such as reference antenna selection, calibrator identification, fringe-fit configuration, and diagnostic logging.
• By reproducing standard AIPS workflows with ParselTongue integration, VIPCALs enables efficient processing of large, heterogeneous archival surveys.

Searching arXiv for the cited VIPCALs paper and closely related VLBI calibration context. arxiv_search.query{"^^^^^^^^2search_query2 OR ti:\2^^^^^^^^"VIPCALs: A fully-automated calibration pipeline for VLBI data\"","max_results":5} Attempting direct arXiv lookup for the VIPCALs paper. Searching arXiv. to=arxiv_search.query code: {"^^^^^^^^2search_query2 VIPCALs, short for VLBI Pipeline for automated data Calibration using AIPS, is a fully automated calibration pipeline for continuum VLBI data. It is designed to convert raw correlated VLBI visibilities into science-ready calibrated datasets without human intervention and without requiring prior knowledge of the observation. Implemented in Python using ParselTongue as an interface to AIPS, VIPCALs reproduces the standard AIPS calibration workflow in a fully unsupervised mode, while also automating decisions that are usually left to expert operators, including reference-antenna selection, calibrator identification, fringe-fit configuration, and diagnostic generation. Its stated motivation is large-scale, heterogeneous archival processing, particularly for projects such as SMILE, where manual or semi-automated reduction would be a bottleneck (&&&^^^^2search_query2^^^^&&&).

^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^. Problem setting and design objective

VLBI calibration is substantially more demanding than calibration for connected-element interferometers because antennas are widely separated, record data independently, and retain antenna-dependent clock, geometric, atmospheric, ionospheric, gain, and bandpass errors after correlation. The paper emphasizes that, unlike facilities such as the VLA or ALMA, VLBI users are generally not delivered science-ready products; instead, they receive raw correlated visibilities. Existing pipelines are described as largely semi-automated and still dependent on user supervision for steps such as choosing calibrators, selecting a reference antenna, inspecting fringe-fit results, and tuning parameters.

VIPCALs is presented as a response to that operational gap. Its target regime is survey-scale continuum VLBI, especially heterogeneous archival data spanning many years, observing bands, and scheduling conventions. The motivating example is the Search for Milli-Lenses (SMILE) project, which aims to analyze nearly 5,^^^^2search_query2search_query2search_query2^^^^ radio-loud sources using archival VLBA observations. The pipeline’s design philosophy is explicitly conservative: preserve as much data as possible, avoid risky automatic edits to questionable metadata, and fail transparently when assumptions are violated. Rather than attempting aggressive recovery from malformed inputs, it halts or flags data and records the issue.

A plausible implication is that VIPCALs is not primarily a new calibration formalism; it is an automation framework that encodes established AIPS reduction practice into a reproducible, diagnostics-rich workflow. The novelty lies in how completely the traditional operator role is algorithmized.

2. Software basis and calibration model

VIPCALs uses AIPS as the calibration engine and orchestrates AIPS tasks from Python through ParselTongue. It also uses matplotlib for diagnostics and provides a simple GUI written with PySide6. The software can be installed as a pip package, provided the user has a dedicated Conda environment and a local installation of AIPS 3^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^DEC24 or newer, or it can be run via Docker.

Conceptually, the pipeline follows the standard AIPS table-based calibration model. The paper states that observed visibilities are modeled through the Hamaker-Bregman-Sault measurement equation, in which the true source coherence matrix is corrupted by antenna-based Jones matrices. Calibration then proceeds by estimating those antenna-based corruptions and storing them in AIPS tables. Newly derived corrections are written first to SN tables and then interpolated and accumulated into successive CL tables.

AIPS table	Role in VIPCALs
CL	Accumulated calibration tables
SN	Newly derived calibration solutions
BP	Frequency-dependent gains
TY / GC	Amplitude information
FG	Flags
NX	Indexing/bookkeeping

The pipeline supplements native AIPS logging with a structured summary log and a CSV file containing run metadata, timings, source and calibrator rankings, flagged records, and other diagnostics. This bookkeeping is central to unattended processing because large-scale use requires rapid post hoc inspection of quality and failure modes.

3. End-to-end calibration workflow

The workflow begins with data loading. VIPCALs checks whether a dataset contains widely separated central frequencies, either as distinct IF groups or different frequency IDs, and, if so, loads them into AIPS as separate entries. It can also concatenate multiple uvfits or idifits files when they share an identical frequency setup. To reduce runtime, it loads only the target source or sources plus up to three bright calibrator candidates unless instructed otherwise. These candidates are found automatically by cross-matching source coordinates against the NRAO VLBA calibrator list using a 5 arcsecond radius and selecting the three brightest matched sources in the observed band. Data import uses FITLD, and the calibration table entry interval is set to 6 seconds.

Pre-calibration preparation includes cleaning source and antenna name strings, verifying time-baseline ordering, reordering with UVSRT if necessary, and generating NX^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ and CL^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ with INDXR if the index table is missing. The pipeline deliberately avoids aggressive a priori flagging. Beyond correlator-supplied flags and conservative system-temperature cleaning, most rejection is deferred.

Auxiliary table retrieval is handled automatically where possible. VIPCALs can fetch project-specific FG, TY, and WX tables from NRAO repositories and import them with ANTAB. System temperatures are smoothed with TYSMO, with a conservative rejection rule: values are discarded if they are negative, exceed ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2(Álvarez-Ortega et al., 18 Aug 2025)2search_query2search_query2^^^^ K, or differ by more than 25^^^^2search_query2^^^^ K from the mean value for a given source. VLBA gain curves are parsed from the common gain-curve repository by matching epoch and antenna. If non-VLBA antennas lack TY or GC support, they are flagged in FG2.

The pipeline can also apply a phase-center shift with UVFIX when more accurate source positions are available. The paper notes that offsets larger than roughly ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ arcsecond from the correlated phase center can significantly degrade fringe fitting and increase smearing. Optional averaging is available: frequency averaging is applied when channel widths are below 5^^^^2search_query2search_query2^^^^ kHz, and time averaging is applied only if the original integration time is ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ second or shorter, in which case the data are averaged into 2-second bins.

Propagation and geometric calibration then proceed in classical AIPS order. VIPCALs retrieves IONEX files and applies ionospheric corrections with TECOR, storing the result in CL2. The paper writes the ionospheric delay as

PRESERVED_PLACEHOLDER_^^^^2search_query2^^^^

where PRESERVED_PLACEHOLDER_^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ is frequency and $\int N_e ds$ is the line-of-sight TEC. CLCOR is then used for Earth Orientation Parameter updates and for parallactic-angle correction, producing CL3 and CL4 where supported.

Instrumental and amplitude calibration follow. VIPCALs applies ACCOR for digital sampling correction, FRING for instrumental delays, BPASS for complex bandpass calibration, ACSCL for autocorrelation-based amplitude renormalization, and APCAL for amplitude calibration using TY and GC tables. The present version does not apply atmospheric opacity correction, a limitation that is noted as particularly relevant above about 22 GHz. Final target fringe fitting is performed with FRING, and calibrated data are exported with SPLIT and FITTP once valid final solutions exist on at least one baseline.

4. Automated decision logic

The most distinctive part of VIPCALs is its automation of decisions that are usually performed interactively by experienced VLBI users.

Reference-antenna selection is fully automated. The pipeline first retains only antennas present in all scans, including calibrator and target scans; if that criterion is too strict, it retains antennas present in all target scans. For VLBA observations, it can preferentially consider geographically central antennas—KP, LA, PT, OV, FD—if they satisfy the coverage criterion. The surviving antennas are then ranked by their average fringe S/N from the FFT stage of FRING. To estimate these scores, VIPCALs runs the FFT stage for up to ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2search_query2^^^^ randomly selected scans per source, using a search window of ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2search_query2search_query2search_query2^^^^ ns in delay and 2^^^^2search_query2search_query2^^^^ mHz in rate, a solution interval equal to the scan length, and aparm(7)=^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ to suppress normal thresholding. The highest-ranked antenna becomes the primary reference antenna, while the remaining ranked antennas are passed to FRING through the search parameter as fallback references.

Calibrator-scan identification is likewise automatic. VIPCALs runs FRING in FFT mode over all scans and all sources, solving for single-band delay and fringe rate relative to the chosen reference antenna, again using ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2search_query2search_query2search_query2^^^^ ns and 2^^^^2search_query2search_query2^^^^ mHz search windows and a fringe S/N threshold of 5. For each antenna, the scan with the highest fringe S/N is chosen as that antenna’s calibrator scan. If the best S/N for an antenna does not exceed 5, that antenna is flagged and removed from subsequent calibration, with the result written to FG3.

Target fringe fitting is automated through a solution-interval search. VIPCALs runs a preliminary FRING on a short data segment and chooses the shortest interval that yields fringe detections with S/N $\ge 5$ on all baselines; if no such interval is found, it uses the full scan length. For the SMILE use case, the default minimum and maximum intervals are ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ minute and ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2search_query2^^^^ minutes. During science-target fringe fitting, the pipeline again uses the ranked reference-antenna list through aparm(9) and search, with the same ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2search_query2search_query2search_query2^^^^ ns delay window, 2^^^^2search_query2search_query2^^^^ mHz rate window, and S/N threshold of 5. If single-band fitting is insufficient, VIPCALs retries with a multi-band delay fit by setting aparm(5)=^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^, and then chooses between the single-band and multi-band runs according to which produces the higher success rate, defined as the ratio of valid solutions to expected solutions.

This suggests that the automation strategy is heuristic but strongly grounded in standard VLBI practice. Rather than learning calibration policies from data, VIPCALs codifies expert procedural choices into deterministic rules with explicit thresholds and fallback behavior.

5. Validation, sample construction, and performance

Validation was performed on a large and deliberately heterogeneous SMILE-related VLBA sample (&&&^^^^2search_query2^^^^&&&). SMILE itself is based on 4,968 radio sources selected from CLASS with a flux-density threshold of 5^^^^2search_query2^^^^ mJy at 8.4 GHz. For VIPCALs testing, the authors constructed a representative sample of ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^,^^^^2search_query2search_query2search_query2^^^^ sources: 45^^^^2search_query2^^^^ uniformly distributed in flux density between 5^^^^2search_query2^^^^ mJy and ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ Jy, 45^^^^2search_query2^^^^ uniformly distributed in exposure time from ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ minute to ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ hour, and ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2search_query2search_query2^^^^ observed at higher frequencies in U band and K band. Archive selection required at least C-band or X-band data, phase centers within ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^ arcminute of the CLASS coordinates, and compatible files from the same project within a 2-day interval. When multiple candidates existed, the dataset with the longest on-source integration was selected.

The resulting benchmark comprised ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^,4^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^7 observations—defined as source-frequency pairs—of ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^,^^^^2search_query2search_query2search_query2^^^^ sources, drawn from 36^^^^2search_query2^^^^ VLBA projects, spread across 2,589 files totaling ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^9 TB, and spanning ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^994 to 2^^^^2search_query22^^^^5. A source was counted as successfully processed if the pipeline completed all steps and produced calibrated output. By that criterion, VIPCALs successfully calibrated 955 of the ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^,^^^^2search_query2search_query2search_query2^^^^ sources, corresponding to ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^,372 individual observations.

The reported retention and fringe-fitting statistics are central. Across the successfully completed sample, the median final visibility ratio—the ratio of calibrated retained visibilities to the total starting visibilities after excluding unsupported non-VLBA antennas—was ^^^^2search_query2^^^^.87, and the mean was ^^^^2search_query2^^^^.78. 87.4% of observations retained more than half of their original visibilities, while only 76 observations (7.2%) retained fewer than 2^^^^2search_query2^^^^%. The final target-fringe-fit statistics were also strong: 9^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^.6% of calibrated datasets achieved successful fringe fitting on the target in at least half of the attempted solution intervals.

Runtime was compatible with survey-scale use. In single-core mode, calibrating all ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^,372 observations took about 7.5 days. The average runtime was about 9.5 minutes per observation, the median was 4.2 minutes, the minimum was ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^6 seconds, and the maximum was 2 hours. More than half of total wall-clock time was spent in I/O-heavy tasks such as data loading and plotting rather than in the calibration algorithms themselves.

6. Scope, failure modes, and significance

VIPCALs is presently optimized and validated for centimeter-wavelength continuum VLBI, especially radio-loud AGN with flux densities above roughly ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2search_query2^^^^ mJy, and especially for VLBA data (&&&^^^^2search_query2^^^^&&&). It does not yet support polarization calibration, is not designed for spectral-line reduction, does not automatically identify phase-reference calibrators, does not automatically handle subarrays, and omits opacity correction, which limits amplitude accuracy at 22 GHz and above. Support for non-VLBA antennas remains limited, even though arrays such as the HSA can include highly sensitive dishes. At low frequencies, the paper notes that spectral-index effects across wide fractional bandwidths and RFI will require additional automated handling. EVN support is described as feasible in principle, but metadata conventions differ and some geometric-correction steps are array-specific.

The paper’s analysis of the 45 failed sources makes the current boundaries of full unsupervised operation explicit. 2^^^^2search_query2^^^^ sources failed because of non-standard TY/GC tables, including malformed formatting, typos, unrelated bands, or missing files. ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^3 sources failed because of non-ordered IF setups, particularly legacy ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^994–^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2^^^^995 observations in which IFs from different bands were interleaved. ^^^^2id:(Álvarez-Ortega et al., 18 Aug 2025) OR ti:\2search_query2^^^^ sources involved subarrays, often due to archive-labeling issues, and 2 sources failed because of incorrect ordering metadata in the FITS header. In each case, the pipeline preferred transparent failure to unsafe automatic correction.

The significance of VIPCALs lies in the degree of automation it brings to a classical AIPS reduction flow. The paper explicitly contrasts it with semi-automated systems such as VLBARUN and rPICARD, which still require user input for calibrator identification, fringe-fit validation, and parameter choice. VIPCALs automates those expert decisions while preserving standard calibration physics, standard AIPS tasks, and a conservative operational posture. For heterogeneous archival surveys, this combination of full automation, reproducibility, and diagnostics-rich output is the key contribution. It enables large-sample VLBI programs such as SMILE to process thousands of datasets in minutes per observation rather than expert-hours per dataset, thereby shifting VLBI continuum calibration from an artisanal workflow toward scalable infrastructure.