---
title: 'VIPCALs: Automated VLBI Calibration Pipeline'
url: https://www.emergentmind.com/topics/vipcals
type: topic
---

# VIPCALs: Automated VLBI Calibration Pipeline

Searching arXiv for the cited VIPCALs paper and closely related VLBI calibration context.
arxiv_search.query{"search_query":"id:2508.13282 OR ti:\"VIPCALs: A fully-automated calibration pipeline for VLBI data\"","max_results":5}
Attempting direct arXiv lookup for the VIPCALs paper.
Searching arXiv.
to=arxiv_search.query code:
{"search_query":"2508.13282","max_results":5}
VIPCALs, short for **VLBI Pipeline for automated data Calibration using AIPS**, is a fully automated calibration pipeline for **continuum VLBI** data. It is designed to convert raw correlated VLBI visibilities into science-ready calibrated datasets **without human intervention and without requiring prior knowledge of the observation**. Implemented in **Python** using **ParselTongue** as an interface to **AIPS**, VIPCALs reproduces the standard AIPS calibration workflow in a fully unsupervised mode, while also automating decisions that are usually left to expert operators, including reference-antenna selection, calibrator identification, fringe-fit configuration, and diagnostic generation. Its stated motivation is large-scale, heterogeneous archival processing, particularly for projects such as **SMILE**, where manual or semi-automated reduction would be a bottleneck [2508.13282].

## 1. Problem setting and design objective

VLBI calibration is substantially more demanding than calibration for connected-element interferometers because antennas are widely separated, record data independently, and retain antenna-dependent clock, geometric, atmospheric, ionospheric, gain, and bandpass errors after correlation. The paper emphasizes that, unlike facilities such as the VLA or ALMA, VLBI users are generally not delivered science-ready products; instead, they receive raw correlated visibilities. Existing pipelines are described as largely semi-automated and still dependent on user supervision for steps such as choosing calibrators, selecting a reference antenna, inspecting fringe-fit results, and tuning parameters.

VIPCALs is presented as a response to that operational gap. Its target regime is **survey-scale continuum VLBI**, especially heterogeneous archival data spanning many years, observing bands, and scheduling conventions. The motivating example is the **Search for Milli-Lenses (SMILE)** project, which aims to analyze nearly 5,000 radio-loud sources using archival VLBA observations. The pipeline’s design philosophy is explicitly conservative: preserve as much data as possible, avoid risky automatic edits to questionable metadata, and fail transparently when assumptions are violated. Rather than attempting aggressive recovery from malformed inputs, it halts or flags data and records the issue.

A plausible implication is that VIPCALs is not primarily a new calibration formalism; it is an automation framework that encodes established AIPS reduction practice into a reproducible, diagnostics-rich workflow. The novelty lies in how completely the traditional operator role is algorithmized.

## 2. Software basis and calibration model

VIPCALs uses **AIPS** as the calibration engine and orchestrates AIPS tasks from Python through **ParselTongue**. It also uses **matplotlib** for diagnostics and provides a simple GUI written with **PySide6**. The software can be installed as a pip package, provided the user has a dedicated Conda environment and a local installation of **AIPS 31DEC24 or newer**, or it can be run via Docker.

Conceptually, the pipeline follows the standard AIPS table-based calibration model. The paper states that observed visibilities are modeled through the **Hamaker-Bregman-Sault measurement equation**, in which the true source coherence matrix is corrupted by antenna-based Jones matrices. Calibration then proceeds by estimating those antenna-based corruptions and storing them in AIPS tables. Newly derived corrections are written first to **SN** tables and then interpolated and accumulated into successive **CL** tables.

| AIPS table | Role in VIPCALs |
|---|---|
| **CL** | Accumulated calibration tables |
| **SN** | Newly derived calibration solutions |
| **BP** | Frequency-dependent gains |
| **TY** / **GC** | Amplitude information |
| **FG** | Flags |
| **NX** | Indexing/bookkeeping |

The pipeline supplements native AIPS logging with a structured summary log and a CSV file containing run metadata, timings, source and calibrator rankings, flagged records, and other diagnostics. This bookkeeping is central to unattended processing because large-scale use requires rapid post hoc inspection of quality and failure modes.

## 3. End-to-end calibration workflow

The workflow begins with **data loading**. VIPCALs checks whether a dataset contains widely separated central frequencies, either as distinct IF groups or different frequency IDs, and, if so, loads them into AIPS as separate entries. It can also concatenate multiple uvfits or idifits files when they share an identical frequency setup. To reduce runtime, it loads only the target source or sources plus up to three bright calibrator candidates unless instructed otherwise. These candidates are found automatically by cross-matching source coordinates against the NRAO **VLBA calibrator list** using a **5 arcsecond** radius and selecting the three brightest matched sources in the observed band. Data import uses **FITLD**, and the calibration table entry interval is set to **6 seconds**.

Pre-calibration preparation includes cleaning source and antenna name strings, verifying time-baseline ordering, reordering with **UVSRT** if necessary, and generating **NX1** and **CL1** with **INDXR** if the index table is missing. The pipeline deliberately avoids aggressive a priori flagging. Beyond correlator-supplied flags and conservative system-temperature cleaning, most rejection is deferred.

Auxiliary table retrieval is handled automatically where possible. VIPCALs can fetch project-specific **FG**, **TY**, and **WX** tables from NRAO repositories and import them with **ANTAB**. System temperatures are smoothed with **TYSMO**, with a conservative rejection rule: values are discarded if they are negative, exceed **1500 K**, or differ by more than **250 K** from the mean value for a given source. VLBA gain curves are parsed from the common gain-curve repository by matching epoch and antenna. If non-VLBA antennas lack TY or GC support, they are flagged in **FG2**.

The pipeline can also apply a **phase-center shift** with **UVFIX** when more accurate source positions are available. The paper notes that offsets larger than roughly **1 arcsecond** from the correlated phase center can significantly degrade fringe fitting and increase smearing. Optional averaging is available: frequency averaging is applied when channel widths are below **500 kHz**, and time averaging is applied only if the original integration time is **1 second or shorter**, in which case the data are averaged into **2-second** bins.

Propagation and geometric calibration then proceed in classical AIPS order. VIPCALs retrieves **IONEX** files and applies ionospheric corrections with **TECOR**, storing the result in **CL2**. The paper writes the ionospheric delay as
$$
\tau_{\text{iono}} \simeq \frac{40.3}{\nu^2} \cdot \sec z \cdot \int N_e ds,
$$
where $\nu$ is frequency and $\int N_e ds$ is the line-of-sight TEC. **CLCOR** is then used for **Earth Orientation Parameter** updates and for **parallactic-angle** correction, producing **CL3** and **CL4** where supported.

Instrumental and amplitude calibration follow. VIPCALs applies **ACCOR** for digital sampling correction, **FRING** for instrumental delays, **BPASS** for complex bandpass calibration, **ACSCL** for autocorrelation-based amplitude renormalization, and **APCAL** for amplitude calibration using **TY** and **GC** tables. The present version does **not** apply atmospheric opacity correction, a limitation that is noted as particularly relevant above about **22 GHz**. Final target fringe fitting is performed with **FRING**, and calibrated data are exported with **SPLIT** and **FITTP** once valid final solutions exist on at least one baseline.

## 4. Automated decision logic

The most distinctive part of VIPCALs is its automation of decisions that are usually performed interactively by experienced VLBI users.

Reference-antenna selection is fully automated. The pipeline first retains only antennas present in all scans, including calibrator and target scans; if that criterion is too strict, it retains antennas present in all target scans. For VLBA observations, it can preferentially consider geographically central antennas—**KP, LA, PT, OV, FD**—if they satisfy the coverage criterion. The surviving antennas are then ranked by their average fringe S/N from the FFT stage of **FRING**. To estimate these scores, VIPCALs runs the FFT stage for up to **10 randomly selected scans per source**, using a search window of **1000 ns** in delay and **200 mHz** in rate, a solution interval equal to the scan length, and **aparm(7)=1** to suppress normal thresholding. The highest-ranked antenna becomes the primary reference antenna, while the remaining ranked antennas are passed to FRING through the **search** parameter as fallback references.

Calibrator-scan identification is likewise automatic. VIPCALs runs **FRING** in FFT mode over all scans and all sources, solving for single-band delay and fringe rate relative to the chosen reference antenna, again using **1000 ns** and **200 mHz** search windows and a fringe **S/N threshold of 5**. For each antenna, the scan with the highest fringe S/N is chosen as that antenna’s calibrator scan. If the best S/N for an antenna does not exceed **5**, that antenna is flagged and removed from subsequent calibration, with the result written to **FG3**.

Target fringe fitting is automated through a solution-interval search. VIPCALs runs a preliminary FRING on a short data segment and chooses the **shortest interval** that yields fringe detections with **S/N $\ge 5$** on all baselines; if no such interval is found, it uses the full scan length. For the SMILE use case, the default minimum and maximum intervals are **1 minute** and **10 minutes**. During science-target fringe fitting, the pipeline again uses the ranked reference-antenna list through **aparm(9)** and **search**, with the same **1000 ns** delay window, **200 mHz** rate window, and **S/N threshold of 5**. If single-band fitting is insufficient, VIPCALs retries with a **multi-band delay** fit by setting **aparm(5)=1**, and then chooses between the single-band and multi-band runs according to which produces the higher **success rate**, defined as the ratio of valid solutions to expected solutions.

This suggests that the automation strategy is heuristic but strongly grounded in standard VLBI practice. Rather than learning calibration policies from data, VIPCALs codifies expert procedural choices into deterministic rules with explicit thresholds and fallback behavior.

## 5. Validation, sample construction, and performance

Validation was performed on a large and deliberately heterogeneous **SMILE-related VLBA sample** [2508.13282]. SMILE itself is based on **4,968** radio sources selected from CLASS with a flux-density threshold of **50 mJy at 8.4 GHz**. For VIPCALs testing, the authors constructed a representative sample of **1,000 sources**: **450** uniformly distributed in flux density between **50 mJy and 1 Jy**, **450** uniformly distributed in exposure time from **1 minute to 1 hour**, and **100** observed at higher frequencies in **U band and K band**. Archive selection required at least **C-band or X-band** data, phase centers within **1 arcminute** of the CLASS coordinates, and compatible files from the same project within a **2-day** interval. When multiple candidates existed, the dataset with the longest on-source integration was selected.

The resulting benchmark comprised **1,417 observations**—defined as source-frequency pairs—of **1,000 sources**, drawn from **360 VLBA projects**, spread across **2,589 files** totaling **19 TB**, and spanning **1994 to 2025**. A source was counted as successfully processed if the pipeline completed all steps and produced calibrated output. By that criterion, VIPCALs successfully calibrated **955 of the 1,000 sources**, corresponding to **1,372 individual observations**.

The reported retention and fringe-fitting statistics are central. Across the successfully completed sample, the median **final visibility ratio**—the ratio of calibrated retained visibilities to the total starting visibilities after excluding unsupported non-VLBA antennas—was **0.87**, and the mean was **0.78**. **87.4%** of observations retained more than half of their original visibilities, while only **76 observations** (**7.2%**) retained fewer than **20%**. The final target-fringe-fit statistics were also strong: **91.6%** of calibrated datasets achieved successful fringe fitting on the target in at least **half** of the attempted solution intervals.

Runtime was compatible with survey-scale use. In **single-core** mode, calibrating all **1,372 observations** took about **7.5 days**. The **average** runtime was about **9.5 minutes per observation**, the **median** was **4.2 minutes**, the minimum was **16 seconds**, and the maximum was **2 hours**. More than half of total wall-clock time was spent in I/O-heavy tasks such as data loading and plotting rather than in the calibration algorithms themselves.

## 6. Scope, failure modes, and significance

VIPCALs is presently optimized and validated for **centimeter-wavelength continuum VLBI**, especially **radio-loud AGN** with flux densities above roughly **10 mJy**, and especially for **VLBA** data [2508.13282]. It does **not** yet support **polarization calibration**, is not designed for **spectral-line** reduction, does not automatically identify **phase-reference calibrators**, does not automatically handle **subarrays**, and omits **opacity correction**, which limits amplitude accuracy at **22 GHz and above**. Support for non-VLBA antennas remains limited, even though arrays such as the HSA can include highly sensitive dishes. At low frequencies, the paper notes that spectral-index effects across wide fractional bandwidths and **RFI** will require additional automated handling. EVN support is described as feasible in principle, but metadata conventions differ and some geometric-correction steps are array-specific.

The paper’s analysis of the **45 failed sources** makes the current boundaries of full unsupervised operation explicit. **20 sources** failed because of **non-standard TY/GC tables**, including malformed formatting, typos, unrelated bands, or missing files. **13 sources** failed because of **non-ordered IF setups**, particularly legacy **1994–1995** observations in which IFs from different bands were interleaved. **10 sources** involved **subarrays**, often due to archive-labeling issues, and **2 sources** failed because of incorrect ordering metadata in the FITS header. In each case, the pipeline preferred transparent failure to unsafe automatic correction.

The significance of VIPCALs lies in the degree of automation it brings to a classical AIPS reduction flow. The paper explicitly contrasts it with semi-automated systems such as **VLBARUN** and **rPICARD**, which still require user input for calibrator identification, fringe-fit validation, and parameter choice. VIPCALs automates those expert decisions while preserving standard calibration physics, standard AIPS tasks, and a conservative operational posture. For heterogeneous archival surveys, this combination of full automation, reproducibility, and diagnostics-rich output is the key contribution. It enables large-sample VLBI programs such as SMILE to process thousands of datasets in minutes per observation rather than expert-hours per dataset, thereby shifting VLBI continuum calibration from an artisanal workflow toward scalable infrastructure.

Source: https://www.emergentmind.com/topics/vipcals