Caracal: Astronomy & ML Systems

• Caracal in radio astronomy is the automated, end-to-end calibration and reduction pipeline used to process MeerKAT visibilities into science-ready H I data.
• Caracal in machine learning is a decoder-only language model that replaces global self-attention with a Fourier-based Multi-Head Fourier module for efficient long-sequence modeling.
• The shared name 'Caracal' highlights two distinct technical contexts, emphasizing the importance of domain-specific interpretation in both astronomical and neural architectures.

Searching arXiv for the two provided Caracal-related papers to ground the article in the cited literature. Caracal denotes two distinct technical systems in recent literature. In radio astronomy, CARACal is the Containerized Automated Radio Astronomy Calibration pipeline, a calibration/reduction framework used to transform raw MeerKAT interferometric visibilities into science-ready H I products for analysis of Hickson Compact Groups. In machine learning, Caracal is a decoder-only LLM architecture that replaces global self-attention with a Fourier-based token mixer called Multi-Head Fourier (MHF) in order to obtain autoregressive long-sequence modeling with $\mathcal{O}(L \log L)$ sequence mixing. The shared name therefore spans two unrelated technical contexts: end-to-end radio data reduction and efficient causal sequence modeling (Ianjamasimanana et al., 13 Feb 2025, Gan et al., 30 Apr 2026).

1. Disambiguation and nomenclature

The two usages differ in capitalization, domain, and technical role.

Form	Domain	Definition
CARACal	Radio astronomy	Containerized Automated Radio Astronomy Calibration pipeline
Caracal	Machine learning	Causal Architecture via Spectral Mixing

In the MeerKAT Hickson Compact Group study, CARACal is described as a calibration/reduction pipeline that integrates several radio-astronomy software packages and runs them through a Stimela workflow. The paper explicitly states that CARACal is not a single algorithm; it is the orchestrator that automates the full MeerKAT reduction chain. In the long-sequence modeling paper, Caracal is presented as a decoder-only architecture that is structurally close to a standard GPT-style Transformer but replaces masked multi-head self-attention with the Multi-Head Fourier module and removes explicit positional encodings (Ianjamasimanana et al., 13 Feb 2025, Gan et al., 30 Apr 2026).

This distinction matters because the same surface term can otherwise obscure fundamentally different technical objects: one is an end-to-end software framework for H I imaging, whereas the other is a neural architecture for autoregressive language modeling.

2. CARACal as a radio-astronomy calibration and reduction pipeline

CARACal was the main end-to-end software framework used in the MeerKAT Hickson Compact Group study to produce the science-ready H I data products analyzed in the paper. Its documented functions include flagging of bad data and radio-frequency interference, cross-calibration on standard calibrators, continuum imaging and self-calibration, continuum subtraction, spectral-line imaging, and diagnostic plotting and quality checks. The pipeline integrates multiple packages within a Stimela workflow, providing a scripted and reproducible reduction path from raw visibilities to calibrated, continuum-subtracted line data (Ianjamasimanana et al., 13 Feb 2025).

The reduction chain was organized into three main workers. The first worker handled flagging and cross-calibration. Auto-correlations and shadowed antennas were flagged with CASA flagdata inside CARACal, and RFI was flagged using AOFlagger. Cross-calibration on the primary calibrator used the sequence KGBKGB, corresponding to time-dependent delay calibration (K), gain calibration (G), and bandpass calibration (B). Those solutions were then transferred to the secondary calibrator: KB was applied, gain amplitude and phase were solved for, spurious calibrated visibilities were flagged, and the solution was repeated with flux-scale bootstrapping using GAF and calmode: [ap, null, ap]. For the target, CARACal performed on-the-fly application of the primary delay/bandpass and secondary gains while also flagging auto-correlations, shadowed antennas, and RFI (Ianjamasimanana et al., 13 Feb 2025).

The second worker covered continuum imaging and self-calibration. This stage used WSClean for imaging and Cubical for self-calibration. Imaging and self-calibration were repeated iteratively; each iteration used the previous clean mask, multi-scale cleaning was enabled, and scales were selected automatically. The reported parameter choices were an image size of 2° × 2°, a pixel size of 1 arcsec, Briggs -1 weighting, 0 arcsec tapering, and three calibration iterations. WSClean clean-mask thresholds were 30, 20, 10, and 5 × rms, and the WSClean auto-threshold parameter was 0.5. These steps produced continuum models for later subtraction from the line data (Ianjamasimanana et al., 13 Feb 2025).

The third worker implemented continuum subtraction and line imaging. The continuum model was subtracted from the target visibilities, and CARACal additionally used CASA mstransform, Doppler-tracking corrections, and a first-order polynomial fit to the real and imaginary parts of the line-free channels. The mstransform outputs were re-flagged with AOFlagger, and line cubes were made with WSClean using multi-scale cleaning and a major-cycle clean gain of 0.2. The authors first created a cube at 98 arcsec, then used SoFiA outside CARACal to make a better clean mask, reran CARACal using that mask to create the final lowest-resolution cube, and repeated this mask-refinement process stepwise for higher-resolution cubes (Ianjamasimanana et al., 13 Feb 2025).

3. CARACal-generated products, observing setup, and scientific role

The MeerKAT study used CARACal to produce data cubes at multiple angular resolutions, moment maps including moment 0 column density maps and moment 1 velocity fields, integrated spectra for each Hickson Compact Group, and the calibrated, continuum-subtracted line data needed for source finding. From the CARACal-generated cubes, the authors derived RA–velocity slices, channel maps, moment maps, position–velocity diagrams, flux measurements, and H I masses. These cubes formed the basis for comparison with previous VLA and GBT observations (Ianjamasimanana et al., 13 Feb 2025).

Additional imaging details are explicit. Cubes were produced at 60 arcsec and at higher resolution down to about 20 kpc linear resolution, with intermediate-resolution cubes also provided. The 60 arcsec cube was the default unless stated otherwise. Primary-beam attenuation was corrected using the model of Mauch et al. (2020), frequency was converted to velocity with MIRIAD VELSW, and the paper used the optical velocity definition. For comparison to GBT fluxes, the authors modeled the GBT beam as a 2D Gaussian with FWHM 9.1 arcmin:

$f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$

with $C3 = 0$, $C4 = 1$, and $C5 = 9.1 \times 60 / 2.355$ arcsec. This beam-response map was multiplied with each MeerKAT channel before extracting the integrated spectrum. H I mass was computed from the primary-beam-corrected integrated flux as

$$M_{\mathrm{HI}}[M_{\odot}] = 235600 \times D^2 \times \sum S_i \Delta v,$$

where $D$ is distance in Mpc and $\sum S_i \Delta v$ is the integrated flux in Jy km s$^{-1}$. For visual comparison of MeerKAT and GBT spectra, both were smoothed to 20 km s$^{-1}$ with a boxcar kernel (Ianjamasimanana et al., 13 Feb 2025).

The observing configuration is also specified: the MeerKAT L-band receiver covered 856–1711.974 MHz in 32k correlator mode with central observing frequency (1389.1322) MHz, channel width 26.123 kHz, velocity resolution 5.5 km s$f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$0 at 1420.4 MHz, 32768 total channels, 8 s integration time, about 5.13 h per group on source, and 61–63 antennas used (Ianjamasimanana et al., 13 Feb 2025).

The scientific conclusions that depended on these reduced products were sharply comparative. Relative to previous VLA observations, MeerKAT revealed much more extended tidal features in phase 2 groups and some new high surface brightness features in phase 3 groups, but no diffuse H I component was found in phase 3 groups. The difference between phase 2 and phase 3 groups therefore remained substantial, supporting previous findings that the transition between the two phases must be abrupt. The reduced data also enabled the identification of tidal tails, bridges, clumps, surrounding galaxies, and disturbed kinematics in several groups. The paper explicitly notes a known issue with UVLIN-like continuum subtraction—vertical stripes in RA–velocity slices where line channels are excluded from the fit—and reports that this effect was checked in the CARACal products (Ianjamasimanana et al., 13 Feb 2025).

4. Caracal as a decoder-only sequence model

In machine learning, Caracal is introduced as a decoder-only LLM architecture that replaces global self-attention with a Fourier-based token mixer called the Multi-Head Fourier (MHF) module. The model is proposed to address three stated limitations of long-context Transformers: the quadratic cost of attention, $f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$1; the need for explicit positional encodings; and the non-causality of prior FFT-based token mixers such as FNet. Caracal is therefore designed to be efficient, autoregressive, and portable across environments because it uses standard FFT and convolution operators rather than hardware-specialized kernels (Gan et al., 30 Apr 2026).

The architecture remains close to a GPT-style Transformer with two major changes: masked multi-head self-attention is replaced by MHF, and explicit positional encodings are removed. To improve local pattern capture, the model retains a small number of sliding-window attention (SWA) layers. In the default configuration, one SWA layer is interleaved after every two MHF layers, giving a 2:1 MHF:SWA ratio. This makes Caracal a hybrid architecture in which global sequence mixing is handled by Fourier-domain operations while local precision is supported by occasional windowed attention (Gan et al., 30 Apr 2026).

The motivation is framed in explicitly comparative terms. Transformer attention is expensive at long lengths; positional encoding schemes are treated as sophisticated fixes for a permutation-equivariant mechanism; and prior Fourier models are typically non-causal, which prevents direct autoregressive generation. Caracal is proposed as a single response to these issues: FFT-based mixing for efficiency, removal of explicit positional encodings, and a causal frequency-domain procedure for generation (Gan et al., 30 Apr 2026).

5. Multi-Head Fourier mechanics and causal sequence mixing

The MHF forward pass begins with a local causal pre-convolution,

$f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$2

described as a lightweight depthwise causal convolution, typically with kernel size 3 and left padding. This injects local inductive bias and compensates for the removal of positional encoding. After normalization,

$f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$3

the module forms a value stream

$f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$4

and a gate stream

$f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$5

where $f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$6 is SiLU and $f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$7 is a 1x1 grouped convolution with $f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$8 groups. The paper identifies this gate stream as the source of content dependence in the mixing rule (Gan et al., 30 Apr 2026).

Causal mixing is then performed by padding the sequence to length $f(R) = C3 + (C4 - C3)\,\exp[-((0.707/C5)^2\,R^2)]$9, taking FFTs of the value and gate streams,

$C3 = 0$0

multiplying them elementwise,

$C3 = 0$1

and applying inverse FFT followed by truncation,

$C3 = 0$2

with a final output projection

$C3 = 0$3

The paper characterizes this as a “pad-FFT-multiply-iFFT-truncate” pipeline. Its complexity is $C3 = 0$4 because the dominant operations are FFT and inverse FFT rather than the $C3 = 0$5 matrix formation of dense attention (Gan et al., 30 Apr 2026).

The multi-head formulation makes the convolutional interpretation explicit. After reshaping, $C3 = 0$6, and for each batch $C3 = 0$7, head $C3 = 0$8, and channel $C3 = 0$9,

$C4 = 1$0

with

$C4 = 1$1

The resulting matrix is described as a lower-triangular Toeplitz matrix for each channel:

$C4 = 1$2

This means the mixing depends on relative offsets $C4 = 1$3 rather than absolute positions. The paper therefore places MHF between two familiar constructions: Fourier mixing with static structured weights and attention with input-dependent weighted sums (Gan et al., 30 Apr 2026).

The key causal mechanism is not a literal mask in frequency space but an asymmetric padding and truncation strategy. Without zero-padding, FFT multiplication corresponds to circular convolution, and future terms can wrap into earlier outputs. Padding both sequences to $C4 = 1$4 moves those wrap-around terms outside the causal region; truncating to the first $C4 = 1$5 positions yields the desired causal convolution,

$C4 = 1$6

This directly addresses the standard objection that Fourier token mixers are non-causal (Gan et al., 30 Apr 2026).

6. Empirical performance, portability, and limitations of Caracal

The Caracal paper evaluates the architecture on language modeling, commonsense reasoning, long-context extraction/retrieval, and efficiency scaling. Across Tiny, Small, Medium, and Large scales, Caracal is described as competitive with Transformer and SSM baselines rather than uniformly dominant. On the Large setting, the reported results are Lambada perplexity: 29.39, Lambada accuracy: 35.26, and average benchmark accuracy: 49.01. In a broader comparison using a 345M-parameter configuration and the prior-work evaluation protocol, Caracal (Default) attains the best average accuracy in the comparison table with 46.62 (Gan et al., 30 Apr 2026).

The long-context retrieval results are more mixed. On SWDE and FDA, the paper states that Llama performs better on exact retrieval-style tasks, with Llama avg: 9.55 and Caracal avg: 5.37. The authors describe this as a “resolution gap”, and dense attention is said to retain an advantage in fine-grained retrieval. This is one of the principal limitations explicitly acknowledged by the paper (Gan et al., 30 Apr 2026).

Efficiency is the clearest empirical strength. Training time and token throughput were measured from context lengths 256 to 8192. At context length 8192, the reported training times are Llama: 94,258 s, Caracal default: 34,071 s, and Mamba2: 33,956 s. Throughput at the same length is Llama: 106,091 tokens/s, Caracal default: 293,504 tokens/s, and Mamba2: 294,498 tokens/s. The hybrid Caracal with SWA is stated to be slightly faster than pure MHF in some settings because the SWA layers benefit from highly optimized FlashAttention kernels (Gan et al., 30 Apr 2026).

Ablation studies isolate the contributions of local modeling and gating design. The tested variants are w/o SWA, w/o SWA + PC, with PE, and SSLP. The reported conclusions are that SWA and pre-convolution help local modeling, that adding positional encodings does not help much, and that the two-stage gating design outperforms a simpler single-stage projection. The best hybrid ratio in the parameter study is 2:1 MHF:SWA; increasing SWA beyond that slightly hurts performance (Gan et al., 30 Apr 2026).

Portability is framed as a major architectural distinction from Mamba-like SSMs. Caracal uses standard FFT operators, standard convolution, standard linear layers, and standard FlashAttention only for the small SWA component. By contrast, the paper characterizes Mamba and Mamba-2 as architectures whose best implementations often rely on custom CUDA kernels, hardware-aware scan algorithms, and specialized tuning for block sizes and memory layout. This suggests that Caracal seeks a specific balance between scalability and engineering simplicity rather than asymptotic optimality alone (Gan et al., 30 Apr 2026).

The paper is explicit about remaining limitations: retrieval precision is weaker than dense attention on some tasks, large-scale validation remains future work, and inference optimization would require more specialized engineering for production-grade throughput. Proposed future directions include multi-scale gating, hierarchical mixing, and better retrieval-oriented refinements (Gan et al., 30 Apr 2026).