Causal-Adapter: Modular Causal Interfaces

• Causal-Adapter is a family of modular causal augmentation mechanisms that attach to pre-trained backbones, enabling causal discovery and intervention without retraining the entire model.
• It employs learnable decoders, such as in the TabPFN setting, to extract adjacency matrices from frozen embeddings using dual-attention layers and structured pooling methods.
• Variants of Causal-Adapter span tabular data, text-to-image diffusion, and mobility-informed forecasting, highlighting its adaptability and potential for efficient causal model specialization.

Searching arXiv for papers using the term “Causal-Adapter” and closely related usage. Causal-Adapter denotes an adapter-style causal interface attached to an existing model so that causal structure can be extracted, injected, or enforced without retraining the full backbone. In "Does TabPFN Understand Causal Structures?" the term refers to a learnable decoder with causal tokens that reads adjacency matrices from frozen TabPFNv2 embeddings for causal discovery (Swelam et al., 10 Nov 2025). In other arXiv usages, the same or closely related designation describes a modular side-arm for text-to-image diffusion counterfactual generation (Tong et al., 29 Sep 2025), a mobility-informed forecasting module (Guo et al., 16 Jan 2026), and, in descriptive usage, a causal-transfer procedure based on Markov blanket discovery (Javidian et al., 2021). This suggests that the term functions less as a single standardized architecture than as a family of causal augmentation mechanisms built on top of pretrained or lightweight models.

1. Terminological scope and research usage

The term appears in multiple technically distinct settings, but the common motif is architectural modularity: a base model remains frozen or largely unchanged, while a smaller mechanism mediates access to causal structure, invariant features, or intervention effects.

Work	Backbone or setting	Adapter role
TabPFN Causal-Adapter (Swelam et al., 10 Nov 2025)	Frozen TabPFNv2 encoder	Decodes DAG adjacency matrices from embeddings
Diffusion Causal-Adapter (Tong et al., 29 Sep 2025)	Frozen text-to-image diffusion backbone	Injects causal semantic residuals for counterfactual generation
MiCA (Guo et al., 16 Jan 2026)	Lightweight temporal forecasting backbones	Mixes mobility-derived causal priors into hidden states
SCTL (Javidian et al., 2021)	Domain adaptation under covariate shift	Identifies invariant causal feature sets via Markov blanket search

In the tabular setting, the adapter is a probing device: it tests whether a foundation model pretrained on synthetic data from structural causal models has internal representations that encode causal information. In the diffusion setting, the adapter is an intervention interface: it operationalizes Pearl’s abduction–action–prediction within a frozen generative model. In epidemic forecasting, the adapter is a lightweight relational layer driven by causal discovery on mobility time series. In causal domain adaptation, the phrase is used more descriptively for a procedure that selects invariant features rather than a named neural module.

2. TabPFN-based Causal-Adapter: objective and motivation

The TabPFN variant is grounded in Prior-Data-Fitted Networks, which are transformers pre-trained on massive collections of small synthetic tabular datasets, each labeled with the correct prediction. TabPFN is specifically pre-trained on data generated from Structural Causal Models, where for each variable

$x_j \leftarrow f_j(x_{pa(j)}) + \epsilon_j,$

with $\epsilon_j$ as noise and $f_j$ possibly linear or a random-Fourier-feature function (Swelam et al., 10 Nov 2025).

This pre-training regime motivates a precise empirical question: whether TabPFN’s internal representations encode causal structure beyond statistical correlation. The Causal-Adapter addresses that question by freezing TabPFNv2’s encoder and attaching a lightweight, trainable decoder whose task is to read out a causal graph, represented as an adjacency matrix, from frozen embeddings. If the adapter can learn to predict DAGs accurately, that is evidence that causal signals are already present in the base model’s hidden states.

The paper frames this as an interpretability and adaptability problem for tabular foundation models. Rather than retraining TabPFN for causal discovery, it uses the pretrained encoder as a fixed representation learner and asks whether a dedicated readout head can extract structural information. This setup is central to the paper’s claim that causal information is embedded in TabPFN and is not merely an artifact of downstream supervised fitting.

3. Architecture and mathematical formulation in the TabPFN setting

The TabPFN Causal-Adapter has three stages: a shared embedding plus frozen encoder, a learnable decoder driven by causal tokens, and a DAG prediction head. The input is a dataset $X \in \mathbb{R}^{n \times f}$ with $n$ samples and $f$ features. Observational and interventional samples are flagged by a binary side-channel, so the effective input shape is treated as $(n,f,2)$. A projection layer $\phi:\mathbb{R}^2 \to \mathbb{R}^d$ maps each value–flag pair to the embedding space, giving

$$H_0 = \phi(X) \in \mathbb{R}^{n \times f \times d},$$

with $d=192$ in TabPFNv2. The frozen encoder applies $\epsilon_j$0 dual-attention layers over samples and features to produce

$\epsilon_j$1

The decoder introduces $\epsilon_j$2 learnable “universal causal tokens”

$\epsilon_j$3

with $\epsilon_j$4. These tokens query the frozen hidden states through dual-attention blocks that mirror TabPFN’s encoder, except that the decoder performs cross-attention from causal tokens to data tokens. After $\epsilon_j$5 decoder layers,

$\epsilon_j$6

yielding $\epsilon_j$7 (Swelam et al., 10 Nov 2025).

Aggregation is moment-based. For each feature, the model computes element-wise max, min, mean, and std across the token dimension, so that $\epsilon_j$8 pooled summaries are concatenated:

$\epsilon_j$9

Two small linear maps split each feature summary into “parent” and “child” embeddings, and directed edges are scored by a dot product followed by a sigmoid:

$f_j$0

Collecting all $f_j$1 yields the predicted adjacency matrix $f_j$2.

The training objective combines binary cross-entropy over all off-diagonal adjacency entries with an acyclicity penalty. The edge-level term is

$f_j$3

and acyclicity is enforced by constraining the largest eigenvalue proxy $f_j$4 to remain below $f_j$5 using an augmented-Lagrangian penalty tracked by power iteration. The full objective is

$f_j$6

The ground-truth synthetic data are generated from a more general SCM,

$f_j$7

with $f_j$8 drawn i.i.d. from Gaussian, Laplace, or Cauchy distributions. This design makes the adapter’s target explicitly structural rather than merely predictive.

4. Training protocol and empirical findings for causal discovery

The implementation fixes the TabPFN encoder and uses only the first $f_j$9 of its $X \in \mathbb{R}^{n \times f}$0 layers, because ablations showed that layers $X \in \mathbb{R}^{n \times f}$1–$X \in \mathbb{R}^{n \times f}$2 carry the strongest causal signal. The adapter uses $X \in \mathbb{R}^{n \times f}$3 causal tokens, pools them to $X \in \mathbb{R}^{n \times f}$4 summaries by max, min, mean, and std, operates at embedding dimension $X \in \mathbb{R}^{n \times f}$5, and adds approximately $X \in \mathbb{R}^{n \times f}$6 million learnable parameters out of roughly $X \in \mathbb{R}^{n \times f}$7 million total. Optimization uses AdamW with initial learning rate $X \in \mathbb{R}^{n \times f}$8, cosine annealing, and batch size $X \in \mathbb{R}^{n \times f}$9 datasets. Training datasets are sampled on the fly with $n$0 features and mixtures of observational $n$1 and interventional $n$2 or $n$3 points, for $n$4 steps on synthetic SCMs of varied graph topologies (Swelam et al., 10 Nov 2025).

Evaluation uses $n$5 held-out SCM datasets with $n$6, each containing $n$7 observational and $n$8 interventional samples. Baselines are AVICI, GIES, IGSP, and DCDI on a subset of $n$9 datasets. The reported metrics are ROC–AUC and average precision over all possible edges.

The main quantitative findings are specific. On ROC–AUC, the Causal-Adapter matches AVICI and clearly outperforms GIES, IGSP, and DCDI. On average precision, it outperforms classical methods but degrades as the number of features grows, which the paper interprets as difficulty resolving false positives in larger graphs. The layerwise analysis shows that mid-range layers, specifically layers $f$0–$f$1, concentrate most of the implicit causal signal, while using layer-$f$2 or layer-$f$3 representations yields substantially worse DAG recovery. The weight-quality ablation shows that freezing the optimal pre-trained encoder is crucial: random weights or a worse fine-tuned encoder lead to markedly weaker DAG accuracy. Decoder ablations further show that a dedicated learnable decoder is needed; simply reusing TabPFN’s own attention without a decoder underperforms substantially.

Taken together, these results support the paper’s central claim that TabPFNv2 learned structural knowledge of the data-generating SCMs during predictive-task pre-training. The work describes this as a new paradigm of “adapter probing” for tabular foundation models (Swelam et al., 10 Nov 2025).

5. Other architectures named or described as Causal-Adapter

In text-to-image diffusion, "Causal-Adapter: Taming Text-to-Image Diffusion for Faithful Counterfactual Generation" uses the same name for a modular framework that adapts frozen diffusion backbones for counterfactual image generation (Tong et al., 29 Sep 2025). The model combines an explicit SCM over semantic attributes, a half-scale adapter network that injects residuals into a frozen denoiser, Prompt-Aligned Injection, and a Conditioned Token Contrastive loss. Inference follows abduction, action, and prediction: DDIM inversion recovers exogenous noise, a $f$4 intervention is propagated through the SCM, and the denoising process is rerun under updated residuals. Reported results include up to $f$5 MAE reduction on Pendulum and $f$6 FID reduction on ADNI, with the stated goal of faithful attribute modification and strong identity preservation.

In epidemic forecasting, MiCA is explicitly presented as a Mobility-Informed Causal Adapter that integrates mobility relations into temporal forecasting models via gated residual mixing (Guo et al., 16 Jan 2026). Mobility time series are processed with PCMCI and ParCorr tests up to lag $f$7 to obtain a directed influence prior $f$8. MiCA then computes a gated causal message

$f$9

and mixes it residually into the backbone representation through a global gate and output projection. Across COVID incidence, COVID mortality, influenza, and dengue, the paper reports an average relative error reduction of $(n,f,2)$0 across forecasting horizons and performance competitive with state-of-the-art spatio-temporal models while remaining lightweight.

In causal domain adaptation, "Scalable Causal Domain Adaptation" does not title its method Causal-Adapter, but the detailed description characterizes SCTL as one because it identifies invariant causal features and attaches a transfer-relevant causal selection stage to prediction (Javidian et al., 2021). The method formalizes invariance through the criterion

$(n,f,2)$1

in an ADMG over system variables and a context variable. It first recovers the Markov blanket of the target, then searches subsets of that blanket for conditioning sets that render the target independent of context, and finally selects among candidates by source-domain risk. Theoretical results state that any minimal $(n,f,2)$2–$(n,f,2)$3 separator lies inside $(n,f,2)$4 and that standard Markov blanket algorithms recover $(n,f,2)$5 in the large-sample limit.

These usages are technically heterogeneous. The shared element is not a single invariant architecture, but a modular causal interface inserted between representation learning and a downstream objective.

6. Methodological themes, limitations, and significance

Across the tabular, generative, forecasting, and transfer-learning variants, several methodological themes recur. First, the adapter is smaller than the backbone and is attached to a frozen or pre-existing model rather than replacing it. Second, causal information enters through explicit structure: SCM-generated pre-training data and DAG recovery in TabPFN, known attribute graphs in diffusion, PCMCI-derived directed mobility priors in MiCA, and m-separation plus Markov blanket localization in SCTL. Third, each method uses the adapter to target a causal property not guaranteed by the underlying backbone alone: adjacency recovery, counterfactual faithfulness, robustness under noisy mobility structure, or invariant prediction under covariate shift.

The limitations are also domain-specific. In the TabPFN setting, average precision degrades as graph size grows, indicating difficulty with false positives in larger graphs (Swelam et al., 10 Nov 2025). In the diffusion setting, the graph is assumed known, classifier-based evaluation can be brittle for rare counterfactuals, and truly out-of-domain edits may still degrade quality in a fixed backbone (Tong et al., 29 Sep 2025). In MiCA, the approach depends on a leakage-free causal prior extracted from mobility histories and uses only a single spatial adapter layer because epidemic series are short and noisy (Guo et al., 16 Jan 2026). In SCTL, success depends on Markov, faithfulness, and causal domain adaptation assumptions, including the absence of a direct edge $(n,f,2)$6 and preservation of relevant independences across domains (Javidian et al., 2021).

The broader significance lies in what these systems imply about model reuse. The TabPFN results suggest that pretrained tabular models can be repurposed for causal discovery by fitting a small decoder on frozen embeddings. The diffusion results suggest that counterfactual editing can be separated into a frozen generative prior and a causal intervention module. MiCA suggests that lightweight forecasters can incorporate relational structure without graph neural networks or full attention. SCTL suggests that local causal discovery can make invariant feature selection computationally tractable. A plausible implication is that “Causal-Adapter” is emerging as a research idiom for causal specialization without wholesale retraining: the causal component is isolated, trainable, and explicitly structured, while the backbone preserves generic predictive or generative capacity.