FairPFN: Causal Fairness in Tabular Models

• FairPFN is a transformer-based model designed to remove both direct and indirect causal effects of protected attributes, ensuring counterfactual fairness in predictions.
• It leverages pre-training on synthetic data generated from diverse structural causal models, eliminating the need for explicit causal graph specification in real-world tasks.
• Empirical results on synthetic and real-world datasets show competitive predictive accuracy while significantly reducing bias, positioning FairPFN on the Pareto frontier for fairness and performance.

FairPFN is a class of transformer-based tabular prediction models designed to produce counterfactually fair or group-fair predictions by mitigating the causal impact of protected attributes (such as race and gender) in tabular machine learning tasks. By leveraging pre-training on synthetic data generated from structural causal models (SCMs), FairPFN aims to remove both direct and indirect causal effects of protected attributes in its predictions, even when the underlying real-world causal graph is unknown or inaccessible. FairPFN represents a significant paradigm shift in algorithmic fairness, providing a scalable foundation model for causal and counterfactual fairness in tabular data without reliance on domain-specific causal model specification (Robertson et al., 2024, Robertson et al., 8 Jun 2025).

1. Background and Motivation

Algorithmic fairness in machine learning concerns the prevention of unfair bias or discrimination due to protected attributes. Traditional approaches to counterfactual fairness require explicit specification of an SCM describing how observed covariates, protected attributes, and outcomes are causally related. This requirement is a major impediment in practice due to limited domain knowledge and the inaccuracy of automatic causal discovery, often resulting in "fairwashing" or unreliable fairness guarantees.

FairPFN sidesteps this limitation by leveraging the Prior-data Fitted Network (PFN) paradigm—pre-training a transformer on large suites of synthetic datasets drawn from diverse, randomly sampled SCMs. During pre-training, FairPFN learns to integrate out the causal effects of protected attributes by being trained to predict "fair" labels (obtained by intervening on the SCM to force the protected attribute to have no effect) directly from observational data. This enables causal and counterfactually fair predictions at inference time for new datasets, without ever requiring access to the underlying SCM or counterfactuals of the real application domain (Robertson et al., 2024, Robertson et al., 8 Jun 2025).

2. Formalism: Counterfactual Fairness and the PFN Approach

Given an SCM with exogenous background noise $U$, protected attribute $A$, covariates $X$, and target $Y$:

• $U \sim P(U)$,
• $A \leftarrow f_A(U_A)$,
• $X \leftarrow f_X(A, U_X)$,
• $Y \leftarrow f_Y(A, X, U_Y)$.

Counterfactual fairness (Kusner et al., 2017) requires that for any $U$ and for all values $a, a'$ of $A$0, the predictive outcome is invariant under interventions on $A$1: $A$2 A predictor $A$3 is counterfactually fair if the distribution over predictions remains invariant under do-operations on $A$4.

FairPFN addresses this requirement by learning from synthetic SCMs how to erase the total causal effect of $A$5. During pre-training, SCMs are randomly generated: a "biased" dataset $A$6 is constructed via ordinary forward sampling, and a "fair" dataset $A$7 is obtained by re-running with all outgoing edges from $A$8 masked (typically zeroing weights in neural SCMs).

Pre-training then proceeds via minimization of binary cross-entropy between the model's predictions (given the biased data) and the fair labels: $A$9 No explicit additional regularizer for fairness is required, as reproducing $X$0 by construction enforces removal of causal effects (Robertson et al., 2024, Robertson et al., 8 Jun 2025).

3. Model Architecture and Pretraining Regime

FairPFN adopts the transformer-based architecture of TabPFN. The main characteristics are:

• Input encoding: Each tabular dataset is flattened into sequences where every cell (row, column) is embedded with positional, type, and value encodings. Special tokens identify protected attribute columns.
• Transformer core: Multi-head self-attention layers (e.g., $X$1 blocks, $X$2) process the context sequence.
• Prediction head: For each query instance, the model outputs the predictive class probabilities of $X$3; in pre-training this is compared to "fair" counterfactual targets.
• Pretraining protocol: Millions of synthetic SCMs are sampled. Each provides a pair of training datasets: the original with potential unfair effects, and a "fair" version with $X$4's influence structurally removed.

At inference, FairPFN processes only observed biased data and outputs predictions intended to be counterfactually fair. Theoretically, the model approximates: $X$5 where $X$6 is the meta-prior over SCMs and $X$7 edges are masked for "fair" prediction (Robertson et al., 2024, Robertson et al., 8 Jun 2025).

4. Empirical Properties and Results

Synthetic Benchmarks

FairPFN exhibits strong causal effect removal across a diverse suite of synthetic SCMs. On controlled benchmarks with direct, indirect, and complex confounding effects:

• Total Effect (ATE): Reduced from typical $X$8–$X$9 (unfair) to $Y$0 (FairPFN), consistently within $Y$1 (synthetic benchmarks).
• Prediction accuracy: Degradation relative to unconstrained models is moderate; often on the Pareto frontier of AUC versus causal fairness.
• Counterfactual MAE: Comparable to counterfactual predictors requiring full SCM knowledge.

Real-World Datasets

On Law School Admissions (LSAC) and Adult Income (US Census):

• ATE is reduced to below $Y$2 for both datasets (e.g., Law School $Y$3; Adult $Y$4).
• Accuracy (AUC): Remains competitive with unconstrained predictors, slightly below those unconstrained but far above naive averaging or exogenous removal models.
• Interpretability: High concordance with noise-based counterfactual predictors, as measured by Kendall $Y$5.

FairPFN outperforms or matches the accuracy–fairness tradeoffs of specialized causal predictors and reduction-based fairness baselines, despite never accessing the real data's SCM (Robertson et al., 2024, Robertson et al., 8 Jun 2025).

5. Comparison to Related Approaches

Compared to classical causal fairness pipelines, FairPFN avoids the need for:

• Explicit causal graph discovery and validation,
• Accurate recovery of latent variables or functional forms,
• Dataset-specific SCM estimation, which represent major barriers in practice and can enable "fairwashing" if misapplied.

Unlike group-fairness approaches that optimize for demographic parity or equalized odds (e.g., via post-processing or fairness-aware losses), FairPFN is explicitly oriented toward counterfactual fairness, attempting to eliminate all causal influence of $Y$6 (both direct and indirect) (Robertson et al., 2024, Robertson et al., 8 Jun 2025).

Empirical comparisons indicate that, on synthetic datasets and real cases where SCM-based counterfactual predictors are feasible, FairPFN achieves similar or superior ATE removal and comparable predictive performance, often lying on the Pareto front.

6. Limitations and Open Challenges

• Scope of protected attributes: FairPFN assumes a single binary exogenous protected attribute. Extensions to multiple or non-binary attributes, or relaxing the exogeneity assumption, remain open.
• Interpretability: The model does not recover a human-interpretable SCM or offer explicit explanations for fairness. Future developments might involve outputting edge-masks or incorporating explainability tokens.
• Complex fairness desiderata: Path-specific effect removal (for legal criteria such as "business necessity") and intersectional fairness are not directly addressed.
• Regressions and multi-class: The architecture can be straightforwardly extended but is evaluated primarily for binary classification.
• Robustness under real distributional shifts: While synthetic data allows broad coverage, FairPFN's ability to generalize to domains with complex, unmodeled causal phenomena is empirically strong but formally unquantified (Robertson et al., 2024, Robertson et al., 8 Jun 2025, Liu et al., 25 Jan 2026).

7. Prospects and Ongoing Developments

• Integration with group fairness: Hybrid approaches could combine FairPFN-style synthetic causal pre-training with explicit group-fairness regularizers or test-time debiasing (Liu et al., 25 Jan 2026).
• Inference-time adjustments: Reweighting or calibration of predictions in the presence of observed subgroup imbalances could further enhance real-world fairness.
• Causal data augmentation: Employing synthetic counterfactual data generation for under-represented groups may address covariate shift and MNAR biases.
• Generalization to new fairness definitions: FairPFN’s pre-training framework could be adapted to path-specific or intersectional fairness settings by expanding the class of SCMs used in synthetic data generation.

FairPFN delineates a new research direction wherein large-scale, causality-guided pre-training enables algorithmic fairness objectives to be met agnostic to domain-specific SCM availability. The method demonstrates that transformer architectures, when equipped with an appropriate meta-prior, can operationalize counterfactual fairness from observational data alone (Robertson et al., 2024, Robertson et al., 8 Jun 2025, Liu et al., 25 Jan 2026).