Conformal Fairness (CF)

• Conformal Fairness (CF) is a framework that integrates conformal prediction with fairness constraints to ensure balanced uncertainty and equitable treatment.
• It employs methods like threshold search, subgroup discovery, and counterfactual calibration to control disparities across sensitive groups.
• CF navigates trade-offs between prediction efficiency and fairness, impacting applications in risk assessment, regression, and federated learning.

Conformal Fairness (CF) denotes a line of work that augments conformal prediction with fairness constraints on set-valued predictors, prediction intervals, or fairness-aware score exceedance events. In current arXiv usage, the term covers several distinct objectives: controlling coverage gaps across sensitive groups, enforcing adaptive equalized coverage on discovered subgroups, imposing demographic parity on interval endpoints before conformalization, and requiring counterfactual invariance of prediction sets under interventions on protected attributes (2505.16115, Liu et al., 2022, Guldogan et al., 9 Oct 2025). The common thread is that conformal calibration is used not only for uncertainty quantification, but also to regulate fairness-relevant behavior of the predictive object.

1. Scope and development

Early applied work connected conformal prediction to fairness in criminal justice risk assessment. In a sample of 300,000 offenders at arraignment, conformal prediction sets were used alongside a White-trained classifier and optimal transport to compare prediction-set behavior across Black and White offenders; at $95\%$ coverage, the frequencies of $\{0\}$, $\{1\}$, and $\{0,1\}$ were $0.58$, $0.03$, and $0.39$ for White test data and $0.58$, $0.03$, and $0.39$ for transported Black data (Berk et al., 2021). That work treated conformal prediction primarily as a fairness lens on confidence and ambiguity, rather than as a formal group-conditional coverage guarantee.

Later papers made CF explicit as a general framework. A generic formulation for multiclass classification defines fairness by adapting classical group fairness notions to conformal prediction sets and selecting thresholds that control worst-case disparities across sensitive groups (2505.16115). Other papers extended the idea to representation-defined subgroups in classification (Xu et al., 12 May 2026), prediction intervals in regression (Liu et al., 2022), federated learning (Srinivasan et al., 26 Sep 2025), and counterfactual fairness of set-valued predictors (Guldogan et al., 9 Oct 2025).

A recurrent terminological ambiguity is that “CF” also abbreviates “counterfactual fairness” in the causal-fairness literature (Zhou et al., 2024, Zuo et al., 2024). Within conformal prediction, this ambiguity produces two distinct strands: coverage-parity formulations that compare groupwise inclusion probabilities, and causal formulations that require prediction sets themselves to be invariant across protected-attribute interventions (2505.16115, Guldogan et al., 9 Oct 2025).

2. Core mathematical formulations

One major formulation lifts classical parity criteria from point predictions to set membership. In the generic CF framework, Demographic Parity is written as

$\{0\}$0

Equal Opportunity as

$\{0\}$1

and Predictive Equality as

$\{0\}$2

for all groups and relevant labels (2505.16115). In this view, CF is a framework for bounding disparities in label-inclusion probabilities across groups while preserving standard conformal coverage by choosing thresholds $\{0\}$3 (2505.16115).

A second formulation targets adaptive equalized coverage over unfairly treated subgroups that are not known in advance. FaReG writes the target as

$\{0\}$4

where $\{0\}$5 is an adaptively selected subgroup found by minimizing empirical group coverage subject to a minimum-mass constraint (Xu et al., 12 May 2026). The key shift is from predefined demographic groups to representation-defined groups induced by nonlinear feature combinations.

A third formulation is causal. Counterfactually Fair Conformal Prediction defines fairness for a prediction set procedure $\{0\}$6 by

$\{0\}$7

for all $\{0\}$8 and $\{0\}$9 (Guldogan et al., 9 Oct 2025). This is an individual-level criterion: the same latent individual must receive the same set-membership outcome across protected-attribute interventions.

Regression papers formulate CF differently. “Conformalized Fairness via Quantile Regression” imposes Demographic Parity on each quantile predictor by requiring $\{1\}$0, then conformalizes the fair lower and upper quantiles to restore finite-sample marginal validity of the interval (Liu et al., 2022). Fairness applies to endpoint distributions, whereas coverage applies to the final interval.

3. Principal algorithmic families

The recent literature contains several recurring constructions.

Family	Mechanism	Representative papers
Threshold search for coverage parity	Search the smallest $\{1\}$1 satisfying a fairness closeness criterion $\{1\}$2	(2505.16115, Srinivasan et al., 26 Sep 2025)
Learned unfair subgroup discovery	Learn representation-based groups with low conditional coverage and union global and subgroup-calibrated sets	(Xu et al., 12 May 2026)
Counterfactual score symmetrization	Aggregate conformity scores across protected-attribute interventions before calibration	(Guldogan et al., 9 Oct 2025)
Fair-quantile conformalization	Project quantile predictors to a Wasserstein barycenter, then apply conformal inflation	(Liu et al., 2022)
Post-hoc fairness-aware thresholding and repair	Calibrate a fairness-aware score and intervene when the score exceeds threshold	(Fayyazi et al., 5 Feb 2025, Fayyazi et al., 10 Apr 2025)

In threshold-search CF, the key object is a fairness-feasible threshold set $\{1\}$3, and the final choice is $\{1\}$4 (2505.16115). The federated extension preserves this structure, but estimates groupwise fairness gaps through client-local sufficient statistics and server aggregation, then updates $\{1\}$5 with a descent-style rule to reduce communication rounds (Srinivasan et al., 26 Sep 2025).

FaReG uses a deep variational information bottleneck to learn a latent representation $\{1\}$6, models subgroup membership through $\{1\}$7, and repeatedly samples learned subgroups whose empirical conditional coverage is low (Xu et al., 12 May 2026). The final prediction set is

$\{1\}$8

so the global conformal set preserves marginal validity while subgroup-calibrated sets enlarge coverage on discovered unfair groups (Xu et al., 12 May 2026).

CF-CP modifies the conformity score rather than the predictor. It defines

$\{1\}$9

with a permutation-invariant aggregator such as mean, max, or min, and then runs split conformal prediction with this symmetrized score (Guldogan et al., 9 Oct 2025). Under an invertible structural causal model, the symmetrized score is invariant across interventions, and the resulting sets are exactly equal across protected-attribute interventions (Guldogan et al., 9 Oct 2025).

CFQP takes a different route. It transforms raw groupwise quantile predictions to a common Wasserstein-2 barycenter,

$\{0,1\}$0

and then conformalizes the fair endpoints with the standard residual score

$\{0,1\}$1

to form a marginally valid interval (Liu et al., 2022).

Post-hoc thresholding and repair frameworks occupy an adjacent space. FACTER calibrates a scalar score

$\{0,1\}$2

on a held-out calibration set, where $\{0,1\}$3 is predictive error and $\{0,1\}$4 is a cross-group semantic disparity penalty, then uses threshold exceedance to trigger prompt updates in LLM recommendation (Fayyazi et al., 5 Feb 2025). FAIR-SIGHT uses an analogous fairness-aware non-conformity score in computer vision and triggers logit shifts or confidence recalibration when the score exceeds the calibrated threshold (Fayyazi et al., 10 Apr 2025).

4. Empirical domains and reported behavior

The empirical record is heterogeneous but broad. In criminal justice, transport-based fair risk assessment used conformal prediction sets to compare the distribution of confident and ambiguous forecasts across Black and White offenders, making abstention itself a fairness-relevant object (Berk et al., 2021). In that setting, singleton $\{0,1\}$5 sets were rare, and a large fraction of cases fell into the ambiguous set $\{0,1\}$6, highlighting that point-prediction fairness can conceal substantial uncertainty asymmetries (Berk et al., 2021).

In regression, CFQP reports coverage close to nominal $\{0,1\}$7 for $\{0,1\}$8 on LAW, CRIME, MEPS 2016, and GOV, while reducing Demographic Parity disparity of interval endpoints by $\{0,1\}$9 to $0.58$0 relative to ordinary conformalized quantile regression (Liu et al., 2022). The paper emphasizes that fairness here concerns endpoint distributions, not equalized coverage by group (Liu et al., 2022).

In counterfactual set fairness, CF-CP reports that standard split conformal prediction attains target coverage but exhibits large Counterfactual Set Disparity, whereas CF-CP-mean achieves CSD $0.58$1 on synthetic regression, synthetic classification, Law School, and Bias in Bios in oracle-counterfactual settings, with coverage $0.58$2, $0.58$3, $0.58$4, and $0.58$5 respectively (Guldogan et al., 9 Oct 2025). Under noisy counterfactual estimation, exact equality disappears, but the paper reports substantial reductions relative to standard split CP (Guldogan et al., 9 Oct 2025).

For adaptive subgroup fairness, FaReG uses synthetic XNOR-defined unfair groups, Nursery, ACSIncome, and appendix experiments to show that only the representation-based method consistently restores target subgroup coverage around $0.58$6 while keeping average set size competitive (Xu et al., 12 May 2026). Its auxiliary nonlinear auditing metric $0.58$7 is intended to reveal poor conditional coverage on nonlinear subgroups that linear worst-slab metrics miss (Xu et al., 12 May 2026).

In federated settings, FedCF reports that baseline federated conformal predictors often violate the desired fairness closeness criterion as the number of clients grows. On Fitzpatrick with $0.58$8 clients, $0.58$9, Demographic Parity, and RAPS, the baseline yields efficiency/fairness disparity $0.03$0, while FedCF yields $0.03$1 (Srinivasan et al., 26 Sep 2025). The pattern is representative: fairness improves to within the target $0.03$2, but prediction sets enlarge.

5. Trade-offs, critiques, and limitations

A central trade-off is between fairness control and efficiency. In the generic framework, fairness is enforced by enlarging thresholds beyond $0.03$3, which preserves or increases marginal coverage but also enlarges prediction sets (2505.16115). FaReG states the same issue directly: stronger subgroupwise guarantees generally require larger prediction sets, so a fair method should not simply inflate everyone’s set excessively (Xu et al., 12 May 2026).

A second limitation is that different fairness notions regulate different objects. CFQP guarantees exact Demographic Parity of endpoint predictors together with marginal interval validity, but not equalized groupwise coverage (Liu et al., 2022). CF-CP guarantees exact counterfactual set invariance and standard marginal coverage, but not conditional or group-conditional coverage, and its fairness theorem relies on an invertible structural causal model together with access to valid counterfactuals (Guldogan et al., 9 Oct 2025).

A third criticism is that procedural coverage fairness need not coincide with substantive decision fairness. Human-subject experiments on FACET, BiosBias, and RAVDESS show that providing conformal prediction sets can itself cause disparate impact, and that Equalized Coverage can increase disparate impact relative to marginal coverage (Cresswell et al., 2024). That paper argues that humans respond more strongly to set size and singleton frequency than to formal coverage parity, and therefore advocates equalized set size or equalized singleton frequency as more relevant fairness targets in human-in-the-loop settings (Cresswell et al., 2024).

A fourth limitation concerns post-hoc black-box systems. The reproduction study of FACTER finds that adaptive-threshold violation counts fall sharply, but that these reductions are not consistently reflected under the fixed threshold or in global fairness metrics such as SNSR and CFR (López-Feliu et al., 26 Jun 2026). This suggests that conformal control of a fairness-aware score is not equivalent to broad fairness improvement in system behavior.

6. Extensions and current directions

Recent work pushes CF beyond standard split conformal classification. A generic framework explicitly leverages exchangeability rather than IID assumptions, which enables fairness-aware conformal prediction on graph data and other non-IID settings (2505.16115). FedCF carries the same logic into federated learning, deriving client-level lower and upper bounds on fairness-specific conditional coverage and using them both for construction and for auditing of deployed federated conformal predictors (Srinivasan et al., 26 Sep 2025).

Shift-aware and causal extensions are also emerging. Calibrated Counterfactual Conformal Fairness ($0.03$4) targets group-conditional coverage parity under covariate shift by combining group-specific importance-weighted conformal calibration with a counterfactual regularizer based on path-specific effects in a structural causal model (Alpay et al., 29 Sep 2025). The method provides finite-sample lower bounds on groupwise coverage, an upper bound on the equalized conditional coverage gap, and first-order control of a counterfactual coverage-parity surrogate, but only under covariate-shift assumptions and with the usual sensitivity to causal misspecification (Alpay et al., 29 Sep 2025).

Application-oriented work broadens the operational range of CF. FACTER adapts conformal thresholding to black-box LLM recommenders through prompt repair rather than set-valued output construction (Fayyazi et al., 5 Feb 2025). FAIR-SIGHT similarly uses a fairness-aware non-conformity score to trigger post-hoc output repair in image classification and object detection (Fayyazi et al., 10 Apr 2025). These papers suggest a broader interpretation of CF as calibrated fairness monitoring and intervention, even when the final object is not a classical conformal prediction set.

Taken together, the literature treats Conformal Fairness as a modular research area rather than a single theorem. Some papers make fairness a property of groupwise coverage, some of discovered subgroups, some of interval endpoints, and some of counterfactual invariance of prediction sets. What unifies them is the use of conformal calibration to turn fairness constraints into finite-sample thresholding problems with explicit coverage, gap, or score-control semantics (2505.16115, Guldogan et al., 9 Oct 2025).