---
title: Fair Bayesian Classifier
url: https://www.emergentmind.com/topics/fair-bayesian-classifier
type: topic
---

# Fair Bayesian Classifier

A fair Bayesian classifier is a classifier that combines Bayesian or Bayes-optimal decision theory with explicit fairness constraints. In the most common formulation, it minimizes misclassification risk subject to a prescribed disparity bound across sensitive groups, typically using the posterior score $\eta(x,a)=P(Y=1\mid X=x,A=a)$, a likelihood ratio, or an equivalent calibrated score as the sufficient decision statistic. Across the literature, the term also covers more specialized constructions: naive Bayes models trained to eliminate “discrimination patterns” under partial observability, Wasserstein-barycenter post-processing for demographic parity, predictive-parity-constrained thresholding, functional-data classifiers based on Radon–Nikodym derivatives, and posterior-predictive subgroup-consistency rules with abstention [2202.09724][1906.03843][2211.01528][2505.09471][2606.12615].

## 1. Formal setting and problem classes

The canonical setting is binary classification with features $X$, label $Y\in\{0,1\}$, and a sensitive attribute $A$ or $S$. In population analyses, the unconstrained Bayes rule is the familiar threshold at $1/2$: $f^\star(x,a)=1\{\eta_a(x)>1/2\}$, where $\eta_a(x)=P(Y=1\mid X=x,A=a)$ [2402.02817]. Fair Bayesian classification replaces this unconstrained rule by the solution of a constrained optimization problem of the form
$$
\min_f R(f)\quad \text{subject to}\quad |D(f)|\le \epsilon,
$$
where $R(f)$ is misclassification risk and $D(f)$ is a disparity functional such as demographic parity, equality of opportunity, or predictive equality [2505.09471].

The literature uses several related but non-identical probabilistic objects. In finite-dimensional fair Bayes-optimal classification, the central object is usually the group-wise regression function $\eta_a(x)$ [2202.09724][2402.02817]. In functional classification, the central object may instead be a Radon–Nikodym derivative $\eta_a(X)=dP_{a,1}/dP_{a,0}(X)$, because posterior probabilities are intractable in infinite-dimensional spaces [2505.09471]. In naive Bayes under partial observability, fairness is defined through how predictions change when sensitive attributes become observed, so the operative quantities are posterior probabilities under marginalized missing features and the induced decision rule $f(x_{\mathrm{obs}})=\arg\max_y P(Y=y\mid x_{\mathrm{obs}})$ [1906.03843].

A further distinction concerns attribute awareness. Some frameworks assume the sensitive attribute is available at prediction time and permit group-specific thresholds; others derive attribute-blind rules in which test-time decisions depend only on $X$, with fairness adjustments expressed through $P(A\mid X)$ or $P(Y,A\mid X)$ estimated from training data [2310.05725][2402.02817]. With multiple sensitive features, the population object becomes a vector of group-membership posteriors over intersectional groups indexed by $(Y,S)$, and the optimal rule generally ceases to be a simple group-wise constant threshold [2505.00631].

## 2. Fairness notions and disparity representations

Fair Bayesian classifiers have been developed for several group fairness notions. A recurring theme is that many of these notions can be expressed as linear or bilinear functionals of the classifier, which makes constrained Bayes analysis tractable.

| Fairness notion | Representative constraint | Structural form in the literature |
|---|---|---|
| Demographic parity | $P(\hat Y=1\mid A=a)=P(\hat Y=1\mid A=a')$ | Linear or bilinear disparity; threshold shifts [2402.02817] |
| Equality of opportunity / equalized opportunity | Equal true positive rates across groups | Linear in conditional selection rates; thresholding on $\eta$ or LR [2202.09724] |
| Predictive equality | Equal false positive rates across groups | Bilinear in fair Bayes analyses [2505.09471] |
| Equalized odds | Simultaneous EO and PE constraints | Multi-constraint threshold selection [2402.02817] |
| Predictive parity | Equal $P(Y=1\mid \hat Y=1,A=a)$ across groups | Sufficiency-based; may or may not admit threshold rules [2205.07182] |
| Discrimination-pattern fairness | Invariance to observing sensitive attributes under partial observability | Naive Bayes–specific signomial constraints [1906.03843] |

In the unified Bayes-optimal framework of linear and bilinear disparity measures, deviations from demographic parity, equality of opportunity, and predictive equality are bilinear, meaning that the fairness weight can be written as $w_{\mathrm{Dis}}(x,a)=s_{\mathrm{Dis},a}\eta_a(x)+b_{\mathrm{Dis},a}$ [2402.02817]. The functional-data counterpart adopts the same principle with a bilinear disparity
$$
D(f)=\sum_{a\in\{0,1\}}\int f(x,a)\left[s_{D,a}\frac{dP_{a,1}}{dP_{a,0}}(x)+b_{D,a}\right]dP_{a,0}(x),
$$
with explicit coefficients for EO, PE, and DP [2505.09471].

When multiple sensitive features are present, approximate fairness measures based on mean difference and mean ratio become linear transformations of selection rates for groups defined jointly by labels and sensitive features. This yields a matrix form $Ar\le b$ in the unconditional selection vector over $(Y,S)$ groups, covering DP, EO, PE, AP, and composite notions such as equalized odds [2505.00631].

Not all fairness notions are of the same type. Predictive parity is sufficiency-based rather than independence- or separation-based, and the corresponding constrained Bayes problem is nonlinear in the classifier. The resulting theory is therefore qualitatively different from the generalized Neyman–Pearson analyses used for DP, EO, and PE [2205.07182]. Likewise, the “discrimination pattern” notion for naive Bayes is neither standard statistical parity nor equalized odds: it requires that adding observed sensitive attributes should not change the decision under partial observability [1906.03843].

## 3. Bayes-optimal structure

For linear and bilinear group-fairness constraints, the dominant structural result is thresholding. The generalized Neyman–Pearson argument shows that the Bayes-optimal fair rule is obtained by comparing a utility term derived from the posterior score to a fairness-dependent threshold term [2202.09724][2402.02817]. In the bilinear case, this simplifies to a group-wise thresholding rule
$$
f_a^\star(x)=1\{\eta_a(x)\ge H_{\mathrm{Dis},a}(t)\},\qquad
H_{\mathrm{Dis},a}(t)=\frac{1+t b_a}{2-t s_a},
$$
with a scalar parameter $t$ chosen so that the disparity constraint is met with minimal excess risk [2402.02817].

This threshold structure persists in several variants. Under demographic parity, one obtains shifted group thresholds on $\eta_a(x)$; under equality of opportunity, thresholds are chosen so that the true positive rate is common across groups; under equalized odds, the feasible decision lies at a common $(\mathrm{TPR},\mathrm{FPR})$ point on the intersection of group ROC convex hulls, with randomization at boundaries when exact attainment requires it [2202.09724][2005.14621]. In functional classification, the fair Bayes rule has the form
$$
f^\star_{D,\epsilon}(x,a)=1\{\eta_a(x)\ge T_a(\epsilon)\},
$$
where the group-specific threshold $T_a(\epsilon)$ is induced by a scalar $\tau^\star_{D,\epsilon}$ chosen as the smallest $|\tau|$ satisfying the disparity constraint [2505.09471].

A complementary post-hoc characterization replaces threshold search by instance-level flipping. Starting from the unconstrained Bayes classifier $\hat Y(X)=1\{P(Y=1\mid X)>1/2\}$, the optimal modification rule is
$$
K^\star(X)=1\left\{\sum_{k=1}^K z_k s_k(X)>1\right\},
$$
where $s_k(X)=f_k(X)/n(X)$ are “bias scores” measuring fairness gain per unit accuracy loss. For DP and EOp this reduces to thresholding a single bias score; for EO it becomes a linear rule in two scores [2310.05725].

With multiple sensitive features, the threshold becomes instance-dependent. The Bayes-optimal rule under linear approximate constraints takes the form
$$
h^\star(x)=1\{\eta(x)\ge \tau(x)\},\qquad
\tau(x)=\frac{1+\sum_{g\in\mathcal G} W_g\,p_g(x)}{2},
$$
where $p_g(x)=P((Y,S)\in g\mid X=x)$ and $W=A^\top\lambda$ is determined by the dual variables of the fairness constraints [2505.00631]. This is a notable departure from the simpler one-sensitive-feature setting.

A common misconception is that every fair Bayesian classifier is merely a constant group-wise threshold shift. That description is accurate for many linear-disparity settings, but not for all. The threshold can become instance-dependent in multi-sensitive-feature and attribute-blind settings, and predictive parity may fail to admit group-wise thresholding rules when group performance levels vary widely [2505.00631][2205.07182].

## 4. Algorithmic realizations

Several algorithmic families instantiate these structural results. “FairBayes” is a post-processing group-based thresholding method derived from the Neyman–Pearson characterization of Bayes-optimal classifiers under group fairness; it directly controls disparity and targets an essentially optimal fairness-accuracy tradeoff [2202.09724]. The related framework of linear and bilinear disparity measures yields three procedures—Fair Up- and Down-Sampling, Fair Cost-Sensitive Classification, and a Fair Plug-In Rule—covering pre-processing, in-processing, and post-processing respectively [2402.02817].

In functional data analysis, the principal construction is Fair-FLDA. Under homoscedastic Gaussian processes, the log Radon–Nikodym derivative is linear in functional principal component scores, and fairness is enforced by group-wise thresholding of the resulting discriminant. The practical procedure estimates means, covariances, eigenpairs, and a truncated log likelihood-ratio, then calibrates the scalar $\tau$ on a validation set so that the empirical disparity meets the target [2505.09471].

A different line of work studies naive Bayes classifiers under missing features. There, fairness is enforced by discovering and eliminating “discrimination patterns,” defined by changes in favorable-decision probability caused solely by observing sensitive attributes. The learning problem becomes maximum likelihood estimation with signomial fairness constraints, solved by an iterative cutting-plane procedure that alternates between pattern mining and constrained retraining [1906.03843].

For demographic parity in the multi-group, multi-class setting, post-processing can be formulated as a Wasserstein-barycenter problem. Given Bayes-optimal scores $f^\star$, the minimal fair error under $\alpha$-DP equals the optimal value of a barycenter problem over group output distributions, and the optimal fair classifier is obtained by composing $f^\star_a(x)$ with an optimal transport map from the group score distribution to the barycenter-supported label distribution [2211.01528].

Predictive parity leads to yet another post-processing scheme. Under a “moderate variation condition,” all fair Bayes-optimal classifiers under predictive parity are group-wise thresholding rules, and the algorithm FairBayes-DPP adaptively chooses group thresholds so as to equalize positive predictive value while seeking to maximize test accuracy [2205.07182].

The exact phrase “Fair Bayesian Classifier” also denotes a posterior-predictive subgroup-consistency method. In that construction, predictions must satisfy determinism and statistical consistency simultaneously across all complete and partial intersectional subgroups; the method abstains whenever no deterministic prediction is statistically defensible at the chosen significance level [2606.12615].

## 5. Guarantees, consistency, and fairness–accuracy tradeoffs

Theoretical guarantees differ by formulation but are central across the literature. In fairness-aware functional classification, continuity and monotonicity of the disparity curve in the threshold parameter yield existence of a fair Bayes-optimal classifier, while Fair-FLDA satisfies a finite-sample fairness guarantee
$$
|D(\hat f_{D,\epsilon})|\le \epsilon + C\sqrt{\log(1/\eta)/n},
$$
together with excess-risk control relative to the oracle fair Bayes rule [2505.09471].

For equal opportunity, semi-supervised plug-in procedures based on labeled data for $\hat\eta$ and unlabeled data for calibration are statistically consistent in both fairness and risk. Under the paper’s assumptions,
$$
\lim_{n,N\to\infty}E[\Delta(\hat g,P)]=0,\qquad
\lim_{n,N\to\infty}E[R(\hat g)]\le R(g^\star),
$$
showing convergence to the fair optimal EO classifier [1906.05082].

Post-processing based on unconstrained optimization also admits consistency guarantees. In the affine-constraint setting, the learned post-processing rule is Bayes consistent: if the base score is Bayes consistent and the randomization-width parameter $\gamma$ vanishes at an appropriate rate, the learned fair rule converges to the Bayes-optimal fair classifier [2005.14621]. In the Wasserstein-barycenter framework, if a non-Bayes score $f$ is used instead of $f^\star$, the regret over the optimal fair risk is bounded by $E\|f(X,A)-f^\star(X,A)\|_1$ [2211.01528].

PAC-Bayesian work adds high-probability generalization certificates for fairness itself. For stochastic Gibbs classifiers, the fairness discrepancy is bounded by group-wise inverse-KL terms derived from standard PAC-Bayes techniques; for deterministic classifiers, a recent decomposition relating deterministic risk to Gibbs risk extends the framework beyond the stochastic setting, yielding self-bounding optimization objectives that trade off certified risk and certified fairness [2602.11722].

The fairness–accuracy tradeoff is therefore not merely empirical. In linear-disparity frameworks, the tradeoff frontier is induced by the scalar or low-dimensional threshold parameter and is often convex or quasi-convex in the relevant region [2402.02817][2505.09471]. In demographic-parity post-processing, the minimum achievable error is exactly a Wasserstein-barycenter objective [2211.01528]. In predictive parity, the tradeoff can be favorable under moderate variation, but the structure can become pathological when group performance levels differ too much [2205.07182].

## 6. Extensions, limitations, and disputed points

Fair Bayesian classification is not a single doctrine but a family of population-optimal or Bayesian-posterior methods whose fairness notion determines the mathematics. This has two consequences. First, different papers are often solving different constrained decision problems. A naive Bayes classifier that eliminates discrimination patterns under partial observability is addressing a distinct fairness target from a Bayes-optimal equalized-odds classifier or a posterior-predictive subgroup-consistency rule [1906.03843][2202.09724][2606.12615]. Second, formal optimality is always relative to the chosen fairness criterion.

Several limitations recur. Many sharp threshold characterizations rely on calibrated posterior estimates and stable group-conditional quantities; distribution shift or poor calibration can degrade both fairness and utility [2310.05725][2402.02817]. Functional-data theory is analytically driven by homoscedastic Gaussian-process assumptions, eigendecay, and Cameron–Martin structure; heteroscedastic settings are explicitly described as more complex [2505.09471]. Naive Bayes approaches rely on conditional independence and solve signomial programs that are not globally convex [1906.03843]. Attribute-blind methods require estimating $P(A\mid X)$ or $P(Y,A\mid X)$, so fairness control depends on auxiliary probability models as well as on the base classifier [2310.05725][2505.00631].

A prominent controversy concerns predictive parity. Under a sufficient “moderate variation condition,” fair Bayes-optimal classifiers under predictive parity are group-wise thresholding rules. When this condition fails, predictive parity may lead to within-group unfairness and no fair Bayes-optimal classifier is a group-wise thresholding rule [2205.07182]. This places predictive parity in a different conceptual position from DP, EO, and PE, whose Bayes-optimal forms are much more uniformly threshold-based.

Another disputed point is whether fair classification should always return a deterministic label. The subgroup-consistency formulation answers negatively: if both deterministic extremes are rejected by the posterior predictive distribution for a subgroup, the correct action is abstention rather than forced classification. This produces zero consistency error by construction, but it also means that fairness can be obtained by declining to decide on some subgroups [2606.12615]. A plausible implication is that “fair Bayesian classifier” can denote either a constrained classifier or a constrained selective classifier, depending on whether abstention is admitted.

Taken together, these works define the fair Bayesian classifier as a principled intersection of Bayesian decision theory, calibrated probabilistic modeling, and explicit fairness constraints. The unifying result is structural: fairness-aware Bayes rules are rarely arbitrary. They are usually threshold rules, randomized threshold rules, optimal transport maps over Bayes scores, or posterior-predictive consistency rules, each derived from a precise fairness criterion and each carrying a distinct account of what “fairness” means in probabilistic classification [2202.09724][2402.02817][2211.01528][2505.09471].

Source: https://www.emergentmind.com/topics/fair-bayesian-classifier