Towards Provably Fair Machine Learning: Bayesian Approaches For Consistent and Transparent Predictions

Abstract: ML classifiers deployed in high-stakes domains produce predictions whose quality varies systematically across subgroups. For granular subgroups defined by intersections of multiple features, predictions are often inconsistent with the observed data: the model's outputs contradict the evidence available for that subgroup. This problem is exacerbated by regularisation, which improves aggregate performance by collapsing small subgroups into larger groups, disproportionately affecting demographic minorities. We define two requirements for consistent prediction: determinism (identical individuals receive identical predictions) and statistical consistency (we cannot reject, at significance level alpha, the hypothesis that the predictions for a subgroup were drawn from the Bayesian optimal target distribution inferred for that subgroup). From these requirements we derive the Fair Bayesian classifier, which enforces both across every group and subgroup simultaneously and abstains whenever no consistent deterministic prediction is possible. On three benchmark datasets (Adult, COMPAS, and Bank Marketing), standard classifiers produce statistically inconsistent predictions for a substantial proportion of subgroups. Our classifier achieves zero consistency error by construction while exceeding baseline accuracy and multicalibration on every dataset tested. Statistical consistency provides a principled foundation for prediction quality with direct implications for algorithmic fairness. Minority demographics are disproportionately concentrated in small subgroups, precisely where frequentist inference is least reliable; addressing this inference problem is therefore a necessary step toward fair ML. By enforcing Bayesian consistency at the finest resolution the data supports, the our classifier demonstrates that exhaustive subgroup fairness with principled abstention is achievable in practice.

• The paper presents a Fair Bayesian classifier that enforces deterministic predictions by validating subgroup evidence through Bayesian posterior predictive testing.
• It uses an exhaustive subgroup analysis with Beta-Binomial models to determine when to assign definitive labels or abstain if the data is insufficient.
• Empirical results on canonical fairness datasets show that despite abstentions, the approach improves accuracy and accountability, especially for minority subgroups.

Provably Fair Machine Learning via Exhaustive Bayesian Consistency: Analysis of "Towards Provably Fair Machine Learning: Bayesian Approaches For Consistent and Transparent Predictions"

Overview and Problem Statement

The paper "Towards Provably Fair Machine Learning: Bayesian Approaches For Consistent and Transparent Predictions" (2606.12615) introduces a rigorous statistical consistency framework for subgroup-level prediction in machine learning and operationalizes this as the Fair Bayesian (FB) classifier. The motivation is to address a critical, under-examined limitation of existing ML models in high-stakes settings: predictions are often systematically inconsistent with the subgroup-level statistical evidence provided by the data. This inconsistency is most severe for granular, intersectional, and minority subgroups—i.e., those at the intersections of many categorical features and underrepresented demographics—where both frequentist and regularized models (including fairness-improved approaches) may contradict the available subgroup evidence.

Key objectives include:

1. Formalizing statistical consistency for predictions at all subgroup levels, not only for protected attributes or pre-specified slices.
2. Enforcing deterministic, data-justified predictions, abstaining in cases where no deterministic label is consistent with the evidence.
3. Demonstrating, across canonical fairness datasets, that standard classifiers and state-of-the-art multicalibration postprocessors often violate these subgroup-level requirements.

Key Concepts and Methodology

Central to the approach are two requirements:

• Determinism: Identical individuals must receive identical predictions; i.e., all members of a data-unique group ($d$-node) are assigned the same label.
• Statistical Consistency: For every subgroup ($d$- or $v$-node: full or partial feature conjunctions, respectively), no prediction may be statistically rejected at a user-chosen significance level $\alpha$ under the Bayesian posterior predictive model conditioned on the data.

The Subgroup Structure is crucial:

• $d$-node: Group of data points with identical values across all categorical features.
• $v$-node: Group defined by fixing any subset of features (intersectional subgroup, typically large overlap among $d$-nodes).

For each $d$-node, the unknown target rate is inferred via a Beta posterior (uniform prior, for noninformativeness). The future label assignment is tested against the Beta-Binomial predictive distribution; if neither all-positive nor all-negative deterministic predictions can pass a two-tailed hypothesis test (i.e., both are outside the $[\alpha/2,1-\alpha/2]$ credible interval), the model must abstain.

This exhaustive approach is systematically extended to all possible subgroups ($v$-nodes), not just protected or pre-specified groups. The method structures the prediction assignment as a constrained optimization over all $d$0-nodes, with the sum/predicted rate for each subgroup required to lie within its Bayesian consistency interval.

A major technical component is efficiently solving the resulting highly coupled, large-scale constraint satisfaction problem: the authors use Gurobi for MIP optimization, returning feasible assignments that are further selected by $d$1-node log-likelihood ranking.

Heterogeneity in large nodes is addressed by imposing a variance floor for the Beta posterior, mitigating overconfidence due to unmodeled population drift or data aggregation.

Statistical Properties and Categorization

Predictions for each $d$2-node fall into four mutually exclusive categories depending on the intersection of the deterministic requirement and the Bayesian consistency interval:

• $d$3: Only all-negative is statistically justified.
• $d$4: Only all-positive survives hypothesis testing.
• $d$5: Both deterministic labels are consistent; ambiguity is resolved via constraints from broader $d$6-nodes.
• $d$7: Neither label is consistent with the data (e.g., close to 50% empirical rate with tight confidence intervals); must abstain altogether.

Empirical Findings

Distributional Analysis of Subgroups and Minorities

A key empirical result concerns the size distribution of $d$8-nodes. Even in large datasets ($d$9–$v$0 rows), most $v$1-nodes comprise few individuals.

Figure 1: Distribution of $v$2-node sizes in the Adult dataset reveals that the majority contain fewer than 10 individuals, evidencing the pervasiveness of small-sample inference conditions.

These small nodes are heavily enriched for minority and intersectional subgroups.

Figure 2: Non-white individuals in the Adult dataset are highly concentrated in the smallest $v$3-nodes, exposing them to more pronounced small-sample inference and regularization-driven error.

Node-Level Consistency Violation

The authors test FB, decision tree (DT), neural network (NN), and multicalibration-based postprocessing (PMCBoost [lacava2023pmc]) on Adult, COMPAS, and Bank Marketing datasets. Major outcomes:

• Both standard and fairness-aware models (DT, NN, PMC) frequently make predictions that contradict the data-supported Bayesian evidence for a significant proportion of granular subgroups ($v$4/$v$5-nodes).
• The Fair Bayesian classifier incurs 0% consistency error by construction for all $v$6 and $v$7-nodes.
• Abstention is not rare: in Adult, over 50% of individuals reside in $v$8 nodes; i.e., reliable prediction is impossible for half the dataset given the data collected, and any deterministic assignment is necessarily inconsistent.

Performance on Standard Metrics

Despite not being optimized for pointwise accuracy, FB achieves higher accuracy and multiaccuracy than baselines on nearly all subgroups where it does not abstain. Its aggregate performance often exceeds stronger baselines, particularly for subgroups that would otherwise be small and at risk under standard models. FB's multicalibration is competitive with, and in some settings surpasses, models directly optimized for calibration.

Illustrative Case Studies

• For specific subgroups (e.g., non-white married women in office roles), DT and NN predictions can lie outside the Bayesian consistency bounds, systematically under- or over-estimating subgroup target probabilities.
• In recidivism prediction (COMPAS), 10%+ of defendants are in $v$9 nodes: any deterministic model is provably unfair for these cases.
• PMC can yield excessive corrections (e.g., marking 5/737 positives in a subgroup as all-positive), violating consistency as a side-effect of its group calibration objectives.

Theoretical and Practical Implications

Fairness

• Theoretical Implication: Statistical consistency with observed subgroup data is a strictly stronger and more actionable criterion than traditional group fairness, individual fairness, or aggregate calibration—none of which preclude provably contradictory predictions for small or intersectional minorities.
• Abstention as a safeguard: By abstaining only when the evidence rules out any deterministic label, FB aligns prediction reliability with inferential integrity—a property existing models do not (and cannot) guarantee.
• Highlighting data insufficiency: The frequency and demographic distribution of abstentions provides direct guidance for data collection, feature engineering, or case routing for human review.

Regularization and Minority Representation

• Regularization, intended to mitigate overfitting, systematically erases the evidential distinctness of small/minority subgroups, folding them into the majority. FB's framework, built around subgroup-level evidence, prevents this erosion and centers prediction legitimacy on directly observable subgroup data.

Tradeoff: Coverage vs. Consistency

• The model may abstain on a large portion of data, but this abstention is necessary—consistent data-driven prediction is simply not possible otherwise. This does introduce a practical tradeoff between coverage and statistical defensibility, especially in sparse or high-dimensional categoricals.

Transparency and Interpretability

• For every individual, the chain of evidence supporting or forbidding a prediction is transparent: the relevant $\alpha$0 and $\alpha$1-nodes, their counts, empirical rates, and posterior intervals are explicit and inspectable—a requirement for both stakeholder trust and AI regulation.

Scalability and Generalization

• The approach is feasible on medium-scale categorical datasets with thousands of intersectional subgroups. Scaling to high-cardinality features or continuously-valued attributes would require further work (e.g., principled discretization, hierarchical modeling).
• Computational tractability is ensured via careful constraint reduction and staged density estimation for hierarchical $\alpha$2-nodes; however, data volume and feature richness remain limiting factors for very large-scale applications.

Contrasts with Prior Work

• Multicalibration and multiaccuracy methods do not enforce statistical consistency for all subgroups, due to the exponential subgroup explosion and reliance on approximate audit/learner loops restricted to pre-specified attribute classes.
• Bayesian and hierarchical Bayesian fairness methods often use priors over parameters or selected subgroup rates, but do not guarantee exhaustive, deterministic subgroup-level consistency or integrated abstention.
• Selective prediction typically interprets abstention as low model confidence, not as statistical impossibility; existing methods do not identify cases where the evidence rules out every deterministic assignment.

Future Directions

• Extension beyond categorical features: incorporation of continuous variables, regression targets, and structured prior knowledge.
• Alternative priors: especially in cases where observed data encode historical injustices; prior selection becomes an intervention mechanism with fairness consequences.
• Efficient handling of previously-unseen $\alpha$3-nodes at test time: current method requires re-solving constraints per instance, which is feasible in batch but not in streaming or at-scale deployment.
• Integration with human-review loops and hybrid decision systems, routing abstention cases to domain experts or collecting more data/features as needed.

Conclusion

The Fair Bayesian framework makes a significant contribution to fair machine learning methodology by operationalizing exhaustively enforced statistical consistency as the standard for legitimate prediction. This shifts the fairness paradigm from approximate, aggregate, or pre-specified subgroup calibration to direct, data-driven defensibility at all representational granularities. For intersectional, minority, or small-sample subgroups, this model avoids both overconfident unfairness and the erasure of distinctive evidence. The empirical results—zero consistency violation, competitive accuracy, and meaningful abstention as a diagnostic for data sufficiency—demonstrate that such models are practical for real datasets and directly actionable for regulatory, scientific, and deployment contexts in high-stakes domains.

Figure 1: Distribution of $\alpha$4-node sizes in Adult dataset, log scale; the majority of $\alpha$5-nodes are small, underlining the importance of principled small-sample inference.

Figure 2: Concentration of non-white vs. white individuals by $\alpha$6-node size in Adult; the smallest nodes, where inference is hardest, are disproportionately non-white.