Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

FaiREE: Fair Classification with Finite-Sample and Distribution-Free Guarantee (2211.15072v4)

Published 28 Nov 2022 in stat.ML and cs.LG

Abstract: Algorithmic fairness plays an increasingly critical role in machine learning research. Several group fairness notions and algorithms have been proposed. However, the fairness guarantee of existing fair classification methods mainly depends on specific data distributional assumptions, often requiring large sample sizes, and fairness could be violated when there is a modest number of samples, which is often the case in practice. In this paper, we propose FaiREE, a fair classification algorithm that can satisfy group fairness constraints with finite-sample and distribution-free theoretical guarantees. FaiREE can be adapted to satisfy various group fairness notions (e.g., Equality of Opportunity, Equalized Odds, Demographic Parity, etc.) and achieve the optimal accuracy. These theoretical guarantees are further supported by experiments on both synthetic and real data. FaiREE is shown to have favorable performance over state-of-the-art algorithms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Auditing black-box models for indirect influence. Knowledge and Information Systems, 54(1):95–122, 2018.
  2. A reductions approach to fair classification. In International Conference on Machine Learning, pages 60–69. PMLR, 2018.
  3. Learn then test: Calibrating predictive algorithms to achieve risk control. arXiv preprint arXiv:2110.01052, 2021.
  4. Machine bias: There’s software used across the country to predict future criminals. And it’s biased against blacks. ProPublica, 23:77–91, 2016.
  5. Equity of attention: Amortizing individual fairness in rankings. In The 41st international acm sigir conference on research & development in information retrieval, pages 405–414, 2018.
  6. Building classifiers with independency constraints. In 2009 IEEE International Conference on Data Mining Workshops, pages 13–18. IEEE, 2009.
  7. Optimized pre-processing for discrimination prevention. Advances in neural information processing systems, 30, 2017.
  8. Classification with fairness constraints: A meta-algorithm with provable guarantees. In Proceedings of the conference on fairness, accountability, and transparency, pages 319–328, 2019.
  9. A fair classifier using kernel density estimation. Advances in Neural Information Processing Systems, 33:15088–15099, 2020.
  10. The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810, 2018.
  11. Leveraging labeled and unlabeled data for consistent fair binary classification. Advances in Neural Information Processing Systems, 32, 2019.
  12. Kevin A Clarke. A simple distribution-free test for nonnested model selection. Political Analysis, 15(3):347–363, 2007.
  13. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, pages 797–806, 2017.
  14. Happymap: A generalized multicalibration method. In 14th Innovations in Theoretical Computer Science Conference (ITCS 2023). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2023.
  15. Compas risk scales: Demonstrating accuracy equity and predictive parity. Northpointe Inc, 7(4), 2016.
  16. Uci machine learning repository. 2017.
  17. Survey of machine learning algorithms for disease diagnostic. Journal of Intelligent Learning Systems and Applications, 9(01):1, 2017.
  18. Michael Feldman. Computational fairness: Preventing machine-learned discrimination. PhD thesis, 2015.
  19. A confidence-based approach for balancing fairness and accuracy. In Proceedings of the 2016 SIAM international conference on data mining, pages 144–152. SIAM, 2016.
  20. On formalizing fairness in prediction with machine learning. arXiv preprint arXiv:1710.03184, 2017.
  21. Fairness guarantees under demographic shift. In International Conference on Learning Representations, 2022.
  22. Obtaining fairness using optimal transport theory. In International Conference on Machine Learning, pages 2357–2365. PMLR, 2019.
  23. A distribution-free theory of nonparametric regression, volume 1. Springer, 2002.
  24. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
  25. Multicalibration: Calibration for the (computationally-identifiable) masses. In International Conference on Machine Learning, pages 1939–1948. PMLR, 2018.
  26. Classifying without discriminating. In 2009 2nd international conference on computer, control and communication, pages 1–6. IEEE, 2009.
  27. Decision theory for discrimination-aware classification. In 2012 IEEE 12th International Conference on Data Mining, pages 924–929. IEEE, 2012.
  28. Fair decisions despite imperfect predictions. In International Conference on Artificial Intelligence and Statistics, pages 277–287. PMLR, 2020.
  29. Multiaccuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 247–254, 2019.
  30. Fair for all: Best-effort fairness guarantees for classification. In International Conference on Artificial Intelligence and Statistics, pages 3259–3267. PMLR, 2021.
  31. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113(523):1094–1111, 2018.
  32. The variational fair autoencoder. arXiv preprint arXiv:1511.00830, 2015.
  33. Kristian et al. Lum. A statistical framework for fair predictive algorithms. arXiv preprint arXiv:1610.08077, 2016.
  34. Study on a prediction of p2p network loan default based on the machine learning lightgbm and xgboost algorithms according to different high dimensional data cleaning. Electronic Commerce Research and Applications, 31:24–39, 2018.
  35. Learning adversarially fair and transferable representations. In International Conference on Machine Learning, pages 3384–3393. PMLR, 2018.
  36. Johannes S Maritz. Distribution-free statistical methods, volume 17. CRC Press, 1995.
  37. The cost of fairness in binary classification. In Conference on Fairness, Accountability and Transparency, pages 107–118. PMLR, 2018.
  38. On fairness and calibration. Advances in neural information processing systems, 30, 2017.
  39. Sidney I Resnick. Heavy tail modeling and teletraffic data: special invited paper. The Annals of Statistics, 25(5):1805–1869, 1997.
  40. Conformalized quantile regression. Advances in neural information processing systems, 32, 2019.
  41. When worlds collide: integrating different counterfactual assumptions in fairness. Advances in neural information processing systems, 30, 2017.
  42. A tutorial on conformal prediction. Journal of Machine Learning Research, 9(3), 2008.
  43. Preventing undesirable behavior of intelligent machines. Science, 366(6468):999–1004, 2019.
  44. Neyman-pearson classification algorithms and np receiver operating characteristics. Science advances, 4(2):eaao1659, 2018.
  45. Enhancing the accuracy and fairness of human decision making. Advances in Neural Information Processing Systems, 31, 2018.
  46. Fairness definitions explained. In 2018 ieee/acm international workshop on software fairness (fairware), pages 1–7. IEEE, 2018.
  47. Finite-and large-sample inference for model and coefficients in high-dimensional linear regression with repro samples. arXiv preprint arXiv:2209.09299, 2022.
  48. Enforcing delayed-impact fairness guarantees. arXiv preprint arXiv:2208.11744, 2022.
  49. Learning non-discriminatory predictors. In Conference on Learning Theory, pages 1920–1953. PMLR, 2017.
  50. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th international conference on world wide web, pages 1171–1180, 2017.
  51. Fairness constraints: Mechanisms for fair classification. In Artificial intelligence and statistics, pages 962–970. PMLR, 2017.
  52. Learning fair representations. In International conference on machine learning, pages 325–333. PMLR, 2013.
  53. Bayes-optimal classifiers under group fairness. arXiv preprint arXiv:2202.09724, 2022.
  54. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335–340, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Puheng Li (4 papers)
  2. James Zou (232 papers)
  3. Linjun Zhang (70 papers)
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now