Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Using Property Elicitation to Understand the Impacts of Fairness Regularizers (2309.11343v2)

Published 20 Sep 2023 in cs.LG and stat.ML

Abstract: Predictive algorithms are often trained by optimizing some loss function, to which regularization functions are added to impose a penalty for violating constraints. As expected, the addition of such regularization functions can change the minimizer of the objective. It is not well-understood which regularizers change the minimizer of the loss, and, when the minimizer does change, how it changes. We use property elicitation to take first steps towards understanding the joint relationship between the loss and regularization functions and the optimal decision for a given problem instance. In particular, we give a necessary and sufficient condition on loss and regularizer pairs for when a property changes with the addition of the regularizer, and examine some regularizers satisfying this condition standard in the fair machine learning literature. We empirically demonstrate how algorithmic decision-making changes as a function of both data distribution changes and hardness of the constraints.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Fair regression: Quantitative definitions and reduction-based algorithms. In International Conference on Machine Learning, pages 120–129. PMLR, 2019.
  2. G. Arutjothi and C. Senthamarai. Prediction of loan status in commercial bank using machine learning classifier. In 2017 International Conference on Intelligent Sustainable Systems (ICISS), pages 416–419. IEEE, 2017.
  3. Y. Bechavod and K. Ligett. Penalizing unfairness in binary classification. arXiv preprint arXiv:1707.00044, 2017.
  4. A convex framework for fair regression. arXiv preprint arXiv:1706.02409, 2017.
  5. J. Blandin and I. Kash. Fairness over utilities via multi-objective rewards. 2022.
  6. G. W. Brier et al. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1–3, 1950.
  7. Fairness guarantee in multi-class classification. arXiv preprint arXiv:2109.13642, 2021.
  8. Fair generalized linear models with a convex penalty. In International Conference on Machine Learning, pages 5286–5308. PMLR, 2022.
  9. Empirical risk minimization under fairness constraints. Advances in Neural Information Processing Systems, 31, 2018.
  10. An embedding framework for consistent polyhedral surrogates, 2019. URL https://arxiv.org/abs/1907.07330.
  11. T. Fissler. On higher order elicitability and some limit theorems on the Poisson and Wiener space. PhD thesis, 2017.
  12. R. Frongillo and I. Kash. General truthfulness characterizations via convex analysis. In Web and Internet Economics, pages 354–370. Springer, 2014.
  13. General truthfulness characterizations via convex analysis, 2019.
  14. Non-discriminatory machine learning through convex fairness criteria. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  15. Moving beyond regression techniques in cardiovascular risk prediction: applying machine learning to address analytic challenges. European heart journal, 38(23):1805–1814, 2017.
  16. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
  17. Multicalibration: Calibration for the (Computationally-identifiable) masses. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 1939–1948. PMLR, 10–15 Jul 2018. URL https://proceedings.mlr.press/v80/hebert-johnson18a.html.
  18. L. Huang and N. Vishnoi. Stable and fair classification. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2879–2890. PMLR, 09–15 Jun 2019. URL https://proceedings.mlr.press/v97/huang19e.html.
  19. Fair prediction with endogenous behavior. In Proceedings of the 21st ACM Conference on Economics and Computation, pages 677–678, 2020.
  20. Moment multicalibration for uncertainty estimation. In M. Belkin and S. Kpotufe, editors, Proceedings of Thirty Fourth Conference on Learning Theory, volume 134 of Proceedings of Machine Learning Research, pages 2634–2678. PMLR, 15–19 Aug 2021. URL https://proceedings.mlr.press/v134/jung21a.html.
  21. F. Kamiran and T. Calders. Classifying without discriminating. In 2009 2nd international conference on computer, control and communication, pages 1–6. IEEE, 2009.
  22. Fairness-aware classifier with prejudice remover regularizer. In Joint European conference on machine learning and knowledge discovery in databases, pages 35–50. Springer, 2012.
  23. N. Konstantinov and C. H. Lampert. Fairness through regularization for learning to rank. CoRR, abs/2102.05996, 2021. URL https://arxiv.org/abs/2102.05996.
  24. Community- and data-driven homelessness prevention and service delivery: Optimizing for equity. Journal of the American Medical Informatics Association, 30(6):1032–1041, 04 2023. ISSN 1527-974X. doi: 10.1093/jamia/ocad052.
  25. N. S. Lambert. Elicitation and evaluation of statistical forecasts. 2018. URL https://web.stanford.edu/ñlambert/papers/elicitability.pdf.
  26. N. S. Lambert and Y. Shoham. Eliciting truthful answers to multiple-choice questions. In Proceedings of the 10th ACM conference on Electronic commerce, pages 109–118, 2009.
  27. Eliciting properties of probability distributions. In Proceedings of the 9th ACM Conference on Electronic Commerce, pages 129–138, 2008.
  28. D. M. Lloyd-Jones. Cardiovascular risk prediction: basic concepts, current status, and future directions. Circulation, 121(15):1768–1777, 2010.
  29. Privacy regularization: Joint privacy-utility optimization in language models, 2021.
  30. G. Noarov and A. Roth. The statistical scope of multicalibration. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 26283–26310. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/noarov23a.html.
  31. On fairness and calibration. Advances in neural information processing systems, 30, 2017.
  32. R. Rahman. Heart attack analysis and prediction dataset. https://www.kaggle.com/datasets/rashikrahmanpritom/heart-attack-analysis-prediction-dataset, 2021. URL https://www.kaggle.com/datasets/rashikrahmanpritom/heart-attack-analysis-prediction-dataset.
  33. L. J. Savage. Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66(336):783–801, 1971.
  34. An approach for prediction of loan approval using machine learning algorithm. In 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC), pages 490–494. IEEE, 2020.
  35. Prediction of modernized loan approval system based on machine learning approach. In 2021 International Conference on Intelligent Technologies (CONIT), pages 1–4. IEEE, 2021.
  36. Elicitation and Identification of Properties. In Proceedings of The 27th Conference on Learning Theory, pages 482–526, 2014.
  37. R. Williamson and A. Menon. Fairness risk measures. In International Conference on Machine Learning, pages 6786–6797. PMLR, 2019.
  38. Fairness constraints: Mechanisms for fair classification. In Artificial intelligence and statistics, pages 962–970. PMLR, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now