Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fairness under Covariate Shift: Improving Fairness-Accuracy tradeoff with few Unlabeled Test Samples (2310.07535v3)

Published 11 Oct 2023 in cs.LG and cs.AI

Abstract: Covariate shift in the test data is a common practical phenomena that can significantly downgrade both the accuracy and the fairness performance of the model. Ensuring fairness across different sensitive groups under covariate shift is of paramount importance due to societal implications like criminal justice. We operate in the unsupervised regime where only a small set of unlabeled test samples along with a labeled training set is available. Towards improving fairness under this highly challenging yet realistic scenario, we make three contributions. First is a novel composite weighted entropy based objective for prediction accuracy which is optimized along with a representation matching loss for fairness. We experimentally verify that optimizing with our loss formulation outperforms a number of state-of-the-art baselines in the pareto sense with respect to the fairness-accuracy tradeoff on several standard datasets. Our second contribution is a new setting we term Asymmetric Covariate Shift that, to the best of our knowledge, has not been studied before. Asymmetric covariate shift occurs when distribution of covariates of one group shifts significantly compared to the other groups and this happens when a dominant group is over-represented. While this setting is extremely challenging for current baselines, We show that our proposed method significantly outperforms them. Our third contribution is theoretical, where we show that our weighted entropy term along with prediction loss on the training set approximates test loss under covariate shift. Empirically and through formal sample complexity bounds, we show that this approximation to the unseen test loss does not depend on importance sampling variance which affects many other baselines.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. A reductions approach to fair classification. In International Conference on Machine Learning, pages 60–69. PMLR, 2018.
  2. Transferring fairness under distribution shifts via fair consistency regularization, 2022. URL https://arxiv.org/abs/2206.12796.
  3. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019.
  4. Provable benefits of representation learning, 2017. URL https://arxiv.org/abs/1706.04601.
  5. Analysis of representations for domain adaptation. Advances in neural information processing systems, 19, 2006.
  6. Learning bounds for domain adaptation. Advances in neural information processing systems, 20, 2007.
  7. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 2016.
  8. Introduction to statistical learning theory. In Summer school on machine learning, pages 169–207. Springer, 2003.
  9. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77–91. PMLR, 2018.
  10. Optimized pre-processing for discrimination prevention. Advances in neural information processing systems, 30, 2017.
  11. Classification with fairness constraints: A meta-algorithm with provable guarantees. In Proceedings of the conference on fairness, accountability, and transparency, pages 319–328, 2019.
  12. Fair regression with wasserstein barycenters. In Advances in Neural Information Processing Systems, 2020. URL https://proceedings.neurips.cc/paper/2020/file/51cdbd2611e844ece5d80878eb770436-Paper.pdf.
  13. Learning bounds for importance weighting. In J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc., 2010a. URL https://proceedings.neurips.cc/paper/2010/file/59c33016884a62116be975a9bb8257e3-Paper.pdf.
  14. Learning bounds for importance weighting. Advances in neural information processing systems, 23, 2010b.
  15. Optimization with non-differentiable constraints with applications to fairness, recall, churn, and other goals. Journal of Machine Learning Research, 20(172):1–59, 2019. URL http://jmlr.org/papers/v20/18-616.html.
  16. Empirical risk minimization under fairness constraints. Advances in Neural Information Processing Systems, 31, 2018.
  17. Learning models with uniform performance via distributionally robust optimization. The Annals of Statistics, 49(3):1378–1406, 2021.
  18. Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 259–268, 2015.
  19. A confidence-based approach for balancing fairness and accuracy. In Proceedings of the 2016 SIAM international conference on data mining, pages 144–152. SIAM, 2016.
  20. Self-ensembling for visual domain adaptation. arXiv, 2017.
  21. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180–1189. PMLR, 2015.
  22. Covariate Shift by Kernel Mean Matching. In Dataset Shift in Machine Learning. The MIT Press, 12 2008. ISBN 9780262170055. doi: 10.7551/mitpress/9780262170055.003.0008. URL https://doi.org/10.7551/mitpress/9780262170055.003.0008.
  23. Covariate shift by kernel mean matching. Dataset shift in machine learning, 3(4):5, 2009.
  24. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
  25. Wasserstein fair classification. In UAI, 2020. URL https://proceedings.mlr.press/v115/jiang20a.html.
  26. Data preprocessing techniques for classification without discrimination. Knowledge and information systems, 33(1):1–33, 2012.
  27. A least-squares approach to direct importance estimation. Journal of Machine Learning Research, 10(48):1391–1445, 2009. URL http://jmlr.org/papers/v10/kanamori09a.html.
  28. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In International conference on machine learning, pages 2564–2572. PMLR, 2018.
  29. Adam: A method for stochastic optimization, 2014. URL https://arxiv.org/abs/1412.6980.
  30. Stochastic re-weighted gradient descent via distributionally robust optimization. arXiv preprint arXiv:2306.09222, 2023.
  31. Robust importance weighting for covariate shift, 2019. URL https://arxiv.org/abs/1910.06324.
  32. Click-through prediction for advertising in twitter timeline. In KDD, 2015.
  33. Revisiting batch normalization for practical domain adaptation, 2017. URL https://openreview.net/forum?id=BJuysoFeg.
  34. SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=Skq89Scxx.
  35. Learning adversarially fair and transferable representations. In International Conference on Machine Learning, pages 3384–3393. PMLR, 2018.
  36. Ensuring fairness beyond the training data. In Advances in Neural Information Processing Systems, volume 33, 2020. URL https://proceedings.neurips.cc/paper/2020/file/d6539d3b57159babf6a72e106beb45bd-Paper.pdf.
  37. Linking losses for density ratio and class-probability estimation. In Proceedings of The 33rd International Conference on Machine Learning, 20–22 Jun 2016. URL https://proceedings.mlr.press/v48/menon16.html.
  38. Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application, 8:141–163, 2021.
  39. A unifying view on dataset shift in classification. Pattern recognition, 45(1):521–530, 2012.
  40. Evaluating prediction-time batch normalization for robustness under covariate shift. 2020.
  41. Fairness in machine learning. In Recent Trends in Learning From Data, pages 155–196. Springer, 2020.
  42. On fairness and calibration. Advances in neural information processing systems, 30, 2017.
  43. A survey on domain adaptation theory: learning bounds and theoretical guarantees. arXiv preprint arXiv:2004.11829, 2020.
  44. Robust fairness under covariate shift. In AAAI, 2021.
  45. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. 2019.
  46. Improving robustness against common corruptions by covariate shift adaptation. In Advances in Neural Information Processing Systems, 2020. URL https://proceedings.neurips.cc/paper/2020/file/85690f81aadc1749175c187784afc9ee-Paper.pdf.
  47. Gradient matching for domain generalization. arXiv preprint arXiv:2104.09937, 2021.
  48. Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference, 90(2):227–244, 2000.
  49. Opportunities and challenges in deep learning adversarial robustness: A survey. 2020.
  50. Fairness violations and mitigation under covariate shift. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 3–13, 2021.
  51. Fairness warnings and fair-maml: learning fairly with minimal data. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 200–209, 2020.
  52. Dropout: A simple way to prevent neural networks from overfitting. JMLR, 2014. URL http://jmlr.org/papers/v15/srivastava14a.html.
  53. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007a.
  54. Direct importance estimation with model selection and its application to covariate shift adaptation. In Advances in Neural Information Processing Systems, volume 20, 2007b. URL https://proceedings.neurips.cc/paper/2007/file/be83ab3ecd0db773eb2dc1b0a17836a1-Paper.pdf.
  55. Test-time training with self-supervision for generalization under distribution shifts, 2019. URL https://arxiv.org/abs/1909.13231.
  56. Learning fair representations. ICML. PMLR, 2013.
  57. Tent: Fully test-time adaptation by entropy minimization. In ICLR, 2021a. URL https://openreview.net/forum?id=uXl3bZLkr3c.
  58. Equalized robustness: Towards sustainable fairness under distributional shifts. 2021b.
  59. Wasserstein robust classification with fairness constraints, 2021c. URL https://arxiv.org/abs/2103.06828.
  60. A fine-grained analysis on distribution shift. arXiv preprint arXiv:2110.11328, 2021.
  61. A survey of unsupervised deep domain adaptation. ACM Transactions on Intelligent Systems and Technology (TIST), 11(5):1–46, 2020.
  62. Self-training with noisy student improves imagenet classification. In CVPR, 2020.
  63. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In In WWW, 2017.
  64. Mitigating unwanted biases with adversarial learning. AIES ’18, 2018. doi: 10.1145/3278721.3278779. URL https://doi.org/10.1145/3278721.3278779.
  65. Farf: A fair and adaptive random forests classifier. In KDD, 2021.
  66. Han Zhao. Costs and benefits of fair regression, 2021. URL https://arxiv.org/abs/2106.08812.
  67. Inherent tradeoffs in learning fair representations. Advances in neural information processing systems, 32, 2019.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - google/uafcs (4 stars)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets