2000 character limit reached
FAIRM: Learning invariant representations for algorithmic fairness and domain generalization with minimax optimality
Published 2 Apr 2024 in stat.ML, cs.LG, and stat.ME | (2404.01608v1)
Abstract: Machine learning methods often assume that the test data have the same distribution as the training data. However, this assumption may not hold due to multiple levels of heterogeneity in applications, raising issues in algorithmic fairness and domain generalization. In this work, we address the problem of fair and generalizable machine learning by invariant principles. We propose a training environment-based oracle, FAIRM, which has desirable fairness and domain generalization properties under a diversity-type condition. We then provide an empirical FAIRM with finite-sample theoretical guarantees under weak distributional assumptions. We then develop efficient algorithms to realize FAIRM in linear models and demonstrate the nonasymptotic performance with minimax optimality. We evaluate our method in numerical experiments with synthetic data and MNIST data and show that it outperforms its counterparts.
- Invariant risk minimization. arXiv preprint arXiv:1907.02893.
- Evaluating fairness of machine learning models under uncertain and incomplete information. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 206–214.
- Bühlmann, P. (2020). Invariance, causality and robustness. Statistical Science 35(3), 404–426.
- Scaffolding sets. arXiv preprint arXiv:2111.03135.
- Domain adaptation under structural causal models. The Journal of Machine Learning Research 22(1), 11856–11935.
- An empirical study of invariant risk minimization. arXiv preprint arXiv:2004.05007.
- Happymap: A generalized multicalibration method. In 14th Innovations in Theoretical Computer Science Conference (ITCS 2023). Schloss Dagstuhl-Leibniz-Zentrum für Informatik.
- Minimax group fairness: Algorithms and experiments. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 66–76.
- Few-shot learning via learning the representation, provably. In International Conference on Learning Representations.
- Environment invariant linear least squares. arXiv preprint arXiv:2303.03092.
- When race/ethnicity data are lacking. RAND Corporation [Internet].
- Shortcut learning in deep neural networks. Nature Machine Intelligence 2(11), 665–673.
- Multicalibration: Calibration for the (computationally-identifiable) masses. In International Conference on Machine Learning, pp. 1939–1948.
- Does invariant risk minimization capture invariance? In International Conference on Artificial Intelligence and Statistics, pp. 4069–4077. PMLR.
- An empirical study of rich subgroup fairness for machine learning. In Proceedings of the conference on fairness, accountability, and transparency, pp. 100–109.
- Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807.
- Understanding self-training for gradual domain adaptation. In International Conference on Machine Learning, pp. 5468–5479. PMLR.
- Counterfactual fairness. In Advances in neural information processing systems, pp. 4066–4076.
- Predicting what you already know helps: Provable self-supervised learning. Advances in Neural Information Processing Systems 34, 309–323.
- Transfer learning for high-dimensional linear regression: Prediction, estimation and minimax optimality. Journal of the Royal Statistical Society Series B: Statistical Methodology 84(1), 149–173.
- Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision, pp. 3730–3738.
- Minimax pareto fairness: A multi objective perspective. In International Conference on Machine Learning, pp. 6755–6764. PMLR.
- Maximin effects in inhomogeneous large-scale data. The Annals of Statistics 43(4), 1801–1830.
- Namkoong, H. and J. C. Duchi (2016). Stochastic gradient methods for distributionally robust optimization with f-divergences. Advances in neural information processing systems 29.
- Domain adaptation via transfer component analysis. IEEE transactions on neural networks 22(2), 199–210.
- Pearl, J. (2009). Causality. Cambridge university press.
- Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 947–1012.
- Invariant causal prediction for sequential data. Journal of the American Statistical Association 114(527), 1264–1276.
- Stabilizing variable selection and regression. The Annals of Applied Statistics 15(3), 1220–1246.
- A survey on domain adaptation theory: learning bounds and theoretical guarantees. arXiv preprint arXiv:2004.11829.
- Invariant models for causal transfer learning. The Journal of Machine Learning Research 19(1), 1309–1342.
- The risks of invariant risk minimization. In International Conference on Learning Representations, Volume 9.
- Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731.
- Provable meta-learning of linear representations. In International Conference on Machine Learning, pp. 10434–10443. PMLR.
- Zhang, Y. (2018). Assessing fair lending risks using race/ethnicity proxies. Management Science 64(1), 178–197.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.