Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
140 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

The Risks of Recourse in Binary Classification (2306.00497v2)

Published 1 Jun 2023 in cs.LG, cs.CY, and stat.ML

Abstract: Algorithmic recourse provides explanations that help users overturn an unfavorable decision by a machine learning system. But so far very little attention has been paid to whether providing recourse is beneficial or not. We introduce an abstract learning-theoretic framework that compares the risks (i.e., expected losses) for classification with and without algorithmic recourse. This allows us to answer the question of when providing recourse is beneficial or harmful at the population level. Surprisingly, we find that there are many plausible scenarios in which providing recourse turns out to be harmful, because it pushes users to regions of higher class uncertainty and therefore leads to more mistakes. We further study whether the party deploying the classifier has an incentive to strategize in anticipation of having to provide recourse, and we find that sometimes they do, to the detriment of their users. Providing algorithmic recourse may therefore also be harmful at the systemic level. We confirm our theoretical findings in experiments on simulated and real-world data. All in all, we conclude that the current concept of algorithmic recourse is not reliably beneficial, and therefore requires rethinking.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Evaluating robustness of counterfactual explanations. In IEEE Symposium Series on Computational Intelligence, 2021.
  2. The need for uncertainty quantification in machine-assisted medical decision making. Nature Machine Intelligence, 1(1):20–23, 2019.
  3. Performative prediction in a stateful world. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, PMLR, 2022.
  4. On the adversarial robustness of causal algorithmic recourse. In Proceedings of the 39th International Conference on Machine Learning, PMLR, 2022.
  5. Robust counterfactual explanations for tree-based ensembles. In Proceedings of the 39th International Conference on Machine Learning, PMLR, 2022.
  6. Timo Freiesleben. The intriguing relation between counterfactual explanations and adversarial examples. Minds and Machines, 32(1):77–109, 2022.
  7. On the impact of algorithmic recourse on social segregation. In Proceedings of the 40th International Conference on Machine Learning, 2023.
  8. On the ethics of algorithmic decision-making in healthcare. Journal of medical ethics, 46(3):205–211, 2020.
  9. Equalizing recourse across groups. arXiv preprint arXiv:1909.03166, 2019.
  10. Strategic classification. In Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science, 2016.
  11. Towards realistic individual recourse and actionable explanations in black-box decision making systems. arXiv preprint arXiv:1907.09615, 2019.
  12. DACE: distribution-aware counterfactual explanation by mixed-integer linear optimization. In International Joint Conference on Artificial Intelligence, IJCAI, 2020.
  13. Model-agnostic counterfactual explanations for consequential decisions. In International Conference on Artificial Intelligence and Statistics. PMLR, 2020.
  14. Algorithmic recourse: from counterfactual explanations to interventions. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, 2021.
  15. A survey of algorithmic recourse: contrastive explanations and consequential recommendations. ACM Computing Surveys, 2022.
  16. A causal perspective on meaningful and robust algorithmic recourse. arXiv preprint arXiv:2107.07853, 2021.
  17. Improvement-focused causal recourse (icr). In Proceedings of the AAAI Conference on Artificial Intelligence, 2023.
  18. Comparison-based inverse classification for interpretability in machine learning. In Internation Conference of Information Processing and Management of Uncertainty in Knowledge-Based Systems IPMU. Theory and Foundations, Communications in Computer and Information Science. Springer, 2018.
  19. Issues with post-hoc counterfactual explanations: a discussion. arXiv preprint arxiv:1906.04774, 2019.
  20. Strategic classification made practical. In Proceedings of the 38th International Conference on Machine Learning, PMLR, 2021.
  21. Delayed impact of fair machine learning. In Proceedings of the 35th International Conference on Machine Learning, PMLR, 2018.
  22. Strategic classification is causal modeling in disguise. In Proceedings of the 37th International Conference on Machine Learning, PMLR, 2020.
  23. The social cost of strategic classification. In Proceedings of the 2nd Conference on Fairness, Accountability, and Transparency, 2019.
  24. Performative prediction with neural networks. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR, 2023.
  25. Multi-objective evolutionary algorithms for the risk-return trade-off in bank loan management. International Transactions in operational research, 9(5):583–597, 2002.
  26. Fair inference on outcomes. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
  27. Optimal counterfactual explanations in tree ensembles. In International Conference on Machine Learning, ICML, Proceedings of Machine Learning Research. PMLR, 2021.
  28. CARLA: A python library to benchmark algorithmic recourse and counterfactual explanation algorithms. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021.
  29. Exploring counterfactual explanations through the lens of adversarial examples: A theoretical and empirical analysis. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, PMLR, 2022a.
  30. Probabilistically robust recourse: Navigating the trade-offs between costs and robustness in algorithmic recourse. arXiv preprint arXiv:2203.06768, 2022b.
  31. Performative prediction. In Proceedings of the 37th International Conference on Machine Learning, PMLR, 2020.
  32. John Platt et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers, 10(3):61–74, 1999.
  33. Algorithmic recourse in the wild: Understanding the impact of data and model shifts. arXiv preprint arXiv:2012.11788, 2020.
  34. We need fairness and explainability in algorithmic hiring. In International Conference on Autonomous Agents and Multi-Agent Systems, 2020.
  35. Optimal decision making under strategic behavior. arXiv preprint arXiv:1905.09239, 2019.
  36. Actionable recourse in linear classification. In Proceedings of the 2nd Conference on Fairness, Accountability, and Transparency, 2019.
  37. On the robustness of sparse counterfactual explanations to adverse perturbations. Artificial Intelligence, 316, 2023.
  38. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL & Tech., 31:841, 2017.
  39. Jialu Wang Yatong Chen. Linear classifiers that encourage constructive adaptation. Algorithmic Recourse workshop at ICML’21, 2021.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets