Papers
Topics
Authors
Recent
Search
2000 character limit reached

Recourse under Model Multiplicity via Argumentative Ensembling (Technical Report)

Published 22 Dec 2023 in cs.LG and cs.AI | (2312.15097v2)

Abstract: Model Multiplicity (MM) arises when multiple, equally performing machine learning models can be trained to solve the same prediction task. Recent studies show that models obtained under MM may produce inconsistent predictions for the same input. When this occurs, it becomes challenging to provide counterfactual explanations (CEs), a common means for offering recourse recommendations to individuals negatively affected by models' predictions. In this paper, we formalise this problem, which we name recourse-aware ensembling, and identify several desirable properties which methods for solving it should satisfy. We show that existing ensembling methods, naturally extended in different ways to provide CEs, fail to satisfy these properties. We then introduce argumentative ensembling, deploying computational argumentation to guarantee robustness of CEs to MM, while also accommodating customisable user preferences. We show theoretically and experimentally that argumentative ensembling satisfies properties which the existing methods lack, and that the trade-offs are minimal wrt accuracy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. Explainable ensemble classification model based on argumentation. In AAMAS 2023, pages 2367–2369, 2023.
  2. Towards artificial argumentation. AI Magazine, 38(3):25–36, 2017.
  3. Handbook of Formal Argumentation. College Publications, 2018.
  4. Trevor J. M. Bench-Capon. Value-based argumentation frameworks. In NMR 2002, pages 443–454, 2002.
  5. Selective ensembles for consistent predictions. In ICLR 2022, 2022.
  6. Model multiplicity: Opportunities, concerns, and solutions. In FAccT 2022, pages 850–863, 2022.
  7. Consistent counterfactuals for deep models. In ICLR 2022, 2022.
  8. Leo Breiman. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science, 16(3):199–231, 2001.
  9. NICE: an algorithm for nearest instance counterfactual explanations. Data Mining and Knowledge Discovery, pages 1–39, 2023.
  10. Counterfactual plans under distributional ambiguity. In ICLR 2022, 2022.
  11. A natural language bipolar argumentation approach to support users in online debate interactions†. Argument Comput., 4(3):209–230, 2013.
  12. On the acceptability of arguments in bipolar argumentation frameworks. In ECSQARU 2005, pages 378–389, 2005.
  13. An interpretable model with globally consistent explanations for credit risk. CoRR, abs/1811.12615, 2018.
  14. Characterizing fairness over the set of good models under selective labels. In ICML 2021, pages 2144–2155, 2021.
  15. Argumentation for explainable scheduling. In AAAI 2019, pages 2752–2759, 2019.
  16. Argumentative XAI: A survey. In IJCAI 2021, pages 4392–4399, 2021.
  17. Underspecification presents challenges for credibility in modern machine learning. JMLR, 23(1):10237–10297, 2022.
  18. Argflow: A toolkit for deep argumentative explanations for neural networks. In AAMAS 2021, pages 1761–1763, 2021.
  19. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In NeurIPS 2018, pages 590–601, 2018.
  20. Sets attacking sets in abstract argumentation. In NMR 2023, pages 22–31, 2023.
  21. Variable importance clouds: A way to explore variable importance for the set of good models. CoRR, abs/1901.03209, 2019.
  22. Phan Minh Dung. On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artif. Intell., 77(2):321–358, 1995.
  23. Is there a trade-off between fairness and accuracy? A perspective using mismatched hypothesis testing. In ICML 2020, pages 2803–2813, 2020.
  24. Robust counterfactual explanations for tree-based ensembles. In ICML 2022, pages 5742–5756, 17–23 Jul 2022.
  25. Rediscovering argumentation principles utilizing collective attacks. In KR 2022, pages 122–131, 2022.
  26. On computing explanations in abstract argumentation. In ECAI 2014, pages 1005–1006, 2014.
  27. The robustness of counterfactual explanations over time. IEEE Access, 10:82736–82750, 2022.
  28. FICO. Explainable machine learning challenge, 2018.
  29. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. J. Mach. Learn. Res., 20:177:1–177:81, 2019.
  30. A comprehensive study of argumentation frameworks with sets of attacking arguments. Int. J. Approx. Reason., 109:55–86, 2019.
  31. Riccardo Guidotti. Counterfactual explanations and how to find them: literature review and benchmarking. Data Mining and Knowledge Discovery, pages 1–55, 2022.
  32. Robust counterfactual explanations for neural networks with probabilistic guarantees. In ICML 2023, pages 12351–12367, 2023.
  33. Hans Hofmann. Statlog (German Credit Data). UCI Machine Learning Repository, 1994.
  34. Rashomon capacity: A metric for predictive multiplicity in classification. In NeurIPS 2023, pages 28988–29000, 2022.
  35. Forecasting argumentation frameworks. In KR 2022, pages 533–543, 2022.
  36. Provably robust and plausible counterfactual explanations for neural networks via robust optimisation. CoRR, abs/2309.12545, 2023.
  37. Formalising the robustness of counterfactual explanations for neural networks. In AAAI 2023, pages 14901–14909, 2023.
  38. Recourse under model multiplicity via argumentative ensembling. In AAMAS 2024, 2024.
  39. Surya Mattu Julia Angwin, Jeff Larson and Lauren Kirchner. There’s software used across the country to predict future criminals. and it’s biased against blacks., 2016.
  40. Counterfactual explanations and model multiplicity: a relational verification view. In KR 2023, pages 763–768, 2023.
  41. Consistent explanations in the face of model indeterminacy via ensembling. CoRR, abs/2306.06193, 2023.
  42. Predictive multiplicity in classification. In ICML 2020, pages 6765–6774, 2020.
  43. But are you sure? an uncertainty-aware perspective on explainable AI. In AISTATS 2023, pages 7375–7391, 2023.
  44. Individual differences among deep neural network models. Nature communications, 11(1):5725, 2020.
  45. A survey on the robustness of feature importance and counterfactual explanations. CoRR, abs/2111.00358, 2021.
  46. Sanjay Modgil. Reasoning about preferences in argumentation frameworks. Artif. Intell., 173(9-10):901–934, 2009.
  47. Explaining machine learning classifiers through diverse counterfactual explanations. In FAT 2020, pages 607–617, 2020.
  48. Robust bayesian recourse. In UAI 2022, pages 1498–1508, 2022.
  49. A generalization of dung’s abstract framework for argumentation: Arguing with sets of attacking arguments. In ArgMAS 2006, pages 54–73, 2006.
  50. On counterfactual explanations under predictive multiplicity. In UAI 2020, pages 809–818, 2020.
  51. Explaining random forests using bipolar argumentation and markov networks. In AAAI 2023, pages 9453–9460, 2023.
  52. Nico Potyka. Interpreting neural networks as quantitative argumentation frameworks. In AAAI 2021, pages 6463–6470, 2021.
  53. Empirical observation of negligible fairness-accuracy trade-offs in machine learning for public policy. Nat. Mach. Intell., 3(10):896–904, 2021.
  54. Reconciling individual probability forecasts. In FAccT 2023, pages 101–110, 2023.
  55. Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell., 1(5):206–215, 2019.
  56. On the existence of simpler machine learning models. In FAccT 2022, pages 1827–1858, 2022.
  57. Explaining bayesian networks using argumentation. In ECSQARU 2015, pages 83–92, 2015.
  58. Interpretable predictions of tree-based ensembles via actionable feature tweaking. In KDD 2017, pages 465–474, 2017.
  59. Towards robust and reliable algorithmic recourse. In NeurIPS 2021, pages 16926–16937, 2021.
  60. Actionable recourse in linear classification. In FAT 2019, pages 10–19, 2019.
  61. Argumentation and explainable artificial intelligence: a survey. Knowl. Eng. Rev., 36:e5, 2021.
  62. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL & Tech., 31:841, 2017.
  63. Predictive multiplicity in probabilistic classification. In AAAI 2023, pages 10306–10314, 2023.
  64. Unlocking fairness: a trade-off revisited. In NeurIPS 2019, pages 8780–8789, 2019.
  65. Exploring the whole rashomon set of sparse decision trees. In NeurIPS 2022, pages 14071–14084, 2022.
  66. Computing argumentative explanations in bipolar argumentation frameworks. In AAAI 2019, pages 10079–10080, 2019.
Citations (4)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 2 tweets with 11 likes about this paper.

Sign Up for Free