Papers
Topics
Authors
Recent
Search
2000 character limit reached

mlr3summary: Concise and interpretable summaries for machine learning models

Published 25 Apr 2024 in cs.LG | (2404.16899v1)

Abstract: This work introduces a novel R package for concise, informative summaries of machine learning models. We take inspiration from the summary function for (generalized) linear models in R, but extend it in several directions: First, our summary function is model-agnostic and provides a unified summary output also for non-parametric machine learning models; Second, the summary output is more extensive and customizable -- it comprises information on the dataset, model performance, model complexity, model's estimated feature importances, feature effects, and fairness metrics; Third, models are evaluated based on resampling strategies for unbiased estimates of model performances, feature importances, etc. Overall, the clear, structured output should help to enhance and expedite the model selection process, making it a helpful tool for practitioners and researchers alike.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (25)
  1. Apley DW, Zhu J (2020). “Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models.” Journal of the Royal Statistical Society Series B: Statistical Methodology, 82(4), 1059–1086. 10.1111/rssb.12377.
  2. Arel-Bundock V (2022). “modelsummary: Data and Model Summaries in R.” Journal of Statistical Software, 103(1), 1–23. 10.18637/jss.v103.i01.
  3. Baniecki H, Biecek P (2019). “modelStudio: Interactive Studio with Explanations for ML Predictive Models.” Journal of Open Source Software, 4(43), 1798. 10.21105/joss.01798.
  4. Bengtsson H (2021). “A Unifying Framework for Parallel and Distributed Processing in R using Futures.” The R Journal, 13(2), 208–227. 10.32614/RJ-2021-048.
  5. Biecek P (2018). “DALEX: Explainers for Complex Predictive Models in R.” Journal of Machine Learning Research, 19(84), 1–5.
  6. “mlr3pipelines - Flexible Machine Learning Pipelines in R.” Journal of Machine Learning Research, 22(184), 1–7.
  7. Applied machine learning using mlr3 in R. Chapman and Hall/CRC. ISBN 9781003402848. 10.1201/9781003402848.
  8. Breiman L (2001). “Random Forests.” Machine Learning, 45(1), 5–32. ISSN 0885-6125. 10.1023/a:1010933404324.
  9. “All Models are Wrong, but Many are Useful: Learning a Variable’s Importance by Studying an Entire Class of Prediction Models Simultaneously.” Journal of Machine Learning Research, 20(177).
  10. Friedman JH (2001). “Greedy Function Approximation: A Gradient Boosting Machine.” The Annals of Statistics, 29(5). ISSN 0090-5364. 10.1214/aos/1013203451.
  11. “A Simple and Effective Model-Based Variable Importance Measure.” arXiv preprint arXiv:1805.04755. 10.48550/arXiv.1805.04755.
  12. Hofmann H (1994). “Statlog (German Credit Data).” UCI Machine Learning Repository. 10.24432/C5NC77.
  13. Kuhn, Max (2008). “Building Predictive Models in R Using the caret Package.” Journal of Statistical Software, 28(5), 1–26. 10.18637/jss.v028.i05.
  14. “mlr3: A Modern Object-oriented Machine Learning Framework in R.” Journal of Open Source Software. 10.21105/joss.01903.
  15. “iml: An R Package for Interpretable Machine Learning.” JOSS, 3(26), 786. 10.21105/joss.00786.
  16. Quantifying model complexity via functional decomposition for better post-hoc interpretability, p. 193–204. Springer International Publishing. 10.1007/978-3-030-43823-4_17.
  17. “Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Process.” In L Longo (ed.), Explainable Artificial Intelligence, pp. 456–479. Springer Nature Switzerland, Cham. 10.1007/978-3-031-44064-9_24.
  18. “General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models.” In A Holzinger, et al (eds.), xxAI - Beyond Explainable AI: International Workshop, pp. 39–68. Springer International Publishing, Cham. ISBN 978-3-031-04083-2. 10.1007/978-3-031-04083-2_4.
  19. Nadeau C, Bengio Y (1999). “Inference for the Generalization Error.” In S Solla, et al (eds.), Advances in Neural Information Processing Systems, volume 12, pp. 1–7. MIT Press.
  20. mlr3fairness: Fairness auditing and debiasing for ’mlr3’. R package version 0.3.2, URL https://CRAN.R-project.org/package=mlr3fairness.
  21. broom: Convert statistical objects into tidy tibbles. R package version 1.0.5, URL https://CRAN.R-project.org/package=broom.
  22. “modelDown: Automated Website Generator with Interpretable Documentation for Predictive Machine Learning Models.” Journal of Open Source Software, 4(38). 10.21105/joss.01444.
  23. Simon R (2007). Resampling strategies for model assessment and selection, pp. 173–186. Springer US, Boston, MA. 10.1007/978-0-387-47509-7_8.
  24. Stephen Bates TH, Tibshirani R (2023). “Cross-Validation: What Does It Estimate and How Well Does It Do It?” Journal of the American Statistical Association, pp. 1–12. 10.1080/01621459.2023.2197686.
  25. Zargari Marandi R (2023). explainer: Machine learning model explainer. R package version 1.0.0, URL https://CRAN.R-project.org/package=explainer.

Summary

No one has generated a summary of this paper yet.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Collections

Sign up for free to add this paper to one or more collections.

Tweets

Sign up for free to view the 2 tweets with 7 likes about this paper.