Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
92 tokens/sec
Gemini 2.5 Pro Premium
46 tokens/sec
GPT-5 Medium
19 tokens/sec
GPT-5 High Premium
32 tokens/sec
GPT-4o
87 tokens/sec
DeepSeek R1 via Azure Premium
98 tokens/sec
GPT OSS 120B via Groq Premium
465 tokens/sec
Kimi K2 via Groq Premium
226 tokens/sec
2000 character limit reached

U-Trustworthy Models.Reliability, Competence, and Confidence in Decision-Making (2401.02062v1)

Published 4 Jan 2024 in stat.ML and cs.LG

Abstract: With growing concerns regarding bias and discrimination in predictive models, the AI community has increasingly focused on assessing AI system trustworthiness. Conventionally, trustworthy AI literature relies on the probabilistic framework and calibration as prerequisites for trustworthiness. In this work, we depart from this viewpoint by proposing a novel trust framework inspired by the philosophy literature on trust. We present a precise mathematical definition of trustworthiness, termed $\mathcal{U}$-trustworthiness, specifically tailored for a subset of tasks aimed at maximizing a utility function. We argue that a model's $\mathcal{U}$-trustworthiness is contingent upon its ability to maximize Bayes utility within this task subset. Our first set of results challenges the probabilistic framework by demonstrating its potential to favor less trustworthy models and introduce the risk of misleading trustworthiness assessments. Within the context of $\mathcal{U}$-trustworthiness, we prove that properly-ranked models are inherently $\mathcal{U}$-trustworthy. Furthermore, we advocate for the adoption of the AUC metric as the preferred measure of trustworthiness. By offering both theoretical guarantees and experimental validation, AUC enables robust evaluation of trustworthiness, thereby enhancing model selection and hyperparameter tuning to yield more trustworthy outcomes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Afroogh, S. 2023. A probabilistic theory of trust concerning artificial intelligence: Can intelligent robots trust humans? AI and Ethics, 3(2): 469–484.
  2. Generalization Bounds for the Area Under the ROC Curve. Journal of Machine Learning Research, 6(4).
  3. An experimental comparison of cross-validation techniques for estimating the area under the ROC curve. Computational Statistics & Data Analysis, 55(4): 1828–1844.
  4. Alvarado, R. 2022. What kind of trust does AI deserve, if any? AI and Ethics, 1–15.
  5. Baier, A. 1986. Trust and Antitrust. Ethics, 96: 231.
  6. Fairness in machine learning. Nips tutorial, 1: 2017.
  7. Adult. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5XW20.
  8. The value of ai guidance in human examination of synthetically-generated faces. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 5930–5938.
  9. Brier, G. W. 1950. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1): 1–3.
  10. Toward a taxonomy of trust for probabilistic machine learning. Science Advances, 9(7): eabn3999.
  11. The state of the art in enhancing trust in machine learning models with the use of visualizations. In Computer Graphics Forum, volume 39, 713–756. Wiley Online Library.
  12. Confidence intervals for the area under the ROC curve. Advances in neural information processing systems, 17.
  13. Assessing calibration of prognostic risk scores. Statistical methods in medical research, 25(4): 1692–1706.
  14. Eshete, B. 2021. Making machine learning trustworthy. Science, 373(6556): 743–744.
  15. Collaborative strategies for deploying artificial intelligence to complement physician diagnoses of acute respiratory distress syndrome. NPJ Digital Medicine, 6(1): 62.
  16. On calibration of modern neural networks. In International conference on machine learning, 1321–1330. PMLR.
  17. Empirical comparison of area under ROC curve (AUC) and Mathew correlation coefficient (MCC) for evaluating machine learning algorithms on imbalanced datasets for binary classification. In Proceedings of the 3rd international conference on machine learning and soft computing, 1–6.
  18. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1): 29–36.
  19. Multicalibration: Calibration for the (computationally-identifiable) masses. In International Conference on Machine Learning, 1939–1948. PMLR.
  20. Horsburgh, H. J. N. 1960. The ethics of trust. The Philosophical Quarterly (1950-), 10(41): 343–354.
  21. Using AUC and accuracy in evaluating learning algorithms. IEEE Transactions on knowledge and Data Engineering, 17(3): 299–310.
  22. To trust or not to trust a classifier. Advances in neural information processing systems, 31.
  23. Jones, K. 1996. Trust as an affective attitude. Ethics, 107(1): 4–25.
  24. Algorithmic fairness. In Aea papers and proceedings, volume 108, 22–27. American Economic Association 2014 Broadway, Suite 305, Nashville, TN 37203.
  25. Trainable calibration measures for neural networks from kernel mean embeddings. In International Conference on Machine Learning, 2805–2814. PMLR.
  26. Proportional Multicalibration. arXiv preprint arXiv:2209.14613.
  27. Trustworthy ai: From principles to practices. ACM Computing Surveys, 55(9): 1–46.
  28. Taiwanese Bankruptcy Prediction. UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5004D.
  29. AUC: a misleading measure of the performance of predictive distribution models. Global ecology and Biogeography, 17(2): 145–151.
  30. Learning to predict trustworthiness with steep slope loss. Advances in Neural Information Processing Systems, 34: 21533–21544.
  31. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6): 1–35.
  32. Collaborative strategies for deploying AI-based physician decision support systems: challenges and deployment approaches. npj Digital Medicine, 6(1): 137.
  33. Reliability of subjective probability forecasts of precipitation and temperature. Journal of the Royal Statistical Society Series C: Applied Statistics, 26(1): 41–47.
  34. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence, volume 29.
  35. Probabilistic modeling for crowdsourcing partially-subjective ratings. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, volume 4, 149–158.
  36. Platt, J.; et al. 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers, 10(3): 61–74.
  37. On fairness and calibration. Advances in neural information processing systems, 30.
  38. ” Why should i trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 1135–1144.
  39. Ryan, M. 2020. In AI we trust: ethics, artificial intelligence, and reliability. Science and Engineering Ethics, 26(5): 2749–2767.
  40. Evaluating the Calibration of Knowledge Graph Embeddings for Trustworthy Link Prediction. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).
  41. Setting goals and choosing metrics for recommender system evaluations. In UCERSTI2 workshop at the 5th ACM conference on recommender systems, Chicago, USA, volume 23, 53.
  42. Practices for engineering trustworthy machine learning applications. In 2021 IEEE/ACM 1st Workshop on AI engineering-software engineering for AI (WAIN), 97–100. IEEE.
  43. Towards trustworthy predictions from deep neural networks with fast adversarial calibration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 9886–9896.
  44. The relationship between trust in AI and trustworthy machine learning technologies. In Proceedings of the 2020 conference on fairness, accountability, and transparency, 272–283.
  45. Varshney, K. R. 2019. Trustworthy machine learning and artificial intelligence. XRDS: Crossroads, The ACM Magazine for Students, 25(3): 26–29.
  46. von Eschenbach, W. J. 2021. Transparency and the black box problem: Why we do not trust AI. Philosophy & Technology, 34(4): 1607–1622.
  47. Optimizing Partial Area Under the Top-k Curve: Theory and Practice. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4): 5053–5069.
  48. Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability. Advances in Neural Information Processing Systems, 35: 28335–28349.
  49. Calibration tests in multi-class classification: A unifying framework. Advances in neural information processing systems, 32.
  50. Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5DW2B.
  51. How much can we really trust you? towards simple, interpretable trust quantification metrics for deep neural networks. arXiv preprint arXiv:2009.05835.
  52. On the consistency of top-k surrogate losses. In III, H. D.; and Singh, A., eds., Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, 10727–10735. PMLR.
  53. AUC maximization in the era of big data and AI: A survey. ACM Computing Surveys, 55(8): 1–37.
  54. Optimizing Two-way Partial AUC with an End-to-end Framework. IEEE Transactions on Pattern Analysis and Machine Intelligence.
  55. Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, 694–699.
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to a collection.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now