Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
51 tokens/sec
GPT-4o
60 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

BIASeD: Bringing Irrationality into Automated System Design (2210.01122v3)

Published 1 Oct 2022 in cs.HC and cs.AI

Abstract: Human perception, memory and decision-making are impacted by tens of cognitive biases and heuristics that influence our actions and decisions. Despite the pervasiveness of such biases, they are generally not leveraged by today's AI systems that model human behavior and interact with humans. In this theoretical paper, we claim that the future of human-machine collaboration will entail the development of AI systems that model, understand and possibly replicate human cognitive biases. We propose the need for a research agenda on the interplay between human cognitive biases and Artificial Intelligence. We categorize existing cognitive biases from the perspective of AI systems, identify three broad areas of interest and outline research directions for the design of AI systems that have a better understanding of our own biases.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (127)
  1. Prospect theory: An analysis of decision under risk. Econometrica, 47(2):263, March 1979.
  2. Predictably irrational. HarperCollins New York, 2008.
  3. Herbert A. Simon. A Behavioral Model of Rational Choice. The Quarterly Journal of Economics, 69(1):99–118, 02 1955.
  4. Daniel Kahneman. Thinking, fast and slow. Macmillan, 2011.
  5. Natasha Dow Schüll. Addiction by design. Princeton University Press, 2012.
  6. Nir Eyal. Hooked: How to build habit-forming products. Penguin, 2014.
  7. Dark nudges and sludge in big alcohol: Behavioral economics, cognitive biases, and alcohol industry corporate social responsibility. The Milbank Quarterly, 98(4):1290–1328, September 2020.
  8. The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the National Academy of Sciences, 112(33), August 2015.
  9. Human modeling for human–robot collaboration. The International Journal of Robotics Research, February 2017.
  10. Lessons for artificial intelligence from the study of natural stupidity. Nature Machine Intelligence, 1(4):174–180, 2019.
  11. Deciding fast and slow: The role of cognitive biases in ai-assisted decision-making, 2020.
  12. A review of possible effects of cognitive biases on interpretation of rule-based machine learning models. Artificial Intelligence, 295, June 2021.
  13. Zeynep Akata et al. A research agenda for hybrid intelligence: Augmenting human intellect with collaborative, adaptive, responsible, and explainable artificial intelligence. Computer, 53(8), August 2020.
  14. Buster Benson. Cognitive bias cheat sheet, 2016.
  15. Timothy L. Hubbard. The possibility of an impetus heuristic. Psychonomic Bulletin Review, jun 2022.
  16. Cognitive biases and heuristics in medical decision making. Medical Decision Making, 35(4):539–557, August 2014.
  17. Cognitive biases associated with medical decisions: a systematic review. BMC Medical Informatics and Decision Making, November 2016.
  18. A systematic review of cognitive biases in tourist decisions. Tourism Management, 75:353–369, December 2019.
  19. Cognitive biases within decision making during fire evacuations. Fire Technology, 55(2):465–485, March 2018.
  20. Judgment under uncertainty: Heuristics and biases. Science, 185(4157), September 1974.
  21. The development of rational thought: A taxonomy of heuristics and biases. In Advances in Child Development and Behavior, pages 251–285. Elsevier, 2008.
  22. David Arnott. Cognitive biases and decision support systems development: a design science approach. Information Systems Journal, 16(1), January 2006.
  23. Cognitive illusions: A handbook on fallacies and biases in thinking, judgement and memory. Psychology Press, 2004.
  24. A task-based taxonomy of cognitive biases for information visualization. IEEE Transactions on Visualization and Computer Graphics, 26(2):1413–1432, February 2020.
  25. Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis. Journal of Consumer Research, 9(1):90, June 1982.
  26. The neural correlates of the decoy effect in decisions. Frontiers in Behavioral Neuroscience, 8, August 2014.
  27. Exploiting a cognitive bias promotes cooperation in social dilemma experiments. Nature Communications, 9(1), July 2018.
  28. Consumer choice in context: The decoy effect in travel and tourism. Journal of Travel Research, 34(1):45–50, July 1995.
  29. Minimization of decoy effects in recommender result sets. Web Intelligence and Agent Systems: An International Journal, 10(4):385–395, 2012.
  30. The framing of decisions and the psychology of choice. Science, 211(4481), 1981.
  31. Are experimental economists prone to framing effects? a natural field experiment. Journal of Economic Behavior & Organization, 70(3):443–446, June 2009.
  32. All frames are not created equal: A typology and critical analysis of framing effects. Organizational Behavior and Human Decision Processes, 76(2):149–188, November 1998.
  33. The framing effect in medical decision-making: a review of the literature. Psychology, Health & Medicine, 18(6):645–653, December 2013.
  34. Towards human-robot interaction: A framing effect experiment. In 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 001929–001934. IEEE, October 2016.
  35. Communicating the limitations of AI: The effect of message framing and ownership on trust in artificial intelligence. International Journal of Human–Computer Interaction, pages 1–11, April 2022.
  36. The anchoring effect in business intelligence supported decision-making. Journal of Decision Systems, 28(2):67–81, April 2019.
  37. Fooled by facts: quantifying anchoring bias through a large-scale experiment. Journal of Computational Social Science, 5(1):1001–1021, January 2022.
  38. Do recommender systems manipulate consumer preferences? a study of anchoring effects. Inf. Syst. Res., December 2013.
  39. Induction with uncertain categories: When do people consider the category alternatives? Memory & Cognition, 37(6):730–743, September 2009.
  40. Dmitry Vladimirovich Burakov. Exogenous credit cycle: An experimental study. World Applied Sciences Journal, 26(6), 2013.
  41. Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review, 90(4):293–315, 1983.
  42. The conjunction fallacy: a misunderstanding about conjunction? Cognitive Science, 28(3):467–477, May 2004.
  43. Testing boundary conditions for the conjunction fallacy: Effects of response mode, conceptual focus, and problem type. Cognition, 107(1), April 2008.
  44. Evidential diversity and premise probability in young children's inductive judgment. Cognitive Science, 26(2):181–206, March 2002.
  45. Base-rate respect: From ecological rationality to dual processes. Behavioral and Brain Sciences, 30(3):241–254, June 2007.
  46. Maya Bar-Hillel. The base-rate fallacy in probability judgments. Acta Psychologica, 44(3):211–233, May 1980.
  47. Eric Gold. The gambler’s fallacy. PhD thesis, Carnegie Mellon University, 1997.
  48. The role of experience in the gambler's fallacy. Journal of Behavioral Decision Making, 23(1):117–129, January 2010.
  49. Decision making under the gambler’s fallacy: Evidence from asylum judges, loan officers, and baseball umpires. The Quarterly Journal of Economics, 131(3):1181–1242, March 2016.
  50. Richard Thaler. Some empirical evidence on dynamic inconsistency. Economics Letters, 8(3):201–207, January 1981.
  51. Good things for those who wait: Predictive modeling highlights importance of delay discounting for income attainment. Frontiers in Psychology, 9, September 2018.
  52. George Ainslie. Specious reward: A behavioral theory of impulsiveness and impulse control. Psychological Bulletin, 1975.
  53. Predicting time preference from social media behavior. Future Generation Computer Systems, 130:155–163, May 2022.
  54. Tim Schulz van Endert and Peter N. C. Mohr. Delay discounting of monetary and social media rewards: Magnitude and trait effects. Frontiers in Psychology, 13, February 2022.
  55. What is beautiful is good. Journal of Personality and Social Psychology, 24(3):285–290, 1972.
  56. The halo effect: Evidence for unconscious alteration of judgments. Journal of Personality and Social Psychology, 35(4):250–256, April 1977.
  57. Is he a hero or a weirdo? how norm violations influence the halo effect. Gender Issues, 33(4):299–310, September 2016.
  58. Beauty is talent: Task evaluation as a function of the performer's physical attractiveness. Journal of Personality and Social Psychology, 29(3), 1974.
  59. The IKEA effect: When labor leads to love. Journal of Consumer Psychology, 22(3):453–460, July 2012.
  60. Cooking together: The ikea effect on family vegetable intake. British Journal of Health Psychology, 24(4):896–912, September 2019.
  61. MyPortfolio: The IKEA effect in financial investment decisions. Journal of Banking & Finance, page 106529, May 2022.
  62. John W. Pratt. Risk aversion in the small and in the large. In Uncertainty in Economics, pages 59–79. Elsevier, 1978.
  63. Keith E Stanovich. Decision making and rationality in the modern world. New York, Oxford University Press, 2010.
  64. How safe is safe enough? a psychometric study of attitudes towards technological risks and benefits. Policy Sciences, April 1978.
  65. Money, kisses, and electric shocks: On the affective psychology of risk. Psychological Science, 12(3):185–190, May 2001.
  66. A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24(4):349–354, 1960.
  67. Social desirability bias in dietary self-report may compromise the validity of dietary intake measures. International Journal of Epidemiology, 24(2), 1995.
  68. L Harrison. The validity of self-reported drug use in survey research: An overview and critique of research methods. national institute of drug abuse monograph 167, 2006.
  69. Social desirability bias in family planning studies: a neglected problem. Contraception, 80(2):108–112, August 2009.
  70. Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 13(5):585–589, October 1974.
  71. Elizabeth F. Loftus. Reconstructing memory: The incredible eyewitness. Jurimetrics Journal, 15(3):188–193, 1975.
  72. Memory illusions and consciousness: Examining the phenomenology of true and false memories. Current Psychology, 16(3-4):181–224, September 1997.
  73. The consequences of false memories for food preferences and choices. Perspectives on Psychological Science, 4(2):135–139, March 2009.
  74. Self-reference and the encoding of personal information. Journal of Personality and Social Psychology, 35(9):677–688, 1977.
  75. Ageing and the self-reference effect in memory. Memory, 15(8):822–837, November 2007.
  76. Bennet B. Murdock. The serial position effect of free recall. Journal of Experimental Psychology, 64(5):482–488, November 1962.
  77. Controlled rehearsal in single-trial free recall. Journal of Verbal Learning and Verbal Behavior, 17(3):309–324, June 1978.
  78. S. E. Asch. Forming impressions of personality. The Journal of Abnormal and Social Psychology, 41(3), July 1946.
  79. When more pain is preferred to less: Adding a better end. Psychological Science, 4(6):401–405, November 1993.
  80. The experienced utility of queuing: real time affect and retrospective evaluations of simulated queues. Duke University: Durham, NC, USA, 1996.
  81. Applying the peak-end rule to reference prices. Journal of Product & Brand Management, 2013.
  82. Status quo bias in decision making. Journal of Risk and Uncertainty, 1(1), March 1988.
  83. Anomalies: The endowment effect, loss aversion, and status quo bias. Journal of Economic Perspectives, 5(1), February 1991.
  84. Donelson R. Forsyth. Group dynamics, 1990.
  85. Quality of decision making and group norms. Journal of Personality and Social Psychology, 80(6):918–930, 2001.
  86. Discovery of hidden profiles by decision-making groups: Solving a problem versus making a judgment. Journal of Personality and Social Psychology, 63(3):426–434, September 1992.
  87. Itamar Simonson. The effect of purchase quantity and timing on variety-seeking behavior. Journal of Marketing Research, 27(2):150, May 1990.
  88. Diversification bias: Explaining the discrepancy in variety seeking between combined and separated choices. Journal of Experimental Psychology: Applied, 1(1):34–49, March 1995.
  89. Empirical behavioral finance. Journal of Economic Behavior & Organization, 107:421–427, November 2014.
  90. Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review, 103(4):650–669, October 1996.
  91. Modeling fast-and-frugal heuristics. PsyCh Journal, 11(4):600–611, July 2022.
  92. César A Hidalgo et al. How humans judge machines. MIT Press, 2021.
  93. Designing nudge agents that promote human altruism, 2021.
  94. Are we ready for “them” now? the relationship between human and humanoid robots. In Integrated Science, pages 377–394. Springer International Publishing, Cham, 2021.
  95. Transparency about a robot's lack of human psychological capacities. ACM Transactions on Human-Robot Interaction, 9(2):1–22, February 2020.
  96. Human reliance on machine learning models when performance feedback is limited: Heuristics and risks. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–16, 2021.
  97. Artificial intelligence and the new forms of interaction: Who has the control when interacting with a chatbot? Journal of Business Research, 129:878–890, May 2021.
  98. Designing theory-driven user-centric explainable ai. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, page 1–15, New York, NY, USA, 2019. Association for Computing Machinery.
  99. Tim Miller. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 267, February 2019.
  100. To trust or to think: cognitive forcing functions can reduce overreliance on ai in ai-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 2021.
  101. Homo heuristicus: Why biased minds make better inferences. Topics in cognitive science, 1(1):107–143, 2009.
  102. Application of human cognitive mechanisms to naïve bayes text classifier. AIP Conference Proceedings, 1863(1):360016, 2017.
  103. A search for symmetry in the conditional discriminations of rhesus monkeys, baboons, and children. Journal of the Experimental Analysis of Behavior, 37(1):23–44, January 1982.
  104. The mutual exclusivity bias in children's word learning. Monographs of the Society for Research in Child Development, 54(3/4):i, 1989.
  105. Implementation of human cognitive bias on neural network and its application to breast cancer diagnosis. SICE Journal of Control, Measurement, and System Integration, 12(2), March 2019.
  106. Self-incremental learning vector quantization with human cognitive biases. Scientific Reports, February 2021.
  107. Simple heuristics that make us smart. Oxford University Press, USA, 1999.
  108. Fast and frugal heuristics in medical decision making. Science and medicine in dialogue: Thinking through particulars and universals, 2005.
  109. A survey on bias and fairness in machine learning. ACM Computing Surveys, 54(6):1–35, July 2021.
  110. A clarification of the nuances in the fairness metrics landscape. Scientific Reports, March 2022.
  111. Towards socially responsible AI: Cognitive bias-aware multi-objective learning. Proceedings of the AAAI Conference on Artificial Intelligence, 34(03):2685–2692, April 2020.
  112. Christopher G. Harris. Mitigating cognitive biases in machine learning algorithms for decision making. In Companion Proceedings of the Web Conference 2020, WWW ’20, page 775–781, New York, NY, USA, 2020. Association for Computing Machinery.
  113. Bounded rationality modeling. Marketing Letters, 10(3), 1999.
  114. Bounded rational decision-making in feedforward neural networks, 2016.
  115. Joshua Tenenbaum. Bayesian modeling of human concept learning. Advances in neural information processing systems, 11, 1998.
  116. Optimal predictions in everyday cognition. Psychological Science, 17(9), September 2006.
  117. Probabilistic models of cognition: Conceptual foundations. Trends in Cognitive Sciences, 10(7):287–291, July 2006.
  118. A rational model of the dunning–kruger effect supports insensitivity to evidence in low performers. Nature Human Behaviour, 5(6):756–763, February 2021.
  119. VIP: Incorporating human cognitive biases in a probabilistic model of retweeting. In Social Computing, Behavioral-Cultural Modeling, and Prediction, pages 101–110. Springer International Publishing, 2015.
  120. Niels J. Blunch. Position bias in multiple-choice questions. Journal of Marketing Research, 21(2):216–220, May 1984.
  121. Automated detection of heuristics and biases among pathologists in a computer-based system. Advances in Health Sciences Education, 18(3):343–363, May 2012.
  122. Modeling decision-making biases. Biologically Inspired Cognitive Architectures, 3:39–50, January 2013.
  123. A framework for cognitive bias detection and feedback in a visual analytics environment. In 2016 European Intelligence and Security Informatics Conference (EISIC), pages 148–151. IEEE, August 2016.
  124. Methods for discovering cognitive biases in a visual analytics environment. In Cognitive Biases in Visualizations, pages 61–73. Springer International Publishing, 2018.
  125. The human behaviour-change project: harnessing the power of artificial intelligence and machine learning for evidence synthesis and interpretation. Implementation Science, 12(1), October 2017.
  126. Movipill: Improving medication compliance for elders using a mobile persuasive social game. In Proceedings of the 12th ACM International Conference on Ubiquitous Computing, UbiComp ’10, page 251–260, New York, NY, USA, 2010. Association for Computing Machinery.
  127. Richards J Heuer. Psychology of intelligence analysis. Center for the Study of Intelligence, 1999.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Aditya Gulati (16 papers)
  2. Bruno Lepri (120 papers)
  3. Nuria Oliver (46 papers)
  4. Miguel Angel Lozano (3 papers)
Citations (6)
View on Semantic Scholar