Could Humans Outshine AI in Visual Data Analysis?

Abstract: People often use visualizations not only to explore a dataset but also to draw generalizable conclusions about underlying models or phenomena. While previous research has viewed deviations from rational analysis as problematic, we hypothesize that human reliance on non-normative heuristics may be advantageous in certain situations. In this study, we investigate scenarios where human intuition might outperform idealized statistical rationality. Our experiment assesses participants' accuracy in characterizing the parameters of known data-generating models from bivariate visualizations. Our findings show that, while participants generally demonstrated lower accuracy than statistical models, they often outperformed Bayesian agents, particularly when dealing with extreme samples. These results suggest that, even when deviating from rationality, human gut reactions to visualizations can provide an advantage. Our findings offer insights into how analyst intuition and statistical models can be integrated to improve inference and decision-making, with important implications for the design of visual analytics tools.