Taken By Surprise? Evaluating how Bayesian Weighting Influences Peoples' Takeaways in Map Visualizations

Abstract: Choropleth maps have been studied and extended in many ways to counteract the many biases that can occur when using them. Two recent techniques, Surprise metrics and Value Suppressing Uncertainty Palettes (VSUPs), offer promising solutions but have yet to be tested empirically with users of visualizations. In this paper, we explore how well people can make use of these techniques in map exploration tasks. We report a crowdsourced experiment where n = 300 participants are assigned to one of Choropleth, Surprise (only), and VSUP conditions (depicting rates and Surprise in a suppressed palette). Results show clear differences in map analysis outcomes, e.g. with Surprise maps leading people to significantly higher areas of population, or VSUPs performing similar or better than Choropleths for rate selection. Qualitative analysis suggests that many participants may only consider a subset of the metrics presented to them during exploration and decision-making. We discuss how these results generally support the use of Surprise and VSUP techniques in practice, and opportunities for further technique development. The material for the study (data, study results and code) is publicly available on https://osf.io/exb95/.