From Delays to Densities: Exploring Data Uncertainty through Speech, Text, and Visualization (2404.02317v2)

Abstract: Understanding and communicating data uncertainty is crucial for making informed decisions in sectors like finance and healthcare. Previous work has explored how to express uncertainty in various modes. For example, uncertainty can be expressed visually with quantile dot plots or linguistically with hedge words and prosody. Our research aims to systematically explore how variations within each mode contribute to communicating uncertainty to the user; this allows us to better understand each mode's affordances and limitations. We completed an exploration of the uncertainty design space based on pilot studies and ran two crowdsourced experiments examining how speech, text, and visualization modes and variants within them impact decision-making with uncertain data. Visualization and text were most effective for rational decision-making, though text resulted in lower confidence. Speech garnered the highest trust despite sometimes leading to risky decisions. Results from these studies indicate meaningful trade-offs among modes of information and encourage exploration of multimodal data representations.

• The paper reveals how different modalities—speech, text, and visualization—impact decision-making through varied precision and clarity.
• The paper demonstrates that visualizations and texts support rational choices, though texts yield lower confidence among users.
• The paper finds that speech, while less effective in rational analysis, notably enhances user trust in uncertain data scenarios.

From Delays to Densities: Exploring Data Uncertainty through Speech, Text, and Visualization

Introduction

In the burgeoning field of data-driven decision-making, the challenge of effectively communicating data uncertainty is paramount. This paper seeks to explore the complexities of presenting uncertain data through three different modes of communication: speech, text, and visualization. The objective is to understand how variations within each mode impact decision-making and to identify potential trade-offs in their use. By dissecting the affordances and limitations of each communication mode under a cohesive framework, the research presented here advances our understanding of multimodal data representation and its implications for user decision-making.

Methodology

The research employs a robust methodology comprising two pilot studies followed by two crowdsourced experiments. The initial pilot studies aimed to delineate a detailed design space for communicating data uncertainty across speech, text, and visualization modes, focusing on the spectrum from concreteness to fuzziness. Building on observations from these pilots, two main experiments were conducted to compare how different precision levels within each mode affect decision-making processes. Crucial to this examination was a consistent decision structure that grounded the task in a practical scenario involving the decision to salt roads based on uncertain temperature forecasts.

Findings

The core findings from this research reveal nuanced insights into how information mode and precision affect decision-making with uncertain data. Visualizations and texts were both effective for rational decision-making, albeit with a notable difference in confidence; participants reported lower confidence in decisions made through text despite its rational outcomes. On the other hand, speech, while not as effective in supporting rational decisions as the former modes, garnered higher levels of trust among users. Intriguingly, the intensification of uncertainty indicators within each mode did not substantially alter decision outcomes, highlighting a complex interplay between mode, precision, and user perception.

Implications and Future Directions

The implications of these findings are multifaceted. Practically, they suggest that while visualizations and text are valuable for supporting rational decision-making under uncertainty, the incorporation of speech could enhance user trust—a critical component in many decision contexts. Theoretically, the results contribute to a nuanced understanding of multimodal information presentation and its impact on user decision-making.

Future research directions emerge from this paper's findings and limitations. There is a clear need to explore multimodal representations that combine the strengths of each individual mode, possibly leading to optimized decision-making outcomes and enhanced user trust. Additionally, expanding the design space to include variables such as speaker characteristics in speech modes or interactive elements in visualizations could further refine our understanding of effective uncertainty communication. Lastly, ensuring accessibility and designing equitable interfaces that cater to diverse user needs represent crucial considerations for future investigations.

Conclusion

The exploration of data uncertainty communication through speech, text, and visualization modes uncovers critical insights into the trade-offs and affordances of each approach. By systematically comparing these modes and their variants under a unified framework, this research advances our understanding of effective strategies for uncertainty representation in decision-making scenarios. The findings underscore the importance of considering both rational outcomes and user trust in the design of multimodal data communication tools, setting the stage for future innovations in this vital area of research.