Generating Analytic Specifications for Data Visualization from Natural Language Queries using Large Language Models (2408.13391v2)

Abstract: Recently, LLMs have shown great promise in translating natural language (NL) queries into visualizations, but their "black-box" nature often limits explainability and debuggability. In response, we present a comprehensive text prompt that, given a tabular dataset and an NL query about the dataset, generates an analytic specification including (detected) data attributes, (inferred) analytic tasks, and (recommended) visualizations. This specification captures key aspects of the query translation process, affording both explainability and debuggability. For instance, it provides mappings from the detected entities to the corresponding phrases in the input query, as well as the specific visual design principles that determined the visualization recommendations. Moreover, unlike prior LLM-based approaches, our prompt supports conversational interaction and ambiguity detection capabilities. In this paper, we detail the iterative process of curating our prompt, present a preliminary performance evaluation using GPT-4, and discuss the strengths and limitations of LLMs at various stages of query translation. The prompt is open-source and integrated into NL4DV, a popular Python-based natural language toolkit for visualization, which can be accessed at https://nl4dv.github.io.

• The paper proposes a novel method using a detailed text prompt for LLMs to generate comprehensive analytic specifications from natural language queries for data visualization, focusing on explainability.
• Evaluating the approach with GPT-4 shows a significant accuracy improvement to 87.02% on 740 queries compared to a rule-based baseline, although response time remains a challenge.
• The research incorporates conversational features and integrates with the NL4DV toolkit, highlighting the potential for enhancing interactive, user-friendly data visualization interfaces.

Parsing Natural Language Queries for Data Visualization with LLMs

The paper "Generating Analytic Specifications for Data Visualization from Natural Language Queries using LLMs" addresses a critical capability in data science: transforming natural language queries into visualization specifications. Utilizing LLMs, this work proposes a methodology that enhances both the explainability and debuggability of the process which interprets natural language (NL) queries for data visualization.

Summary of Methodology

The authors introduce a significant enhancement to the field by composing a comprehensive text prompt that aids LLMs in generating detailed analytic specifications from NL queries about tabular datasets. This enhancement includes an effective detection of data attributes, inference of analytic tasks, and visualization recommendations. Unlike previous methods that have remained opaque or limited in transparency, this approach incorporates detailed mappings of query phrases to corresponding data entities, alongside the specific visualization principles that guide these recommendations.

A distinguishing feature of this research is its incorporation of conversational interaction and ambiguity detection capabilities within the query translation process—an advance beyond existing LLM-based solutions. The effort culminates in an integration with NL4DV, an established Python toolkit for natural language-based visualization, thus positioning the work as practically relevant for current data visualization tasks.

Key Results

An empirical evaluation of the prompt is conducted using GPT-4 across a corpus of 740 queries drawn from diverse dataset domains. The results indicate a notable accuracy improvement, achieving 87.02% compared to the 64.05% accuracy of NL4DV's rule-based approach. This outcome positions the newly developed prompt as competitive relative to other neural network and LLM-driven systems, which have previously documented lower performance rates in similar tasks. However, the average 25-second response time, although improved over previous LLM approaches, is noted as a potential usability bottleneck, meriting further attention.

Implications and Future Directions

The findings presented in this paper provide critical insights into enhancing natural language interfaces for visualization. They underscore the necessity of addressing explainability in LLM-driven tools to foster user trust and error identification. The integration with conversational capabilities reflects an adaptation to evolving user needs for interactive and dynamic data analysis environments.

As the field progresses, this research opens avenues for exploring varied LLMs, optimizing response times, and broadening dataset benchmarks for evaluation. Potential expansions could involve integrating feedback loops that allow users to adjust and refine specifications interactively, thus creating more resilient and flexible user-driven systems. Moreover, examining how different LLM architectures and training paradigms influence both the accuracy and explainability of NL2VIS tasks could lead to further advancements.

In conclusion, this research represents a step forward in harnessing LLMs for natural-language-driven data visualizations, broadening the accessibility and usability of visualization tools beyond traditional domains. It substantiates the practical application of LLMs in enhancing user interaction with data visualization, suggesting valuable trajectories for subsequent investigations in this domain.