TableTalk: Scaffolding Spreadsheet Development with a Language Agent

Abstract: Spreadsheet programming is challenging. Programmers use spreadsheet programming knowledge (e.g., formulas) and problem-solving skills to combine actions into complex tasks. Advancements in LLMs have introduced language agents that observe, plan, and perform tasks, showing promise for spreadsheet creation. We present TableTalk, a spreadsheet programming agent embodying three design principles -- scaffolding, flexibility, and incrementality -- derived from studies with seven spreadsheet programmers and 85 Excel templates. TableTalk guides programmers through structured plans based on professional workflows, generating three potential next steps to adapt plans to programmer needs. It uses pre-defined tools to generate spreadsheet components and incrementally build spreadsheets. In a study with 20 programmers, TableTalk produced higher-quality spreadsheets 2.3 times more likely to be preferred than the baseline. It reduced cognitive load and thinking time by 12.6%. From this, we derive design guidelines for agentic spreadsheet programming tools and discuss implications on spreadsheet programming, end-user programming, AI-assisted programming, and human-agent collaboration.

• The paper presents a novel language agent, TableTalk, that scaffolds spreadsheet creation through iterative guidance, including requirements gathering and schema prototyping.
• The system’s flexible design enables users to choose context-specific actions and rapid prototyping, promoting modular and user-driven spreadsheet construction.
• Evaluation reveals that TableTalk reduces cognitive load and improves spreadsheet quality, with preferred outputs over baseline agents in complex tasks.

Scaffolding Spreadsheet Development with TableTalk

The paper "TableTalk: Scaffolding Spreadsheet Development with a Language Agent" presents a novel approach to assisting spreadsheet programmers. It introduces TableTalk, a language agent designed to alleviate common challenges in spreadsheet programming by guiding users through a structured, iterative process based on expert practices.

System Overview and Design

TableTalk is built to enhance the productivity and quality of spreadsheet programming by incorporating three major design principles: scaffolding, flexibility, and incrementality.

Scaffolding

TableTalk adopts a structured approach to guide users through the spreadsheet creation process:

1. Requirements Gathering: The system begins by understanding the user's goals, prompting detailed questions to elicit necessary requirements such as the spreadsheet's audience, context, and timescale.
2. Schema Definition: It proposes a data table structure, prototyping the schema as a Markdown table within the chat interface to efficiently test and refine ideas.
3. Insight Extraction: The tool suggests various analyses based on the provided data, helping users derive actionable insights.

   Figure 1: An overview of the themes and codes of Excel spreadsheet template features from the spreadsheet template study...

Flexibility

TableTalk's design allows users to adapt the proposed steps to their specific context:

• Suggestion Pills: It offers multiple, contextually relevant next steps to users, enabling them to choose the most appropriate action and maintain control over the planning process.

  Figure 2: An overview of TableTalk. Interact/Observe: A spreadsheet programmer interacts with TableTalk and TableTalk observes the current state of the spreadsheet...

• Rapid Prototyping: By previewing tables in Markdown before implementation, users can iteratively refine their approach without committing changes to the spreadsheet.

Incrementality

TableTalk emphasizes building spreadsheets piece-by-piece:

• Tool Use: It utilizes tools to create atomic spreadsheet components, such as tables and charts, thereby facilitating a modular and incremental construction method.

This incremental approach not only clarifies the development process but also makes the final output more understandable and adaptable for the programmer.

Implementation and Evaluation

The implementation of TableTalk leverages GPT-4 through OpenAI Assistants API for reasoning and action selection, tailored with a custom system prompt. This includes JSON representations of spreadsheet states and guidelines for generating responses and suggestions. The tool undergoes a two-step reasoning process: generating user responses and planning actions.

The evaluation involved 20 spreadsheet programmers completing tasks using TableTalk and a baseline agent. Results showed that TableTalk-produced spreadsheets were preferred by evaluators, attributed to higher quality in polish, usability, schema design, and correctness. Moreover, programmers using TableTalk exhibited reduced cognitive load and more focused attention on high-level problem-solving.

Implications and Future Directions

Design Implications

1. Task Complexity Assessment: Future tools should balance the degree of scaffolding based on task complexity, potentially using heuristic methods to adjust tool involvement dynamically.
2. User Control: Providing controls for stopping or undoing actions is critical to maintaining user agency during tool interaction.
3. Customization of Proactivity: Allowing users to adjust the level of tool proactivity can improve interaction tailored to personal preferences and project needs.
4. Evaluation of Output: Incorporating features to evaluate the quality of agent-generated outputs can mitigate overreliance and improve end-user debugging processes.

Impact on Spreadsheet Programming

TableTalk demonstrates the potential for agents to significantly aid spreadsheet programming by automating routine tasks and providing structured guidance. This can enhance end-user productivity, improve spreadsheet quality, and reduce cognitive exertion. Future work should explore broader applications and refine interaction models to further integrate proactive language agents into daily spreadsheet use.

Overall, TableTalk exemplifies how AI can transform spreadsheet programming from a daunting task into a streamlined and collaborative process, paving the way for more sophisticated AI-driven productivity tools in various computational domains.

Figure 3: A screenshot of the TableTalk interface.