Functional Flexibility in Generative AI Interfaces: Text Editing with LLMs through Conversations, Toolbars, and Prompts (2410.10644v1)

Abstract: Prompting-based user interfaces (UIs) shift the task of defining and accessing relevant functions from developers to users. However, how UIs shape this flexibility has not yet been investigated explicitly. We explored interaction with LLMs over four years, before and after the rise of general-purpose LLMs: (1) Our survey (N=121) elicited how users envision to delegate writing tasks to AI. This informed a conversational UI design. (2) A user study (N=10) revealed that people regressed to using short command-like prompts. (3) When providing these directly as shortcuts in a toolbar UI, in addition to prompting, users in our second study (N=12) dynamically switched between specified and flexible AI functions. We discuss functional flexibility as a new theoretical construct and thinking tool. Our work highlights the value of moving beyond conversational UIs, by considering how different UIs shape users' access to the functional space of generative AI models.

• The paper introduces the concept of a functional space, defining how UIs act as gateways to the range of text functionalities provided by LLMs.
• It demonstrates that conversational UIs encourage command-like prompts and parallel workflows, while toolbar UIs promote rapid acceptance of AI suggestions.
• The paper shows that balancing free prompting with pre-determined tools enhances both usability and creative task delegation in text editing.

Functional Flexibility in Generative AI Interfaces: Text Editing with LLMs

The paper "Functional Flexibility in Generative AI Interfaces: Text Editing with LLMs through Conversations, Toolbars, and Prompts" (2410.10644) explores the integration of LLMs into text editors, focusing on how different user interfaces (UIs) shape users' access to LLMs' functional capabilities. The research involved a longitudinal paper over four years and included surveys, user studies, and theoretical analysis.

Introduction and Research Motivation

The research aims to understand how UIs can leverage the flexibility of LLMs. Many existing systems grant users maximum flexibility through natural language prompts but shift the task of identifying specific functions from designers to end-users. This paper identifies the need to balance flexibility with usability in designing AI tools within text editors.

Methodology

Survey

A formative survey collected data from 121 participants regarding their text editing, collaboration, and task delegation habits. Participants provided phrases they would use to delegate writing tasks to an AI, offering insights into potential AI capabilities.

Figure 1: We investigated text editing with generative text models in a project that covered four years from 2020 to 2024 and included the moment when generative AI became widely popular.

Prototypes

1. Conversational UI Prototype: Participants used a comment-based interface to delegate tasks to AI, reflecting a conversational model.
2. Toolbar UI Prototype: Based on feedback from the first paper, this UI offered pre-determined AI functions with buttons for tasks like extending, summarizing, and translating text, along with free prompt input for custom tasks.

Findings

User Interaction Patterns

1. Task Delegation: Participants in the conversational UI paper defaulted to short, command-like prompts, indicating a "regression to commands" effect. This contrasts with the more elaborate prompts collected during the survey.
2. Parallel Workflow: The conversational UI allowed for more parallel workflows, where users could perform other tasks while waiting for AI responses. In contrast, the toolbar UI saw less short-term parallelism due to faster AI response times.

   Figure 2: Lengths of AI generations and the lengths of the users selected text, measured as number of words.

Acceptance and Rejection

AI suggestions were generally accepted more often when provided through the toolbar UI, with acceptance decisions occurring more rapidly than rejections. Free prompting supported flexible user-defined functions, leading some participants to chain functions for more complex task execution.

Figure 3: Visualization of the time users spent to decide whether to accept or reject an AI-generated text.

Theoretical Construct: Functional Space

The paper introduces the concept of a "functional space," representing the LLM's range of text functionalities. UIs act as gateways that allow users to navigate this space, varying in the degree of functional flexibility they offer. The paper emphasizes the critical role of UIs in defining how users access and explore this functional space.

UI Design Implications

1. Pre-determined Tools: Offering specific functions as toolbar buttons can lower the barrier to using AI technologies by providing clear entry points into the functional space.
2. Free Prompting as a Research Tool: The variability in user-defined prompts can inform future tool development, serving as a continuous feedback mechanism to refine AI capabilities.

Conclusion

The paper highlights the importance of designing UIs that balance functional flexibility with user accessibility. It suggests that while conversational UIs are excellent for exploring the functional space, toolbars with specific functions are more effective for routine tasks. This dual approach allows users to engage with LLMs efficiently while providing avenues for creativity and customization.

Overall, the research provides a blueprint for integrating generative AI into text editors, emphasizing the significance of UI design in shaping user experience and interaction with AI technologies. Future research can further explore the application of these findings in domains beyond text editing, enhancing the usability and accessibility of generative AI systems.