Papers
Topics
Authors
Recent
2000 character limit reached

PROMISE: A Framework for Developing Complex Conversational Interactions (Technical Report) (2312.03699v3)

Published 6 Dec 2023 in cs.CL

Abstract: The advent of increasingly powerful LLMs has raised expectations for language-based interactions. However, controlling these models is a challenge, emphasizing the need to be able to investigate the feasibility and value of their application. We present PROMISE, a framework that facilitates the development of complex language-based interactions with information systems. Its use of state machine modeling concepts enables model-driven, dynamic prompt orchestration across hierarchically nested states and transitions. This improves the control of the behavior of LLMs and thus enables their effective and efficient use. In this technical report we show the benefits of PROMISE in the context of application scenarios within health information systems and demonstrate its ability to handle complex interactions. We also include code examples and present default user interfaces available as part of PROMISE.

Summary

  • The paper introduces a state machine-driven framework that segments dialogues into manageable states to ensure more predictable language model behavior.
  • It demonstrates the framework’s versatility with implementations in Python and Java, enabling rapid prototyping and real-world validation through REST APIs and database integration.
  • Its application in healthcare validates immersive simulations that support compassionate communication between physicians and patients.

Introduction

Discussions about language-based interactions with information systems have heightened with advancements in LMs, such as GPT-3. From voice assistants to complex applications, the potential for nuanced and personalized interactions is becoming a reality. LMs like ChatGPT have made strides in question-answering and command execution, with applicability extending to intricate domains such as healthcare. Nonetheless, creating and controlling these interactions remains a significant challenge.

The emergence of LMs has not only offered flexibility for open-ended conversations but has also surfaced the difficulty in directing their behavior, particularly in complex interactions. Despite various prompting strategies, ensuring consistent LM behavior requires more than crafting sophisticated prompts. Frameworks such as LangChain provide basic LM application support but lack in managing complex nested interactions. Dialogue management research also offers insights; however, solutions don't fully explore controlling LMs through prompts. This has paved the way for the development of a framework like PROMISE (Prompt-Orchestrating Model-driven Interaction State Engineering) that seeks to fill the gap in building intricate language-based interactions.

PROMISE Framework

PROMISE is a model-driven framework that utilizes the state machine concept, enhancing the development of language-based interactions with LMs. The framework breaks down complex interactions into state models where each state corresponds to a specific part of the conversation. By segmenting conversations and assigning states and transitions, it helps establish a more predictable flow, thus improving LM control. The framework is versatile, supporting single-state scenarios as well as multi-state ones with nested conversations for detailed interactions. This level of orchestration is demonstrated in PROMISE's application within healthcare systems, where patient-therapist dialogues can unfold seamlessly and adaptively.

Implementation and Validation

PROMISE offers a development approach that is accessible to both researchers and industry practitioners. The framework's adaptability is showcased through implementations available in both Python for prototyping and Java for web-based applications. With REST API support and database integration, PROMISE allows for rapid prototyping and live experiments that engage users directly through web applications. The framework's effectiveness is validated through its application in creating immersive experiences for physicians to practice compassionate communication with simulated patients, highlighting its ability to manage contextual nuances within conversations.

In conclusion, PROMISE not only addresses the limitations of current LM application development but also encourages collaborative design processes, enabling domain experts to contribute effectively to complex interaction scenarios. Its robust model-driven approach embodies a promising direction for future developments in LM applications across various domains.

Whiteboard

Paper to Video (Beta)

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Collections

Sign up for free to add this paper to one or more collections.