Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants (2402.06229v1)

Published 9 Feb 2024 in cs.HC, cs.AI, and cs.SE

Abstract: The widespread availability of LLMs within Integrated Development Environments (IDEs) has led to their speedy adoption. Conversational interactions with LLMs enable programmers to obtain natural language explanations for various software development tasks. However, LLMs often leap to action without sufficient context, giving rise to implicit assumptions and inaccurate responses. Conversations between developers and LLMs are primarily structured as question-answer pairs, where the developer is responsible for asking the the right questions and sustaining conversations across multiple turns. In this paper, we draw inspiration from interaction patterns and conversation analysis -- to design Robin, an enhanced conversational AI-assistant for debugging. Through a within-subjects user study with 12 industry professionals, we find that equipping the LLM to -- (1) leverage the insert expansion interaction pattern, (2) facilitate turn-taking, and (3) utilize debugging workflows -- leads to lowered conversation barriers, effective fault localization, and 5x improvement in bug resolution rates.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (19)
  1. An Exploratory Study of Sharing Strategic Programming Knowledge. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 66, 15 pages. https://doi.org/10.1145/3491102.3502070
  2. Chapter Two - The Self-Organization of Human Interaction. Psychology of Learning and Motivation, Vol. 59. Academic Press, 43–95. https://doi.org/10.1016/B978-0-12-407187-2.00002-2
  3. Forbes. 2023. The Age of Copilots. https://www.forbes.com/sites/forbestechcouncil/2023/09/08/the-age-of-co-pilots/
  4. GitHub. 2023. GitHub Copilot Chat. https://docs.github.com/en/copilot/github-copilot-chat/using-github-copilot-chat-in-your-ide
  5. Andreas Göldi and Roman Rietsche. 2023. Insert-expansions for Tool-enabled Conversational Agents. arXiv:2307.01644 [cs.HC]
  6. Towards Designing Conversational Agents for Pair Programming: Accounting for Creativity Strategies and Conversational Styles. In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 1–11. https://doi.org/10.1109/VL/HCC50065.2020.9127276
  7. A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges. In 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE). IEEE Computer Society, Los Alamitos, CA, USA, 605–617. https://doi.ieeecomputersociety.org/
  8. Amanda Liu and Michael Coblenz. 2023. Debugging Techniques in Professional Programming. (3 2023). https://doi.org/10.1184/R1/22277365.v1
  9. Microsoft. 2023. Visual Studio. https://visualstudio.microsoft.com/#vs-section
  10. Robert J. Moore and Raphael Arar. 2019. Conversational UX Design: A Practitioner’s Guide to the Natural Conversation Framework. Association for Computing Machinery, New York, NY, USA.
  11. OpenAI. 2022. ChatGPT. https://chat.openai.com
  12. Charles Rich and Richard C. Waters. 1990. The programmer’s apprentice. Association for Computing Machinery, New York, NY, USA.
  13. The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development. In Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI ’23). Association for Computing Machinery, New York, NY, USA, 491–514. https://doi.org/10.1145/3581641.3584037
  14. A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language 50, 4 Pt 1 (Dec 1974), 696–735. ERIC Number: EJ109512.
  15. Itamar Shatz. [n. d.]. Grice’s Maxims of Conversation: The Principles of Effective Communication. https://effectiviology.com/principles-of-effective-communication/
  16. Jack Sidnell. 2016. Conversation Analysis. https://doi.org/10.1093/acrefore/9780199384655.013.40
  17. Tabnine. 2023. Tabnine AI Assistant. https://marketplace.eclipse.org/content/tabnine-ai-assistant-chat-software-developers
  18. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, Article 332, 7 pages. https://doi.org/10.1145/3491101.3519665
  19. Wikipedia contributors. 2023. Conversation analysis — Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=Conversation_analysis&oldid=1185118449 [Online; accessed 26-January-2024].
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Bhavya Chopra (5 papers)
  2. Yasharth Bajpai (6 papers)
  3. Param Biyani (1 paper)
  4. Gustavo Soares (21 papers)
  5. Arjun Radhakrishna (23 papers)
  6. Chris Parnin (19 papers)
  7. Sumit Gulwani (55 papers)
Citations (5)
View on Semantic Scholar