2000 character limit reached
Advancing GUI for Generative AI: Charting the Design Space of Human-AI Interactions through Task Creativity and Complexity (2401.02494v3)
Published 4 Jan 2024 in cs.HC
Abstract: Technological progress has persistently shaped the dynamics of human-machine interactions in task execution. In response to the advancements in Generative AI, this paper outlines a detailed study plan that investigates various human-AI interaction modalities across a range of tasks, characterized by differing levels of creativity and complexity. This exploration aims to inform and contribute to the development of Graphical User Interfaces (GUIs) that effectively integrate with and enhance the capabilities of Generative AI systems. The study comprises three parts: exploring fixed-scope tasks through news headline generation, delving into atomic creative tasks with analogy generation, and investigating complex tasks via data visualization. Future work aims to extend this exploration to linearize complex data analysis results into narratives understandable to a broader audience, thereby enhancing the interpretability of AI-generated content.
- Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM, San Francisco CA USA, 1–22. https://doi.org/10.1145/3586183.3606719
- Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Models. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM, San Francisco CA USA, 1–14. https://doi.org/10.1145/3586183.3606725
- Formulating or Fixating: Effects of Examples on Problem Solving Vary as a Function of Example Presentation Interface Design. http://arxiv.org/abs/2401.11022 arXiv:2401.11022 [cs].
- Semantically Far Inspirations Considered Harmful?: Accounting for Cognitive States in Collaborative Ideation. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (C&C ’17). ACM, New York, NY, USA, 93–105. https://doi.org/10.1145/3059454.3059455
- Mapping the Design Space of Human-AI Interaction in Text Summarization. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Seattle, United States, 431–455. https://doi.org/10.18653/v1/2022.naacl-main.33
- John Joon Young Chung and Eytan Adar. 2023. PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM, San Francisco CA USA, 1–17. https://doi.org/10.1145/3586183.3606777
- All That’s ’Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text. arXiv:2107.00061 [cs] (July 2021). http://arxiv.org/abs/2107.00061 00008 arXiv: 2107.00061.
- Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories. In 23rd International Conference on Intelligent User Interfaces. ACM, Tokyo Japan, 329–340. https://doi.org/10.1145/3172944.3172983
- Zijian Ding and Joel Chan. 2023. Mapping the Design Space of Interactions in Human-AI Text Co-creation Tasks. http://arxiv.org/abs/2303.06430 arXiv:2303.06430 [cs].
- TalkTive: A Conversational Agent Using Backchannels to Engage Older Adults in Neurocognitive Disorders Screening. In CHI Conference on Human Factors in Computing Systems. ACM, New Orleans LA USA, 1–19. https://doi.org/10.1145/3491102.3502005
- Harnessing the Power of LLMs: Evaluating Human-AI Text Co-Creation through the Lens of News Headline Generation. http://arxiv.org/abs/2310.10706 arXiv:2310.10706 [cs].
- Fluid Transformers and Creative Analogies: Exploring Large Language Models’ Capacity for Augmenting Cross-Domain Analogical Creativity. arXiv preprint arXiv:2302.12832 (2023).
- Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Dublin, Ireland, 7250–7274. https://doi.org/10.18653/v1/2022.acl-long.501
- Karl Duncker. 1945. On problem-solving. Psychological Monographs 58, 5 (1945), i–113. https://doi.org/10.1037/h0093599
- Sparks: Inspiration for Science Writing using Language Models. In Designing Interactive Systems Conference. ACM, Virtual Event Australia, 1002–1019. https://doi.org/10.1145/3532106.3533533
- News Summarization and Evaluation in the Era of GPT-3. http://arxiv.org/abs/2209.12356 arXiv:2209.12356 [cs].
- An exploratory study of how abundant display space may support data analysis. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design. ACM, Copenhagen Denmark, 558–567. https://doi.org/10.1145/2399016.2399102
- Evaluating Human-Language Model Interaction. http://arxiv.org/abs/2212.09746 arXiv:2212.09746 [cs] version: 2.
- Zhicheng Lin. 2023. Why and how to embrace AI such as ChatGPT in your academic life. preprint. PsyArXiv. https://doi.org/10.31234/osf.io/sdx3j
- Finding Place in a Design Space: Challenges for Supporting Community Design Efforts at Scale. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (April 2021), 1–30. https://doi.org/10.1145/3449246
- ProbMap: Automatically constructing design galleries through feature extraction and semantic clustering. In The Adjunct Publication of the 34th Annual ACM Symposium on User Interface Software and Technology. ACM, Virtual Event USA, 134–136. https://doi.org/10.1145/3474349.3480203
- Framing Creative Work: Helping Novices Frame Better Problems through Interactive Scaffolding. In Creativity and Cognition (C&C ’21). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3450741.3465261
- Freeform Templates: Combining Freeform Curation with Structured Templates. In Creativity and Cognition. 478–488. https://doi.org/10.1145/3591196.3593337 arXiv:2305.00937 [cs].
- Generating Diverse Code Explanations using the GPT-3 Large Language Model. In Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 2. ACM, Lugano and Virtual Event Switzerland, 37–39. https://doi.org/10.1145/3501709.3544280
- The ”Active Search” Hypothesis: How Search Strategies Relate to Creative Learning. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval. ACM, Canberra ACT Australia, 325–329. https://doi.org/10.1145/3406522.3446046
- CoNotate: Suggesting Queries Based on Notes Promotes Knowledge Discovery. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3411764.3445618
- R. Keith Sawyer. 2012. Explaining creativity: the science of human innovation (2nd ed.). Oxford University Press, New York.
- Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence. ACM Transactions on Computer-Human Interaction (Feb. 2022), 3511599. https://doi.org/10.1145/3511599
- XCreation: A Graph-based Crossmodal Generative Creativity Support Tool. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM, San Francisco CA USA, 1–15. https://doi.org/10.1145/3586183.3606826
- Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces. ACM, Helsinki Finland, 841–852. https://doi.org/10.1145/3490099.3511105
- Q. Zhu and J. Luo. 2022. Generative Pre-Trained Transformer for Design Concept Generation: An Exploration. Proceedings of the Design Society 2 (May 2022), 1825–1834. https://doi.org/10.1017/pds.2022.185 Publisher: Cambridge University Press.