Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Understanding Nonlinear Collaboration between Human and AI Agents: A Co-design Framework for Creative Design (2401.07312v4)

Published 14 Jan 2024 in cs.HC

Abstract: Creative design is a nonlinear process where designers generate diverse ideas in the pursuit of an open-ended goal and converge towards consensus through iterative remixing. In contrast, AI-powered design tools often employ a linear sequence of incremental and precise instructions to approximate design objectives. Such operations violate customary creative design practices and thus hinder AI agents' ability to complete creative design tasks. To explore better human-AI co-design tools, we first summarize human designers' practices through a formative study with 12 design experts. Taking graphic design as a representative scenario, we formulate a nonlinear human-AI co-design framework and develop a proof-of-concept prototype, OptiMuse. We evaluate OptiMuse and validate the nonlinear framework through a comparative study. We notice a subconscious change in people's attitudes towards AI agents, shifting from perceiving them as mere executors to regarding them as opinionated colleagues. This shift effectively fostered the exploration and reflection processes of individual designers.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Adobe Firefly – Generative AI for Everyone. https://www.adobe.com/sensei/generative-ai/firefly.html. Last accessed on 14.09.2023.
  2. Dynamicland. https://dynamicland.org/. Last accessed on 14.09.2023.
  3. Framework for Innovation - Design Council. https://www.designcouncil.org.uk/our-resources/framework-for-innovation/. Last accessed on 14.09.2023.
  4. Midjourney. https://www.midjourney.com/home/?callbackUrl=%2Fapp%2F. Last accessed on 14.09.2023.
  5. https://scratch.mit.edu/. Last accessed on 14.09.2023.
  6. System Usability Scale (SUS) | Usability.gov. https://uiuxtrend.com/sus-calculator/. Last accessed on 14.09.2023.
  7. The effects of it-enabled cognitive stimulation tools on creative problem solving: A dual pathway to creativity. Journal of Management Information Systems 33, 1 (2016), 11–44.
  8. Empirical research in requirements engineering: trends and opportunities. Requirements Engineering 23 (2018), 63–95.
  9. Human-ai collaboration in a cooperative game setting: Measuring social perception and outcomes. Proc. ACM Hum.-Comput. Interact. 4, CSCW2 (oct 2020).
  10. Bruckman, A. Community support for constructionist learning. Computer Supported Cooperative Work (CSCW) 7 (1998), 47–86.
  11. Nine potential pitfalls when designing human-ai co-creative systems. arXiv preprint arXiv:2104.00358 (2021).
  12. The antecedents of remix. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (New York, NY, USA, 2014), CSCW ’14, ACM, p. 1011–1022.
  13. Choice overload: A conceptual review and meta-analysis. Journal of Consumer Psychology 25, 2 (2015), 333–358.
  14. Creative writing with a machine in the loop: Case studies on slogans and stories. In 23rd International Conference on Intelligent User Interfaces (2018), pp. 329–340.
  15. Wizard of oz studies–why and how. Knowledge-based systems 6, 4 (1993), 258–266.
  16. Remixing as a pathway to computational thinking. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (2016), pp. 1438–1449.
  17. Software requirement analysis: Research challenges and technical approaches. In 2018 6th International Symposium on Digital Forensic and Security (ISDFS) (2018), pp. 1–6.
  18. Fischer, G. Social creativity: turning barriers into opportunities for collaborative design. In Proceedings of the eighth conference on Participatory design: Artful integration: interweaving media, materials and practices-Volume 1 (2004), pp. 152–161.
  19. A Cognitive Process Theory of Writing. College Composition and Communication 32, 4 (1981), 365–387.
  20. Simulating speech systems. Computer Speech & Language (1991), 81–99.
  21. Mapping the landscape of creativity support tools in hci. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019), p. 1–18.
  22. Gero, J. S. Design prototypes: a knowledge representation schema for design. AI magazine 11, 4 (1990), 26–26.
  23. The situated function–behaviour–structure framework. Design studies 25, 4 (2004), 373–391.
  24. Girotto, V. Collective creativity through a micro-tasks crowdsourcing approach. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (New York, NY, USA, 2016), CSCW ’16 Companion, ACM, p. 143–146.
  25. Google. Imagen: Text-to-image diffusion models, 2022. https://imagen.research.google/. Last accessed on 14.09.2023.
  26. Google. Parti: Pathways autoregressive text-to-image model, 2022. https://parti.research.google/. Last accessed on 14.09.2023.
  27. Applying the wizard-of-oz framework to cooperative service discovery and configuration. In 13th IEEE International Symposium on Robot and Human Interactive Communication (Ro-MAN) (2004), pp. 575–580.
  28. An interaction framework for studying co-creative ai. arXiv preprint arXiv:1903.09709 (2019).
  29. A new approach of innovative Design: an introduction to CK theory. In DS 31: Proceedings of ICED 03, the 14th International Conference on Engineering Design, Stockholm (2003).
  30. Describing the creative design process by the integration of engineering design and cognitive psychology literature. Design Studies 29, 2 (2008), 160–180.
  31. Inakage, M. Collective creativity: Toward a new paradigm for creative culture. In Proceedings of the 2nd International Conference on Digital Interactive Media in Entertainment and Arts (New York, NY, USA, 2007), DIMEA ’07, ACM, p. 8.
  32. Modes for creative human-computer collaboration: Alternating and task-divided co-creativity. In Proceedings of the seventh international conference on computational creativity (2016), pp. 77–84.
  33. A word is worth a thousand pictures: Prompts as ai design material. arXiv preprint arXiv:2303.12647 (2023).
  34. Why is ai not a panacea for data workers? an interview study on human-ai collaboration in data storytelling. arXiv preprint arXiv:2304.08366 (2023).
  35. Where are we so far? understanding data storytelling tools from the perspective of human-ai collaboration, 2023.
  36. Evodesigner: Towards aiding creativity in graphic design. In Artificial Intelligence in Music, Sound, Art and Design (2022), pp. 162–178.
  37. Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (2020), pp. 1–13.
  38. Lubart, T. How can computers be partners in the creative process: classification and commentary on the special issue. International journal of human-computer studies 63, 4-5 (2005), 365–369.
  39. User requirements analysis: a review of supporting methods. In IFIP World Computer Congress, TC 13 (2002), pp. 133–148.
  40. Maher, M. L. Computational and collective creativity: Who’s being creative? In ICCC (2012), pp. 67–71.
  41. From rookie to all-star: professional development in a graphic design social networking site. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (2014), pp. 922–933.
  42. Microsoft. Microsoft 365 copilot, 2023. https://www.microsoft.com/en-us/microsoft-365/blog/2023/03/16/introducing-microsoft-365-copilot-a-whole-new-way-to-work/. Last accessed on 14.09.2023.
  43. Creativity syndrome: Integration, application, and innovation. Psychological Bulletin 103, 1 (1988), 27–43.
  44. Leaving the beaten tracks in creative work–A design theory for systems that support convergent and divergent thinking. Journal of the Association for Information Systems 12, 11 (2011), 2.
  45. The apprentice framework: planning and assessing creativity. Fifth International Conference on Computational Creativity.
  46. Patterns of physical design remixing in online maker communities. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (New York, NY, USA, 2015), CHI ’15, ACM, p. 639–648.
  47. I Lead, You Help but Only with Enough Details: Understanding User Experience of Co-Creation with Artificial Intelligence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (2018), pp. 1–13.
  48. OpenAI. Dall·e 2, 2022. https://openai.com/dall-e-2/. Last accessed on 14.09.2023.
  49. OpenAI. Dall·e: Creating images from text, 2022. https://openai.com/blog/dall-e/. Last accessed on 31.08.2022.
  50. O’Toole, K. Collaborative creativity in tiktok music duets. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2023), CHI ’23, ACM.
  51. A human-computer collaborative editing tool for conceptual diagrams. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (2023), pp. 1–29.
  52. Pugh, S. Total design: integrated methods for successful product engineering. (No Title) (1991).
  53. A Framework for Open Civic Design: Integrating Public Participation, Crowdsourcing, and Design Thinking. 31:1–31:22.
  54. Understanding user perceptions, collaborative experience and user engagement in different human-ai interaction designs for co-creative systems. In Proceedings of the 14th Conference on Creativity and Cognition (New York, NY, USA, 2022), C&C ’22, ACM, p. 38–48.
  55. Riek, L. D. Wizard of oz studies in hri: A systematic review and new reporting guidelines. J. Hum.-Robot Interact. (2012), 119–136.
  56. Royce, W. W. Managing the development of large software systems: concepts and techniques. In Proceedings of the 9th international conference on Software Engineering (1987), pp. 328–338.
  57. Visualizing biological pathways: requirements analysis, systems evaluation and research agenda. Information Visualization 4, 3 (2005), 191–205.
  58. Understanding design collaboration between designers and artificial intelligence: A systematic literature review. Proc. ACM Hum.-Comput. Interact. 7, CSCW2 (oct 2023).
  59. Conflict or collaboration—the impact of knowledge endowment heterogeneity on remix in open collaborative communities. Frontiers in Psychology 13 (2022), 941448.
  60. From Human-Human Collaboration to Human-AI Collaboration: Designing AI Systems That Can Work Together with People. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (2020), pp. 1–6.
  61. A literature review on individual creativity support systems. Computers in Human Behavior 74 (2017), 139–151.
  62. Yatani, K. hcistats:wilcoxonsigned [Koji Yatani’s Course Webpage]. https://yatani.jp/teaching/doku.php?id=hcistats:wilcoxonsigned. Last accessed on 14.09.2023.
  63. Improving makers’ remix contribution behavior in the open innovation communities——an example of thingiverse. In 2019 4th International Conference on Social Sciences and Economic Development (ICSSED 2019) (2019), Atlantis Press, pp. 224–229.
  64. Yusuf, S. From creativity to innovation. Technology in society 31, 1 (2009), 1–8.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Junxiu Tang (4 papers)
  2. Tan Tang (6 papers)
  3. Haotian Li (72 papers)
  4. Weiwei Cui (53 papers)
  5. Yingcai Wu (47 papers)
  6. Jiayi Zhou (24 papers)
  7. Renzhong Li (2 papers)
Citations (11)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets