Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Exploring the Potential of Large Language Models in Artistic Creation: Collaboration and Reflection on Creative Programming (2402.09750v1)

Published 15 Feb 2024 in cs.HC and cs.AI

Abstract: Recently, the potential of LLMs has been widely used in assisting programming. However, current research does not explore the artist potential of LLMs in creative coding within artist and AI collaboration. Our work probes the reflection type of artists in the creation process with such collaboration. We compare two common collaboration approaches: invoking the entire program and multiple subtasks. Our findings exhibit artists' different stimulated reflections in two different methods. Our finding also shows the correlation of reflection type with user performance, user satisfaction, and subjective experience in two collaborations through conducting two methods, including experimental data and qualitative interviews. In this sense, our work reveals the artistic potential of LLM in creative coding. Meanwhile, we provide a critical lens of human-AI collaboration from the artists' perspective and expound design suggestions for future work of AI-assisted creative tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–22.
  2. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023 (2023).
  3. The search dashboard: how reflection and comparison impact search behavior. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1785–1794.
  4. Eric PS Baumer. 2015. Reflective informatics: conceptual dimensions for designing technologies of reflection. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 585–594.
  5. Reviewing reflection: on the use of reflection in interactive system design. In Proceedings of the 2014 conference on Designing interactive systems. 93–102.
  6. A critical evaluation of the usefulness of a coding scheme to categorise levels of reflective thinking. Assessment & Evaluation in Higher Education 36, 7 (2011), 797–815.
  7. Cam: A large language model-based creative analogy mining framework. In Proceedings of the ACM Web Conference 2023. 3903–3914.
  8. Davide Castelvecchi. 2022. Are ChatGPT and AlphaCode going to replace programmers? Nature (2022).
  9. Art or artifice? large language models and the false promise of creativity. arXiv preprint arXiv:2309.14556 (2023).
  10. NatGen: generative pre-training by “naturalizing” source code. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 18–30.
  11. Kathy Charmaz. 2014. Constructing grounded theory. sage.
  12. Zenan Chen and Jason Chan. 2023. Large language model in creative work: The role of collaboration modality and user expertise. Available at SSRN 4575598 (2023).
  13. Github copilot ai pair programmer: Asset or liability? Journal of Systems and Software (2023), 111734.
  14. Supporting reflection in and on design processes. In Proceedings of the Designing Interactive Systems Conference. 803–804.
  15. Touch Designer. 2023. Derivative. https://derivative.ca/
  16. The Idea Machine: LLM-based Expansion, Rewriting, Combination, and Suggestion of Ideas. In Proceedings of the 14th Conference on Creativity and Cognition. 623–627.
  17. NOOT: a tool for sharing moments of reflection during creative meetings. In Proceedings of the 8th ACM conference on Creativity and cognition. 157–164.
  18. Corey Ford and Nick Bryan-Kinns. 2023. Towards a Reflection in Creative Experience Questionnaire. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–16.
  19. Incoder: A generative model for code infilling and synthesis. arXiv preprint arXiv:2204.05999 (2022).
  20. Philip Galanter. 2016. Generative art theory. A companion to digital art (2016), 146–180.
  21. John E Gedo. 1996. The artist & the emotional world: creativity and personality. Columbia University Press.
  22. TEAM STORM: demonstrating an interaction model for working with multiple ideas during creative group work. In Proceedings of the 6th ACM SIGCHI Conference on Creativity & Cognition. 193–202.
  23. Evaluating large language models in generating synthetic hci research data: a case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–19.
  24. Nicolai Brodersen Hansen and Peter Dalsgaard. 2012. The productive role of material design artefacts in participatory design events. In Proceedings of the 7th nordic conference on human-computer interaction: Making sense through design. 665–674.
  25. Reflection enhances creativity: Beneficial effects of idea evaluation on idea generation. Brain and cognition 103 (2016), 30–37.
  26. Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
  27. The Dimensions of Reflection Coding Scheme: A New Tool for Measuring the Impact of Designing for Reflection in Early Childhood. In Proceedings of the 15th Conference on Creativity and Cognition. 519–528.
  28. Caroline Hummels and Joep Frens. 2009. The reflective transformative design process. In CHI’09 Extended Abstracts on Human Factors in Computing Systems. 2655–2658.
  29. A four-category scheme for coding and assessing the level of reflection in written work. Assessment & evaluation in higher education 33, 4 (2008), 369–379.
  30. Human-centered approaches to fair and responsible AI. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–8.
  31. Streaming video to enhance students’ reflection in dance education. Computers & Education 52, 1 (2009), 169–176.
  32. A conversation analysis of non-progress and coping strategies with a banking task-oriented chatbot. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.
  33. Evaluating the logical reasoning ability of chatgpt and gpt-4. arXiv preprint arXiv:2304.03439 (2023).
  34. Getting started with P5. js: Making interactive graphics in JavaScript and processing. Maker Media, Inc.
  35. Freed: a system for creating multiple views of a digital collection during the design process. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1481–1490.
  36. A conversational paradigm for program synthesis. arXiv e-prints (2022), arXiv–2203.
  37. Kylie Peppler and Yasmin Kafai. 2005. Creative coding: Programming for personal expression. Retrieved August 30, 2008 (2005), 314.
  38. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446 (2021).
  39. Casey Reas and Ben Fry. 2007. Processing: a programming handbook for visual designers and artists. Mit Press.
  40. The programmer’s assistant: Conversational interaction with a large language model for software development. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 491–514.
  41. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950 (2023).
  42. Johnny Saldaña. 2021. The coding manual for qualitative researchers. The coding manual for qualitative researchers (2021), 1–440.
  43. Design and evaluation of a short version of the user experience questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence, 4 (6), 103-108. (2017).
  44. Role of interface manipulation style and scaffolding on cognition and concept learning in learnware. ACM Transactions on Computer-Human Interaction (TOCHI) 8, 1 (2001), 34–59.
  45. Reflective design. In Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility. 49–58.
  46. Moushumi Sharmin and Brian P Bailey. 2011. ” I reflect to improve my design” investigating the role and process of reflection in creative design. In Proceedings of the 8th ACM Conference on Creativity and Cognition. 389–390.
  47. Moushumi Sharmin and Brian P Bailey. 2013. ReflectionSpace: an interactive visualization tool for supporting reflection-on-action in design. In Proceedings of the 9th ACM Conference on Creativity & Cognition. 83–92.
  48. Ben Shneiderman. 2002. Creativity support tools. Commun. ACM 45, 10 (2002), 116–120.
  49. Putting GPT-3’s Creativity to the (Alternative Uses) Test. arXiv preprint arXiv:2206.08932 (2022).
  50. Story centaur: Large language model few shot learning as a creative writing tool. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. 244–256.
  51. Explicit reflection in prolog-tutor. International Journal of Artificial Intelligence in Education 17, 2 (2007), 169–215.
  52. Michael Tempel. 2017. Generative art for all. Journal of Innovation and Entrepreneurship 6, 1 (2017), 1–14.
  53. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 (2022).
  54. Adrian Ward and Geoff Cox. 1999. How I Drew One of My Pictures: or, The Authorship of Generative Art. In International Conference on Generative Art. Generative Design Lab Milan.
  55. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 (2022).
  56. Better together? an evaluation of ai-supported code translation. In 27th International Conference on Intelligent User Interfaces. 369–391.
  57. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382 (2023).
  58. In-ide code generation from natural language: Promise and challenges. ACM Transactions on Software Engineering and Methodology (TOSEM) 31, 2 (2022), 1–47.
  59. How to guide task-oriented chatbot users, and when: A mixed-methods study of combinations of chatbot guidance types and timings. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–16.
  60. A value sensitive action-reflection model: evolving a co-design space with stakeholder and designer prompts. In Proceedings of the SIGCHI conference on human factors in computing systems. 419–428.
  61. Arturo Castro et al Zach Lieberman, Theodore Watson. 2009. Openframeworks. http://openframeworks.cc/about
  62. Productivity assessment of neural code completion. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming. 21–29.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Anqi Wang (26 papers)
  2. Zhizhuo Yin (4 papers)
  3. Yulu Hu (1 paper)
  4. Yuanyuan Mao (6 papers)
  5. Pan Hui (155 papers)
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets