Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Metamorpheus: Interactive, Affective, and Creative Dream Narration Through Metaphorical Visual Storytelling (2403.00632v1)

Published 1 Mar 2024 in cs.HC, cs.AI, cs.CL, and cs.CY

Abstract: Human emotions are essentially molded by lived experiences, from which we construct personalised meaning. The engagement in such meaning-making process has been practiced as an intervention in various psychotherapies to promote wellness. Nevertheless, to support recollecting and recounting lived experiences in everyday life remains under explored in HCI. It also remains unknown how technologies such as generative AI models can facilitate the meaning making process, and ultimately support affective mindfulness. In this paper we present Metamorpheus, an affective interface that engages users in a creative visual storytelling of emotional experiences during dreams. Metamorpheus arranges the storyline based on a dream's emotional arc, and provokes self-reflection through the creation of metaphorical images and text depictions. The system provides metaphor suggestions, and generates visual metaphors and text depictions using generative AI models, while users can apply generations to recolour and re-arrange the interface to be visually affective. Our experience-centred evaluation manifests that, by interacting with Metamorpheus, users can recall their dreams in vivid detail, through which they relive and reflect upon their experiences in a meaningful way.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (92)
  1. Kathleen Adams. 1999. A Brief History of Journal Writing. Retrieved Aug 18, 2023 from https://web.archive.org/web/20130205033640/http://journaltherapy.com/journaltherapy/journal-cafe/journal-writing-history
  2. Nazanin Andalibi and Justin Buss. 2020. The human in emotion recognition on social media: Attitudes, outcomes, risks. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–16.
  3. Emoballoon-conveying emotional arousal in text chats with speech balloons. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–16.
  4. Trackly: A customisable and pictorial self-tracking app to support agency in multiple sclerosis self-care. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–15.
  5. Flexible and mindful self-tracking: Design implications from paper bullet journals. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–14.
  6. Lisa Feldman Barrett. 2006. Solving the emotion paradox: Categorization and the experience of emotion. Personality and social psychology review 10, 1 (2006), 20–46.
  7. Lisa Feldman Barrett. 2017a. How emotions are made: The secret life of the brain. Pan Macmillan.
  8. Lisa Feldman Barrett. 2017b. The theory of constructed emotion: an active inference account of interoception and categorization. Social cognitive and affective neuroscience 12, 1 (2017), 1–23.
  9. Of mice and men: Natural kinds of emotions in the mammalian brain? A response to Panksepp and Izard. Perspectives on psychological science 2, 3 (2007), 297–312.
  10. Lisa Feldman Barrett and James A Russell. 2014. The psychological construction of emotion. Guilford Publications.
  11. Lisa Feldman Barrett and Ajay Bhaskar Satpute. 2013. Large-scale brain networks in affective and social neuroscience: towards an integrative functional architecture of the brain. Current opinion in neurobiology 23, 3 (2013), 361–372.
  12. Alan Paul Bell and Calvin Springer Hall. 2011. The personality of a child molester: An analysis of dreams. Transaction Publishers.
  13. Cynthia L Bennett and Daniela K Rosner. 2019. The promise of empathy: Design, disability, and knowing the” other”. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–13.
  14. Adriel Boals. 2012. The use of meaning making in expressive writing: When meaning is beneficial. Journal of Social and Clinical Psychology 31, 4 (2012), 393–409.
  15. Affect: from information to interaction. In Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility. 59–68.
  16. Relation of dreams to waking concerns. Psychiatry research 141, 3 (2006), 261–270.
  17. Who is the” human” in human-centered machine learning: The case of predicting mental health from social media. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–32.
  18. Palette-based photo recoloring. ACM Trans. Graph. 34, 4 (2015), 139–1.
  19. Trauma-informed computing: Towards safer technology experiences for all. In Proceedings of the 2022 CHI conference on human factors in computing systems. 1–20.
  20. TaleBrush: Sketching stories with generative pretrained language models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–19.
  21. Juliet Corbin and Anselm Strauss. 2014. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications.
  22. Van de Castle and RL Our Dreaming Mind. 1994. Ballantine Books. New York (1994).
  23. G William Domhoff. 1996. Finding meaning in dreams: A quantitative approach. (1996).
  24. G William Domhoff. 2017. The invasion of the concept snatchers: The origins, distortions, and future of the continuity hypothesis. Dreaming 27, 1 (2017), 14.
  25. DeepThInk: Designing and probing human-AI co-creation in digital art therapy. International Journal of Human-Computer Studies 181 (2024), 103139.
  26. Paul Ekman. 1992. An argument for basic emotions. Cognition & emotion 6, 3-4 (1992), 169–200.
  27. Paul Ekman et al. 1999. Basic emotions. Handbook of cognition and emotion 98, 45-60 (1999), 16.
  28. It’s just my history isn’t it? Understanding smart journaling practices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2819–2831.
  29. Vyvyan Evans. 2017. The emoji code: How smiley faces, love hearts and thumbs up are changing the way we communicate. Michael O’Mara Books.
  30. Bob Fleshman and Jerry L Fryrear. 1981. The arts in therapy. (No Title) (1981).
  31. Charles Forceville. 2011. Pictorial runes in Tintin and the Picaros. Journal of Pragmatics 43, 3 (2011), 875–890.
  32. Maria Gendron and Lisa Feldman Barrett. 2018. Emotion perception as conceptual synchrony. Emotion Review 10, 2 (2018), 101–110.
  33. Katy Ilonka Gero and Lydia B Chilton. 2019. Metaphoria: An algorithmic companion for metaphor creation. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–12.
  34. Aetiology and treatment of nightmare disorder: State of the art and future perspectives. Journal of sleep research 28, 4 (2019), e12820.
  35. Soudabeh Givrad. 2016. Dream theory and science: A review. Psychoanalytic Inquiry 36, 3 (2016), 199–213.
  36. Samuel T Gladding. 1992. Counseling as an art: The creative arts in counseling. ERIC.
  37. Calvin Hall and Robert Van de Castle. 1966. The content analysis of dreams. (1966).
  38. Gordon Halliday. 1987. Direct psychological therapies for nightmares: A review. Clinical psychology review 7, 5 (1987), 501–523.
  39. Brett A Halperin and Stephanie M Lukin. 2023. Envisioning Narrative Intelligence: A Creative Visual Storytelling Anthology. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21.
  40. The three paradigms of HCI. In Alt. Chi. Session at the SIGCHI Conference on human factors in computing systems San Jose, California, USA. 1–18.
  41. Clara E Hill. 1996. Working with dreams in psychotherapy. Guilford Press.
  42. Dream interpretation sessions: Who volunteers, who benefits, and what volunteer clients view as most and least helpful. Journal of Counseling Psychology 44, 1 (1997), 53.
  43. Personal Dream Informatics: A Self-Information Systems Model of Dream Engagement. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–16.
  44. Erik Hoel. 2021. The overfitted brain: Dreams evolved to assist generalization. Patterns 2, 5 (2021).
  45. Kristina Höök. 2004. User-centred design and evaluation of affective interfaces: A two-tiered model. From brows to trust: evaluating embodied conversational agents (2004), 127–160.
  46. MetaMap: Supporting visual metaphor ideation through multi-dimensional example-based exploration. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.
  47. Use of dreams in therapy: A survey of clinicians in private practice. Psychological Reports 76, 3_suppl (1995), 1288–1290.
  48. Imagery rehearsal treatment for chronic nightmares. Behaviour Research and Therapy 33, 7 (1995), 837–843.
  49. Barry Krakow and Antonio Zadra. 2006. Clinical management of chronic nightmares: imagery rehearsal therapy. Behavioral sleep medicine 4, 1 (2006), 45–70.
  50. Neuroimaging of dreaming: state of the art and limitations. International Review of Neurobiology 92 (2010), 87–99.
  51. Affect labeling disrupts amygdala activity in response to affective stimuli. Psychological Science 18, 5 (2007), 421–428.
  52. The brain basis of emotion: a meta-analytic review. Behavioral and brain sciences 35, 3 (2012), 121–143.
  53. Glancee: An adaptable system for instructors to grasp student learning status in synchronous online classes. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–25.
  54. Cathy A Malchiodi. 2013. Expressive therapies. Guilford Publications.
  55. Music therapy for depression. Cochrane database of systematic reviews 1 (2008).
  56. John McCarthy and Peter Wright. 2004. Technology as experience. interactions 11, 5 (2004), 42–43.
  57. Shaun McNiff. 1981. The arts and psychotherapy. Charles C. Thomas.
  58. Shaun McNiff. 1992. Art as medicine: Creating a therapy of the imagination. Shambhala Publications.
  59. Elisa D Mekler and Kasper Hornbæk. 2019. A framework for the experience of meaning in human-computer interaction. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–15.
  60. The value of experience-centred design approaches in dementia research contexts. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 1326–1338.
  61. Clark Moustakas. 1994. Phenomenological research methods. Sage publications.
  62. Affectivespotlight: Facilitating the communication of affective responses from audience members during online presentations. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.
  63. Alicia Ostriker. 2018. Poetry and Healing: Some Moments of Wholeness. The American Poetry Review 47, 2 (2018), 9–12.
  64. Laura Palagini and Nicholas Rosenlicht. 2011. Sleep, dreaming, and mental health: a review of historical and neurobiological perspectives. Sleep Medicine Reviews 15, 3 (2011), 179–186.
  65. Mark Pearson and Helen Wilson. 2008. Using expressive counselling tools to enhance emotional literacy, emotional wellbeing and resilience: Improving therapeutic outcomes with Expressive Therapies. Counselling, Psychotherapy and Health 4, 1 (2008), 1–19.
  66. James W Pennebaker. 1997. Writing about emotional experiences as a therapeutic process. Psychological science 8, 3 (1997), 162–166.
  67. Making history: intentional capture of future memories. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 1723–1732.
  68. Barbara J Phillips and Edward F McQuarrie. 2004. Beyond visual metaphor: A new typology of visual rhetoric in advertising. Marketing theory 4, 1-2 (2004), 113–136.
  69. Rosalind W Picard. 2000. Affective computing. MIT press.
  70. Terry Pifalo. 2007. Jogging the cogs: Trauma-focused art therapy and cognitive behavioral therapy with sexually abused children. Art Therapy 24, 4 (2007), 170–175.
  71. Articulating experience: Reflections from experts applying micro-phenomenology to design research in HCI. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.
  72. Nina Rajcic and Jon McCormack. 2020a. Mirror ritual: An affective interface for emotional self-reflection. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–13.
  73. Nina Rajcic and Jon McCormack. 2020b. Mirror Ritual: human-machine co-construction of emotion. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction. 697–702.
  74. Negotiating Experience and Communicating Information Through Abstract Metaphor. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–16.
  75. The effectiveness of art therapy: does it work? Art Therapy 17, 3 (2000), 207–213.
  76. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684–10695.
  77. James A Russell. 2003. Core affect and the psychological construction of emotion. Psychological review 110, 1 (2003), 145.
  78. James A Russell. 2015. The greater constructionist project for emotion. The psychological construction of emotion (2015), 429–447.
  79. What life events are disclosed on social media, how, when, and by whom?. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1–22.
  80. The functional role of dreaming in emotional processes. Frontiers in psychology 10 (2019), 459.
  81. Trauma-Informed Social Media: Towards Solutions for Reducing and Healing Online Harm. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–20.
  82. Outcome studies on the efficacy of art therapy: A review of findings. Art therapy 27, 3 (2010), 108–118.
  83. Jonathan A Smith. 2007. Hermeneutics, human sciences and health: Linking theory and practice. International Journal of Qualitative Studies on health and Well-being 2, 1 (2007), 3–11.
  84. Painful metaphors: enactivism and art in qualitative research. Medical humanities 47, 2 (2021), 235–247.
  85. Heather L Stuckey and Jeremy Nobel. 2010. The connection between art, healing, and public health: A review of current literature. American journal of public health 100, 2 (2010), 254–263.
  86. MEMEories: internet memes as means for daily journaling. In Designing Interactive Systems Conference 2021. 538–548.
  87. Jakob Tholander and Maria Normark. 2020. Crafting personal information-resistance, imperfection, and self-creation in bullet journaling. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
  88. Jared B Torre and Matthew D Lieberman. 2018. Putting feelings into words: Affect labeling as implicit emotion regulation. Emotion Review 10, 2 (2018), 116–124.
  89. Timothy D Wilson and Jonathan W Schooler. 1991. Thinking too much: introspection can reduce the quality of preferences and decisions. Journal of personality and social psychology 60, 2 (1991), 181.
  90. Aesthetics and experience-centered design. ACM Transactions on Computer-Human Interaction (TOCHI) 15, 4 (2008), 1–21.
  91. Investigating the Effects of Self-Avatars and Story-Relevant Avatars on Children’s Creative Storytelling. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–11.
  92. StoryDrawer: A Child–AI Collaborative Drawing System to Support Children’s Creative Visual Storytelling. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–15.
Citations (3)
View on Semantic Scholar

Summary

  • The paper introduces an affective AI interface that co-constructs metaphorical visuals and textual narratives from users’ dream emotions.
  • It employs a phenomenological evaluation approach, revealing enhanced emotional recollection and mindful self-reflection in participants.
  • The study highlights potential applications in digital journaling and therapeutic settings by bridging technology with creative emotional expression.

Interactive Affective Narration with Metamorpheus: A New Avenue for Generative AI in Dream Storytelling

Exploring Metamorpheus: An Interface for Dream Narration

In the field of human-computer interaction (HCI), exploring the nuances of human emotions through technology has always been a path filled with intrigue and potential. A paper presents Metamorpheus, an affective interface designed to engage users in a creative visual storytelling process, focusing on emotional experiences during dreams. Metamorpheus stands out by arranging storylines based on the emotional arc of dreams and encouraging the creation of metaphorical images and text depictions. It leverages generative AI models to provide metaphor suggestions, generating visual metaphors and textual narratives that users can further refine. This system enables recoloring and re-arrangement of interfaces to enhance visual affectivity, presenting a unique approach to interactive and affective computing.

The Quest for Emotional Expression and Self-Reflection

The core of Metamorpheus lies in its capability to provoke self-reflection and memory recall regarding dreams. Dreams, often a direct reflection of our unconscious experiences and emotions, provide fertile ground for exploring personal meaning and affective mindfulness. Metamorpheus taps into this by facilitating a creative process where users narrate their dreams through metaphorical visuals and texts. This process not only aids in vivid dream recall but also allows users to relive and reflect upon their experiences in a structured and meaningful manner. The blend of AI-generated suggestions and user creativity forms an intimate space for users to explore and articulate emotions that are otherwise hard to express.

Engaging in Meaning Co-construction

One of the pivotal explorations of this paper is the interactive meaning co-construction between the user and the AI. This process begins with the user narrating emotional scenes from dreams, prompting the AI to generate metaphorical visuals and texts. Users iteratively refine these outputs, imbuing them with personal meaning and context. This co-creative process underscores a significant shift from conventional AI roles, positioning the AI as a partner in the creative process rather than merely a tool. It underscores the potential of generative AI models in facilitating a deep exploration of emotional experiences, bridging the gap between technology and the intricate landscape of human emotions.

Methodology and Insights

The paper employed a phenomenological approach in its evaluation, focusing on the experiential aspects of interaction with Metamorpheus. Participants reported a heightened awareness of their emotions and a more vivid recollection of their dreams. This points towards the interface's effectiveness in fostering a reflective and mindful engagement with one’s own affective states. Interestingly, the process of creating and interacting with metaphorical representations offered a non-invasive, therapeutic-like experience, enabling participants to revisit emotional experiences from a place of safety and detachment.

Potential Applications and Future Directions

The Metamorpheus paper opens several avenues for future research and application. The interface’s approach to dream narration could be extended to other forms of personal informatics and digital journaling, offering a novel method for emotional tracking and reflection. Additionally, the concept of AI-facilitated creative narration holds promise for therapeutic applications, offering a new tool for expressing and processing complex emotional experiences.

Conclusion

Metamorpheus represents an innovative step forward in the field of affective computing and HCI. By combining generative AI with interactive design, it offers a unique platform for individuals to explore and narrate their emotional experiences through the creative storytelling of dreams. This paper not only highlights the potential of AI in facilitating meaningful self-exploration but also paves the way for new forms of interaction between humans and technology, centered around the profound and personal field of emotions and dreams.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown