Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Indexing Analytics to Instances: How Integrating a Dashboard can Support Design Education (2404.05417v1)

Published 8 Apr 2024 in cs.HC, cs.AI, and cs.CY

Abstract: We investigate how to use AI-based analytics to support design education. The analytics at hand measure multiscale design, that is, students' use of space and scale to visually and conceptually organize their design work. With the goal of making the analytics intelligible to instructors, we developed a research artifact integrating a design analytics dashboard with design instances, and the design environment that students use to create them. We theorize about how Suchman's notion of mutual intelligibility requires contextualized investigation of AI in order to develop findings about how analytics work for people. We studied the research artifact in 5 situated course contexts, in 3 departments. A total of 236 students used the multiscale design environment. The 9 instructors who taught those students experienced the analytics via the new research artifact. We derive findings from a qualitative analysis of interviews with instructors regarding their experiences. Instructors reflected on how the analytics and their presentation in the dashboard have the potential to affect design education. We develop research implications addressing: (1) how indexing design analytics in the dashboard to actual design work instances helps design instructors reflect on what they mean and, more broadly, is a technique for how AI-based design analytics can support instructors' assessment and feedback experiences in situated course contexts; and (2) how multiscale design analytics, in particular, have the potential to support design education. By indexing, we mean linking which provides context, here connecting the numbers of the analytics with visually annotated design work instances.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (85)
  1. Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: A survey on Explainable Artificial Intelligence (XAI). IEEE Access 6 (2018), 52138–52160.
  2. Nancy E Adams. 2015. Bloom’s taxonomy of cognitive learning objectives. Journal of the Medical Library Association: JMLA 103, 3 (2015), 152.
  3. Characterizing the work of coaching during design reviews. Design Studies 45 (2016), 30–67.
  4. Christopher Alexander. 1964. Notes on the Synthesis of Form. Vol. 5. Harvard University Press.
  5. Patricia Armstrong. 2016. Bloom’s taxonomy. Vanderbilt University Center for Teaching (2016).
  6. Kimberly E Arnold and Matthew D Pistilli. 2012. Course signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd international conference on learning analytics and knowledge. ACM, 267–270.
  7. Yaneer Bar-Yam. 2006. Engineering complex systems: multiscale analysis and evolutionary engineering. In Complex engineered systems. Springer, 22–39.
  8. Evan Barba. 2019. Cognitive Point of View in Recursive Design. She Ji: The Journal of Design, Economics, and Innovation 5, 2 (2019), 147–162.
  9. Benjamin B Bederson. 2011. The promise of zoomable user interfaces. Behaviour & Information Technology 30, 6 (2011), 853–866.
  10. Benjamin B Bederson and Angela Boltman. 1999. Does animation help users build mental maps of spatial information?. In Proceedings 1999 IEEE Symposium on Information Visualization (InfoVis’ 99). IEEE, 28–35.
  11. Victoria Bellotti and Keith Edwards. 2001. Intelligibility and accountability: human considerations in context-aware systems. Human–Computer Interaction 16, 2-4 (2001), 193–212.
  12. Melanie Birks and Jane Mills. 2015. Grounded theory: A practical guide. Sage.
  13. Paulo Blikstein. 2011. Using learning analytics to assess students’ behavior in open-ended programming tasks. In Proceedings of the 1st international conference on learning analytics and knowledge. ACM, 110–116.
  14. Benjamin S Bloom et al. 1956. Taxonomy of educational objectives. Vol. 1: Cognitive domain. New York: McKay 20 (1956), 24.
  15. Motivating project-based learning: Sustaining the doing, supporting the learning. Educational psychologist 26, 3-4 (1991), 369–398.
  16. Design is (A)live: An Environment Integrating Ideation and Assessment. In CHI Late-Breaking Work. ACM, 1–8.
  17. Ann L Brown. 1992. Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The journal of the learning sciences 2, 2 (1992), 141–178.
  18. The role of explanations on trust and reliance in clinical decision support systems. In 2015 International Conference on Healthcare Informatics. IEEE, 160–169.
  19. Kathy Charmaz. 2014. Constructing grounded theory. Sage.
  20. Bo T Christensen and Linden J Ball. 2016. Dimensions of creative evaluation: Distinct design and reasoning strategies for aesthetic, functional and originality judgments. Design Studies 45 (2016), 116–136.
  21. A review of overview+ detail, zooming, and focus+ context interfaces. ACM Computing Surveys (CSUR) 41, 1 (2009), 1–31.
  22. Students’ talk about the climate of feedback interventions in the critique. Communication Education 60, 1 (2011), 95–114.
  23. PBL across the disciplines: research into best practice. In The 3rd International Research Symposium on PBL. Aalborg: Aalborg Universitetsforlag.
  24. Using technology to encourage self-directed learning: The Collaborative Lecture Annotation System (CLAS). In Australasian Society for Computers in Learning in Tertiatry Education. 246–255.
  25. Assessment focus in studio: What is most prominent in architecture, art and design? International Journal of Art & Design Education 28, 1 (2009), 37–51.
  26. Ashley Deeks. 2019. The Judicial Demand for Explainable Artificial Intelligence. Columbia Law Review 119, 7 (2019), 1829–1850.
  27. Erik Duval. 2011. Attention please!: learning analytics for visualization and recommendation. In Proceedings of the 1st international conference on learning analytics and knowledge. ACM, 9–17.
  28. Engineering Design Thinking, Teaching, and Learning. Journal of Engineering Education 94, 1 (jan 2005), 103–120. https://doi.org/10.1002/j.2168-9830.2005.tb00832.x
  29. Charles Eames and Ray Eames. 1968. Powers of ten. Pyramid Films (1968).
  30. Vladimir Estivill-Castro and Ickjai Lee. 2002. Multi-level clustering and its visualization for exploratory spatial analysis. GeoInformatica 6, 2 (2002), 123–152.
  31. William Gaver. 2012. What should we expect from research through design?. In Proceedings of the SIGCHI conference on human factors in computing systems. 937–946.
  32. Dedre Gentner and Albert L Stevens. 2014. Mental models. Psychology Press.
  33. John S Gero and Mary Lou Maher. 1993. Modeling creativity and knowledge-based creative design. Psychology Press.
  34. Gabriela Goldschmidt. 1994. On visual design thinking: the vis kids of architecture. Design studies 15, 2 (1994), 158–174.
  35. Collaborative Live Media Curation: Shared Context for Participation in Online Learning. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 1–14.
  36. Artificial intelligence-enhanced decision support for informing global sustainable development: A human-centric AI-thinking approach. Information 11, 1 (2020), 39.
  37. Technology probes: inspiring design for and with families. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 17–24.
  38. Ajit Jain. 2017. Measuring Creativity: Multi-Scale Visual and Conceptual Design Analysis. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. 490–495.
  39. Ajit Jain. 2021. How to Support Situated Design Education through AI-Based Analytics. Ph. D. Dissertation.
  40. Recognizing creative visual design: multiscale design characteristics in free-form web curation documents. In Proceedings of the 21st ACM Symposium on Document Engineering. 1–10.
  41. Evaluating tweetbubble with ideation metrics of exploratory browsing. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition. 53–62.
  42. David H Jonassen. 1994. Thinking technology: Toward a constructivist design model. Educational technology 34, 4 (1994), 34–37.
  43. Strategies of Free-Form Web Curation: Processes of Creative Engagement with Prior Work. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. 380–392.
  44. Using metrics of curation to evaluate information-based ideation. ACM Transactions on Computer-Human Interaction (TOCHI) 21, 3 (2014), 1–48.
  45. Scaling up analogical innovation with crowds and AI. Proceedings of the National Academy of Sciences 116, 6 (2019), 1870–1877.
  46. David R Krathwohl. 2002. A revision of Bloom’s taxonomy: An overview. Theory into practice 41, 4 (2002), 212–218.
  47. Critique style guide: Improving crowdsourced design feedback with a natural language model. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 4627–4639.
  48. Hajin Lim. 2018. Design for Computer-Mediated Multilingual Communication with AI Support. In Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing. 93–96.
  49. What changes, and for whom? A study of the impact of learning analytics-based process feedback in a large course. Learning and Instruction (2019), 101202.
  50. Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist 57, 10 (2013), 1439–1459.
  51. Multiscale Design Strategies in a Landscape Architecture Classroom. In Proceedings of the 2018 on Designing Interactive Systems Conference 2018. ACM, 1081–1093.
  52. Multiscale Design Curation: Supporting Computer Science Students’ Iterative and Reflective Creative Processes. In Proceedings of the 2019 on Creativity and Cognition. ACM, 233–245.
  53. Patterns of free-form curation: Visual thinking with web content. Proceedings of the 2016 ACM on Multimedia Conference. http://dl.acm.org/citation.cfm?id=2964303
  54. Gary Marchionini. 2006. Exploratory search: from finding to understanding. Commun. ACM 49, 4 (2006), 41–46.
  55. Richard E Mayer and Roxana Moreno. 2002. Animation as an aid to multimedia learning. Educational psychology review 14, 1 (2002), 87–99.
  56. National Academy of Engineering. 2010. Rising Above the Gathering Storm, Revisited: Rapidly Approaching Category 5. The National Academies Press.
  57. A theoretical framework of design critiquing in architecture studios. Design Studies 34, 3 (2013), 302–325.
  58. Jane Osmond and Michael Tovey. 2015. The Threshold of Uncertainty in Teaching Design. Design and Technology Education 20, 2 (2015), 50–57.
  59. Aalto Interface Metrics (AIM) A Service and Codebase for Computational GUI Evaluation. In The 31st Annual ACM Symposium on User Interface Software and Technology Adjunct Proceedings. 16–19.
  60. Using learning analytics to scale the provision of personalised feedback. British Journal of Educational Technology 50, 1 (2019), 128–138.
  61. Ken Perlin and David Fox. 1993. Pad: an alternative approach to the computer interface. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques. 57–64.
  62. Metadata type system: Integrate presentation, data models and extraction to enable exploratory browsing interfaces. In Proc. EICS. ACM, 107–116.
  63. XAlgo: a Design Probe of Explaining Algorithms’ Internal States via Question-Answering. In 26th International Conference on Intelligent User Interfaces. 329–339.
  64. Predicting users’ first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. In Proc. CHI. ACM, 2049–2058.
  65. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296 (2017).
  66. Elizabeth B-N Sanders and Pieter Jan Stappers. 2008. Co-creation and the new landscapes of design. Co-design 4, 1 (2008), 5–18.
  67. Ben Shneiderman. 2020. Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy. International Journal of Human–Computer Interaction (2020), 1–10.
  68. Simon Buckingham Shum and Ruth Deakin Crick. 2012. Learning dispositions and transferable competencies: pedagogy, modelling and learning analytics. In Proceedings of the 2nd international conference on learning analytics and knowledge. 92–101.
  69. Katrina Sin and Loganathan Muthu. 2015. Application of Big Data in Education Data Mining and Learning Analytics–A Literature Review. ICTACT journal on soft computing 5, 4 (2015).
  70. The creative cognition approach. MIT press.
  71. Lucy A Suchman. 1987. Plans and situated actions: The problem of human-machine communication. Cambridge university press.
  72. Joshua D Summers and Jami J Shah. 2010. Mechanical engineering design complexity metrics: size, coupling, and solvability. Journal of Mechanical Design 132, 2 (2010).
  73. Envisioning information. Vol. 126. Graphics press Cheshire, CT.
  74. David Turnbull and Helen Watson. 1993. Maps Are Territories Science is an Atlas: A Portfolio of Exhibits. University of Chicago Press.
  75. Animation: can it facilitate? International journal of human-computer studies 57, 4 (2002), 247–262.
  76. Innovative InterLabs system for smart learning analytics in engineering education. In 2019 IEEE Global Engineering Education Conference (EDUCON). IEEE, 1363–1369.
  77. Learning analytics dashboard applications. American Behavioral Scientist 57, 10 (2013), 1500–1509.
  78. Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing 18, 6 (2014), 1499–1514.
  79. A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure–ground organization. Psychological bulletin 138, 6 (2012), 1172.
  80. Designing theory-driven user-centric explainable AI. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–15.
  81. Alyssa Friend Wise. 2014. Designing pedagogical interventions to support student use of learning analytics. In Proceedings of the fourth international conference on learning analytics and knowledge. 203–211.
  82. A classroom study of using crowd feedback in the iterative design process. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing. 1637–1648.
  83. Robert E Yager. 1991. The constructivist learning model. The science teacher 58, 6 (1991), 52.
  84. Research through design as a method for interaction design research in HCI. In Proceedings of the ACM CHI. ACM, 493–502.
  85. An analysis and critique of Research through Design: towards a formalization of a research approach. In proceedings of the 8th ACM conference on designing interactive systems. 310–319.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now