Papers
Topics
Authors
Recent
2000 character limit reached

Designing Conversational Multimodal 3D Printed Models with People who are Blind

Published 12 Jun 2023 in cs.HC | (2306.06811v1)

Abstract: 3D printed models have been used to improve access to graphical information by people who are blind, offering benefits over conventional accessible graphics. Here we investigate an interactive 3D printed model (I3M) that combines a conversational interface with haptic vibration and touch to provide more natural and accessible experiences. Specifically, we co-designed a multimodal model of the Solar System with nine blind people and evaluated the prototype with another seven blind participants. We discuss our journey from a design perspective, focusing on touch, conversational and multimodal interactions. Based on our experience, we suggest design recommendations that consider blind users' desire for independence and control, customisation, comfort and use of prior experience

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. Nazatul Naquiah Abd Hamid and Alistair D.N. Edwards. 2013. Facilitating Route Learning Using Interactive Audio-tactile Maps for Blind and Visually Impaired People. In CHI ’13 Extended Abstracts on Human Factors in Computing Systems (Paris, France) (CHI EA ’13). ACM, New York, NY, USA, 37–42. https://doi.org/10.1145/2468356.2468364
  2. ”Siri Talks at You”: An Empirical Investigation of Voice-Activated Personal Assistant (VAPA) Usage by Individuals Who Are Blind. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (Galway, Ireland) (ASSETS ’18). ACM, New York, NY, USA, 249–258. https://doi.org/10.1145/3234695.3236344
  3. Towards a Multisensory Augmented Reality Map for Blind and Low Vision People: A Participatory Design Approach. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3173574.3174203
  4. Frances K. Aldrich and Linda Sheppard. 2001. Tactile graphics in school education: perspectives from pupils. British Journal of Visual Impairment 19, 2 (2001), 69–73. https://doi.org/10.1177/026461960101900204 arXiv:https://doi.org/10.1177/026461960101900204
  5. Shiri Azenkot and Nicole B. Lee. 2013. Exploring the Use of Speech Input by Blind People on Mobile Devices. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (Bellevue, Washington) (ASSETS ’13). ACM, New York, NY, USA, Article 11, 8 pages. https://doi.org/10.1145/2513383.2513440
  6. Map Use by Adults with Visual Impairments. The Professional Geographer 51, 4 (1999), 539–553. https://doi.org/10.1111/0033-0124.00191 arXiv:https://doi.org/10.1111/0033-0124.00191
  7. Richard A. Bolt. 1980. “Put-That-There”: Voice and Gesture at the Graphics Interface. In Proceedings of the 7th Annual Conference on Computer Graphics and Interactive Techniques (Seattle, Washington, USA) (SIGGRAPH ’80). Association for Computing Machinery, New York, NY, USA, 262–270. https://doi.org/10.1145/800250.807503
  8. Stacy M. Branham and Antony Rishin Mukkath Roy. 2019. Reading Between the Guidelines: How Commercial Voice Assistant Guidelines Hinder Accessibility for Blind Users. In Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (Pittsburgh, PA, USA) (ASSETS ’19). Association for Computing Machinery, New York, NY, USA, 446–458. https://doi.org/10.1145/3308561.3353797
  9. Design and User Satisfaction of Interactive Maps for Visually Impaired People. Computers Helping People with Special Needs 7383 (2012), 544–551. https://doi.org/10.1007/978-3-642-31534-3_80
  10. Craig Brown and Amy Hurst. 2012. VizTouch: Automatically Generated Tactile Visualizations of Coordinate Spaces. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (Kingston, Ontario, Canada) (TEI ’12). ACM, New York, NY, USA, 131–138. https://doi.org/10.1145/2148131.2148160
  11. MapSense: Multi-Sensory Interactive Maps for Children Living with Visual Impairments. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). ACM, New York, NY, USA, 445–457. https://doi.org/10.1145/2858036.2858375
  12. Investigating the Implications of 3D Printing in Special Education. ACM Trans. Access. Comput. 8, 3, Article 11 (March 2016), 28 pages. https://doi.org/10.1145/2870640
  13. Understanding the graphical challenges faced by vision-impaired students in Australian universities. Higher Education Research & Development 36, 1 (2017), 59–72. https://doi.org/10.1080/07294360.2016.1177001 arXiv:https://doi.org/10.1080/07294360.2016.1177001
  14. Technology Developments in Touch-Based Accessible Graphics: A Systematic Review of Research 2010-2020. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 278, 15 pages. https://doi.org/10.1145/3411764.3445207
  15. Exploring Human-Robot Interaction with the Elderly: Results from a Ten-Week Case Study in a Care Home. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376402
  16. Justine Cassell. 2000. More than just another pretty face: Embodied conversational interface agents. Commun. ACM 43, 4 (2000), 70–78.
  17. Jido: A Conversational Tactile Map for Blind People. In Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (Pittsburgh, PA, USA) (ASSETS ’19). Association for Computing Machinery, New York, NY, USA, 682–684. https://doi.org/10.1145/3308561.3354600
  18. An Interactive Multimodal Guide to Improve Art Accessibility for Blind People. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (Galway, Ireland) (ASSETS ’18). Association for Computing Machinery, New York, NY, USA, 346–348. https://doi.org/10.1145/3234695.3241033
  19. Accessible Visual Artworks for Blind and Visually Impaired People: Comparing a Multimodal Approach with Tactile Graphics. Electronics 10, 3 (2021). https://doi.org/10.3390/electronics10030297
  20. ”Nobody Speaks That Fast!” An Empirical Study of Speech Rate in Conversational Agents for People with Vision Impairments. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376569
  21. TangibleCircuits: An Interactive 3D Printed Circuit Education Tool for People with Visual Impairments. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376513
  22. Exploring Map Orientation with Interactive Audio-Tactile Maps. In Human-Computer Interaction – INTERACT 2015, Julio Abascal, Simone Barbosa, Mirko Fetter, Tom Gross, Philippe Palanque, and Marco Winckler (Eds.). Springer International Publishing, Cham, 72–79.
  23. The Cross-Sensory Globe: Participatory Design of a 3D Audio-Tactile Globe Prototype for Blind and Low-Vision Users to Learn Geography. In Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, CA, USA) (DIS ’19). Association for Computing Machinery, New York, NY, USA, 399–412. https://doi.org/10.1145/3322276.3323686
  24. Map learning with a 3D printed interactive small-scale model: Improvement of space and text memorization in visually impaired students. Frontiers in Psychology 8, 930 (2017). https://doi.org/10.3389/fpsyg.2017.00930
  25. The Making of Accessibility to Rural Place for Blind People: The Relational Design of an Interactive Map. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (Eindhoven, Netherlands) (DIS ’20). Association for Computing Machinery, New York, NY, USA, 1419–1431. https://doi.org/10.1145/3357236.3395527
  26. Cagatay Goncu and Kim Marriott. 2011. GraVVITAS: Generic Multi-touch Presentation of Accessible Graphics. In Human-Computer Interaction – INTERACT 2011, Pedro Campos, Nicholas Graham, Joaquim Jorge, Nuno Nunes, Philippe Palanque, and Marco Winckler (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 30–48.
  27. A Personal Computer-based Approach for 3D Printing Accessible to Blind People. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments (Island of Rhodes, Greece) (PETRA ’17). ACM, New York, NY, USA, 1–4. https://doi.org/10.1145/3056540.3064954
  28. Visual Impairment and urban orientation. Pilot study with tactile maps produced through 3D Printing. Psyecology 3, 2 (2012), 239–250.
  29. Methods for Creating and Evaluating 3D Tactile Images to Teach STEM Courses to the Visually Impaired. Journal of College Science Teaching 44, 6 (2015), 92–99. http://www.jstor.org/stable/43632001
  30. Animations at Your Fingertips: Using a Refreshable Tactile Display to Convey Motion Graphics for People Who Are Blind or Have Low Vision. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (Athens, Greece) (ASSETS ’22). Association for Computing Machinery, New York, NY, USA, Article 32, 16 pages. https://doi.org/10.1145/3517428.3544797
  31. Accessible maps for the blind: Comparing 3D printed models with tactile graphics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montréal, QC, Canada) (CHI ’18). ACM. https://doi.org/10.1145/3173574.3173772
  32. Making Sense of Art: Access for Gallery Visitors with Vision Impairments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300250
  33. 3D Printed Maps and Icons for Inclusion: Testing in the Wild by People Who Are Blind or Have Low Vision. In Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (Pittsburgh, PA, USA) (ASSETS ’19). Association for Computing Machinery, New York, NY, USA, 183–195. https://doi.org/10.1145/3308561.3353790
  34. Michele Hu. 2015. Exploring New Paradigms for Accessible 3D Printed Graphs. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (Lisbon, Portugal) (ASSETS ’15). ACM, New York, NY, USA, 365–366. https://doi.org/10.1145/2700648.2811330
  35. Exploring Art with a Voice Controlled Multimodal Guide for Blind People. In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction (Tempe, Arizona, USA) (TEI ’19). Association for Computing Machinery, New York, NY, USA, 383–390. https://doi.org/10.1145/3294109.3300994
  36. Shaun K. Kane and Jeffrey P. Bigham. 2014. Tracking@ stemxcomet: teaching programming to blind students via 3D printing, crisis management, and twitter. In Proceedings of the 45th ACM technical symposium on Computer science education. ACM, 247–252.
  37. Jill Keeffe. 2005. Psychosocial Impact of Vision Impairment. International Congress Series 1282 (2005), 167–173.
  38. Jeeeun Kim and Tom Yeh. 2015. Toward 3D-printed movable tactile pictures for children with visual impairments. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 2815–2824.
  39. Accessible Visualization: Design Space, Opportunities, and Challenges. Computer Graphics Forum 40, 3 (2021), 173–188. https://doi.org/10.1111/cgf.14298
  40. What’s This? A Voice and Touch Multimodal Approach for Ambiguity Resolution in Voice Assistants. In Proceedings of the 2021 International Conference on Multimodal Interaction (Montréal, QC, Canada) (ICMI ’21). Association for Computing Machinery, New York, NY, USA, 512–520. https://doi.org/10.1145/3462244.3479902
  41. Re-Embodiment and Co-Embodiment: Exploration of Social Presence for Robots and Conversational Agents. In Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, CA, USA) (DIS ’19). Association for Computing Machinery, New York, NY, USA, 633–644. https://doi.org/10.1145/3322276.3322340
  42. Improving Humans’ Ability to Interpret Deictic Gestures in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376340
  43. Tactile Aids for Visually Impaired Graphical Design Education. In Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (Rochester, New York, USA) (ASSETS ’14). ACM, New York, NY, USA, 275–276. https://doi.org/10.1145/2661334.2661392
  44. Betsy Phillips M.S. and Hongxin Zhao Ph.D. 1993. Predictors of Assistive Technology Abandonment. Assistive Technology 5, 1 (1993), 36–45. https://doi.org/10.1080/10400435.1993.10132205 arXiv:https://doi.org/10.1080/10400435.1993.10132205 PMID: 10171664.
  45. 3D Building Plans: Supporting Navigation by People Who Are Blind or Have Low Vision in Multi-Storey Buildings. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 539, 19 pages. https://doi.org/10.1145/3544548.3581389
  46. National Federation of the Blind. 2009. The Braille Literacy Crisis in America: Facing the Truth, Reversing the Trend, Empowering the Blind. (2009). Available from https://nfb.org/images/nfb/documents/pdf/braille_literacy_report_web.pdf.
  47. TeDUB: A System for Presenting and Exploring Technical Drawings for Blind People. In Computers Helping People with Special Needs, Klaus Miesenberger, Joachim Klaus, and Wolfgang Zagler (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 537–539.
  48. Tactile Materials in Practice: Understanding the Experiences of Teachers of the Visually Impaired. ACM Transactions on Accessible Computing (TACCESS) (2021). https://doi.org/10.1145/3508364
  49. TouchOver Map: Audio-tactile Exploration of Interactive Maps. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (Stockholm, Sweden) (MobileHCI ’11). ACM, New York, NY, USA, 545–550. https://doi.org/10.1145/2037373.2037458
  50. ”Accessibility Came by Accident”: Use of Voice-Controlled Intelligent Personal Assistants by People with Disabilities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). ACM, New York, NY, USA, Article 459, 13 pages. https://doi.org/10.1145/3173574.3174033
  51. Guidelines for Multimodal User Interface Design. Commun. ACM 47, 1 (Jan. 2004), 57–59. https://doi.org/10.1145/962081.962106
  52. Gesture-Based Interactive Audio Guide on Tactile Reliefs. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (Reno, Nevada, USA) (ASSETS ’16). ACM, New York, NY, USA, 91–100. https://doi.org/10.1145/2982142.2982176
  53. ”Hey Model!” - Natural User Interactions and Agency in Accessible Interactive 3D Models. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376145
  54. L. Penny Rosenblum and Tina S. Herzberg. 2015. Braille and Tactile Graphics: Youths with Visual Impairments Share Their Experiences. Journal of Visual Impairment & Blindness 109, 3 (2015), 173–184. https://doi.org/10.1177/0145482X1510900302 arXiv:https://doi.org/10.1177/0145482X1510900302
  55. Jonathan Rowell and Simon Ungar. 2005. Feeling our way: tactile map user requirements – a survey. In In Proceedings of 22nd International Cartographic Conference. International Cartographic Association.
  56. Face Value? Exploring the Effects of Embodiment for a Group Facilitation Agent. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3173965
  57. Rebecca Sheffield. 2016. International Approaches to Rehabilitation Programs for Adults who are Blind or Visually Impaired: Delivery Models, Services, Challenges and Trends.
  58. Designing Interactive 3D Printed Models with Teachers of the Visually Impaired. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). ACM, New York, NY, USA, Article 197, 14 pages. https://doi.org/10.1145/3290605.3300427
  59. Tickers and Talker: An accessible labeling toolkit for 3D printed models. In Proceedings of the 34rd Annual ACM Conference on Human Factors in Computing Systems (CHI’16). https://doi.org/10.1145/2858036.2858507
  60. Designing Interactions for 3D Printed Models with Blind People. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS ’17). ACM, New York, NY, USA, 200–209. https://doi.org/10.1145/3132525.3132549
  61. Markit and Talkit: A Low-Barrier Toolkit to Augment 3D Printed Models with Audio Annotations. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST ’17). ACM, New York, NY, USA, 493–506. https://doi.org/10.1145/3126594.3126650
  62. Molder: An Accessible Design Tool for Tactile Maps. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376431
  63. Issues Relating to Trust in Care Agents for the Elderly. In Proceedings of the 1st International Conference on Conversational User Interfaces (Dublin, Ireland) (CUI ’19). Association for Computing Machinery, New York, NY, USA, Article 20, 3 pages. https://doi.org/10.1145/3342775.3342808
  64. Transcribing Across the Senses: Community Efforts to Create 3D Printable Accessible Tactile Pictures for Young Children with Visual Impairments. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (Lisbon, Portugal) (ASSETS ’15). ACM, New York, NY, USA, 127–137. https://doi.org/10.1145/2700648.2809854
  65. TactileMaps.net: A web interface for generating customized 3D-printable tactile maps. In Proc. ACM SIGACCESS Conference on Computers & Accessibility. ACM, 427–428. https://doi.org/10.1145/2700648.2811336
  66. Creating Accessible Interactive Audio-Tactile Drawings Using Spatial Augmented Reality. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces (Daejeon, Republic of Korea) (ISS ’19). Association for Computing Machinery, New York, NY, USA, 17–28. https://doi.org/10.1145/3343055.3359711
  67. Applied computational chemistry for the blind and visually impaired. Journal of Chemical Education 89, 11 (2012), 1400–1404.
Citations (4)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up