Papers
Topics
Authors
Recent
Search
2000 character limit reached

ChartA11y: Designing Accessible Touch Experiences of Visualizations with Blind Smartphone Users

Published 27 Oct 2024 in cs.HC | (2410.20545v1)

Abstract: We introduce ChartA11y, an app developed to enable accessible 2-D visualizations on smartphones for blind users through a participatory and iterative design process involving 13 sessions with two blind partners. We also present a design journey for making accessible touch experiences that go beyond simple auditory feedback, incorporating multimodal interactions and multisensory data representations. Together, ChartA11y aimed at providing direct chart accessing and comprehensive chart understanding by applying a two-mode setting: a semantic navigation framework mode and a direct touch mapping mode. By re-designing traditional touch-to-audio interactions, ChartA11y also extends to accessible scatter plots, addressing the under-explored challenges posed by their non-linear data distribution. Our main contributions encompass the detailed participatory design process and the resulting system, ChartA11y, offering a novel approach for blind users to access visualizations on their smartphones.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (76)
  1. NV Access. 2024-04-05. NVDA. https://www.nvaccess.org/
  2. SeeChart: Enabling Accessible Visualizations Through Interactive Natural Language Interface For People with Visual Impairments. In Proceedings of the International Conference on Intelligent User Interfaces. 46–64. https://doi.org/10.1145/3581641.3584099 arXiv:2302.07742 [cs].
  3. Towards Digital Image Accessibility for Blind Users Via Vibrating Touch Screen: A Feasibility Test Protocol. In 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems. IEEE, 547–554. https://doi.org/10.1109/SITIS.2012.85
  4. Maryam Bandukda and Catherine Holloway. 2020. Audio AR to support nature connectedness in people with visual disabilities. In Adjunct Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the ACM International Symposium on Wearable Computers (UbiComp/ISWC ’20 Adjunct). ACM, New York, NY, USA, 204–207. https://doi.org/10.1145/3410530.3414332
  5. Beagle: Automated Extraction and Interpretation of Visualizations from the Web. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–8. https://doi.org/10.1145/3173574.3174168
  6. Semantic pointing: improving target acquisition with control-display ratio adaptation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’04). ACM, New York, NY, USA, 519–526. https://doi.org/10.1145/985692.985758
  7. Olli: An Extensible Visualization Library for Screen Reader Accessibility. (2022).
  8. Paul Blenkhorn and D Gareth Evans. 1998. Using speech and touch to enable blind people to access schematic diagrams. Journal of Network and Computer Applications 21, 1 (Jan. 1998), 17–29. https://doi.org/10.1006/jnca.1998.0060
  9. Design guidelines for audio presentation of graphs and tables. In 9th International Conference on Auditory Display (ICAD). 284–287. http://eprints.gla.ac.uk/3196/
  10. Technology Developments in Touch-Based Accessible Graphics: A Systematic Review of Research 2010-2020. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’21). ACM, New York, NY, USA, 1–15. https://doi.org/10.1145/3411764.3445207
  11. Visualizing for the Non-Visual: Enabling the Visually Impaired to Use Visualization. Computer Graphics Forum 38, 3 (2019), 249–260. https://doi.org/10.1111/cgf.13686 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13686.
  12. Towards Understanding Sensory Substitution for Accessible Visualization: An Interview Study. IEEE Transactions on Visualization and Computer Graphics 28, 1 (Jan. 2022), 1084–1094. https://doi.org/10.1109/TVCG.2021.3114829
  13. TactualPlot: spatializing data as sound using sensory substitution for touchscreen accessibility. IEEE Transactions on Visualization and Computer Graphics (2023).
  14. Apple Developer Documentation. 2023-06-16. Audio graphs. https://developer.apple.com/documentation/accessibility/audio_graphs
  15. How accessible is my visualization? Evaluating visualization accessibility with Chartability. Computer Graphics Forum 41, 3 (June 2022), 57–70. https://doi.org/10.1111/cgf.14522
  16. The Accessibility of Data Visualizations on the Web for Screen Reader Users: Practices and Experiences During COVID-19. ACM Transactions on Accessible Computing 16, 1 (March 2023), 1–29. https://doi.org/10.1145/3557899
  17. Catherine Feng. 2016. Designing Wearable Mobile Device Controllers for Blind People: A Co-Design Approach. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’16). ACM, New York, NY, USA, 341–342. https://doi.org/10.1145/2982142.2982144
  18. Evaluating a Tool for Improving Accessibility to Charts and Graphs. ACM Transactions on Computer-Human Interaction 20, 5 (Nov. 2013), 1–32. https://doi.org/10.1145/2533682.2533683
  19. Cross-Modal Equivalence of Visual and Auditory Scatterplots for Exploring Bivariate Data Samples. Human Factors 39, 3 (Sept. 1997), 341–351. https://doi.org/10.1518/001872097778827151 Publisher: SAGE Publications Inc.
  20. Barrier pointing: using physical edges to assist target acquisition on mobile device touch screens. In Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’07). ACM, New York, NY, USA, 19–26. https://doi.org/10.1145/1296843.1296849
  21. Playing With Others: Depicting Multiplayer Gaming Experiences of People With Visual Impairments. In Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility. ACM, New York, NY, USA, 1–12. https://doi.org/10.1145/3373625.3418304
  22. Design Guidelines and Recommendations for Multimodal, Touchscreen-based Graphics. ACM Transactions on Accessible Computing 13, 3 (Sept. 2020), 1–30. https://doi.org/10.1145/3403933
  23. Tovi Grossman and Ravin Balakrishnan. 2005. The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor’s activation area. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 281–290. https://doi.org/10.1145/1054972.1055012
  24. Highcharts. 2024-04-05. Interactive charting library. https://www.highcharts.com/blog/homepage21may/
  25. Infosonics: Accessible Infographics for People who are Blind using Sonification and Voice. In CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–13. https://doi.org/10.1145/3491102.3517465
  26. Accessible Data Representation with Natural Sound. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’23). ACM, New York, NY, USA, Article 826, 19 pages. https://doi.org/10.1145/3544548.3581087
  27. Freedom Scientific Inc. 2024-04-05. JAWS®. https://www.freedomscientific.com/products/software/jaws/
  28. “Customization is Key”: Reconfigurable Content Tokens for Accessible Data Visualizations. (2024).
  29. Visualization Accessibility in the Wild: Challenges Faced by Visualization Designers. In CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–19. https://doi.org/10.1145/3491102.3517630
  30. Communicating Visualizations without Visuals: Investigation of Visualization Alternative Text for People with Visual Impairments. IEEE Transactions on Visualization and Computer Graphics 28, 1 (Jan. 2022), 1095–1105. https://doi.org/10.1109/TVCG.2021.3114846
  31. ChartSense: Interactive Data Extraction from Chart Images. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’17). ACM, New York, NY, USA, 6706–6717. https://doi.org/10.1145/3025453.3025957
  32. Slide rule: making mobile touch screens accessible to blind people using multi-touch interaction techniques. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility (ASSETS ’08). ACM, New York, NY, USA, 73–80. https://doi.org/10.1145/1414471.1414487
  33. Usable gestures for blind people: understanding preference and performance. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’11). ACM, New York, NY, USA, 413–422. https://doi.org/10.1145/1978942.1979001
  34. Edward Kim and Kathleen F. McCoy. 2018. Multimodal Deep Learning using Images and Text for Information Graphic Classification. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’18). ACM, New York, NY, USA, 143–148. https://doi.org/10.1145/3234695.3236357
  35. Information Graphic Summarization using a Collection of Multimodal Deep Neural Networks. In 2020 25th International Conference on Pattern Recognition (ICPR). 10188–10195. https://doi.org/10.1109/ICPR48806.2021.9412146 ISSN: 1051-4651.
  36. Accessible Visualization: Design Space, Opportunities, and Challenges. Computer Graphics Forum 40, 3 (June 2021), 173–188. https://doi.org/10.1111/cgf.14298
  37. Richard E. Ladner. 2015. Design for user empowerment. Interactions 22, 2 (Feb. 2015), 24–29. https://doi.org/10.1145/2723869
  38. What Frustrates Screen Reader Users on the Web: A Study of 100 Blind Users. International Journal of Human–Computer Interaction 22, 3 (May 2007), 247–269. https://doi.org/10.1080/10447310709336964
  39. CollabAlly: Accessible Collaboration Awareness in Document Editing. In CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–17. https://doi.org/10.1145/3491102.3517635
  40. Toucha11y: Making Inaccessible Public Touchscreens Accessible. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–13. https://doi.org/10.1145/3544548.3581254
  41. BrushLens: Hardware Interaction Proxies for Accessible Touchscreen Interface Actuation. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 1–17. https://doi.org/10.1145/3586183.3606730
  42. Data Extraction from Charts via Single Deep Neural Network. http://arxiv.org/abs/1906.11906
  43. Sociotechnical Considerations for Accessible Visualization Design. In 2019 IEEE Visualization Conference (VIS). 16–20. https://doi.org/10.1109/VISUAL.2019.8933762
  44. Alan Lundgard and Arvind Satyanarayan. 2022. Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content. IEEE Transactions on Visualization and Computer Graphics 28, 1 (Jan. 2022), 1073–1083. https://doi.org/10.1109/TVCG.2021.3114770
  45. Towards an Efficient Framework for Data Extraction from Chart Images. In Document Analysis and Recognition – ICDAR 2021, Josep Lladós, Daniel Lopresti, and Seiichi Uchida (Eds.). Springer International Publishing, Cham, 583–597.
  46. Inclusive data visualization for people with disabilities: a call to action. Interactions 28, 3 (May 2021), 47–51. https://doi.org/10.1145/3457875
  47. David K. McGookin and Stephen A. Brewster. 2006. SoundBar: exploiting multiple views in multimodal graph browsing. In Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles (NordiCHI ’06). ACM, New York, NY, USA, 145–154. https://doi.org/10.1145/1182475.1182491
  48. Collaborative Tabletops for Blind People: The Effect of Auditory Design on Workspace Awareness. Proceedings of the ACM on Human-Computer Interaction 4, ISS (Nov. 2020), 197:1–197:19. https://doi.org/10.1145/3427325
  49. Microsoft Research. 2021. Microsoft Soundscape – A map delivered in 3D sound. https://www.microsoft.com/en-us/research/product/soundscape/.
  50. Social Sensemaking with AI: Designing an Open-ended AI Experience with a Blind Child. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’21). ACM, New York, NY, USA, 1–14. https://doi.org/10.1145/3411764.3445290
  51. Martez E. Mott and Jacob O. Wobbrock. 2014. Beating the bubble: using kinematic triggering in the bubble lens for acquiring small, dense targets. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 733–742. https://doi.org/10.1145/2556288.2557410
  52. Surface haptic interactions with a TPad tablet. In Adjunct Proceedings of the 26th annual ACM symposium on User interface software and technology (UIST ’13 Adjunct). ACM, New York, NY, USA, 7–8. https://doi.org/10.1145/2508468.2514929
  53. Jason Obeid and Enamul Hoque. 2020. Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model. http://arxiv.org/abs/2010.09142
  54. Haptic Information Access Using Touchscreen Devices: Design Guidelines for Accurate Perception of Angular Magnitude and Line Orientation. In Universal Access in Human-Computer Interaction. Methods, Technologies, and Users, Margherita Antona and Constantine Stephanidis (Eds.). Vol. 10907. Springer International Publishing, Cham, 243–255. https://doi.org/10.1007/978-3-319-92049-8_18 Series Title: Lecture Notes in Computer Science.
  55. Spatial Audio-Enhanced Multimodal Graph Rendering for Efficient Data Trend Learning on Touchscreen Devices. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24). ACM, New York, NY, USA, Article 206, 12 pages. https://doi.org/10.1145/3613904.3641959
  56. ReVision: automated classification, analysis and redesign of chart images. In Proceedings of the 24th annual ACM symposium on User interface software and technology. ACM, New York, NY, USA, 393–402. https://doi.org/10.1145/2047196.2047247
  57. AUDITORY AUGMENTATION OF HAPTIC GRAPHS: DEVELOPING A GRAPHIC TOOL FOR TEACHING PRECALCULUS SKILL TO BLIND STUDENTS. (2005).
  58. MAIDR: Making Statistical Visualizations Accessible with Multimodal Data Representation. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24). ACM, New York, NY, USA, Article 211, 22 pages. https://doi.org/10.1145/3613904.3642730
  59. Understanding Screen-Reader Users’ Experiences with Online Data Visualizations. In Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility. ACM, New York, NY, USA, 1–16. https://doi.org/10.1145/3441852.3471202
  60. “What Makes Sonification User-Friendly?” Exploring Usability and User-Friendliness of Sonified Responses. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, New York, NY, USA, 1–5. https://doi.org/10.1145/3517428.3550360
  61. VoxLens: Making Online Data Visualizations Accessible with an Interactive JavaScript Plug-In. In CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–19. https://doi.org/10.1145/3491102.3517431
  62. Understanding and Improving Drilled-Down Information Extraction from Online Data Visualizations for Screen-Reader Users. In Proceedings of the 20th International Web for All Conference. ACM, New York, NY, USA, 18–31. https://doi.org/10.1145/3587281.3587284
  63. Conveying Uncertainty in Data Visualizations to Screen-Reader Users Through Non-Visual Means. In The 25th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, New York, NY, USA, 1–6. https://doi.org/10.1145/3597638.3614502
  64. Designing Interactive 3D Printed Models with Teachers of the Visually Impaired. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300427
  65. Tensor Fields for Data Extraction from Chart Images: Bar Charts and Scatter Plots. http://arxiv.org/abs/2010.02319
  66. Chart Reader: Accessible Visualization Experiences Designed with Screen Reader Users. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’23). ACM, New York, NY, USA, 1–18. https://doi.org/10.1145/3544548.3581186
  67. Shape2Vibe: A Tangible Tool for Vibrotactile Co-Design with People with Deafblindness. In Proceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’24). ACM, New York, NY, USA, 1–6. https://doi.org/10.1145/3623509.3635264
  68. W3C. 2024. Accessible Rich Internet Applications (WAI-ARIA) 1.2. https://www.w3.org/TR/wai-aria/
  69. Toward accessible 3D virtual environments for the blind and visually impaired. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts (DIMEA ’08). ACM, New York, NY, USA, 134–141. https://doi.org/10.1145/1413634.1413663
  70. Jacob Wobbrock. 2003. The benefits of physical edges in gesture-making: Empirical support for an edge-based unistroke alphabet. In CHI’03 Extended Abstracts on Human Factors in Computing Systems. 942–943.
  71. Developing and Deploying a Real-World Solution for Accessible Slide Reading and Authoring for Blind Users. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’23). ACM, New York, NY, USA, 1–15. https://doi.org/10.1145/3597638.3608418
  72. Zhuohao (Jerry) Zhang and Jacob O. Wobbrock. 2023. A11yBoard: Making Digital Artboards Accessible to Blind and Low-Vision Users. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–17. https://doi.org/10.1145/3544548.3580655
  73. Data Sonification for Users with Visual Impairment: A Case Study with Georeferenced Data. ACM Transactions on Computer-Human Interaction 15, 1 (May 2008), 4:1–4:28. https://doi.org/10.1145/1352782.1352786
  74. TADA: Making Node-link Diagrams Accessible to Blind and Low-Vision People. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24). ACM, New York, NY, USA, Article 45, 20 pages. https://doi.org/10.1145/3613904.3642222
  75. Rich Screen Reader Experiences for Accessible Data Visualization. Computer Graphics Forum 41, 3 (June 2022), 15–27. https://doi.org/10.1111/cgf.14519
  76. Umwelt: Accessible Structured Editing of Multi-Modal Data Representations. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24). ACM, New York, NY, USA, Article 46, 20 pages. https://doi.org/10.1145/3613904.3641996
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up