Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Agnostic Visual Recommendation Systems: Open Challenges and Future Directions (2302.00569v2)

Published 1 Feb 2023 in cs.LG

Abstract: Visualization Recommendation Systems (VRSs) are a novel and challenging field of study aiming to help generate insightful visualizations from data and support non-expert users in information discovery. Among the many contributions proposed in this area, some systems embrace the ambitious objective of imitating human analysts to identify relevant relationships in data and make appropriate design choices to represent these relationships with insightful charts. We denote these systems as "agnostic" VRSs since they do not rely on human-provided constraints and rules but try to learn the task autonomously. Despite the high application potential of agnostic VRSs, their progress is hindered by several obstacles, including the absence of standardized datasets to train recommendation algorithms, the difficulty of learning design rules, and defining quantitative criteria for evaluating the perceptual effectiveness of generated plots. This paper summarizes the literature on agnostic VRSs and outlines promising future research directions.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (92)
  1. B. Chin-Yee and R. Upshur, “Three problems with big data and artificial intelligence in medicine,” Perspectives in Biology and Medicine, vol. 62, no. 2, pp. 237–256, 2019.
  2. C. Rudin, C. Chen, Z. Chen, H. Huang, L. Semenova, and C. Zhong, “Interpretable machine learning: Fundamental principles and 10 grand challenges,” Statistic Surveys, vol. 16, pp. 1–85, 2022.
  3. Y. Zhang, P. Tiňo, A. Leonardis, and K. Tang, “A survey on neural network interpretability,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 5, no. 5, pp. 726–742, 2021.
  4. G. Peake and J. Wang, “Explanation mining: Post hoc interpretability of latent factor models for recommendation systems,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 2060–2069.
  5. Z. Chen, F. Silvestri, J. Wang, Y. Zhang, Z. Huang, H. Ahn, and G. Tolomei, “Grease: Generate factual and counterfactual explanations for gnn-based recommendations,” arXiv preprint arXiv:2208.04222, 2022.
  6. B. Saket, D. Moritz, H. Lin, V. Dibia, C. Demiralp, and J. Heer, “Beyond heuristics: Learning visualization design,” arXiv preprint arXiv:1807.06641, 2018.
  7. Q. Wang, Z. Chen, Y. Wang, and H. Qu, “A survey on ml4vis: Applying machinelearning advances to data visualization,” IEEE Transactions on Visualization and Computer Graphics, 2021.
  8. S. Zhu, G. Sun, Q. Jiang, M. Zha, and R. Liang, “A survey on automatic infographics and visualization recommendations,” Visual Informatics, vol. 4, no. 3, pp. 24–40, 2020.
  9. V. Graf, V. Graf-Drasch, V. Tiefenbeck, R. Weitzel, and G. Fridgen, “Supporting citizens’ political decision-making using information visualisation,” in Proceedings of the 28th European Conference on Information Systems (ECIS), 2020.
  10. A. Mottus, Kinshuk, S. Graf, and N.-S. Chen, “Use of dashboards and visualization techniques to support teacher decision making,” Ubiquitous learning environments and technologies, pp. 181–199, 2015.
  11. A. Shen-Hsieh and M. Schindl, “Data visualization for strategic decision making,” in Case studies of the CHI2002— AIGA Experience Design FORUM, 2002, pp. 1–17.
  12. M. Oghbaie, M. J. Pennock, and W. B. Rouse, “Understanding the efficacy of interactive visualization for decision making for complex systems,” in 2016 Annual IEEE Systems Conference (SysCon).   IEEE, 2016, pp. 1–6.
  13. B. Goertzel, “Artificial general intelligence: Concept, state of the art, and future prospects,” Journal of Artificial General Intelligence, vol. 0, 01 2014.
  14. A. Wu, Y. Wang, X. Shu, D. Moritz, W. Cui, H. Zhang, D. Zhang, and H. Qu, “Ai4vis: Survey on artificial intelligence approaches for data visualization,” IEEE Transactions on Visualization and Computer Graphics, 2021.
  15. M. E. Ellen, G. Léon, G. Bouchard, M. Ouimet, J. M. Grimshaw, and J. N. Lavis, “Barriers, facilitators and views about next steps to implementing supports for evidence-informed decision-making in health systems: a qualitative study,” Implementation Science, vol. 9, no. 1, pp. 1–12, 2014.
  16. C. Packer, J. J. McAuley, and A. Ramisa, “Visually-aware personalized recommendation using interpretable image representations,” CoRR, vol. abs/1806.09820, 2018. [Online]. Available: http://arxiv.org/abs/1806.09820
  17. F. Long, “Improved personalized recommendation algorithm based on context-aware in mobile computing environment,” Wirel. Commun. Mob. Comput., vol. 2020, pp. 8 857 576:1–8 857 576:10, 2020.
  18. A. Livne, E. S. Tov, A. Solomon, A. Elyasaf, B. Shapira, and L. Rokach, “Evolving context-aware recommender systems with users in mind,” Expert Syst. Appl., vol. 189, p. 116042, 2022.
  19. N. Torres, “A multimodal user-adaptive recommender system,” Electronics, vol. 12, no. 17, 2023.
  20. A. C. Valdez, M. Ziefle, K. Verbert, A. Felfernig, and A. Holzinger, “Recommender systems for health informatics: State-of-the-art and future perspectives,” in Machine Learning for Health Informatics - State-of-the-Art and Future Challenges, ser. Lecture Notes in Computer Science, A. Holzinger, Ed.   Springer, 2016, vol. 9605, pp. 391–414.
  21. M. Sharaf, E. E. Hemdan, A. El-Sayed, and N. A. El-Bahnasawy, “A survey on recommendation systems for financial services,” Multim. Tools Appl., vol. 81, no. 12, pp. 16 761–16 781, 2022.
  22. W. Liang, C. Huang, T. Tseng, and Z. Wang, “The effect of visualisation on user experience in recommender systems,” Inf. Res., vol. 26, no. 3, 2021.
  23. J. C. Rasmussen, K. Ehrlich, S. I. Ross, S. E. Kirk, D. M. Gruen, and J. F. Patterson, “Nimble cybersecurity incident management through visualization and defensible recommendations,” in 7th International Symposium on Visualization for Cyber Security, VizSec 2010, Ottawa, ON, Canada, September 14, 2010, J. Gerth, Ed.   ACM, 2010, pp. 102–113.
  24. M. F. Franco, B. Rodrigues, and B. Stiller, “MENTOR: the design and evaluation of a protection services recommender system,” in 15th International Conference on Network and Service Management, CNSM 2019, Halifax, NS, Canada, October 21-25, 2019, H. Lutfiyya, Y. Diao, A. N. Zincir-Heywood, R. Badonnel, and E. R. M. Madeira, Eds.   IEEE, 2019, pp. 1–7.
  25. M. Vartak, S. Huang, T. Siddiqui, S. Madden, and A. G. Parameswaran, “Towards visualization recommendation systems,” SIGMOD Rec., vol. 45, no. 4, pp. 34–39, 2016.
  26. A. O. Afolabi and P. J. Toivanen, “Integration of recommendation systems into connected health for effective management of chronic diseases,” IEEE Access, vol. 7, pp. 49 201–49 211, 2019.
  27. S. Zhang, L. Yao, A. Sun, and Y. Tay, “Deep learning based recommender system: A survey and new perspectives,” ACM computing surveys (CSUR), vol. 52, no. 1, pp. 1–38, 2019.
  28. S. C. S. Joyner, A. Riegelhuth, K. Garrity, Y.-S. Kim, and N. W. Kim, “Visualization accessibility in the wild: Challenges faced by visualization designers,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022, pp. 1–19.
  29. A. Suh, G. Appleby, E. W. Anderson, L. Finelli, R. Chang, and D. Cashman, “Are metrics enough? guidelines for communicating and visualizing predictive models to subject matter experts,” IEEE Transactions on Visualization and Computer Graphics, 2023.
  30. M. Vartak, S. Rahman, S. Madden, A. Parameswaran, and N. Polyzotis, “Seedb: Efficient data-driven visualization recommendations to support visual analytics,” in Proceedings of the VLDB Endowment International Conference on Very Large Data Bases, vol. 8, no. 13.   NIH Public Access, 2015, p. 2182.
  31. Ç. Demiralp, P. J. Haas, S. Parthasarathy, and T. Pedapati, “Foresight: Recommending visual insights,” Proc. VLDB Endow., vol. 10, no. 12, pp. 1937–1940, 2017.
  32. J. O. Spicer and C. G. Coleman, “Creating effective infographics and visual abstracts to disseminate research and facilitate medical education on social media,” Clinical Infectious Diseases, vol. 74, no. Supplement_3, pp. e14–e22, 2022.
  33. M. Evagorou, S. Erduran, and T. Mäntylä, “The role of visual representations in scientific practices: from conceptual understanding and knowledge generation to ‘seeing’ how science works,” International journal of Stem education, vol. 2, no. 1, pp. 1–13, 2015.
  34. V. T. Nguyen, K. Jung, and V. Gupta, “Examining data visualization pitfalls in scientific publications,” Visual Computing for Industry, Biomedicine, and Art, vol. 4, no. 1, pp. 1–15, 2021.
  35. W. Mengist, T. Soromessa, and G. Legese, “Method for conducting systematic literature review and meta-analysis for environmental science research,” MethodsX, vol. 7, p. 100777, 2020.
  36. S. Keele et al., “Guidelines for performing systematic literature reviews in software engineering,” 2007.
  37. Y. Luo, X. Qin, N. Tang, and G. Li, “Deepeye: Towards automatic data visualization,” in 2018 IEEE 34th international conference on data engineering (ICDE).   IEEE, 2018, pp. 101–112.
  38. K. Hu, M. A. Bakker, S. Li, T. Kraska, and C. Hidalgo, “Vizml: A machine learning approach to visualization recommendation,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 2019, pp. 1–12.
  39. V. Dibia and C. Demiralp, “Data2vis: Automatic generation of data visualizations using sequence-to-sequence recurrent neural networks,” IEEE computer graphics and applications, vol. 39, no. 5, pp. 33–46, 2019.
  40. A. Wu, Y. Wang, M. Zhou, X. He, H. Zhang, H. Qu, and D. Zhang, “Multivision: Designing analytical dashboards with deep learning based recommendation,” IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 1, pp. 162–172, 2021.
  41. V. Dibia, “LIDA: A tool for automatic generation of grammar-agnostic visualizations and infographics using large language models,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), D. Bollegala, R. Huang, and A. Ritter, Eds.   Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 113–126.
  42. L. Podo and P. Velardi, “Plotly. plus, an improved dataset for visualization recommendation,” in Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 2022, pp. 4384–4388.
  43. D. Moritz, C. Wang, G. L. Nelson, H. Lin, A. M. Smith, B. Howe, and J. Heer, “Formalizing visualization design knowledge as constraints: Actionable and extensible models in draco,” IEEE Transactions on Visualization and Computer Graphics, vol. 25, no. 1, pp. 438–448, 2018.
  44. M. A. Borkin, A. Vo, Z. Bylinskii, P. Isola, S. Sunkavalli, A. Oliva, and H. Pfister, “What makes a visualization memorable?” IEEE Transactions on Visualization and Computer Graphics, vol. 19, no. 12, pp. 2306–2315, 2013.
  45. S. Cole and E. Balcetis, “Motivated perception for self-regulation: How visual experience serves and is served by goals,” in Advances in Experimental Social Psychology.   Elsevier, 2021, vol. 64, pp. 129–186.
  46. K. Taylor and J. Rodriguez, “Visual discrimination,” in StatPearls [Internet].   StatPearls Publishing, 2022.
  47. P. Garrigan and P. J. Kellman, “Perceptual learning depends on perceptual constancy,” Proceedings of the National Academy of Sciences, vol. 105, no. 6, pp. 2248–2253, 2008.
  48. E. Bertini and G. Santucci, “Give chance a chance: modeling density to enhance scatter plot quality through random data sampling,” Information Visualization, vol. 5, no. 2, pp. 95–110, 2006.
  49. M. Behrisch, M. Blumenschein, N. W. Kim, L. Shao, M. El-Assady, J. Fuchs, D. Seebacher, A. Diehl, U. Brandes, H. Pfister et al., “Quality metrics for information visualization,” in Computer Graphics Forum, vol. 37.   Wiley Online Library, 2018, pp. 625–662.
  50. F. Hayes-Roth, “Rule-based systems,” Communications of the ACM, vol. 28, no. 9, pp. 921–932, 1985.
  51. J. Heer, N. Kong, and M. Agrawala, “Sizing the horizon: the effects of chart size and layering on the graphical perception of time series visualizations,” in Proceedings of the SIGCHI conference on human factors in computing systems, 2009, pp. 1303–1312.
  52. K. Reda, P. Nalawade, and K. Ansah-Koi, “Graphical perception of continuous quantitative maps: the effects of spatial frequency and colormap design,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 2018, pp. 1–12.
  53. H. Kennedy, R. L. Hill, W. Allen, and A. Kirk, “Engaging with (big) data visualizations: Factors that affect engagement and resulting new definitions of effectiveness,” First Monday, vol. 21, no. 11, 2016.
  54. R. J. Ziarani and R. Ravanmehr, “Serendipity in recommender systems: a systematic literature review,” Journal of Computer Science and Technology, vol. 36, no. 2, pp. 375–396, 2021.
  55. Z. Zeng and L. Battle, “A review and collation of graphical perception knowledge for visualization recommendation,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023, pp. 1–16.
  56. W. S. Cleveland and R. McGill, “Graphical perception: Theory, experimentation, and application to the development of graphical methods,” Journal of the American statistical association, vol. 79, no. 387, pp. 531–554, 1984.
  57. J. Mackinlay, “Automating the design of graphical presentations of relational information,” Acm Transactions On Graphics (Tog), vol. 5, no. 2, pp. 110–141, 1986.
  58. S. F. Roth, J. Kolojejchick, J. Mattis, and J. Goldstein, “Interactive graphic design using automatic presentation knowledge,” in Proceedings of the SIGCHI conference on Human factors in computing systems, 1994, pp. 112–117.
  59. S. M. Casner, “Task-analytic approach to the automated design of graphic presentations,” ACM Transactions on Graphics (ToG), vol. 10, no. 2, pp. 111–151, 1991.
  60. J. Mackinlay, P. Hanrahan, and C. Stolte, “Show me: Automatic presentation for visual analysis,” IEEE Transactions on Visualization and Computer Graphics, vol. 13, no. 6, pp. 1137–1144, 2007.
  61. L. Shen, E. Shen, Z. Tai, Y. Xu, J. Dong, and J. Wang, “Visual data analysis with task-based recommendations,” Data Science and Engineering, vol. 7, no. 4, pp. 354–369, 2022.
  62. A. Chakrabarti, F. Ahmad, and C. Quix, “Towards a rule-based visualization recommendation system.” in KDIR, 2021, pp. 57–68.
  63. K. Z. Hu, D. Orghian, and C. A. Hidalgo, “DIVE: A mixed-initiative system supporting integrated data exploration workflows,” in Proceedings of the Workshop on Human-In-the-Loop Data Analytics, HILDA@SIGMOD 2018, Houston, TX, USA, June 10, 2018, C. Binnig, J. Freire, and E. Wu, Eds.   ACM, 2018, pp. 5:1–5:7.
  64. F. Viégas, M. Wattenberg, D. Smilkov, J. Wexler, and D. Gundrum, “Generating charts from data in a data table,” US 20180088753 A, vol. 1, p. 2018, 2018.
  65. K. Wongsuphasawat, D. Moritz, A. Anand, J. Mackinlay, B. Howe, and J. Heer, “Voyager: Exploratory analysis via faceted browsing of visualization recommendations,” IEEE Transactions on Visualization and Computer Graphics, vol. 22, no. 1, pp. 649–658, 2015.
  66. K. Wongsuphasawat, Z. Qu, D. Moritz, R. Chang, F. Ouk, A. Anand, J. Mackinlay, B. Howe, and J. Heer, “Voyager 2: Augmenting visual analysis with partial view specifications,” in Proceedings of the 2017 chi conference on human factors in computing systems, 2017, pp. 2648–2659.
  67. B. Saket, A. Endert, and C. Demiralp, “Task-based effectiveness of basic visualizations,” IEEE Transactions on Visualization and Computer Graphics, vol. 25, no. 7, pp. 2505–2512, 2018.
  68. A. Satyanarayan, D. Moritz, K. Wongsuphasawat, and J. Heer, “Vega-lite: A grammar of interactive graphics,” IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 1, pp. 341–350, 2016.
  69. X. Qian, R. A. Rossi, F. Du, S. Kim, E. Koh, S. Malik, T. Y. Lee, and J. Chan, “Ml-based visualization recommendation: Learning to recommend visualizations from data,” arXiv preprint arXiv:2009.12316, 2020.
  70. H. Li, Y. Wang, S. Zhang, Y. Song, and H. Qu, “Kg4vis: A knowledge graph-based approach for visualization recommendation,” IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 1, pp. 195–205, 2021.
  71. A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, and O. Yakhnenko, “Translating embeddings for modeling multi-relational data,” Advances in neural information processing systems, vol. 26, 2013.
  72. M. Q. W. Baldonado, A. Woodruff, and A. Kuchinsky, “Guidelines for using multiple views in information visualization,” in Proceedings of the working conference on Advanced visual interfaces, AVI 2000, Palermo, Italy, May 23-26, 2000.   ACM Press, 2000, pp. 110–119.
  73. M. Zhou, Q. Li, X. He, Y. Li, Y. Liu, W. Ji, S. Han, Y. Chen, D. Jiang, and D. Zhang, “Table2charts: Recommending charts by learning shared table representations,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021, pp. 2389–2399.
  74. D. Deng, A. Wu, H. Qu, and Y. Wu, “Dashbot: Insight-driven dashboard generation based on deep reinforcement learning,” IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 1, pp. 690–700, 2022.
  75. G. Wang, B. Tan, Z. Wang, J. Wang, H. Guo, and Y. Wu, “A machine learning approach to visual insight discovery in multidimensional hierarchical data,” in 2022 5th International Conference on Pattern Recognition and Artificial Intelligence (PRAI).   IEEE, 2022, pp. 1304–1310.
  76. OpenAI, “Gpt-4 technical report,” arXiv, 2023.
  77. J. White, Q. Fu, S. Hays, M. Sandborn, C. Olea, H. Gilbert, A. Elnashar, J. Spencer-Smith, and D. C. Schmidt, “A prompt pattern catalog to enhance prompt engineering with chatgpt,” arXiv preprint arXiv:2302.11382, 2023.
  78. Z. Shen, T. Tao, L. Ma, W. Neiswanger, J. Hestness, N. Vassilieva, D. Soboleva, and E. Xing, “Slimpajama-dc: Understanding data combinations for llm training,” arXiv preprint arXiv:2309.10818, 2023.
  79. P. Maddigan and T. Susnjak, “Chat2vis: Generating data visualisations via natural language using chatgpt, codex and gpt-3 large language models,” arXiv preprint arXiv:2302.02094, 2023.
  80. C. J. Burges, “From ranknet to lambdarank to lambdamart: An overview,” Learning, vol. 11, no. 23-581, p. 81, 2010.
  81. M. B. Fazi, “Beyond human: Deep learning, explainability and representation,” Theory, Culture & Society, vol. 38, no. 7-8, pp. 55–77, 2021.
  82. C. Olah, A. Satyanarayan, I. Johnson, S. Carter, L. Schubert, K. Ye, and A. Mordvintsev, “The building blocks of interpretability,” Distill, vol. 3, no. 3, p. e10, 2018.
  83. K. Hu, N. Gaikwad, M. Bakker, M. Hulsebos, E. Zgraggen, C. Hidalgo, T. Kraska, G. Li, A. Satyanarayan, and Ç. Demiralp, “Viznet: Towards a large-scale visualization learning and benchmarking repository,” in Proceedings of the 2019 Conference on Human Factors in Computing Systems (CHI).   ACM, 2019.
  84. J. Bobadilla, F. Ortega, A. Hernando, and A. Gutiérrez, “Recommender systems survey,” Knowledge-based systems, vol. 46, pp. 109–132, 2013.
  85. Y. Luo, J. Tang, G. Li, and C. Chai, “Empowering natural language to visualization neural translation using synthesized benchmarks,” in VIS, 2021.
  86. E. Commission, “On artificial intelligence—a european approach to excellence and trust,” 2020.
  87. S. Dabir, S. Babakoohi, A. Kluge, J. J. Morrow, A. Kresak, M. Yang, D. MacPherson, G. Wildey, and A. Dowlati, “Ret mutation and expression in small-cell lung cancer,” Journal of Thoracic Oncology, vol. 9, no. 9, pp. 1316–1323, 2014.
  88. J. Donnelly, A. J. Barnett, and C. Chen., “Deformable protopnet: An interpretable image classifier using deformable prototypes,” Computer Vision and Pattern Recognition, 2021.
  89. Y. Chen, R. Li, A. Mac, T. Xie, T. Yu, and E. Wu, “Nl2interface: Interactive visualization interface generation from natural language queries,” arXiv preprint arXiv:2209.08834, 2022.
  90. Y. Chen and E. Wu, “Pi2: End-to-end interactive visualization interface generation from queries,” in Proceedings of the 2022 International Conference on Management of Data, 2022, pp. 1711–1725.
  91. A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with CLIP latents,” CoRR, vol. abs/2204.06125, 2022.
  92. A. Q. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “GLIDE: towards photorealistic image generation and editing with text-guided diffusion models,” in International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, ser. Proceedings of Machine Learning Research, vol. 162.   PMLR, 2022, pp. 16 784–16 804. [Online]. Available: https://proceedings.mlr.press/v162/nichol22a.html
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Luca Podo (5 papers)
  2. Bardh Prenkaj (16 papers)
  3. Paola Velardi (12 papers)
Citations (5)
View on Semantic Scholar