Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Robustness of Text-to-Visualization Translation against Lexical and Phrasal Variability (2404.07135v2)

Published 10 Apr 2024 in cs.CL and cs.AI

Abstract: Text-to-Vis is an emerging task in the NLP area that aims to automatically generate data visualizations from natural language questions (NLQs). Despite their progress, existing text-to-vis models often heavily rely on lexical matching between words in the questions and tokens in data schemas. This overreliance on lexical matching may lead to a diminished level of model robustness against input variations. In this study, we thoroughly examine the robustness of current text-to-vis models, an area that has not previously been explored. In particular, we construct the first robustness dataset nvBench-Rob, which contains diverse lexical and phrasal variations based on the original text-to-vis benchmark nvBench. Then, we found that the performance of existing text-to-vis models on this new dataset dramatically drops, implying that these methods exhibit inadequate robustness overall. Finally, we propose a novel framework based on Retrieval-Augmented Generation (RAG) technique, named GRED, specifically designed to address input perturbations in these two variants. The framework consists of three parts: NLQ-Retrieval Generator, Visualization Query-Retrieval Retuner and Annotation-based Debugger, which are used to tackle the challenges posed by natural language variants, programming style differences and data schema variants, respectively. Extensive experimental evaluations show that, compared to the state-of-the-art model RGVisNet in the Text-to-Vis field, GRED performs better in terms of model robustness, with a 32% increase in accuracy on the proposed nvBench-Rob dataset.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Palm 2 technical report. arXiv preprint arXiv:2305.10403.
  2. Skeleton-to-response: Dialogue generation guided by retrieval memory. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1219–1228, Minneapolis, Minnesota. Association for Computational Linguistics.
  3. Retrieval-guided dialogue response generation via a matching-to-generation framework. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1866–1875, Hong Kong, China. Association for Computational Linguistics.
  4. Can rationalization improve robustness? In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3792–3805, Seattle, United States. Association for Computational Linguistics.
  5. Improving the robustness of summarization systems with dual augmentation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6846–6857, Toronto, Canada. Association for Computational Linguistics.
  6. Chatlaw: Open-source legal large language model with integrated external knowledge bases. Preprint, arXiv:2306.16092.
  7. Text-to-viz: Automatic generation of infographics from proportion-related natural language statements. IEEE transactions on visualization and computer graphics, 26(1):906–916.
  8. Victor Dibia and Çağatay Demiralp. 2019. Data2vis: Automatic generation of data visualizations using sequence-to-sequence recurrent neural networks. IEEE computer graphics and applications, 39(5):33–46.
  9. Multispider: towards benchmarking multilingual text-to-sql semantic parsing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 12745–12753.
  10. Information visualization in data mining and knowledge discovery. Morgan Kaufmann.
  11. Language-agnostic bert sentence embedding. arXiv preprint arXiv:2007.01852.
  12. Automatic data visualization generation from chinese natural language questions. In The 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation.
  13. Virgilio Gómez-Rubio. 2017. ggplot2 - elegant graphics for data analysis (2nd edition). Journal of Statistical Software, 77:1–3.
  14. Textbooks are all you need. arXiv preprint arXiv:2306.11644.
  15. Pat Hanrahan. 2006. Vizql: a language for query, analysis and visualization. In Proceedings of the 2006 ACM SIGMOD international conference on Management of data, pages 721–721.
  16. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning (ICML).
  17. Pretrained transformers improve out-of-distribution robustness. In Association for Computational Linguistics (ACL).
  18. Visualization support for a user-centered kdd process. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 519–524.
  19. Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open domain question answering. In EACL 2021-16th Conference of the European Chapter of the Association for Computational Linguistics, pages 874–880. Association for Computational Linguistics.
  20. Tree of clarifications: Answering ambiguous questions with retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 996–1009, Singapore. Association for Computational Linguistics.
  21. Large language models with controllable working memory. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1774–1793, Toronto, Canada. Association for Computational Linguistics.
  22. Echarts: A declarative framework for rapid construction of web-based visualization. Vis. Informatics, 2:136–146.
  23. Adapt in contexts: Retrieval-augmented domain adaptation via in-context learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6525–6542, Singapore. Association for Computational Linguistics.
  24. Deepeye: Creating good data visualizations by keyword search. In Proceedings of the 2018 International Conference on Management of Data, pages 1733–1736.
  25. Synthesizing natural language to visualization (nl2vis) benchmarks from nl2sql benchmarks. In Proceedings of the 2021 International Conference on Management of Data, SIGMOD ’21, page 1235–1247, New York, NY, USA. Association for Computing Machinery.
  26. Natural language to visualization by neural machine translation. IEEE Transactions on Visualization and Computer Graphics, 28(1):217–226.
  27. Query rewriting in retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5303–5315, Singapore. Association for Computational Linguistics.
  28. Retrieval-augmented chain-of-thought in semi-structured domains. In Proceedings of the Natural Legal Language Processing Workshop 2023, pages 178–191, Singapore. Association for Computational Linguistics.
  29. Formalizing visualization design knowledge as constraints: Actionable and extensible models in draco. IEEE transactions on visualization and computer graphics, 25(1):438–448.
  30. Nl4dv: A toolkit for generating analytic specifications for data visualization from natural language queries. IEEE Transactions on Visualization and Computer Graphics, 27(2):369–379.
  31. OpenAI. 2024. Gpt-4 technical report. Preprint, arXiv:2303.08774.
  32. Check your facts and try again: Improving large language models with external knowledge and automated feedback. Preprint, arXiv:2302.12813.
  33. Goodtriever: Adaptive toxicity mitigation with retrieval-augmented models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5108–5125, Singapore. Association for Computational Linguistics.
  34. Learning to recommend visualizations from data. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1359–1369.
  35. Nils Reimers and Iryna Gurevych. 2020. Making monolingual sentence embeddings multilingual using knowledge distillation. arXiv preprint arXiv:2004.09813.
  36. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950.
  37. Vega-lite: A grammar of interactive graphics. IEEE Transactions on Visualization and Computer Graphics, 23:341–350.
  38. Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9248–9274, Singapore. Association for Computational Linguistics.
  39. Effortless data exploration with zenvisage: An expressive and interactive visual analytics system. Proc. VLDB Endow., 10:457–468.
  40. Rgvisnet: A hybrid retrieval-generation neural framework towards automatic data visualization generation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1646–1655.
  41. Sevi: Speech-to-visualization through neural machine translation. In Proceedings of the 2022 International Conference on Management of Data, pages 2353–2356.
  42. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  43. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10014–10037, Toronto, Canada. Association for Computational Linguistics.
  44. Towards visualization recommendation systems. Acm Sigmod Record, 45(4):34–39.
  45. Attention is all you need. Advances in neural information processing systems, 30.
  46. ReCode: Robustness evaluation of code generation models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13818–13843, Toronto, Canada. Association for Computational Linguistics.
  47. Measure and improve robustness in NLP models: A survey. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4569–4586, Seattle, United States. Association for Computational Linguistics.
  48. On the robustness of reading comprehension models to entity renaming. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 508–520, Seattle, United States. Association for Computational Linguistics.
  49. Retrieval-augmented few-shot text classification. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6721–6735, Singapore. Association for Computational Linguistics.
  50. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3911–3921, Brussels, Belgium. Association for Computational Linguistics.
  51. RobuT: A systematic study of table QA robustness against human-annotated adversarial perturbations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6064–6081, Toronto, Canada. Association for Computational Linguistics.
  52. Docprompting: Generating code by retrieving the docs. In The Eleventh International Conference on Learning Representations.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Jinwei Lu (5 papers)
  2. Yuanfeng Song (27 papers)
  3. Haodi Zhang (6 papers)
  4. Chen Zhang (404 papers)
  5. Raymond Chi-Wing Wong (29 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now