Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SQLformer: Deep Auto-Regressive Query Graph Generation for Text-to-SQL Translation (2310.18376v4)

Published 27 Oct 2023 in cs.CL and cs.LG

Abstract: In recent years, the task of text-to-SQL translation, which converts natural language questions into executable SQL queries, has gained significant attention for its potential to democratize data access. Despite its promise, challenges such as adapting to unseen databases and aligning natural language with SQL syntax have hindered widespread adoption. To overcome these issues, we introduce SQLformer, a novel Transformer architecture specifically crafted to perform text-to-SQL translation tasks. Our model predicts SQL queries as abstract syntax trees (ASTs) in an autoregressive way, incorporating structural inductive bias in the encoder and decoder layers. This bias, guided by database table and column selection, aids the decoder in generating SQL query ASTs represented as graphs in a Breadth-First Search canonical order. Our experiments demonstrate that SQLformer achieves state-of-the-art performance across six prominent text-to-SQL benchmarks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. Translating synthetic natural language to database queries with a polyglot deep learning framework. Scientific Reports, 11(1).
  2. Global Reasoning over Database Structures for Text-to-SQL Parsing. ArXiv:1908.11214 [cs].
  3. An Encoder-Decoder Framework Translating Natural Language to Database Queries. ArXiv:1711.06061 [cs].
  4. SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQL. ArXiv:2111.00653 [cs].
  5. Yitao Cai and Xiaojun Wan. 2020. IGSQL: Database schema interaction graph based neural model for context-dependent text-to-SQL generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6903–6912, Online. Association for Computational Linguistics.
  6. Star: Sql guided pre-training for context-dependent text-to-sql parsing.
  7. LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations. ArXiv:2106.01093 [cs].
  8. RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases. ArXiv:2004.03125 [cs].
  9. ELECTRA: Pre-training text encoders as discriminators rather than generators. In ICLR.
  10. Expanding the Scope of the ATIS Task: The ATIS-3 Corpus. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.
  11. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. ArXiv:1810.04805 [cs].
  12. Language to Logical Form with Neural Attention. ArXiv:1601.01280 [cs].
  13. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ArXiv:2010.11929 [cs].
  14. Matthias Fey and Jan Eric Lenssen. 2019. Fast Graph Representation Learning with PyTorch Geometric. ArXiv:1903.02428 [cs, stat].
  15. Towards robustness of text-to-sql models against synonym substitution.
  16. Exploring underexplored limitations of cross-domain text-to-sql generalization.
  17. Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation. ArXiv:1905.08205 [cs].
  18. X-SQL: reinforce schema representation with context. ArXiv:1908.08113 [cs].
  19. The ATIS Spoken Language Systems Pilot Corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990.
  20. Dynamic hybrid relation network for cross-domain context-dependent semantic parsing.
  21. S$^2$SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers. ArXiv:2203.06958 [cs].
  22. A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization. ArXiv:1902.01069 [cs].
  23. Graphix-t5: Mixing pre-trained transformers with graph-aware layers for text-to-sql parsing. arXiv preprint arXiv:2301.07507.
  24. RoBERTa: A Robustly Optimized BERT Pretraining Approach. ArXiv:1907.11692 [cs].
  25. PyTorch: An Imperative Style, High-Performance Deep Learning Library. ArXiv:1912.01703 [cs, stat].
  26. RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL. ArXiv:2205.06983 [cs].
  27. Stanza: A Python Natural Language Processing Toolkit for Many Human Languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 101–108, Online. Association for Computational Linguistics.
  28. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. ArXiv:1910.10683 [cs, stat].
  29. Ohad Rubin and Jonathan Berant. 2021. SmBoP: Semi-autoregressive bottom-up semantic parsing. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 311–324, Online. Association for Computational Linguistics.
  30. PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models. ArXiv:2109.05093 [cs].
  31. Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? ArXiv:2010.12725 [cs].
  32. Learning contextual representations for semantic parsing with generation-augmented pre-training. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15):13806–13814.
  33. Exploring Unexplored Generalization Challenges for Cross-Database Semantic Parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8372–8388, Online. Association for Computational Linguistics.
  34. Attention Is All You Need. ArXiv:1706.03762 [cs].
  35. Graph Attention Networks. ArXiv:1710.10903 [cs, stat].
  36. RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers. ArXiv:1911.04942 [cs].
  37. Relational Graph Attention Network for Aspect-based Sentiment Analysis. ArXiv:2004.12362 [cs].
  38. Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing. ArXiv:2206.14017 [cs].
  39. Transformers: State-of-the-Art Natural Language Processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
  40. Optimizing deeper transformers on small datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2089–2102, Online. Association for Computational Linguistics.
  41. SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning. ArXiv:1711.04436 [cs].
  42. SQLizer: query synthesis from natural language. Proceedings of the ACM on Programming Languages, 1(OOPSLA):63:1–63:26.
  43. Pengcheng Yin and Graham Neubig. 2017a. A Syntactic Neural Model for General-Purpose Code Generation. ArXiv:1704.01696 [cs].
  44. Pengcheng Yin and Graham Neubig. 2017b. A syntactic neural model for general-purpose code generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 440–450, Vancouver, Canada. Association for Computational Linguistics.
  45. TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data. ArXiv:2005.08314 [cs].
  46. Do Transformers Really Perform Bad for Graph Representation? ArXiv:2106.05234 [cs].
  47. GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models. ArXiv:1802.08773 [cs].
  48. TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generation. ArXiv:1804.09769 [cs].
  49. GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing. ArXiv:2009.13845 [cs].
  50. Cosql: A conversational text-to-sql challenge towards cross-domain natural language interfaces to databases.
  51. {SC}ore: Pre-training for context representation in conversational semantic parsing. In International Conference on Learning Representations.
  52. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. ArXiv:1809.08887 [cs].
  53. Sparc: Cross-domain semantic parsing in context.
  54. John M. Zelle and Raymond J. Mooney. 1996. Learning to parse database queries using inductive logic programming. In Proceedings of the thirteenth national conference on Artificial intelligence - Volume 2, AAAI’96, pages 1050–1055, Portland, Oregon. AAAI Press.
  55. Luke S. Zettlemoyer and Michael Collins. 2012. Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars. ArXiv:1207.1420 [cs].
  56. Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions. ArXiv:1909.00786 [cs].
  57. Hie-sql: History information enhanced network for context-dependent text-to-sql semantic parsing.
  58. Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. ArXiv:1709.00103 [cs].
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Adrián Bazaga (10 papers)
  2. Pietro Liò (270 papers)
  3. Gos Micklem (7 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now