Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios (2403.19318v2)

Published 28 Mar 2024 in cs.CL

Abstract: We introduce TableLLM, a robust LLM with 13 billion parameters, purpose-built for proficiently handling tabular data manipulation tasks, whether they are embedded within documents or spreadsheets, catering to real-world office scenarios. We propose a distant supervision method for training, which comprises a reasoning process extension strategy, aiding in training LLMs to understand reasoning patterns more effectively as well as a cross-way validation strategy, ensuring the quality of the automatically generated data. To evaluate the performance of TableLLM, we have crafted a benchmark tailored to address both document and spreadsheet formats as well as constructed a well-organized evaluation pipeline capable of handling both scenarios. Thorough evaluations underscore the advantages of TableLLM when compared to various existing general-purpose and tabular data-focused LLMs. We have publicly released the model checkpoint, source code, benchmarks, and a web application for user interaction.Our codes and data are publicly available at https://github.com/TableLLM/TableLLM.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Table-to-text generation and pre-training with tabt5. arXiv preprint arXiv:2210.09162 (2022).
  2. PaLM 2 Technical Report. CoRR abs/2305.10403 (2023). https://doi.org/10.48550/arXiv.2305.10403 arXiv:2305.10403
  3. Anonymous. 2024. Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=4L0xnS4GQM
  4. Transformers for Tabular Data Representation: A Survey of Models and Applications. TACL (2023).
  5. DeepSeek LLM: Scaling Open-Source Language Models with Longtermism. arXiv preprint arXiv:2401.02954 (2024).
  6. Open Question Answering over Tables and Text. Proceedings of ICLR 2021 (2021).
  7. Binding Language Models in Symbolic Languages. ICLR (2023).
  8. TURL: table understanding through representation learning. Proceedings of the VLDB Endowment 14, 3 (2020), 307–319.
  9. Thomas G Dietterich et al. 2002. Ensemble learning. The handbook of brain theory and neural networks 2, 1 (2002), 110–125.
  10. Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Tasks. In IJCAI.
  11. C3: Zero-shot Text-to-SQL with ChatGPT. CoRR abs/2307.07306 (2023). arXiv:2307.07306
  12. MATE: Multi-view Attention for Table Transformer Efficiency. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 7606–7619.
  13. TableGPT: Few-shot Table-to-Text Generation with Table Structure Reconstruction and Content Matching. In Proceedings of the 28th International Conference on Computational Linguistics, Donia Scott, Nuria Bel, and Chengqing Zong (Eds.). International Committee on Computational Linguistics, Barcelona, Spain (Online), 1978–1988. https://doi.org/10.18653/v1/2020.coling-main.179
  14. TaPas: Weakly Supervised Table Parsing via Pre-training. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.). Association for Computational Linguistics, Online, 4320–4333. https://doi.org/10.18653/v1/2020.acl-main.398
  15. ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory. arXiv preprint arXiv:2306.03901 (2023).
  16. InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks. arXiv:2401.05507 [cs.CL]
  17. Gittables: A large-scale corpus of relational tables. Proceedings of the ACM on Management of Data 1, 1 (2023), 1–17.
  18. Tabbie: Pretrained representations of tabular data. arXiv preprint arXiv:2105.02584 (2021).
  19. StructGPT: A General Framework for Large Language Model to Reason over Structured Data. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 9237–9251. https://doi.org/10.18653/v1/2023.emnlp-main.574
  20. Ziqi Jin and Wei Lu. 2023. Tab-CoT: Zero-shot Tabular Chain of Thought. arXiv preprint arXiv:2305.17812 (2023).
  21. CritiqueLLM: Scaling LLM-as-Critic for Effective and Explainable Evaluation of Large Language Model Generation. arXiv:2311.18702 [cs.CL]
  22. DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation. In Proceedings of the 40th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 202), Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.). PMLR, 18319–18345.
  23. SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Models. In Thirty-seventh Conference on Neural Information Processing Systems. https://openreview.net/forum?id=tfyr2zRVoK
  24. Table-GPT: Table-tuned GPT for Diverse Table Tasks. arXiv:2310.09263 [cs.CL]
  25. Holistic Evaluation of Language Models. arXiv:2211.09110 [cs.CL]
  26. TAPEX: Table Pre-training via Learning a Neural SQL Executor. In International Conference on Learning Representations. https://openreview.net/forum?id=O50443AsCP
  27. FeTaQA: Free-form Table Question Answering. Transactions of the Association for Computational Linguistics 10 (2022), 35–49.
  28. OpenAI. 2023. GPT-4 Technical Report. CoRR abs/2303.08774 (2023). https://doi.org/10.48550/arXiv.2303.08774 arXiv:2303.08774
  29. Training language models to follow instructions with human feedback. In NeurIPS. http://papers.nips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html
  30. Panupong Pasupat and Percy Liang. 2015. Compositional Semantic Parsing on Semi-Structured Tables. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Chengqing Zong and Michael Strube (Eds.). Association for Computational Linguistics, Beijing, China, 1470–1480. https://doi.org/10.3115/v1/P15-1142
  31. Mohammadreza Pourreza and Davood Rafiei. 2023. Din-sql: Decomposed in-context learning of text-to-sql with self-correction. arXiv preprint arXiv:2304.11015 (2023).
  32. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research 21, 140 (2020), 1–67. http://jmlr.org/papers/v21/20-074.html
  33. Code Llama: Open Foundation Models for Code. ArXiv abs/2308.12950 (2023). https://api.semanticscholar.org/CorpusID:261100919
  34. Llama 2: Open Foundation and Fine-Tuned Chat Models. CoRR abs/2307.09288 (2023). https://doi.org/10.48550/arXiv.2307.09288 arXiv:2307.09288
  35. Tuta: Tree-based transformers for generally structured table pre-training. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1780–1790.
  36. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In NeurIPS. http://papers.nips.cc/paper_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html
  37. UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 602–631. https://doi.org/10.18653/v1/2022.emnlp-main.39
  38. DB-GPT: Empowering Database Interactions with Private Large Language Models. arXiv preprint arXiv:2312.17449 (2023).
  39. TableFormer: Robust Transformer Modeling for Table-Text Encoding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, Dublin, Ireland, 528–537. https://doi.org/10.18653/v1/2022.acl-long.40
  40. Large Language Models Are Versatile Decomposers: Decomposing Evidence and Questions for Table-Based Reasoning. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (¡conf-loc¿, ¡city¿Taipei¡/city¿, ¡country¿Taiwan¡/country¿, ¡/conf-loc¿) (SIGIR ’23). Association for Computing Machinery, New York, NY, USA, 174–184. https://doi.org/10.1145/3539618.3591708
  41. TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.). Association for Computational Linguistics, Online, 8413–8426. https://doi.org/10.18653/v1/2020.acl-main.745
  42. Grappa: Grammar-augmented pre-training for table semantic parsing. arXiv preprint arXiv:2009.13845 (2020).
  43. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium.
  44. TableLlama: Towards Open Large Generalist Models for Tables. arXiv:2311.09206 [cs.CL]
  45. Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow. arXiv preprint arXiv:2306.07209 (2023).
  46. ReAcTable: Enhancing ReAct for Table Question Answering. arXiv:2310.00815 [cs.DB]
  47. Large Language Models are Complex Table Parsers. arXiv preprint arXiv:2312.11521 (2023).
  48. Localize, retrieve and fuse: A generalized framework for free-form question answering over tables. arXiv preprint arXiv:2309.11049 (2023).
  49. Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. CoRR abs/1709.00103 (2017).
  50. TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association for Computational Linguistics, Online, 3277–3287. https://doi.org/10.18653/v1/2021.acl-long.254
  51. TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data. arXiv preprint arXiv:2401.13223 (2024).
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (14)
  1. Xiaokang Zhang (42 papers)
  2. Jing Zhang (730 papers)
  3. Zeyao Ma (4 papers)
  4. Yang Li (1140 papers)
  5. Bohan Zhang (31 papers)
  6. Guanlin Li (31 papers)
  7. Zijun Yao (50 papers)
  8. Kangli Xu (3 papers)
  9. Jinchang Zhou (3 papers)
  10. Daniel Zhang-Li (10 papers)
  11. Jifan Yu (49 papers)
  12. Shu Zhao (31 papers)
  13. Juanzi Li (144 papers)
  14. Jie Tang (302 papers)
Citations (14)
View on Semantic Scholar