Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning to Reduce: Optimal Representations of Structured Data in Prompting Large Language Models (2402.14195v1)

Published 22 Feb 2024 in cs.CL

Abstract: LLMs have been widely used as general-purpose AI agents showing comparable performance on many downstream tasks. However, existing work shows that it is challenging for LLMs to integrate structured data (e.g. KG, tables, DBs) into their prompts; LLMs need to either understand long text data or select the most relevant evidence prior to inference, and both approaches are not trivial. In this paper, we propose a framework, Learning to Reduce, that fine-tunes a LLM to generate a reduced version of an input context, given a task description and context input. The model learns to reduce the input context using On-Policy Reinforcement Learning and aims to improve the reasoning performance of a fixed LLM. Experimental results illustrate that our model not only achieves comparable accuracies in selecting the relevant evidence from an input context, but also shows generalizability on different datasets. We further show that our model helps improve the LLM's performance on downstream tasks especially when the context is long.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (23)
  1. HybridQA: A dataset of multi-hop question answering over tabular and textual data. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1026–1036, Online. Association for Computational Linguistics.
  2. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416.
  3. RLPrompt: Optimizing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3369–3391, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  4. Tabllm: Few-shot classification of tabular data with large language models. In International Conference on Artificial Intelligence and Statistics, pages 5549–5581. PMLR.
  5. Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645.
  6. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213.
  7. Guiding large language models via directional stimulus prompting. arXiv preprint arXiv:2302.11520.
  8. Lost in the middle: How language models use long contexts. arXiv preprint arXiv:2307.03172.
  9. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  10. OpenAI. 2023a. Chatgpt. https://chat.openai.com/chat.
  11. OpenAI. 2023b. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  12. Panupong Pasupat and Percy Liang. 2015. Compositional semantic parsing on semi-structured tables. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1470–1480, Beijing, China. Association for Computational Linguistics.
  13. GrIPS: Gradient-free, edit-based instruction search for prompting large language models. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3845–3864, Dubrovnik, Croatia. Association for Computational Linguistics.
  14. Is reinforcement learning (not) for natural language processing?: Benchmarks, baselines, and building blocks for natural language policy optimization. arXiv preprint arXiv:2210.01241.
  15. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
  16. On the potential of lexico-logical alignments for semantic parsing to SQL queries. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1849–1864, Online. Association for Computational Linguistics.
  17. Make a choice! knowledge base question answering with in-context learning. arXiv preprint arXiv:2305.13972.
  18. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  19. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  20. QA-GNN: Reasoning with language models and knowledge graphs for question answering. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 535–546, Online. Association for Computational Linguistics.
  21. Large language models are versatile decomposers: Decompose evidence and questions for table-based reasoning. arXiv preprint arXiv:2301.13808.
  22. Generate rather than retrieve: Large language models are strong context generators. arXiv preprint arXiv:2209.10063.
  23. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Younghun Lee (6 papers)
  2. Sungchul Kim (65 papers)
  3. Tong Yu (119 papers)
  4. Ryan A. Rossi (124 papers)
  5. Xiang Chen (343 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets