Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars (2405.16122v2)

Published 25 May 2024 in cs.AI, cs.CL, cs.LG, and stat.ML

Abstract: LLMs have shown impressive capabilities in real-world applications. The capability of in-context learning (ICL) allows us to adapt an LLM to downstream tasks by including input-label exemplars in the prompt without model fine-tuning. However, the quality of these exemplars in the prompt greatly impacts performance, highlighting the need for an effective automated exemplar selection method. Recent studies have explored retrieval-based approaches to select exemplars tailored to individual test queries, which can be undesirable due to extra test-time computation and an increased risk of data exposure. Moreover, existing methods fail to adequately account for the impact of exemplar ordering on the performance. On the other hand, the impact of the instruction, another essential component in the prompt given to the LLM, is often overlooked in existing exemplar selection methods. To address these challenges, we propose a novel method named EASE, which leverages the hidden embedding from a pre-trained LLM to represent ordered sets of exemplars and uses a neural bandit algorithm to optimize the sets of exemplars while accounting for exemplar ordering. Our EASE can efficiently find an ordered set of exemplars that performs well for all test queries from a given task, thereby eliminating test-time computation. Importantly, EASE can be readily extended to jointly optimize both the exemplars and the instruction. Through extensive empirical evaluations (including novel tasks), we demonstrate the superiority of EASE over existing methods, and reveal practical insights about the impact of exemplar selection on ICL, which may be of independent interest. Our code is available at https://github.com/ZhaoxuanWu/EASE-Prompt-Optimization.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. A survey on data selection for language models. arXiv:2402.16827, 2024.
  2. Language models are few-shot learners. In Proc. NeurIPS, pages 1877–1901, 2020.
  3. Data curation alone can stabilize in-context learning. In Proc. ACL, pages 8123–8144, 2023.
  4. InstructZero: Efficient instruction optimization for black-box large language models. arXiv:2306.03082, 2023a.
  5. How is ChatGPT’s behavior changing over time? arXiv:2307.09009, 2023b.
  6. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv:2309.16797, 2023.
  7. Ambiguity-aware in-context learning with large language models. arXiv:2309.07900, 2024.
  8. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. In Proc. ICLR, 2024.
  9. Coverage-based example selection for in-context learning. In Proc. EMNLP, pages 13924–13950, 2023.
  10. Localized zeroth-order prompt optimization. arXiv:2403.02993, 2024.
  11. Diverse demonstrations improve in-context compositional generalization. In Proc. ACL, pages 1401–1422, 2023.
  12. Finding support examples for in-context learning. In Proc. EMNLP, pages 6219–6235, 2023.
  13. Unified demonstration retriever for in-context learning. In Proc. ACL, pages 4644–4668, 2023.
  14. Use your INSTINCT: Instruction optimization using neural bandits coupled with transformers. In NeurIPS Workshop on Instruction Tuning and Instruction Following, 2023.
  15. What makes good in-context examples for GPT-3? In Proc. DeeLIO: Deep Learning Inside Out, pages 100–114, 2022.
  16. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proc. ACL, pages 8086–8098, 2022.
  17. In-context learning with retrieved demonstrations for language models: A survey. arXiv:2401.11624, 2024.
  18. The AI index 2024 annual report. Technical report, AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, 2024.
  19. Rethinking the role of demonstrations: What makes in-context learning work? In Proc. EMNLP, pages 11048–11064, 2022.
  20. Large language models: A survey. arXiv:2402.06196, 2024.
  21. Few-shot fine-tuning vs. in-context learning: A fair comparison and evaluation. In Proc. ACL Findings, pages 12284–12314, 2023.
  22. Privacy issues in large language models: A survey. arXiv:2312.06717, 2024.
  23. In-context example selection with influences. arXiv:2302.11042, 2023.
  24. GPT-4 technical report. arXiv:2303.08774, 2024.
  25. True few-shot learning with language models. In Proc. NeurIPS, pages 11054–11070, 2021.
  26. Learning transferable visual models from natural language supervision. In Proc. ICML, pages 8748–8763, 2021.
  27. Sentence-BERT: Sentence embeddings using siamese bert-networks. In Proc. EMNLP, pages 3982–3992, 2019.
  28. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr., 3(4):333–389, 2009.
  29. Learning to retrieve prompts for in-context learning. In Proc. NAACL, pages 2655–2671, 2022.
  30. Equivalence of distance-based and RKHS-based statistics in hypothesis testing. Annals of Statistics, 41(5):2263–2291, 2013.
  31. MPNet: masked and permuted pre-training for language understanding. In Proc. NeurIPS, pages 16857–16867, 2020.
  32. Gemini: A family of highly capable multimodal models. arXiv:2312.11805, 2024.
  33. Cédric Villani. Topics in optimal transportation, volume 58. American Mathematical Soc., 2021.
  34. MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. In Proc. NeurIPS, pages 5776–5788, 2020.
  35. Large language models are latent variable models: Explaining and finding good demonstrations for in-context learning. In Proc. NeurIPS, pages 15614–15638, 2023.
  36. Jeremy White. How strangers got my email address from ChatGPT’s model. The New York Times, 2023. URL https://www.nytimes.com/interactive/2023/12/22/technology/openai-chatgpt-privacy-exploit.html.
  37. Large language models as optimizers. In Proc. ICLR, 2024.
  38. Compositional exemplars for in-context learning. In Proc. ICML, pages 39818–39833, 2023.
  39. Active example selection for in-context learning. In Proc. EMNLP, pages 9134–9148, 2022.
  40. A survey of large language models. arXiv:2303.18223, 2023.
  41. Calibrate before use: Improving few-shot performance of language models. In Proc. ICML, pages 12697–12706, 2021.
  42. Neural contextual bandits with UCB-based exploration. In Proc. ICML, pages 11492–11502, 2020.
  43. Large language models are human-level prompt engineers. In Proc. ICLR, 2023.
Citations (3)
View on Semantic Scholar

Summary

  • The paper introduces EASE, a novel approach employing neural bandits and optimal transport to optimize in-context exemplar selection and ordering.
  • It jointly optimizes exemplars and instructions, reducing test-time computation and enhancing LLM performance on diverse tasks.
  • Extensive experiments validate EASE's robustness on noisy data and its effectiveness in 17 out of 19 benchmark tasks.

Analyzing Efficient Ordering-Aware Automated Selection of Exemplars for Prompt Optimization

Introduction

The paper "Prompt Optimization with EASE: Efficient Ordering-aware Automated Selection of Exemplars" introduces a novel approach named EASE to optimize exemplar selection in prompts for LLMs without fine-tuning their parameters. The primary objective is to enhance in-context learning (ICL) performance by judiciously selecting and ordering in-context exemplars. The EASE methodology incorporates the hidden embeddings from pre-trained LLMs and leverages a neural bandit algorithm to optimize the selection and ordering of exemplars efficiently.

Context and Motivation

LLMs, such as GPT-4 and others, exhibit impressive capabilities across various tasks through ICL, where exemplar quality significantly impacts performance. Traditional methods, mostly retrieval-based, tailor exemplars for each test query but suffer from drawbacks such as increased test-time computation and potential data exposure risks. These methods also overlook the impact of exemplar ordering, crucial for optimal performance. Additionally, current studies inadequately address the interplay between exemplars and instructions within prompts. EASE addresses these gaps by offering a method to optimize a fixed, ordered set of exemplars for all test queries of a given task, eliminating the need for test-time computation and enhancing practical deployment.

Methodology

  1. Optimization Framework: Exemplar selection is formulated as a black-box optimization problem. The objective function quantifies ICL performance by scoring the output generated by the LLM for a given validation set.
  2. Neural Bandits: A neural network (NN) is trained to predict the ICL performance based on the embeddings of exemplar sequences. The NeuralUCB (neural upper confidence bound) algorithm is employed to balance exploration and exploitation, efficiently selecting the next query based on past observations.
  3. Optimal Transport (OT): To manage the large search space of exemplar permutations, OT is utilized to refine the sampled domain of exemplar sequences, retaining only relevant sequences, thus reducing computational cost while maintaining performance integrity.
  4. Joint Optimization: EASE extends to jointly optimize both exemplars and instructions in the prompt. This holistic optimization ensures better performance by simultaneously considering both components’ interaction effects on the ICL.

Experimental Results

  1. Benchmark Tasks: EASE outperformed existing methods on a variety of benchmark tasks, achieving the highest performance in 17 out of 19 tasks tested. The results validate the efficacy of considering exemplar ordering and fixed exemplar sets over traditional retrieval-based methods.
  2. Novel Rule-based Tasks: In proposed out-of-distribution tasks such as linear regression (LR), language puzzles (LP), and remapped label tasks (e.g., AG News Remap), EASE demonstrated substantial improvements, affirming its robustness in novel scenarios where LLMs have limited pre-learned knowledge.
  3. Noisy Data: EASE showed resilience in handling noisy datasets—common in real-world applications—by effectively filtering out irrelevant or misleading exemplars and thus improving ICL performance.
  4. Joint Optimization: Enhancing prompts by jointly optimizing exemplars and instructional components resulted in notable performance gains for several challenging tasks, underscoring the method's comprehensive applicability.

Implications and Future Directions

The practical implications of EASE are significant, potentially transforming how LLMs are utilized for ICL across diverse domains without fine-tuning. It offers a scalable solution to the computational and privacy challenges associated with traditional exemplar selection methods.

From a theoretical perspective, EASE’s incorporation of neural bandits and OT demonstrates a sophisticated interplay between reinforcement learning, optimal transport, and deep learning. This interdisciplinary approach suggests a promising direction for optimizing complex models and adapting them to varied tasks efficiently.

Future research might explore further extending EASE’s capabilities:

  1. Integration with other forms of context optimization (e.g., dynamic context adaptation based on real-time performance feedback).
  2. Scalability to larger exemplar sets: Addressing the bottleneck of embedding computation, potentially using more efficient embedding techniques or distributed computation paradigms.
  3. Fine-grained control over noise handling: Developing more sophisticated methods to identify and mitigate data noise dynamically during exemplar selection.

Conclusion

EASE introduces a robust, computationally efficient framework that significantly advances exemplar selection for ICL in LLMs. By addressing critical limitations of existing methods, particularly regarding exemplar ordering and joint exemplar-instruction optimization, EASE paves the way for enhanced performance in both common and novel real-world scenarios. The synergy between neural bandits and optimal transport in EASE highlights an innovative approach, potentially influencing future research trajectories in automated prompt optimization and beyond.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown