Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Localized Zeroth-Order Prompt Optimization (2403.02993v1)

Published 5 Mar 2024 in cs.AI

Abstract: The efficacy of LLMs in understanding and generating natural language has aroused a wide interest in developing prompt-based methods to harness the power of black-box LLMs. Existing methodologies usually prioritize a global optimization for finding the global optimum, which however will perform poorly in certain tasks. This thus motivates us to re-think the necessity of finding a global optimum in prompt optimization. To answer this, we conduct a thorough empirical study on prompt optimization and draw two major insights. Contrasting with the rarity of global optimum, local optima are usually prevalent and well-performed, which can be more worthwhile for efficient prompt optimization (Insight I). The choice of the input domain, covering both the generation and the representation of prompts, affects the identification of well-performing local optima (Insight II). Inspired by these insights, we propose a novel algorithm, namely localized zeroth-order prompt optimization (ZOPO), which incorporates a Neural Tangent Kernel-based derived Gaussian process into standard zeroth-order optimization for an efficient search of well-performing local optima in prompt optimization. Remarkably, ZOPO outperforms existing baselines in terms of both the optimization performance and the query efficiency, which we demonstrate through extensive experiments.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. On exact computation with an infinitely wide neural net. In NeurIPS, pp.  8139–8148, 2019.
  2. InstructZero: Efficient instruction optimization for black-box large language models. arXiv preprint arXiv:2306.03082, 2023.
  3. RLPrompt: Optimizing discrete text prompts with reinforcement learning. In Proc. EMNLP, pp.  3369–3391, 2022.
  4. Black-box prompt learning for pre-trained language models. Transactions on Machine Learning Research, 2023.
  5. Benchmarking optimization software with performance profiles. Mathematical programming, 91:201–213, 2002.
  6. Online convex optimization in the bandit setting: Gradient descent without a gradient. In Proc. SODA, 2005.
  7. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. In ICLR, 2024.
  8. Instruction induction: From few examples to natural language task descriptions. In Proc. ACL, pp.  1935–1952, 2023.
  9. Neural Tangent Kernel: Convergence and generalization in neural networks. In Proc. NeurIPS, pp.  8580–8589, 2018.
  10. Large language models are zero-shot reasoners. In Proc. NeurIPS, volume 35, pp.  22199–22213, 2022.
  11. Universal approximation theorems for differentiable geometric deep learning. The Journal of Machine Learning Research, 23(1):8896–8968, 2022.
  12. Wide neural networks of any depth evolve as linear models under gradient descent. In Proc. NeurIPS, pp.  8572–8583, 2019.
  13. Prefix-tuning: Optimizing continuous prompts for generation. Proc. ACL, pp.  4582–4597, 2021.
  14. Use Your INSTINCT: INSTruction optimization usIng Neural bandits Coupled with Transformers. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following, 2023.
  15. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1–35, 2023.
  16. Reframing instructional prompts to gptk’s language. ACL Findings, pp.  589–612, 2021.
  17. High-dimensional bayesian optimization using low-dimensional feature spaces. Machine Learning, 109:1925–1943, 2020.
  18. Random gradient-free minimization of convex functions. Found. Comput. Math., 17(2):527–566, 2017.
  19. OpenAI. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
  20. OpenAI. ChatGPT. https://openai.com/blog/chatgpt, 2024a.
  21. OpenAI. Documentation of OpenAI’s text embeddings. https://platform.openai.com/docs/guides/embeddings, 2024b.
  22. Training language models to follow instructions with human feedback. Proc. NeurIPS, pp.  27730–27744, 2022a.
  23. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155, 2022b.
  24. Sentence-BERT: Sentence embeddings using siamese bert-networks. In Proc. EMNLP-IJCNLP, pp.  3982–3992, 2019.
  25. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp.  1–7, 2021.
  26. Optimal approximation rate of relu networks in terms of width and depth. Journal de Mathématiques Pures et Appliquées, 157:101–135, 2022.
  27. NASI: Label- and data-agnostic neural architecture search at initialization. In Proc. ICLR, 2022a.
  28. Unifying and boosting gradient-based training-free neural architecture search. In Proc. NeurIPS, pp.  33001–33015, 2022b.
  29. Zeroth-order optimization with trajectory-informed derivative estimation. In Proc. ICLR, 2023a.
  30. Federated zeroth-order optimization using trajectory-informed surrogate gradients. arXiv preprint arXiv:2308.04077, 2023b.
  31. Bbtv2: Pure black-box optimization can be comparable to gradient descent for few-shot learning. arXiv preprint arXiv:2205.11200, 2022a.
  32. Black-box tuning for language-model-as-a-service. In International Conference on Machine Learning, pp.  20841–20855. PMLR, 2022b.
  33. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023.
  34. TEMPERA: Test-time prompt editing via reinforcement learning. In Proc. ICLR, 2023.
  35. Large language models are human-level prompt engineers. In ICLR, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Wenyang Hu (9 papers)
  2. Yao Shu (29 papers)
  3. Zongmin Yu (1 paper)
  4. Zhaoxuan Wu (15 papers)
  5. Xiangqiang Lin (1 paper)
  6. Zhongxiang Dai (39 papers)
  7. See-Kiong Ng (103 papers)
  8. Bryan Kian Hsiang Low (77 papers)
Citations (4)
View on Semantic Scholar