Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
123 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
3 tokens/sec
DeepSeek R1 via Azure Pro
51 tokens/sec
2000 character limit reached

Iterative Experience Refinement of Software-Developing Agents (2405.04219v1)

Published 7 May 2024 in cs.CL, cs.AI, cs.MA, and cs.SE

Abstract: Autonomous agents powered by LLMs show significant potential for achieving high autonomy in various scenarios such as software development. Recent research has shown that LLM agents can leverage past experiences to reduce errors and enhance efficiency. However, the static experience paradigm, reliant on a fixed collection of past experiences acquired heuristically, lacks iterative refinement and thus hampers agents' adaptability. In this paper, we introduce the Iterative Experience Refinement framework, enabling LLM agents to refine experiences iteratively during task execution. We propose two fundamental patterns: the successive pattern, refining based on nearest experiences within a task batch, and the cumulative pattern, acquiring experiences across all previous task batches. Augmented with our heuristic experience elimination, the method prioritizes high-quality and frequently-used experiences, effectively managing the experience space and enhancing efficiency. Extensive experiments show that while the successive pattern may yield superior results, the cumulative pattern provides more stable performance. Moreover, experience elimination facilitates achieving better performance using just 11.54% of a high-quality subset.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Toward an Assessment of Software Development Risk. In Journal of Management Information Systems, volume 10, pages 203–225.
  2. Large Language Models in Machine Translation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 858–867.
  3. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 1877–1901.
  4. Sparks of Artificial General Intelligence: Early Experiments with GPT-4. In arXiv preprint arXiv:2303.12712.
  5. Large Language Models as Tool Makers. In The Twelfth International Conference on Learning Representations (ICLR).
  6. Chateval: Towards Better LLM-based Evaluators through Multi-agent Debate. In The Twelfth International Conference on Learning Representations (ICLR).
  7. GameGPT: Multi-agent Collaborative Framework for Game Development. In arXiv preprint arXiv:2310.08067.
  8. Evaluating Large Language Models Trained on Code. In arXiv preprint arXiv:2107.03374.
  9. Agentverse: Facilitating Multi-agent Collaboration and Exploring Emergent Behaviors in Agents. In The Twelfth International Conference on Learning Representations (ICLR).
  10. LM vs LM: Detecting Factual Errors via Cross Examination. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 12621–12640.
  11. Katherine Compton and Scott Hauck. 2002. Reconfigurable Computing: a Survey of Systems and Software. In ACM Computing Surveys (csuR), volume 34, pages 171–210.
  12. DesignGPT: Multi-Agent Collaboration in Design. In arXiv preprint arXiv:2311.11591.
  13. MindAgent: Emergent Gaming Interaction. In arXiv preprint arXiv:2309.09971.
  14. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. In The Twelfth International Conference on Learning Representations (ICLR).
  15. War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars. In arXiv preprint arXiv:2311.17227.
  16. Scaling Laws for Neural Language Models. In arXiv preprint arXiv:2001.08361.
  17. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 9459–9474.
  18. CAMEL: Communicative Agents for ”Mind” Exploration of Large Language Model Society. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS).
  19. Metaagents: Simulating Interactions of Human Behaviors for LLM-based Task-oriented Coordination via Collaborative Generative Agents. In arXiv preprint arXiv:2310.06500.
  20. BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents. In arXiv preprint arXiv:2308.05960.
  21. LASER: LLM agent with state-space exploration for web navigation. In NeurIPS 2023 Foundation Models for Decision Making Workshop.
  22. Harlan D Mills. 1976. Software development. In IEEE Transactions on Software Engineering, 4, pages 265–273.
  23. Anton Osika. 2023. GPT-Engineer. In https://github.com/AntonOsika/gpt-engineer.
  24. Training Language Models to Follow Instructions with Human Feedback. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 27730–27744. Curran Associates, Inc.
  25. Generative Agents: Interactive Simulacra of Human Behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST), pages 1–22.
  26. Communicative Agents for Software Development. In arXiv preprint arXiv:2307.07924.
  27. Experiential co-learning of software-developing agents. In arXiv preprint arXiv:2312.17025.
  28. Toolllm: Facilitating Large Language Models to Master 16000+ Real-World APIs. In The Twelfth International Conference on Learning Representations (ICLR).
  29. Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting. In arXiv preprint arXiv:2306.17563.
  30. Language Models are Unsupervised Multitask Learners. In OpenAI blog, volume 1, page 9.
  31. Toran Bruce Richards. 2023. AutoGPT. In https://github.com/Significant-Gravitas/AutoGPT.
  32. TPTU: Task Planning and Tool Usage of Large Language Model-based AI Agents. In NeurIPS 2023 Foundation Models for Decision Making Workshop.
  33. Toolformer: Language Models Can Teach Themselves to Use Tools. In arXiv preprint arXiv:2302.04761.
  34. Role Play with Large Language Models. In Nature, volume 623, pages 493–498.
  35. The curse of recursion: Training on generated data makes models forget. In arXiv preprint arXiv:2305.17493.
  36. Cognitive Architectures for Language Agents. In arXiv preprint arXiv:2309.02427.
  37. Llama: Open and Efficient Foundation Language Models. In arXiv preprint arXiv:2302.13971.
  38. Attention is All You Need. In Advances in Neural Information Processing Systems (NeurIPS), volume 30.
  39. Voyager: An Open-Ended Embodied Agent with Large Language Models. In Intrinsically-Motivated and Open-Ended Learning Workshop @NeurIPS2023.
  40. A survey on large language model based autonomous agents. In arXiv preprint arXiv:2308.11432, volume abs/2308.11432.
  41. When Large Language Model based Agent Meets User Behavior Analysis: A Novel User Simulation Paradigm. In arXiv preprint arXiv:2306.02552.
  42. Avalon’s Game of Thoughts: Battle Against Deception through Recursive Contemplation. In arXiv preprint arXiv:2310.01320.
  43. PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization. In The Twelfth International Conference on Learning Representations (ICLR).
  44. Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pages 13484–13508.
  45. Humanoid Agents: Platform for Simulating Human-like Generative Agents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP), pages 167–176.
  46. Emergent Abilities of Large Language Models. In Transactions on Machine Learning Research.
  47. Chain-of-thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 24824–24837.
  48. Lilian Weng. 2023. LLM-powered Autonomous Agents. In lilianweng.github.io.
  49. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework. In arXiv preprint arXiv:2308.08155.
  50. Large Language Models as Optimizers. In The Twelfth International Conference on Learning Representations (ICLR).
  51. GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS).
  52. On Generative Agents in Recommendation. In arXiv preprint arXiv:2310.10108.
  53. ExpeL: LLM Agents Are Experiential Learners. In arXiv preprint arXiv:2308.10144.
  54. Memorybank: Enhancing large language models with long-term memory. In arXiv preprint arXiv:2305.10250.
  55. Webarena: A realistic Web Environment for Building Autonomous Agents. In arXiv preprint arXiv:2307.13854.
  56. Agents: An Open-source Framework for Autonomous Language Agents. In arXiv preprint arXiv:2309.07870.
  57. Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory. In arXiv preprint arXiv:2305.17144.
Citations (10)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now