INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair (2311.09868v5)
Abstract: This paper introduces INTERVENOR (INTERactiVE chaiN Of Repair), a system designed to emulate the interactive code repair processes observed in humans, encompassing both code diagnosis and code repair. INTERVENOR prompts LLMs to play distinct roles during the code repair process, functioning as both a Code Learner and a Code Teacher. Specifically, the Code Learner is tasked with adhering to instructions to generate or repair code, while the Code Teacher is responsible for crafting a Chain-of-Repair (CoR) to serve as guidance for the Code Learner. During generating the CoR, the Code Teacher needs to check the generated codes from Code Learner and reassess how to address code bugs based on error feedback received from compilers. Experimental results demonstrate that INTERVENOR surpasses baseline models, exhibiting improvements of approximately 18% and 4.3% over GPT-3.5 in code generation and code translation tasks, respectively. Our further analyses show that CoR is effective to illuminate the reasons behind bugs and outline solution plans in natural language. With the feedback of code compilers, INTERVENOR can accurately identify syntax errors and assertion errors and provide precise instructions to repair codes. All data and codes are available at https://github.com/NEUIR/INTERVENOR
- Anthropic. 2023. Model card and evaluations for claude models.
- Program synthesis with large language models.
- Codet: Code generation with generated tests. In Proceedings of ICLR.
- Evaluating large language models trained on code.
- Teaching large language models to self-debug. ArXiv preprint.
- Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, pages 240:1–240:113.
- Self-collaboration code generation via chatgpt.
- Incoder: A generative model for code infilling and synthesis. In Proceedings of ICLR.
- Fixeval: Execution-based evaluation of program fixes for programming problems. In 2023 IEEE/ACM International Workshop on Automated Program Repair (APR), pages 11–18. IEEE.
- Codecot and beyond: Learning to program and test like a developer. ArXiv preprint.
- Review4repair: Code review aided automatic program repairing. Information and Software Technology, page 106765.
- Fault-aware neural code rankers. In Proceedings of NeurIPS.
- Challenges and applications of large language models. ArXiv preprint.
- Large language models are zero-shot reasoners. In Proceedings of NeurIPS, pages 22199–22213.
- Structured chain-of-thought prompting for code generation.
- Starcoder: may the source be with you! ArXiv preprint.
- Competition-level code generation with AlphaCode. Science, (6624):1092–1097.
- Encouraging divergent thinking in large language models through multi-agent debate. ArXiv preprint.
- Codexglue: A machine learning benchmark dataset for code understanding and generation. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1).
- Wizardcoder: Empowering code large language models with evol-instruct. ArXiv preprint.
- Self-refine: Iterative refinement with self-feedback.
- William J McGuire. 1960. Cognitive consistency and attitude change. The Journal of Abnormal and Social Psychology, (3):345.
- Octopack: Instruction tuning code large language models. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following.
- Lever: Learning to verify language-to-code generation with execution. In Proceedings of ICML, pages 26106–26128.
- Codegen: An open large language model for code with multi-turn program synthesis. In Proceedings of ICLR.
- Is self-repair a silver bullet for code generation?
- OpenAI. 2022. Chatgpt: Optimizing language models for dialogue.
- OpenAI. 2023. Gpt-4 technical report.
- Communicative agents for software development.
- Creator: Tool creation for disentangling abstract and concrete reasoning of large language models. In Proceedings of EMNLP Findings, pages 6922–6939.
- Code llama: Open foundation models for code. ArXiv preprint.
- Pangu-coder2: Boosting large language models for code with ranking feedback.
- Natural language to code translation with execution. In Proceedings of EMNLP, pages 3533–3546.
- Reflexion: Language agents with verbal reinforcement learning. In Proceedings of NeurIPS.
- Llama: Open and efficient foundation language models.
- Compilable neural code generation with compiler feedback. In Proceedings of ACL Findings, pages 9–19.
- Self-instruct: Aligning language model with self generated instructions. In Proceedings of ACL, pages 13484–13508.
- Codet5+: Open code large language models for code understanding and generation. pages 1069–1088.
- Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of NeurIPS, pages 24824–24837.
- Generating sequences by learning to self-correct. In Proceedings of ICLR.
- Lemur: Harmonizing natural language and code for language agents.
- Michihiro Yasunaga and Percy Liang. 2021. Break-it-fix-it: Unsupervised learning for program repair. In Proceedings of ICML, pages 11941–11952.
- Self-edit: Fault-aware code editor for code generation. In Proceedings of ACL, pages 769–787.
- Coder reviewer reranking for code generation. In Proceedings of ICML, pages 41832–41846.
- Codegeex: A pre-trained model for code generation with multilingual evaluations on humaneval-x.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.