Metacognition-Enhanced Few-Shot Prompting With Positive Reinforcement (2312.08642v2)

Abstract: Few-shot prompting elicits the remarkable abilities of LLMs by equipping them with a few demonstration examples in the input. However, the traditional method of providing LLMs with all demonstration input-output pairs at once may not effectively guide LLMs to learn the specific input-output mapping relationship. In this paper, inspired by the regulatory and supportive role of metacognition in students' learning, we propose a novel metacognition-enhanced few-shot prompting, which guides LLMs to reflect on their thought processes to comprehensively learn the given demonstration examples. Furthermore, considering that positive reinforcement can improve students' learning motivation, we introduce positive reinforcement into our metacognition-enhanced few-shot prompting to promote the few-shot learning of LLMs by providing response-based positive feedback. The experimental results on two real-world datasets show that our metacognition-enhanced few-shot prompting with positive reinforcement surpasses traditional few-shot prompting in classification accuracy and macro F1.