Prompt Engineering and Calibration for Zero-Shot Commonsense Reasoning

Abstract: Prompt engineering and calibration make LLMs excel at reasoning tasks, including multiple choice commonsense reasoning. From a practical perspective, we investigate and evaluate these strategies on smaller LLMs. Through experiments on five commonsense reasoning benchmarks, we find that each strategy favors certain models, but their joint effects are mostly negative.