LLMs Struggle with Code Generation when Function Names Swapped (arxiv.org)

Summary:

• Large Language Models (LLMs) fail to generate correct Python code when default function names are changed.
• As model size increases, some LLMs become more confident in incorrect predictions, a phenomenon called Inverse Scaling.

Tags: