Impact of prompt reading level on ChatGPT response accuracy

Ascertain how the reading level of the user’s prompt influences the accuracy of responses generated by ChatGPT (GPT3.5, GPT4, and GPT4o-mini) to statistics questions, and quantify the relationship between prompt readability and correctness.

Background

The paper measured readability (Flesch–Kincaid and SMOG) of AI-generated answers and illustrated that prompting at different reading levels changes the readability of the output. However, the authors note that the effect of prompt reading level on answer accuracy has not been quantified in this work.

They explicitly identify this as a topic for future investigation, linking it to practical classroom usage where students may request simpler explanations that could affect correctness.