Determine GPT-4o performance on complex handwritten math and visual statistics tasks

Determine how OpenAI’s GPT-4o responds to image-based inputs involving systems of hand-written equations, complicated multiple integrals, and identification of differences in medians from hand-drawn boxplots, and assess whether it produces correct and useful solutions for these tasks in the context of mathematics and statistics education.

Background

The paper demonstrates that ChatGPT4 substantially outperforms ChatGPT3.5 on statistics and data science exam questions, with the performance gap especially large when questions include images, since ChatGPT3.5 cannot process images. This motivates evaluating newer multimodal models for their image-understanding capabilities in educational contexts.

OpenAI announced GPT-4o with claims of GPT-4-level intelligence and the ability to read/upload images, and showcased a demonstration on a simple handwritten algebra problem. However, the authors explicitly note that it is unclear how GPT-4o handles more complex handwritten or visual tasks common in mathematics and statistics (e.g., systems of equations, complicated integrals, or interpreting hand-drawn boxplots). Clarifying GPT-4o’s performance on these tasks is important for understanding its utility and potential equity implications in education.