A Tutorial on Teaching Data Analytics with Generative AI (2411.07244v1)

Abstract: This tutorial addresses the challenge of incorporating LLMs, such as ChatGPT, in a data analytics class. It details several new in-class and out-of-class teaching techniques enabled by AI. For example, instructors can parallelize instruction by having students interact with different custom-made GPTs to learn different parts of an analysis and then teach each other what they learned from their AIs. For another example, instructors can turn problem sets into AI tutoring sessions, whereby a custom-made GPT guides a student through the problems, and the student uploads the chatlog for their homework submission. For a third example, you can assign different labs to each section of your class and have each section create AI assistants to help the other sections work through their labs. This tutorial advocates the programming in the English paradigm, in which students express the desired data transformations in prose and then use AI to generate the corresponding code. Students can wrangle data more effectively by programming in English than by manipulating in Excel. However, some students will program in English better than others, so you will still derive a robust grade distribution (at least with current LLMs).

• The paper demonstrates that integrating ChatGPT in analytics teaching increases productivity, evidenced by a tenfold improvement using R compared to Excel.
• The paper outlines innovative methods such as programming in English and turning traditional homework into interactive tutoring sessions.
• The paper advocates for incorporating LLMs in curricula to foster computational thinking, enhance engagement, and modernize assessment approaches.

A Tutorial on Teaching Data Analytics with Generative AI

The paper "A Tutorial on Teaching Data Analytics with Generative AI" by Robert L. Bray explores the integration of LLMs, specifically ChatGPT, into analytics education. The author addresses both the challenges and opportunities posed by these models, particularly in teaching data science to MBA students.

The transformative role of LLMs in educational settings is a central theme of the paper. Bray demonstrates that LLMs, such as ChatGPT, can dramatically enhance learning environments by fostering new teaching methods and facilitating a shift from traditional instruction techniques. The paper provides various examples of how instructors can leverage these models, like turning homework into tutoring sessions or using AI to create interactive learning experiences. Moreover, the author contends that programming in English (PIE) is a viable approach when working with LLMs, highlighting its efficacy in helping students articulate data transformations in natural language rather than through conventional software like Excel.

One significant insight from the paper is that students can be more productive when programming in English compared to using spreadsheets. Bray's in-class experiment, which compared the proficiency of students using R with ChatGPT against those using Excel with ChatGPT, yielded striking results. The students using R solved ten times as many questions as those using Excel. Such findings assert that code-based data manipulation not only exploits the full potential of LLMs but also changes the dynamic of class exercises, encouraging deeper computational thinking skills.

Additionally, the author presents a "10000% ChatGPT enabled" approach to classroom assessment without collapsing grade distributions. This approach involved permitting LLM use in quizzes and recognizing that while LLMs can enhance accuracy, they do not lead to perfect scores due to model limitations and students' over-reliance on such AI tools. In fact, students reported a sense of brittleness in their understanding when relying heavily on AI, which was corroborated by an observed reduction in paper time and effort.

The paper also argues for the necessity of continuous adaptation in teaching to include LLMs, suggesting AI retrofitting of curricula as a strategic response to the evolving educational landscape. Instructors are advised to integrate AI confidently and leverage it for new modalities of learning, such as interactive sessions and tailored AI assistants, to complement traditional learning methods effectively.

Looking towards the future, the author speculates that consistent refinements in the application of AI in teaching could offer unprecedented enhancements to analytics education. The paper posits that integrating generative AI in the classroom will elevate not just technical skills but also the overall learning experience by instigating more engaging and empowering educational strategies.

In conclusion, Bray's paper underscores the potential of LLMs to reshape education, particularly within analytics domains, by encouraging innovative instructional designs, fostering a greater appreciation and understanding of programming languages, and improving student engagement and satisfaction. Combining AI with traditional pedagogies promises to broaden the horizons of teaching methodologies and redefine educational frameworks.