Text to Automata Diagrams: Comparing TikZ Code Generation with Direct Image Synthesis

Abstract: Diagrams are widely used in teaching computer science courses. They are useful in subjects such as automata and formal languages, data structures, etc. These diagrams, often drawn by students during exams or assignments, vary in structure, layout, and correctness. This study examines whether current vision-language and LLMs can process such diagrams and produce accurate textual and digital representations. In this study, scanned student-drawn diagrams are used as input. Then, textual descriptions are generated from these images using a vision-LLM. The descriptions are checked and revised by human reviewers to make them accurate. Both the generated and the revised descriptions are then fed to a LLM to generate TikZ code. The resulting diagrams are compiled and then evaluated against the original scanned diagrams. We found descriptions generated directly from images using vision-LLMs are often incorrect and human correction can substantially improve the quality of vision LLM generated descriptions. This research can help computer science education by paving the way for automated grading and feedback and creating more accessible instructional materials.