Teaching Language Models How to Code Like Learners: Conversational Serialization for Student Simulation

Abstract: Artificial models that simulate how learners act and respond within educational systems are a promising tool for evaluating tutoring strategies and feedback mechanisms at scale. However, many existing approaches in programming education rely on prompting large, proprietary LLMs, raising concerns around privacy, cost, and dependence. In this work, we propose a method for training open-weight artificial programming learners using authentic student process data. Our approach serializes temporal log traces into a conversational format, representing each student's problem-solving process as a dialogue between the learner and their automated assessment system. Student code submissions and environment feedback, such as test outcomes, grades, and error traces, form alternating conversational turns, enabling models to learn from the iterative debugging process. We additionally introduce a training pipeline combining supervised fine-tuning with preference optimization to align models with authentic student debugging behavior. We evaluate our framework by training Qwen models at 4B and 8B scales on a large-scale dataset of real student submissions to Python programming assignments. Our results show that incorporating environment feedback strengthens the models' ability to replicate student debugging behavior, improving over both prior code-only approaches and prompted LLMs baselines in functional alignment and code similarity. We release our code to support reproducibility.

• The paper's main contribution is a novel framework that serializes student debugging dialogues for accurate simulation of iterative coding behavior.
• It employs supervised fine-tuning and direct preference optimization on authentic student logs, achieving improved coverage and grade proximity.
• The approach demonstrates scalable, privacy-conscious student simulation applicable for educational feedback and dynamic tutoring systems.

Conversational Serialization for Student Simulation: Training LLMs to Emulate Learner Debugging Behavior

Introduction

The synthesis and evaluation of artificial learners, or "student simulators," has grown increasingly relevant in computational education research as scalable alternatives for evaluating feedback policies, tutor strategies, and problem difficulty. Conventional models leveraging large proprietary LLMs for synthetic student code generation confront obstacles in realism, iteration fidelity, privacy, and cost. The work "Teaching LLMs How to Code Like Learners: Conversational Serialization for Student Simulation" (2604.10720) presents a technically novel framework for training open-weight LLMs on authentic student process data, capturing the fine-grained dynamics of iterative debugging as conversational exchanges. The approach also introduces a robust preference optimization pipeline and delivers a thorough ablation on multiple model scales, establishing methodological conclusions pertinent for both educational AI and sequence modeling.

Conversational Representation of Process Data

This research advances the serialization of student interaction logs into conversational dialogues, wherein assistant turns encode code submissions and user turns encode the corresponding automated assessment feedback (test results, errors, grades). This mapping enables the use of general-purpose chat-oriented models without architectural changes for simulating authentic, stepwise problem-solving. The alternation naturally conditions predicted code revisions on explicit feedback, contrasting with prior code-only or prompt-based paradigms. This structure allows direct modeling of the causal dependency between formative feedback and student revisions—a critical property for effective student emulation and data-efficient training.

Training Pipeline: Supervised Finetuning and Preference Optimization

The training protocol is a multi-stage pipeline. First, supervised fine-tuning (SFT) is performed using next-token prediction on serialized dialogues, focusing gradients only on assistant (submission) turns. This step imparts sequence modeling knowledge of code edit trajectories. To further align policies with authentic student behavior, including the avoidance of unrealistic solution jumps, the work leverages Direct Preference Optimization (DPO). Rather than using synthetic or human-annotated preference pairs, DPO is driven by contrastive pairs mined directly from real student logs: at a given step, the preferred candidate is the student's immediate next submission; the counterfactual (dispreferred) is the first subsequent submission with a different grade, thus operationalizing functional progression as implicit feedback.

A variant of online preference optimization is explored via Group Relative Preference Optimization (GRPO/DAPO), adding reward-based evaluation leveraging syntactic and functional (grade) similarity for sampled next-step programs—although under computational limitations only a subset of prefixes and candidates is used.

Experimental Design and Evaluation

The principal empirical setting is the FalconCode dataset, providing ~1.7k training trajectories and 1k evaluation trajectories from a large-scale CS1 course, with full code, grade, and autograder feedback. The primary evaluation metric suite comprises coverage (fraction of student revision steps that the model can simulate before terminating), grade proximity (functional alignment to student submissions), and CodeBLEU (syntactic/semantic similarity).

Both 4B and 8B Qwen models are compared under multiple settings: base (prompted), PARA (code-only, no feedback, as in prior work), SFT, DPO, and DAPO, with dynamic context window management to allow direct comparison.

Results and Analysis

The results provide several salient outcomes:

• Activation of environment feedback (i.e., using the conversational serialization with grader message context) consistently outperforms code-only variants on both functional and syntactic measures. SFT models using serialized feedback achieve higher coverage and grade proximity than PARA, despite PARA's ability to leverage slightly more code context within fixed sequence limits.
• Offline DPO tuning yields the strongest gains on core dimensions of student simulation, achieving coverage of 0.982 (4B) and 0.957 (8B) and grade proximity up to 0.797, compared to SFT's 0.922 and 0.782 (4B), respectively.
• GPT-5-mini demonstrates poor coverage (0.340) and low similarity, indicating that untrained or even highly capable generic LLMs, when prompted as students, degenerate to expert- or benchmark-oriented behaviors (e.g., solving in one step), failing to reproduce realistic debugging sequences.
• Analysis of rollout performance reveals that DPO-based models maintain functional alignment for longer trajectories, with minimal coverage degradation per rollout step, while SFT and code-only models degrade more rapidly.
• The grade progression dynamics predicted by DPO-aligned models closely match real student trajectories, as visualized in normalized trajectory position plots, capturing incremental progression and abrupt multi-test-case jumps.

  Figure 1: Coverage, grade proximity, and CodeBLEU as functions of autoregressive rollout depth for various training algorithms and model scales.

  

  Figure 2: Grade progression trajectories for models and human students, with DPO-aligned artificial students best tracking empirical student learning curves.

Theoretical and Practical Implications

The demonstrated improvements with conversational serialization and preference optimization have several implications:

• Technical: Off-policy preference optimization anchored in real process data serves as a powerful alignment signal for process emulation. This approach leverages naturally occurring functional "mistakes" and corrections, circumventing the cost and bias of teacher/human preference annotation, and avoids the model's tendency toward "oracle" behaviors.
• Educational AI: Artificial student simulators trained in this fashion can be deployed for scalable tutor and feedback policy evaluation, curriculum stress-testing, and possibly dynamic assignment generation. Their close match to actual learning curves also renders them promising candidates for downstream knowledge tracing and personalized support pipelines.
• Future AI development: The findings suggest that interaction-aware serialization (feedback-sensitive turn structure) is essential for any process simulation domain where agent actions are immediately conditioned on environmental or external signals. This is relevant for synthesis, code repair, or any behavioral modeling with strong action-feedback couplings.
• Privacy and openness: Reliance on open-weight models addresses institutional and privacy constraints, supporting adoption in sensitive educational settings.

Limitations and Prospective Directions

Limitations include a current focus on a single dataset and programming language, and over-representation of successful students. Extrapolation to low-proficiency or disengaged learner simulation, multidomain process modeling, and real-time integration with intelligent tutoring systems remains to be investigated. Further, computational constraints limited the full exploration of online preference optimization; prior work suggests chain-of-thought augmentation would likely improve DAPO/GRPO.

Several actionable research lines emerge:

• Integration of hierarchical process data (cross-assignment, personalized history) with per-problem simulation for more granular knowledge tracing and simulation.
• Expansion to agent-based artificial students utilizing fine-grained interactive tools and iterative reasoning traces for richer internal state modeling or explainability.
• Application of the approach to live tutoring/feedback systems, where student simulation is used adversarially to optimize tutor feedback via reinforcement learning.

Conclusion

The methodology of conversational serialization of rich process data, paired with offline preference alignment on authentic student behaviors, establishes a new SOTA for realistic artificial student simulation in programming education. The demonstrated improvements in functional and syntactic alignment, as well as fine-grained emulation of progression curves, support the value of incorporating explicit environment feedback and real behavioral counterfactuals for LLM-based learner emulation. This work motivates an expanded research agenda combining process data modeling, automatic tutor training, and privacy-conscious, institutionally deployable LLM solutions for educational support systems.