CLASS Framework for Intelligent Tutoring

• CLASS is a design framework that structures LLM tutoring by decomposing problems and enabling guided, multi-turn dialogues based on learning-science principles.
• It employs curated synthetic scaffolding and conversational datasets to train systems like SPOCK, ensuring explicit decision-making and pedagogical structure.
• The SPOCK instantiation in introductory biology shows high expert ratings in factual correctness, hint quality, and effective subproblem breakdown.

Searching arXiv for the specific CLASS framework paper and closely related ITS work.

Using the provided paper as the primary source and checking arXiv metadata.

CLASS, short for Conversational Learning with Analytical Step-by-Step Strategies, is a design framework for building LLM-based Intelligent Tutoring Systems (ITS) that are intended to behave more like tutors than general-purpose question-answering chatbots. It organizes LLM tutoring around two coupled capabilities: tutor-like, step-by-step problem-solving support, and natural-language dialogue management grounded in learning-science principles. The framework is instantiated in the paper through SPOCK, a proof-of-concept ITS for introductory college-level biology built on Vicuna-13B and trained with synthetic tutoring data generated from curricular materials (Wang et al., 2023).

1. Definition, scope, and positioning

CLASS is presented as a framework rather than as a single model architecture. Its central purpose is to structure the development of advanced ITS so that an LLM can decompose problems, guide learners through subproblems, diagnose student responses, provide hints and feedback, and sustain supportive dialogue over multiple turns. The framework is explicitly motivated by the view that effective tutoring requires more than fluent generation: it requires scaffolding, adaptive feedback, error handling, and interaction patterns aligned with educational practice (Wang et al., 2023).

The framework’s acronym is functional. “Conversational” refers to free-form natural-language interaction; “Learning” indicates that the system is designed to support conceptual acquisition rather than merely produce answers; “Analytical” refers to analysis of both problem structure and student responses; and “Step-by-Step Strategies” denotes decomposition of difficult problems into manageable subproblems. In the paper’s formulation, these capabilities are not left to inference-time prompting alone. They are induced through curated synthetic datasets and structured prompting during deployment.

A common simplification is to treat CLASS as a prompt template. The paper presents a broader system: base-model selection, domain adaptation on educational text, generation of a scaffolding dataset, generation of a conversational dataset, fine-tuning on both, optional retrieval augmentation, and structured inference outputs that expose decision variables such as tutoring action and current subproblem (Wang et al., 2023).

2. Pedagogical foundations and core principles

CLASS is explicitly grounded in several learning-science principles. The most prominent is scaffolding: complex tasks are decomposed into intermediate steps, and support is provided progressively rather than by immediate answer revelation. The framework also draws on problem decomposition as a computational-thinking principle and on a socio-constructivist view of learning in which understanding is built through guided interaction (Wang et al., 2023).

These principles are operationalized through structured tutoring behaviors. The tutor is expected to break a main problem into subproblems, validate each step before advancing, provide hints rather than solutions when possible, and adapt its response to whether a student answer is correct, incorrect, partially correct, ambiguous, off-topic, or a direct request for help. The framework also incorporates formative feedback and encouraging responses. The paper ties this to findings on effective praise, emphasizing responses that are immediate, specific, truthful, and process-focused.

An important implication is that CLASS does not rely on an explicit model-tracing student model in the style of some classical ITS. Instead, it uses a practical dialogue-state-like structure centered on the current subproblem, the evaluation of the latest student message, and the next tutoring action. This suggests a lightweight but inspectable alternative to richer long-horizon learner modeling, though the paper does not claim that long-term knowledge tracing is solved.

3. Data construction and training workflow

The CLASS pipeline begins with a base instruction-following LLM and domain-specific educational content. In the SPOCK instantiation, the base model is Vicuna-13B, itself derived from LLaMA and instruction-tuned on ShareGPT conversations. Domain adaptation is then performed on 60 LibreTexts biology textbooks using causal language modeling loss to improve biology knowledge before tutoring-specific fine-tuning (Wang et al., 2023).

The next stage is the creation of a scaffolding dataset. Using OpenStax Biology 2e as the curricular source, GPT-4 is prompted with section-level learning objectives to generate a challenging main problem, a decomposition into subproblems, hints, an incorrect response for each subproblem, targeted feedback, supporting facts, and a final solution. For SPOCK, this process used 648 learning objectives spanning 207 sections and 47 chapters, producing 648 main problems and 2198 subproblems (Wang et al., 2023).

A separate conversational dataset is then generated. Here GPT-4 simulates dialogues between a student and a tutorbot around the scaffolded problems. These dialogues encode not only student and tutor utterances but also internal tutoring variables such as “Thoughts of Tutorbot,” “Decision by Tutorbot,” “Subproblem,” and, in a later prompt version, “Evaluation of Student Response,” “Action Based on Evaluation,” and “Subproblem State.” The resulting SPOCK conversational set contains 648 conversations and about 20K student–tutor interactions, with average conversation length of about 400 words counting both student and tutor fields (Wang et al., 2023).

After domain adaptation, the model is fine-tuned on the scaffolding and conversational datasets, with 10 epochs each in the reported implementation. The system was trained with Hugging Face Transformers, DeepSpeed, and FastChat on 8 NVIDIA RTX 48-GB A6000 GPUs for 3 days. The reported GPT-4 data-generation cost was about $50** for the scaffolding dataset and about **$50 for the conversational dataset (Wang et al., 2023).

4. Scaffolding and conversational tutoring mechanisms

The scaffolding component teaches the tutor how to organize reasoning. Each training instance links a learning objective to a main problem and then to a sequence of subproblems, each with its own answer, hint, incorrect response, and corrective feedback. The paper’s examples include biology questions such as identifying the structure responsible for capturing sunlight in photosynthesis, where the hint points to “a specialized organelle found in plant cells” and the incorrect response is “mitochondria,” followed by feedback explaining the difference between respiration and photosynthesis (Wang et al., 2023).

The conversational component teaches the model how to use those pedagogical elements in interaction. The framework enumerates response categories such as incorrect, correct, partially correct, ambiguous or unclear, off-topic, student inquiry, guidance through subproblems, and continuation of conversation. Different actions are associated with these categories: acknowledge correct content, identify missing pieces, ask clarifying questions, redirect the conversation, provide hints, or eventually provide a solution after repeated difficulty.

This division of labor is central to CLASS. The scaffolding dataset teaches what instructional structure should exist; the conversational dataset teaches how that structure is deployed turn by turn. The paper therefore treats tutor-like reasoning and natural dialogue as separable but interdependent capabilities. This suggests a modular view of tutoring competence in which decomposition and interaction are jointly necessary.

5. Runtime behavior, structured outputs, and retrieval

At deployment time, CLASS uses structured prompting so that the model does not merely emit a response string. The tutor is prompted to produce internal fields such as Decision, Subproblem, and Tutorbot, and in richer variants also evaluation and action fields. These fields are intended to expose the tutor’s internal decision-making process: how it categorized the student response, which subproblem is active, and what pedagogical move it selected (Wang et al., 2023).

This structured output is one of the framework’s defining features. It makes the tutoring policy inspectable and creates a basis for targeted refinement. Errors can be localized to response evaluation, tutoring action selection, subproblem progression, or response generation. The paper frames this as an affordance for future user-feedback integration and possibly future loss functions over decision errors, although it does not yet implement an online feedback-learning loop.

CLASS also supports optional retrieval augmentation from indexed educational content. The paper reports a subproblem-augmented dual retrieval setup in which evidence is retrieved not only for the main problem but also for generated subproblems. On MMLU biology subsets, retrieving for the main problem alone gave about +3% accuracy for LLaMA, while retrieving for both main problem and subproblems gave about +3.5% accuracy (Wang et al., 2023). This indicates that decomposition is used not only for pedagogy but also for factual grounding.

6. SPOCK instantiation, evaluation, and limitations

SPOCK is the proof-of-concept ITS built with CLASS for introductory college-level biology. It is deployed through a Gradio chat interface and was evaluated in two stages: quality assessment of the synthetic scaffolding data and expert review of the tutoring system itself. The scaffolding dataset was reviewed by 4 biology subject matter experts over 60 main problems and 209 subproblems. Reported scores were 100% for factual correctness of main solutions and for whether subproblems represented key aspects of the main problem; among subproblem-specific criteria, factual correctness was 98.5%, hint helpfulness 96.2%, incorrect-response relevance 97.6%, incorrect-response incorrectness 97.6%, feedback adequacy 99.0%, and subproblem relatedness 100% (Wang et al., 2023).

SPOCK itself was evaluated by 4 SMEs in virtual sessions lasting 90–120 minutes, using a think-aloud protocol over 3–5 random biology sections each. The rating rubric covered factual correctness, relevance, completeness, and motivation on a 1–5 scale. Reported averages were F1 = 4.50, F2 = 4.83, F3 = 4.33 for factual correctness; R1 = 4.33, R2 = 4.33, R3 = 4.00 for relevance; C1 = 3.83, C2 = 4.83 for completeness; and M1 = 4.00, M2 = 4.67 for motivation (Wang et al., 2023). Expert comments particularly emphasized SPOCK’s ability to break problems into manageable subproblems and to maintain an encouraging tone.

The framework’s limitations are also explicit. The paper notes factual inaccuracies and hallucinations, weakness on numerical or mathematical tasks, and the need for stronger guardrails. Its evaluation remains preliminary: it is small-scale, expert-based, and does not establish downstream student learning gains. CLASS also does not implement a rich long-term student model, even though it adapts to turn-level response categories. A further implication is that the framework’s main novelty lies in structured data generation, prompting, and pedagogical design rather than in a new model architecture or a mathematically formal tutoring objective. The paper explicitly contains very little mathematical formalism beyond causal language modeling loss, binary dataset-quality percentages, and the 1–5 human rating scales (Wang et al., 2023).

Overall, CLASS defines an LLM-centered ITS design framework in which pedagogically structured synthetic data, structured inference outputs, and optional retrieval are combined to produce step-by-step tutoring dialogue. Its significance lies in turning tutoring behavior into an explicit design target rather than leaving it to generic conversational competence, and in showing that open-source LLM tutors can be shaped around scaffolding, feedback, and inspectable decision policies in a domain-specific setting (Wang et al., 2023).