Vocabulary Assistant LLM (GPT-4o)

• Vocabulary Assistant LLM (GPT-4o) is a data-driven system for vocabulary acquisition that uses a ‘Learning by Teaching’ approach to generate contextual errors and adaptive questions.
• It integrates five modular components—including a Materials Generator, Question Generator, and Learner Model—to personalize vocabulary learning based on real-time performance metrics.
• Empirical evaluations demonstrate a 5-15 percentage point improvement in retention rates, validating its effectiveness over static flashcard methods.

A Vocabulary Assistant LLM based on GPT-4o implements a data-driven “Learning by Teaching” (LbT) approach for vocabulary acquisition. This system utilizes GPT-4o to dynamically generate pedagogically effective errors, context-sensitive learner questioning, and tailored tutoring feedback, all orchestrated in a scalable pipeline. Empirical evidence supports measurable improvements in memory retention and shows that learner-specific adaptation is important for maximizing learning outcomes (Uchida et al., 20 Apr 2026).

1. System Architecture and Workflow

The core architecture comprises five integrated modules:

• Materials Generator (MG): A GPT-4o prompt ingests a target word or idiom and produces a plausible sentence with a misuse, an error explanation, and a list of possible corrections.
• Question Generator (QG): A GPT-4o prompt, acting as a “student,” asks context-sensitive questions derived from the misuse sentence, the learner’s previous explanation, and interaction history.
• Learner Model (LM): Maintains a per-word proficiency parameter ($\theta_w$) and a global cognitive load estimate ($C$). Parameters are updated after each learner response.
• Feedback Engine (FE): Delivers targeted hints, explanations, and gradually increased scaffolding when prompted by learner errors or explicit requests.
• Data Store / Analytics (DS): Records all interactions (including word, question, answer, correctness, and input length), supporting post hoc analytics and algorithmic adaptation.

The data flow is as follows:

1. Pretesting uses MCQs to identify unfamiliar words ($W_u$).
2. For each $w \in W_u$, the MG produces a misuse sentence $s_{\text{wrong}}$, supporting evidence, and correction set.
3. The learner reads $s_{\text{wrong}}$ and submits an explanation $E_\ell$.
4. The QG, conditioned on $s_{\text{wrong}}$, supporting evidence, $E_\ell$, recent question history, and learner profile, generates a follow-up question $q$.
5. The learner responds; FE provides hints or confirmation, while DS logs $C$0.
6. LM updates $C$1 and $C$2 according to observed performance.
7. Steps 4–6 iterate until the module’s stopping criterion is met (e.g., five corrections or three minutes elapsed per word).
8. Retention is assessed immediately and at intervals ($C$3 and $C$4 days).

Selection of training batches is governed by a scoring function:

$C$5

where $C$6 are tunable, $C$7 is proficiency, and $C$8 time since last review. The top $C$9 words by score are selected per review batch.

2. Dynamic Question Generation and Adaptive Difficulty

Question generation leverages both prompt engineering and live adaptation:

• Prompt Templating: System messages define the “student” persona, target vocabulary, and correction roles. User messages inject $W_u$0, supporting evidence, $W_u$1, and the last $W_u$2 questions, minimizing repetition.
• Sampling Parameters: Sampling diversity is maintained with temperature $W_u$3 and top-p $W_u$4, adjusted for output variability and control.
• Difficulty Adaptation: The QG prompt reflects learner proficiency through a dynamic target difficulty:

$W_u$5

As $W_u$6, the questions transition from basic definitions to novel contextual usage.

• Redundancy Control: Recent question history is embedded in the prompt to explicitly prevent repeated questioning.

This method addresses major limitations of prior work, in which question-generation relied on static templates and incurred significant development costs (Uchida et al., 20 Apr 2026).

3. Experimental Protocol and Evaluation Metrics

Pilot evaluation employed a within-subject, cross-over design with $W_u$7 university student participants. Each participant completed both the LbT-GPT-4o condition and a static flashcard (multiple-choice) baseline. The study sequence:

• Pretest: 30 Eiken-level vocabulary MCQs to determine $W_u$8.
• Posttest-1: Immediately follows learning of 10 assigned words ($W_u$9).
• Posttest-2: Three days later, assessment for the next decile of words.
• Posttest-3: Seven days after initial exposure, for the final 10.

Retention rate per word at time $w \in W_u$0 is calculated:

$w \in W_u$1

Mean retention rate:

$w \in W_u$2

Learning gain is immediate posttest minus pretest:

$w \in W_u$3

Statistical significance is evaluated by paired two-tailed t-tests on per-participant $w \in W_u$4; $w \in W_u$5 indicates significance.

Key Empirical Findings

• Immediate Posttest-1: LbT confers a modest $w \in W_u$6 percentage point advantage.
• Posttest-2 (3 days): Retention rate $w \in W_u$7 percentage points higher under LbT.
• Posttest-3 (7 days): $w \in W_u$8 percentage point retention advantage versus baseline.

4. Personalization and Learner Modeling

Personalization derives from both real-time interaction logs and learner-reflective self-reports.

Traits Associated with Learning Outcomes

• Active Engagement: Measured as high average input length; positively correlated with $w \in W_u$9 (LbT outperformance).
• Cognitive Load Sensitivity: Participants with lower $s_{\text{wrong}}$0 (lower average time per correction) sustain greater retention.
• Metacognitive Awareness & Intrinsic Motivation: Higher self-rated “efficiency” and “motivation” (survey Q3/Q4) predict increased learning gain.

Adaptive Strategies

Challenge Detected	System Adaptation	Metric/Trigger
Low engagement (input $s_{\text{wrong}}$1 $s_{\text{wrong}}$2)	Simpler, multiple-choice questions	Avg. words per response
High cognitive load ($s_{\text{wrong}}$3 high)	Reduced question frequency, scaffolding “hint mode”	Response time or cognitive load $s_{\text{wrong}}$4
Reported high effort	Increase review interval, lower question difficulty	Self-report (1–5 scale after each block)

The system uses a lightweight Bayesian update for word proficiency:

$s_{\text{wrong}}$5

with learning rate $s_{\text{wrong}}$6, for $s_{\text{wrong}}$7.

5. Module Specifications and Prompt-Engineering Recommendations

Question Generator: Issues GPT-4o API calls with dynamically programmed persona and dialogue context, maintaining a context window of $s_{\text{wrong}}$8 turns.

Learner Model: Implements continuous $s_{\text{wrong}}$9 and $s_{\text{wrong}}$0 estimation, with Bayesian updating for each word and session.

Feedback Engine:

• Upon hint request, GPT-4o is prompted for scaffolded clues tailored to the actual learner answer and the corresponding misuse sentence.
• Repeated errors on a word trigger insertion of additional usage examples in subsequent reviews.

Prompt Engineering:

• Incorporate few-shot examples spanning question difficulty.
• Dynamically adjust temperature and top-p (e.g., begin with $s_{\text{wrong}}$1 for diverse output; decrease to $s_{\text{wrong}}$2 where $s_{\text{wrong}}$3).
• Explicit redundancy prevention (“do not repeat previous questions”), with recent question history passed to GPT-4o context.

Cost is managed by limiting corrections (e.g., $s_{\text{wrong}}$4 per word) and batching API calls.

6. Impact, Scalability, and Empirical Outcomes

Deployment is entirely based on cloud-accessed GPT-4o prompts without the need for manually authored templates, facilitating scalable and cost-effective operation. The pilot study anticipates an average $s_{\text{wrong}}$5–$s_{\text{wrong}}$6 percentage point improvement in 3-day and 7-day retention rates per learner. All system modules are orchestrated via API and interaction data is centrally logged, supporting both real-time personalization and large-scale analytics.

The architecture thus operationalizes the LbT paradigm for vocabulary acquisition, overcoming legacy limitations in question diversity, development cost, and adaptability, and yielding quantitatively validated gains in retention and learner engagement (Uchida et al., 20 Apr 2026).