LegalStories Dataset: Legal Doctrines Narratives

• LegalStories Dataset is a structured resource combining legal doctrine definitions, LLM-generated narratives, and MCQs to simplify complex legal concepts for educational use.
• It employs a tripartite architecture of canonical definitions, narrative explanations, and graded MCQs to enhance comprehension and retention.
• A human–machine iterative review process ensures high accuracy, improved engagement, and effective learning, especially for legal novices.

LegalStories Dataset

The LegalStories dataset is a structured educational resource designed for the pedagogical exploration of complex legal doctrines via narrative and active learning methodologies. Introduced in the context of leveraging LLMs for legal education, LegalStories targets the translation of abstract jurisprudential concepts into accessible, story-driven format. The dataset is optimized for both instructional utility and empirical evaluation of comprehension among non-expert learners, especially those without formal legal background (Jiang et al., 26 Feb 2024). Its architecture and associated methodologies reflect current trends in educational NLP, with particular emphasis on explainability, expert oversight, and validated efficacy.

1. Dataset Foundation and Structure

LegalStories is anchored in legal doctrine taxonomy sourced from the Wikipedia page “legal doctrines and principles.” The dataset comprises 295 distinct doctrines, each represented by a tripartite structure:

• The doctrinal definition (the Wikipedia introductory paragraph) forms the canonical source text for each legal concept.
• An LLM-generated narrative (“story”) provides an interpretive account of the doctrine, generally constrained to 500 words. The story’s role is to recontextualize complex concepts into relatable, scenario-based form.
• Three styles of multiple-choice questions (MCQs) for each concept: a “concept question” targeting definitional knowledge; a “prediction question” requiring application of the concept; and a “limitation question” probing boundaries and exceptions.

Table 1. LegalStories Content Schema

Component	Source	Description
Definition	Wikipedia page	Original legal doctrine definition
Story	GPT-3.5/GPT-4/LLaMA2	500-word narrative explanation
MCQ Set	LLM + expert iteration	Concept, prediction, limitation (4 options)

A curated subset of 102 doctrines, with definitions in the 100–200 word range, is optimized for generating concise yet complete stories and was used for controlled downstream evaluation.

2. Story Generation Methodology

Narrative generation utilizes instruction prompting of LLMs, with GPT-3.5, GPT-4, and LLaMA 2 variants investigated for story synthesis fidelity. The prompt paradigm specifies both word count boundaries and explicit simplification objectives:

“Tell a story within 500 words to simplify the concept explanation below for ‘{CONCEPT}’. Start your answer with ‘Concept Simplified:’. Concept: ‘{DEFINITION}’.”

Stories are further assessed along multi-dimensional criteria: readability, relevance, cohesiveness, completeness, factuality, likeability, and believability. Empirical ratings—such as GPT-4 achieving RoS > 4.5/5—indicate high output clarity and alignment with educational objectives.

A plausible implication is that model selection—especially the choice of GPT-4—plays a key role in maximizing readability and coherence, essential for user engagement and retention.

3. Expert-in-the-loop Content Assurance

LegalStories employs a cyclical human–machine pipeline for MCQ design and quality assurance. Legal experts (JD-holders or advanced law students) review both the generated narrative and corresponding questions. Roles include direct fact-checking, ambiguity identification, correctness validation, and feedback provisioning for iterative question refinement.

Experts verify that (1) MCQ distractors are plausible yet incorrect, (2) explanations of correct answers are logically sound, and (3) all questions derive from the narrative and definition. Feedback cycles integrate expert suggestions into revised prompts for re-generation or amendment of problematic content. Figure 1 (Jiang et al., 26 Feb 2024) details this feedback loop.

This suggests that maintaining content fidelity, especially in legal education, requires systematic expert engagement even when leveraging advanced LLM outputs.

4. Evaluation: Human Ratings and Controlled Experiments

The efficacy of the LegalStories dataset is empirically substantiated along two axes:

• Story and Question Quality: Human raters (crowdworkers with legal background) evaluate story quality and MCQ reliability. GPT-4 outputs notably lead in all metrics, with minimal need for expert-driven correction.
• RCT-Based Comprehension Gains: A randomized controlled trial with two arms—one receiving only definitions, the other both definitions and narratives—assesses comprehension and retention among legal novices. MCQ performance, relevance scores, self-reported interest, and delayed retention are tracked. Non-native English speakers demonstrate statistically significant improvement across prediction and limitation questions, and higher engagement and retention when exposed to narrative explanations. The significance of the results is confirmed by chi-squared and Mann–Whitney U tests.

A plausible implication is that narrative explanation is particularly advantageous for non-native speakers and legal novices, especially in tasks requiring generalization or conceptual boundary probing. In contrast, native speakers show improvement primarily in higher-order (“limitation”) questions.

5. Educational and NLP Significance

LegalStories demonstrates that leveraging LLM-driven storytelling is an effective method in legal education for broad audiences. Supplementing formal doctrine definitions with narrative and expertly curated MCQs promotes deeper comprehension, retention, and perceived relevance. The expert-in-the-loop methodology ensures integrity of educational content—critical for high-stakes domains.

More broadly, this approach is extensible to other domains requiring translation of technical, jargon-laden material into accessible forms. The dataset’s structure, combining canonical definitions, LLM-generated stories, and iterative expert refinement of active tasks, establishes an architectural template for educational NLP pipelines.

6. Implications and Future Directions

The dataset highlights several methodological and research trajectories for future work:

• Comparative studies with alternative simplification techniques (e.g., “explain like I’m 5”) could delineate efficacy boundaries of narrative vs. didactic simplification.
• Further automation and scaling of expert-in-the-loop workflows may reduce curation costs and enable larger datasets without loss of fidelity.
• Integration into broader legal literacy programs and cross-domain educational initiatives is plausible, particularly for populations under-served by traditional legal education.
• Investigation into retention mechanisms (narrative structure vs. factual priming) may illuminate general principles of learning optimization via LLMs.

The LegalStories dataset thus not only operationalizes contemporary pedagogical theory in legal education but also provides a validated empirical platform for research in legal NLP and educational technology. Its comprehensive structure, human–machine synergy, and rigorous evaluation set a robust benchmark for future datasets targeting complex concept explanation (Jiang et al., 26 Feb 2024).