Protocol-Guided Instruction Data Generation

• Protocol-guided instruction data generation is a method that constrains language model outputs via explicit procedures to produce precise instruction-response pairs.
• It employs techniques such as student-aware loops, document grounding, and scene-graph programming to reduce noise and improve data reliability.
• These protocols transform raw model output into measurable training signals, resulting in scalable and controlled synthetic instruction data for various domains.

Protocol-guided instruction data generation is a family of data-construction methods in which instruction-response pairs are not produced by unconstrained self-generation, but by an explicit protocol that constrains what is generated, how it is grounded, how it is filtered, and what optimization signal defines “good” data. Across recent work, the protocol may be a student-aware optimization loop, a document-grounded task designer, a taxonomy-and-syllabus curriculum, a retrieval-grounded checker, a scene-graph program library, or a monograph-derived diagnostic workflow. The shared objective is to reduce noisy, non-informative, misleading, hallucinated, or weakly useful supervision while making synthetic instruction corpora more controllable, auditable, and effective for downstream tuning (Li et al., 2024, Zhang et al., 2024, Jin et al., 4 Feb 2026).

1. Definitional scope and distinguishing characteristics

In this literature, “protocol-guided” does not denote a single algorithm. It denotes a structured generation regime in which the latent freedom of a LLM is narrowed by an external procedure. That procedure may specify a generation order, a grounding source, an explicit decision rule, a preference signal, a schema, or a validation layer. The essential shift is from asking a model to generate “high quality” instruction data in the abstract to forcing generation through measurable intermediate constraints.

Several representative systems make this shift explicit. Montessori-Instruct is described as a student-aware synthetic data generation loop in which the teacher is tuned toward the student’s learning process rather than treated as a free-form generator (Li et al., 2024). InstructLR is a pipeline for building instruction-following datasets in low-resource languages (LRLs) by combining language-model generation, automated quality control, and human verification (Keita et al., 1 Dec 2025). GLAN replaces seed-example imitation with a taxonomy of human knowledge and capabilities that is recursively decomposed into disciplines, subjects, syllabi, class sessions, and key concepts (Li et al., 2024). ProVision uses scene graphs and human-written programs so that multimodal instruction generation becomes transparent, controllable, factual, and scalable (Zhang et al., 2024). ECG-R1 uses a monograph-derived ECG reading protocol with measurable features, quantitative thresholds, and differential exclusion logic to generate structured clinical reasoning data (Jin et al., 4 Feb 2026).

A concise comparison of representative protocols is useful because the term spans multiple modalities and supervision regimes.

System	Guiding protocol	Primary aim
Montessori-Instruct	local data influence, preference pairs, DPO	generate synthetic data tailored toward student learning preferences
GLAN	taxonomy $\rightarrow$ subjects $\rightarrow$ syllabus $\rightarrow$ sessions $\rightarrow$ concepts	generate large-scale synthetic instruction data across all disciplines
InstructLR	French seed instructions, RAG-based checker, human validation	create instruction datasets for low-resource languages
ProVision	augmented scene graphs, human-written programs, templates	synthesize vision-centric instruction data
ISSE	paired speech synthesis, quality evaluation, instruction design	construct instruction-aligned speech editing pairs
ECG-R1	FeatureDB, ProtocolGuider, six-step reasoning schema	reliable ECG interpretation

A recurring misconception is that protocol-guided generation is simply “better prompting” or “using a stronger teacher.” The record is narrower and more specific. Montessori-Instruct reports that teacher strength alone is not enough; teacher data must be tailored to the student (Li et al., 2024). InstructLR explicitly states that the model is not merely translating a finished pair; it first generates seed instructions in French and then jointly produces target-language instruction, response, and, for reasoning tasks, CoT_lrl (Keita et al., 1 Dec 2025). ProVision contrasts LLMs that guess good tasks with a programmatic pipeline that executes a protocol over symbolic image facts (Zhang et al., 2024). This suggests that protocol guidance is best understood as a shift from unconstrained generation to constrained supervision design.

2. Formal mechanisms of guidance

The strongest versions of protocol guidance specify not only a workflow but also a measurable objective. Montessori-Instruct is exemplary in this regard. It defines local data influence by asking how one-step training on a synthetic pair changes the student’s loss on a held-out reference set $\mathcal{D}_{\text{ref}}$. Positive influence means the example improves the student’s reference performance; negative influence means it hurts performance. These influence values are converted into preference triplets $(p, x^+, x^-)$ in which both examples come from the same seed prompt but one has positive influence and the other negative influence; the teacher is then optimized with DPO to prefer the positively influential instruction over the negatively influential one (Li et al., 2024). The protocol therefore turns downstream student utility into a teacher-training signal.

A different formalization appears in open-source output-first generation. “Harnessing the Power of David against Goliath: Exploring Instruction Data Generation without Using Closed-Source Models” selects an instruction for a chosen output passage by minimizing

$\arg\min_I \, ppl(O \mid I),$

treating instruction selection as a discrimination problem rather than a pure generation problem (Wang et al., 2023). The extract-then-generate stage further constrains the output distribution by first extracting a fragment from long passages and then generating candidate instructions for that fragment. Here, the protocol is a scoring-and-selection rule over candidate tasks.

REInstruct uses a different decomposition. It trains a reverse model on seed instruction-response pairs to predict the instruction given the response-like text, with objective

$$\mathcal{L}_{\text{reverse}} = \sum_{(x_s, y_s)\in \mathcal{D}_{\text{seed}}} -\log p(x_s \mid y_s),$$

and then trains a rewriting model to generate a cleaner assistant-style response conditioned on the passage and generated instruction, with objective

$$\mathcal{L}_{\text{rewrite}} = \sum_{(x_g, y_u, y_r)\in \mathcal{D}_{\text{rewrite}}} -\log p(y_r \mid y_u, x_g).$$

The resulting synthetic pairs are mixed with seed data for final instruction tuning (Chen et al., 2024). In this case, the protocol is a staged transformation from unlabeled corpus text to latent task to rewritten response.

Structured latent representations provide a further class of formal guidance. ProVision represents an image as an augmented scene graph

$G=(V,E),$

where nodes are objects and edges are directed relations between object pairs; program libraries then query object labels, attributes, relations, segmentation masks, and depth annotations to generate question-answer pairs (Zhang et al., 2024). ECG-R1 analogously formalizes clinical data generation as

$\rightarrow$0

so that raw ECG time-series are converted into deterministic physiological measurements, then embedded in a standardized clinical protocol, and only then rendered into a six-step reasoning trace plus diagnosis (Jin et al., 4 Feb 2026).

GLAN’s notation emphasizes hierarchical decomposition rather than pointwise scoring. It defines leaf disciplines $\rightarrow$1, subject sets $\rightarrow$2, a syllabus $\rightarrow$3, class sessions $\rightarrow$4, and key concepts $\rightarrow$5, from which one or two sessions and one to five concepts are sampled to generate homework-like instructions (Li et al., 2024). This suggests a broad formal taxonomy: some protocols define utility, some define grounding structure, and some define curriculum traversal.

3. Language-centered pipelines

In text-only settings, protocol guidance has taken at least five distinct forms: output-first generation, corpus-grounded reverse generation, document-grounded task design, curriculum construction, and training-feedback loops.

The open-source pipeline in “Harnessing the Power of David against Goliath: Exploring Instruction Data Generation without Using Closed-Source Models” begins with corpus passages rather than seed instructions. It adopts LongForm-style output-first generation, trains a LLaMA-7B instruction-generation model on open-source instruction data, applies optional fragment extraction, generates multiple candidate instructions with nucleus sampling, and filters them by minimizing $\rightarrow$6. The best combined dataset reaches 44.88 on SUPERNI and 18.84 on LongForm, compared with Alpaca’s 38.62 and 14.06, and the paper also reports that the generated data can slightly outperform Alpaca in GPT-4 evaluation on unseen Vicuna instructions (Wang et al., 2023).

REInstruct starts from unlabeled web text, specifically C4, and imposes a strict three-stage protocol: select candidate texts, generate instructions reversely from the text, and rewrite responses to better match instruction-following style. The selection stage uses rule-based filters over length, paragraph structure, first-person frequency, punctuation patterns, capitalization, and question marks. The headline result is that training Llama-7b on 3k seed data plus 32k synthetic data yields a 65.41% AlpacaEval win rate against text-davinci-003, outperforming other open-source, non-distilled instruction data construction methods (Chen et al., 2024).

Document-grounded task design is articulated most explicitly in “DoG-Instruct: Towards Premium Instruction-Tuning Data via Text-Grounded Instruction Wrapping” (Chen et al., 2023). The paper defines a task as

$\rightarrow$7

where a human-written document is wrapped into instruction, input, and output fields rather than treated as a fixed answer blob. The meta-training set is built with dual-view ICL from document-view seed examples and task-view seed examples, followed by relevance filtering and a logical consistency check. The reported human-evaluation values for the resulting data are $\rightarrow$8, $\rightarrow$9, $\rightarrow$0, $\rightarrow$1, and $\rightarrow$2 (Chen et al., 2023).

GLAN generalizes protocol guidance from document structure to curricular structure. Rather than sampling around a few exemplars, it traverses a curated taxonomy of human knowledge and capabilities, generates subjects for each discipline, designs a syllabus for each subject, extracts class sessions and key concepts, and generates homework-like instruction-response pairs from sampled sessions and concepts. The reported scale is 126 disciplines, around 100–200 subjects per discipline on average, and 10 million instruction-response pairs overall; a Mistral 7B trained on this corpus reaches 48.8 on HumanEval, 57.6 on MBPP, 80.8 on GSM8K, 32.7 on MATH, 60.7 on BBH, 90.7 on ARC-E, 81.1 on ARC-C, and 62.9 on MMLU, without using task-specific training data from those benchmarks (Li et al., 2024).

Low-resource language generation introduces an additional layer of protocolization. InstructLR begins from a curated set of 20 topics inspired by MMLU, generates seed instructions in French, uses an LLM to produce instr_lrl, resp_lrl, and CoT_lrl in the target language, and then applies a dual-layer quality filter: a RAG-based checker followed by native-speaker validation. The framework produces ZarmaInstruct-50k, BambaraInstruct-50k, and FulfuldeInstruct-50k, each with 50,000 instruction-response pairs, and reports an 88% cost reduction compared with full human correction (Keita et al., 1 Dec 2025).

SeDi-Instruct adds a feedback loop from training dynamics back into generation. It relaxes the ROUGE-L filtering threshold from 0.7 to 0.85, forms locally diverse batches by clustering in a PCA-reduced space, and updates the seed pool every ten iterations by mining the batch with the highest gradient norm and replacing the seed with the lowest score $\rightarrow$3. The reported outcome is 5.2% higher accuracy on average than traditional methods together with 36% fewer API invocations to generate 10,000 kept instructions (Kim et al., 7 Feb 2025).

Across these systems, protocol guidance serves different purposes: corpus recovery in REInstruct, task wrapping in DoG-Instruct, curriculum expansion in GLAN, multilingual quality control in InstructLR, and self-directed seed evolution in SeDi-Instruct. The common pattern is that generation is not left to model priors alone.

4. Multimodal and domain-specific instantiations

Multimodal work has extended protocol guidance by replacing text-only prompts with symbolic representations, synthesis pipelines, and domain rules. ProVision is the clearest programmatic example. It first constructs an augmented scene graph for each image by combining object detection via YOLO-World, segmentation via SAM-2, attribute detection via fine-tuned CoCa and especially LLaVA-1.5-13B, relation detection via finetuned Osprey, and depth estimation via Depth Anything V2. A library of 24 single-image and 14 multi-image instruction generators then queries this graph to produce short-answer and multiple-choice question-answer pairs about objects, attributes, relations, segmentation, depth, counting, and cross-image comparison. Applied to Visual Genome and DataComp, ProVision produces over 10 million instruction data points, including VG-S: 1.5M, VG-M: 4.2M, DC-S: 2.3M, and DC-M: 4.2M (Zhang et al., 2024).

The practical importance of this programmatic turn is visible in the benchmarks. When used in instruction tuning, ProVision-derived single-image data yields up to +7% on CVBench 2D, up to +8% on CVBench 3D, and about +3% on QBench2, RealWorldQA, and MMMU. Its multi-image data improves Mantis-Eval by about 8 points, and incorporating ProVision in both pre-training and fine-tuning of xGen-MM-4B yields an averaged improvement of 1.6% across 11 benchmarks (Zhang et al., 2024). The article’s central distinction is factual grounding by construction: if the scene graph is correct, the question-answer pair is grounded by construction.

ISSE applies analogous logic to speech style editing. Its pipeline has three stages: Paired Speech Generation, Quality Evaluation, and Instruction Design. The system samples an anchor speech and several style reference speeches from Ears and Expresso, uses EmoCapTTS to synthesize a target speech that preserves the anchor transcript while adopting the reference style, applies Chatterbox voice conversion to match the anchor speaker identity, and then uses Qwen3-8B to generate natural-language editing instructions describing the style transformation. Filtering keeps only samples with WER $\rightarrow$4 10, $\rightarrow$5, and $\rightarrow$6 (Chen et al., 29 Sep 2025).

The resulting ISSE dataset contains approximately 382 hours of speech, over 100,000 source-target pairs, and 28 styles. In the in-domain setting, LlasaEdit trained on ISSE achieves WER: 8.06, $\rightarrow$7: 0.68, $\rightarrow$8: 0.58, and UTMOS: 4.29, compared with 10.07, 0.64, 0.49, and 4.01 for the ESD-trained version (Chen et al., 29 Sep 2025). Here the protocol is not only generative but transformational: the instruction is generated from the actual difference between aligned source and target speech.

ECG-R1 is the most rigidly protocolized example in the corpus. It builds a clinical interpretation corpus from MIMIC-IV-ECG, a deterministic non-trainable feature extractor (FeatureDB), a monograph-derived ProtocolGuider, and DeepSeek-V3.1-Terminus as interpretation generator. The protocol is derived from Chapter 23 of ECG from Basics to Essentials: Step by Step and is reorganized into five phases—Technical, Rate & Rhythm; Conduction, Axis & Intervals; Chamber Hypertrophy & Voltage; Ischemia, Infarction & Mimics; and Electrolytes & QT—plus a sixth final synthesis step. The paper reports 30,000 protocol-guided instruction samples and later constructs an RL subset of size

$\rightarrow$9

from the top-500 most frequent report texts (Jin et al., 4 Feb 2026).

The clinical protocol is unusually explicit. It enforces standard 25 mm/s speed, standard 10 mm/mV calibration, rate thresholds of < 60 bpm, 60–100 bpm, and > 100 bpm, PR interval thresholds of 0.12–0.20 s, QRS thresholds of < 0.10 s, 0.10–0.12 s, and > 0.12 s, QTc normality at $\rightarrow$0 ms, LVH criteria such as $\rightarrow$1 mm, ST elevation thresholds of $\rightarrow$2 mm in limb leads and $\rightarrow$3 mm in precordial leads, and pathologic Q-wave thresholds of > 0.03 s or > 0.1 mV (Jin et al., 4 Feb 2026). This is protocol guidance in a strict diagnostic sense: the generator must reason through measurable evidence, lead groups, and explicit exclusions rather than free-associate from sparse labels.

5. Grounding, filtering, and demonstrated effects

The major justification for protocol guidance is empirical rather than merely conceptual. The strongest evidence comes from methods that compare the guiding signal against weaker alternatives. Montessori-Instruct reports that local data influence outperforms both LLM-as-a-Judge scoring and raw training loss as a measure of example usefulness. For the 8B student, local influence yields 54.92% LC-WR / 58.59% WR / 6.903 MT-Bench, versus 53.42 / 54.93 / 6.731 for LLM-as-a-Judge and 52.34 / 54.99 / 6.656 for training loss. The full method reaches 56.82% LC-WR / 60.23% WR on Alpaca Eval and 7.092 on MT-Bench after the second iteration, compared with Self-Instruct’s 50% / 50% and 6.490, summarized as 18.35% relative improvement on Alpaca Eval and 46.24% relative improvement on MT-Bench (Li et al., 2024).

In low-resource language construction, the key evidence is that automated filtering can absorb most of the annotation burden without overwhelming false positives. InstructLR reports, for the RAG checker on 300 Zarma sentences, GLEU = 0.8978, M² score = 0.3400, False-positive rate = 0.0, and native-speaker fluency 4.3/5. At pipeline scale, 85.8% accepted without correction, 5.07% corrected by RAG, and 9.13% sent to top-priority human review. Human agreement for Zarma reaches Krippendorff’s alpha = 0.793 on 351 samples (Keita et al., 1 Dec 2025).

In vision-centric multimodal tuning, ProVision’s evidence is benchmark-based and largely supports the claim that symbolic programmatic generation targets under-served visual skills such as counting, depth, localization, and inter-image reasoning. The paper also reports that human-annotated graphs are generally cleaner and slightly better for training than automatically generated graphs, but automatically generated graphs still help (Zhang et al., 2024). This matters because protocol guidance here is not simply about synthetic scale; it is about producing supervision tied to object count, attribute label, relation edge, mask, or depth comparison.

ISSE shows the effect of tightly aligned pair construction. Beyond in-domain gains, the paper reports that the ISSE-trained model on ESD data gives WER 7.69, while the ESD-trained model evaluated on ISSE gives WER 68.17. The paper attributes these gains to larger scale, more diverse styles, fine-grained style instructions, and better alignment between instruction and paired speech (Chen et al., 29 Sep 2025). This suggests that explicit transformation protocols can improve generalization when the target task is editing rather than unconditional generation.

Corpus-grounded text methods show similar patterns. REInstruct’s ablation at the 8k scale gives 63.78 for the full method, 61.42 without unlabeled text selection, 49.19 without response rewriting, 56.63 for Self-Rewrite, and 59.76 for Self-Training. The paper also reports 77.63% word overlap between rewritten responses and the unlabeled source text, supporting the claim that rewriting remains grounded in the source passage (Chen et al., 2024). GLAN provides a different form of evidence: breadth rather than filtering. Its gains across coding, mathematical reasoning, academic exams, and instruction following suggest that curricular decomposition can substitute for benchmark-specific data under a sufficiently broad taxonomy (Li et al., 2024).

SeDi-Instruct’s findings are narrower but still relevant. The paper reports that relaxed filtering plus iterative feedback task generation improves the average benchmark score from 46.1 for Llama-3-8B + Self-Instruct to 46.9 for Llama-3-8B + SeDi-Instruct, while reducing API cost by 1.6× for 10,000 kept instructions (Kim et al., 7 Feb 2025). This supports a broader point: in protocol-guided generation, the governing procedure can target cost efficiency and data retention as well as quality.

6. Limitations, misconceptions, and open research directions

The literature is explicit that protocol guidance is not a universal remedy. Montessori-Instruct identifies cost and scale as primary limitations. Collecting local influence is expensive because it requires per-example one-step training and reference-set evaluation; the paper reports substantially higher per-data overhead than Self-Instruct, demonstrates the method at the 10K-data scale, and notes that behavior at 100K+ production scales remains uncertain. The authors also warn that redundancy could become a problem as scale grows and that more work is needed on balancing diversity with utility (Li et al., 2024).

In low-resource settings, protocol guidance does not remove linguistic bottlenecks. InstructLR still depends on some LRL coverage in an existing LLM, is demonstrated on languages that share French as a contact language, and notes that human validators focus on grammar, orthography, fluency, naturalness, and preservation of semantic intent rather than factual verification (Keita et al., 1 Dec 2025). This suggests that protocolized fluency control and protocolized factual grounding are separable problems.

Text-grounded methods retain their own fragilities. REInstruct states that heuristic selection is shallow, may miss good texts or admit bad ones, is designed around natural language web text, and still requires further alignment to reduce hallucination and toxicity (Chen et al., 2024). GLAN notes that not all domains are equally strong, that the current synthetic data is mostly question-answer pairs, that coverage depends on the taxonomy, and that the resulting models still trail GPT-3.5 and GPT-4 on instruction following and open-ended quality (Li et al., 2024). SeDi-Instruct explicitly reports that it does not incorporate safety filtering and that models trained on SeDi-Instruct may answer harmful queries without refusal (Kim et al., 7 Feb 2025).

A second misconception is that protocolization necessarily means hand-crafted rigidity at the expense of diversity. The surveyed methods suggest a more nuanced picture. GLAN uses taxonomic breadth and combinatorial concept sampling to expand coverage (Li et al., 2024); InstructLR uses controlled French seed generation plus topic diversity (Keita et al., 1 Dec 2025); Montessori-Instruct and SeDi-Instruct couple generation to downstream utility signals (Li et al., 2024, Kim et al., 7 Feb 2025). A plausible implication is that protocol guidance and diversity are not opposites; rather, the protocol often determines where diversity is allowed to enter.

The most consequential open direction is reliability in high-stakes domains. ECG-R1 argues that severe hallucinations are widespread in ECG interpretation across proprietary, open-source, and medical MLLMs and states that the public should not directly trust these outputs without independent verification (Jin et al., 4 Feb 2026). This warning generalizes beyond cardiology only as an inference, but it highlights an important boundary condition: protocol-guided instruction data generation is most valuable where unsupported fluency is dangerous and where explicit evidence, thresholds, or structured validation can be encoded.

Taken together, these systems define protocol-guided instruction data generation as a shift in the locus of supervision design. The central design question is no longer only how to sample more instruction-response pairs, but how to specify the procedure that makes a pair worth generating at all.