Evaluate requirement coverage in end-to-end generated dialogue trees

Evaluate whether end-to-end generated NPC dialogue trees produced from Knudge quest specifications satisfy all quest requirements and complete all user-provided specifications, thereby "checking all the boxes" of required objectives.

Background

The paper primarily evaluates next-utterance prediction and conducts a limited case study of full tree generation, acknowledging that these generated trees do not approach the complexity of actual in-game dialogue structures. While the dataset, Knudge, encapsulates granular quest objectives and lore constraints, the authors note that they have not yet performed a comprehensive evaluation of whether end-to-end generated dialogues meet all listed requirements.

Consequently, a systematic evaluation protocol to verify that generated dialogue trees fully cover and comply with the specified quest objectives and user-provided specifications remains to be developed, and the authors explicitly leave this task for future work.