Evaluate whether the rules-versus-LLM ranking generalizes to real plant text

Determine whether the finding that rule-based extraction outperforms independent large-language-model extraction remains valid when the rules are applied to unfamiliar maintenance text from a real plant.

Background

The reported comparison between rule-based extraction and the LLM uses one model, one prompt, and synthetic text whose vocabulary is aligned with the system's rules. The paper therefore identifies evaluation on hand-labelled real-plant work orders as necessary to determine whether the observed performance ranking persists under unfamiliar language and terminology.