Generalization to other settings and judgment dimensions

Determine whether the reported relationships between reference-based coherence/diversity measures and human judgments generalize beyond the small English sample, single generator, fixed decoding configurations, and overall-quality ratings used in the study.

Background

The evaluation is restricted to one generator, fixed decoding configurations, a small English-language sample, and two annotators providing overall-quality ratings. These design choices limit conclusions about other generation settings and more specific evaluation dimensions.

The limitations section explicitly identifies both generalization and dimension-specific judgments as unresolved areas requiring further study.