Independent validation of perceived reading difficulty

Determine whether outputs produced by the closed-loop readability controller are perceived by human readers as genuinely easier or harder, using an independent human- or expert-based evaluation criterion rather than the readability metric optimized by the controller.

Background

The closed-loop architecture optimizes a composite readability value and evaluates success using that same value. Consequently, its high hit rate demonstrates that a computable metric can be driven into a target interval, but it does not establish that readers actually perceive the resulting passages as having the requested difficulty. The authors note that standard readability formulas rely on surface features such as sentence length and word complexity and do not capture discourse structure, concept density, or reader background.

Resolving this issue requires an evaluation criterion independent of the optimization loop, such as human or expert judgments of the accepted outputs. This problem is explicitly unresolved because the present experimental design cannot determine whether metric control corresponds to genuine human-perceived readability.

References

The question this design cannot answer is whether the output reads as genuinely easier or harder to a person, and answering it needs an independent, non-optimized criterion, that is, human or expert judgment on the accepted outputs.

— A Closed-Loop Control Architecture for Reliable Constraint Satisfaction in LLM Text Generation  (2609.19710 - Zhou et al., 17 Sep 2026) in Section 6, Threats to Validity, subsection “Construct validity: metric circularity”