Evaluate the effect of ordinal objectives on readability classification

Determine whether fine-tuning transformer-based and feature-based readability classifiers with an explicit ordinal objective provides less benefit than is commonly assumed, given that flat cross-entropy training already produces smoothly graded CEFR predictions.

Background

The paper finds that both feature-based classifiers and transformer encoders produce an ordinal pattern across the six CEFR levels, including a smooth progression in feature-based SHAP values and an inversion between the easiest level, A1, and the hardest level, C2, in TCAV results. This structure emerges even though the models are trained with flat cross-entropy objectives rather than losses that explicitly encode the ordering of CEFR categories.

Because the task itself may induce ordinal behavior, the authors suggest that adding an explicit ordinal loss might contribute less than expected. Establishing whether ordinal training objectives improve readability classification, and quantifying the magnitude of any improvement, requires dedicated fine-tuning experiments with ordinal objectives.

References

One task-level property cuts across both analyses. The feature-based beeswarms grade smoothly across all six CEFR levels, and the transformer TCAV results show the same inversion between A1 and C2 at the extremes, despite flat cross-entropy training in both cases. We designed neither analysis to produce ordinal structure, so the ordinal shape comes from the task itself. This pattern hints that explicit ordinal losses may add less than sometimes assumed. Testing that possibility requires fine-tuning with an ordinal objective, which we leave to future work.

— Analyzing Traditional and Neural Approaches to Multilingual Readability Assessment  (2609.10792 - Wong et al., 9 Sep 2026) in Section 6, Discussion