Establish reproducible outcome-statement classification with contextual encoders

Establish whether a contextual encoder can classify changes in course learning-outcome statements reproducibly enough for inclusion in an accreditation record, while evaluating it alongside classification accuracy.

Background

OBER+ detects and classifies revisions to course learning-outcome statements by combining static sentence embeddings with a lookup of the leading action verb against a Revised Bloom’s Taxonomy list. The paper reports that this approach can fail when the institutional verb list assigns a verb to multiple cognitive levels, and it notes that the real statement pairs were not independently labelled.

The authors propose replacing the ambiguous verb-levelling step with a classifier trained on complete outcome statements and comparing contextual encoders with the static vectors used in OBER+. Because the resulting classifications could enter accreditation records, reproducibility is identified as a criterion in addition to predictive accuracy; the paper explicitly characterizes whether this can be achieved as an open question.

References

A contextual encoder should be compared against the static vectors used here under the same reproducibility constraint. The levelling step needs replacing rather than tuning, since a classifier trained on outcome statements levels a whole statement and is therefore not defeated by a verb that the institutional list places at two levels, which is the failure that produced the single missed redefinition reported above. Whether such a classifier can be made reproducible enough for an accreditation record is the open question, and it should be evaluated on that criterion alongside accuracy.

— OBER+: Continuity-Aware Reporting and Traceable Continuous Improvement in Outcome-Based Education  (2609.03770 - Rajasekar, 3 Sep 2026) in Section Future Work