Formulating a setup that jointly studies demonstrations and evaluative feedback

Formulate a learning setup that jointly models demonstration-based supervision and evaluative feedback (e.g., whether a generated response is good), enabling analysis of how overlap among consistent hypotheses and additional feedback interact, and characterization of the feedback requirements to guarantee performance.

Background

After showing that MLE over a restricted policy class can attain non-trivial overlap with the true supports, the authors suggest this overlap might be transformed into strong performance if one could also leverage evaluative feedback (such as correctness of generated responses).

They propose capturing this via a parameter quantifying overlap and feedback needs, but explicitly leave open the formulation of a suitable setup to paper both feedback sources together for their problem.