Unfiltered-query evaluation and robust final-answer generation

Develop unfiltered-query evaluations and robust final-answer generation beyond the acceptance-filtered, separated-prompt retrieval-augmented question-answering pipeline evaluated in the paper.

Background

The evaluation uses only questions for which a teacher model produced an accepted trajectory, so the reported action-prediction and end-to-end question-answering results do not characterize performance on an unfiltered query distribution. In addition, the end-to-end setup uses a specific separated final-answer prompt and reports only modest controller-only improvements in final-answer quality, leaving the broader problem of robust final-answer generation unresolved.

The paper’s conclusion explicitly identifies both unfiltered-query evaluation and robust final-answer generation as remaining open issues, rather than claiming that the experiments resolve them.