Exact completion-state selection in evidence-guided reporting

Determine how to reliably select the correct Answer, Qualify, or Block completion state for high-performance computing operational reports, particularly when evidence is missing, invalid, contradictory, or otherwise insufficient.

Background

REPORT-OPS requires an operational reporting agent to select among three completion states: Answer when the requested claim is supported, Qualify when only a supported subset can be reported with limitations, and Block when the central value or explanation would require unsupported inference. The evaluation reports that evidence gating substantially improves claim-support precision, but all configurations continue to have difficulty selecting the exact completion state. The unresolved problem is therefore to improve the decision boundary among Answer, Qualify, and Block while preserving evidence validity and avoiding unsupported unqualified answers.

References

Exact completion-state selection remains an open problem.

— REPORT-OPS: Hybrid Evidence-Guided Reporting with Human-Supervised Refinement for HPC  (2610.19222 - Konate et al., 13 Oct 2026) in Section Evaluation, subsection “Results,” paragraph “RQ1: Evidence validity and selective completion”

These approaches leave open how to evaluate answer-or-refuse decisions when the consequences vary across inputs and refusal is appropriate only for some of them.

— UtilityBench: Measuring Language Model Utility When Errors Have Different Costs  (2610.25924 - Lei et al., 19 Oct 2026) in Appendix D, Related Work, subsection “Decision costs and evaluation”