Effect of pipeline quality on users’ ability to evaluate and review goals

Determine how the quality (accuracy/precision) of OnGoal’s goal pipeline affects users’ ability to evaluate and review conversational goals during multi-turn LLM dialogues, including impacts on task performance, confidence, and decision-making.

Background

The pipeline’s outputs directly inform users’ judgments of goal alignment and progress. Variations in pipeline accuracy could influence how users interpret and act on feedback.

Identifying the relationship between pipeline quality and user outcomes would guide improvements to evaluation logic and interface design.

References

For example, how the quality of the goal pipeline might affect a user's ability to evaluate and review goals remains unclear.

— OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Models (2508.21061 - Coscia et al., 28 Aug 2025) in Section 7.2 Limitations and Future Work