Open problems in task-aware relational question answering

Develop relational reasoning methods that jointly address schema grounding, multi-hop composition, and answer-form control; extend predictive question answering beyond binary classification to ranking, set-valued prediction, and scalar regression under weak temporal signal; construct lightweight routers that combine schema grounding with reliable capability selection; and establish multi-turn relational question answering in which follow-up questions depend on previous answers or evolving temporal assumptions.

Background

RDL-Bench evaluates natural-language question answering over relational databases across two capability tracks: current-state reasoning and future-oriented prediction. The experiments show that existing reasoning systems struggle to jointly perform schema grounding, multi-hop relational composition, and answer normalization, while predictive systems are substantially weaker beyond binary classification, particularly for scalar regression under weak temporal signals.

The routing experiments further show that LLMs can identify the required capability reliably, whereas the smaller Llama8B router may identify relevant tables without correctly distinguishing reasoning from prediction. The benchmark therefore motivates lightweight routing systems that can combine schema grounding with reliable capability selection. The authors also identify multi-turn relational QA involving answer-dependent follow-up questions and changing temporal assumptions as a natural unresolved extension.