Create a Video View Paper

Navigating Rifts in Human-LLM Grounding

This lightning talk explores a critical gap in language model capabilities: their inability to establish mutual understanding through collaborative dialogue. The researchers analyze thousands of human-LLM interactions to reveal how models fail at the conversational grounding acts humans use naturally, introduce the Rifts benchmark to measure these deficits, and propose initial interventions to help models ask clarifying questions and follow up effectively.

Script

Language models excel at following instructions, but when it comes to real conversation, they miss something fundamental: the ability to establish mutual understanding through grounding.

The researchers analyzed datasets including WildChat and MultiWOZ and found striking asymmetries. Language models initiate clarification three times less often than humans, and follow-up requests sixteen times less often. Instead of asking, they generate verbose responses packed with irrelevant information.

When ambiguity arises, humans naturally pause to clarify. Language models plow ahead, leading to interaction breakdowns that range from user frustration to failures in high-stakes scenarios where shared understanding is critical.

To measure this gap, the authors built the Rifts benchmark with 1,800 tasks from real interaction logs, each requiring a grounding act. Most existing models performed poorly, revealing that current training approaches fail to teach collaborative dialogue behavior.

The researchers tested a grounding forecaster that predicts when clarification is needed and prompts the model to ask. It helped, but only marginally, suggesting that fixing this problem requires rethinking both training foundations and dialogue management from the ground up.

Fixing grounding in language models means building systems that don't just respond but actively cooperate, asking questions and confirming understanding the way humans do naturally. If you want to explore this research further and create your own explainer videos, visit EmergentMind.com.