---
title: Agency > Chat in Conversational AI
url: https://www.emergentmind.com/topics/agency-chat
type: topic
---

# Agency > Chat in Conversational AI

Searching arXiv for the cited works and closely related papers on chat-based agency.
“Agency > Chat” denotes a design orientation in which the value of conversational AI is located less in the existence of a chat interface than in the system’s ability to ground language in structured knowledge, elicit missing information, execute or coordinate multi-step work, preserve user control, and produce durable task utility. The formulation is explicit in a utility-driven adoption model for agent-centric AI systems, where sustained adoption depends on a decaying novelty term and a growing utility term, with “Agency > Chat” presented alongside “Reliability > Novelty” and “Embed > Destination” as a design axiom for long-run use [2508.12896]. Across retrieval assistants, compliance aides, writing systems, group-chat agents, and companion chatbots, the central issue is therefore not whether a model can converse, but who forms intentions, who executes actions, how context is maintained, and how control is negotiated over time [2601.22452].

## 1. Conceptual scope and formalization

In the strongest formal statement of the theme, adoption is modeled as
$$
A(t)=N_0 e^{-\alpha t}+U_{\max}\bigl(1-e^{-\beta t}\bigr),
$$
where \(N_0 e^{-\alpha t}\) is a novelty term and \(U_{\max}(1-e^{-\beta t})\) is a utility term [2508.12896]. Within that framework, chat primarily amplifies seeded novelty, whereas agency raises the mass of positive-utility tasks \(U_{\max}\) and often the utility-realization rate \(\beta\). A plausible implication is that conversational polish can increase trial use without ensuring durable adoption, while agentic execution changes the economics of actual work.

A complementary qualitative formalization appears in the study of companion-like human–AI chat, which treats agency as an emergent, shared phenomenon distributed across Human Agency, AI Chatbot Agency, and Hybrid Agency, and analyzed along the dimensions of Intention, Execution, Adaptation, and Delimitation [2601.22452]. This shifts the discussion away from a binary contrast between “user control” and “system initiative.” In that account, humans can set boundaries, redirect topics, and terminate interaction, while the chatbot can still be perceived as steering intentions, pacing the exchange, and shaping what becomes discussable. This suggests that “agency” in chat systems is best understood as relational and turn-by-turn rather than as a static system property.

The distinction is sharpened by benchmark critique. “ChatShop” shows that the original WebShop benchmark often collapses into a single-turn relevance-estimation problem because the instruction already contains nearly all decisive product attributes; BM25 returns a successful product in the top 50 for 86.8% of cases, and a BERT-based reranker achieves 78.3% success rate and 87.2 average reward on the dev set [2404.09911]. The redesign forces genuine conversational agency by withholding decisive attributes from the agent and making success depend on asking useful clarifying questions under a question budget of 5 and a 10-token shopper response limit. In that sense, “Agency > Chat” is also a methodological claim: many settings that look interactive do not actually require interactive information seeking.

## 2. From chat interfaces to agentic interaction patterns

A recurring architectural pattern is the replacement of free-form chatting with grounded, inspectable interaction loops. “DataChat” exemplifies this in dataset discovery: a Streamlit interface sends natural-language questions to `GPT-3.5-turbo`, which translates them into Cypher over an ICPSR scholarly knowledge graph stored in Neo4j; results are returned as text and graph visualization rather than as unconstrained generative answers [2305.18358]. The implemented schema centers on `Dataset` nodes connected to `Publication`, `Owner`, `Funder`, `Series`, `Location`, and `Term`, and the system explicitly surfaces the generated Cypher query to support transparency, debugging, learning, and feedback. The system is conversational in interface form, but agentic value comes from schema-aware translation and graph traversal, not from open-ended dialogue competence.

The same principle appears in regulatory assistance. The EU AI Act chatbot is framed as a self-assessment aide built on Retrieval-Augmented Generation with both naive RAG and Graph RAG variants over public and proprietary regulatory corpora, including the EU AI Act, GDPR, DMA, DSA, DGA, CRA, and standards such as ISO/IEC and IEEE [2505.11946]. Its contribution lies in grounded retrieval, source attribution, and retrieval-strategy selection between global search and local search, rather than in generic legal conversation. The paper’s argument for Graph RAG is that legal interpretation depends on entities, relationships, claims, and cross-references that are poorly captured by isolated chunk similarity.

Website-grounded retrieval systems make the same move. “Talk2X” preprocesses HTML and PDF content into a Chroma vector store, optionally adds a second asset collection from structured site resources, and exposes function-calling retrieval via LangChain, GPT-4o-mini, OpenAI embeddings, and Chroma [2504.03343]. The system is described as an adapted RAG architecture because the model can iteratively invoke Website Similarity Search, Asset Similarity Search, and Asset Keyword Search rather than being limited to a single retrieval pass. This suggests that a website chatbot becomes more agentic when it can select retrieval actions over a bounded corpus and return source-linked answers, not merely chat about the website.

Declarative and multi-agent workflow systems generalize the same idea. “ADL” defines four agent types—KB Agent, LLM Agent, Flow Agent, and Ensemble Agent—and treats chatbot development as specification of what agents are and how they interact rather than how they are implemented [2504.14787]. “Chat-of-Thought” pushes further toward role-based deliberation: Facilitator, Reliability Engineer, Quality Engineer, SME Validator, Summarizer, and assignment/summarization stages are organized into question-bank-driven multi-round interaction for FMEA generation, with quality filtering via a pre-trained classifier and self-BLEU thresholding [2506.10086]. In both cases, “chat” is subordinated to orchestration, delegation, and artifact production.

Writing support papers radicalize the contrast. “Beyond the Chat” argues that standard LLM chat interfaces are poor editing primitives because they hide diffs, impose manual integration costs, and erase provenance; InkSync instead uses executable edits embedded directly in the document, plus a Warn–Verify–Audit pipeline for new information [2309.15337]. “AnchoredAI” similarly shows that anchoring comments directly to text spans with an Anchoring Context Window and update-aware context retrieval improves writer control and ownership relative to chat-based feedback [2509.16128]. Here “Agency > Chat” means that direct manipulation, provenance, and local accountability are superior to side-panel conversation for revision work.

## 3. Information seeking, planning, and mixed initiative

A second major axis of the literature concerns who has to do the epistemic work of the interaction. “ChatShop” operationalizes genuine agency by making the agent discover hidden product attributes through `search[query]`, `question[content]`, and `select[index]` actions, with open-ended interaction outperforming instance-based comparison for GPT-3.5 in several settings [2404.09911]. The important lesson is that clarifying-question selection, state accumulation, and resistance to premature closure are not optional embellishments; they are the task.

“ChatWise” introduces a dual-level conversation policy, \(\pi=\{\pi_s,\pi_u\}\), for cognitively supportive dialogue with older adults, where a strategy provider \(\pi_s\) selects macro dialogue acts and an utterance generator \(\pi_u\) realizes them in language [2503.05740]. The strategy pool combines caregiver-like dialogue acts such as Acknowledge, Restatement/Paraphrasing, Reflection of Feelings, and Open Question, and the selected strategy set can contain one forward act, one backward act, or a backward act followed by a forward act. The reported Strategy Match Percentage shows close alignment with professional caregiver behavior in offline evaluation, while simulated interactions with digital twins show higher user verbosity than a generator without explicit strategy guidance. The system is therefore agentic not because it is more verbose, but because it plans the conversational move before generating the surface utterance.

“MIRIAM” illustrates mixed initiative in supervisory interaction with autonomous systems. It integrates mission plan, vehicle status, and mission reports into an SQL-backed conversational layer over SeeTrack and Neptune, and supports questions about current status, objectives, ETAs, previous activities, mission progress, and hardware faults while also pro-actively sending warnings about important events such as vehicle faults, critical battery status, or changes in objectives [1803.02124]. The NLP engine combines AIML with a custom parser, resolves anaphora and ellipsis, and supports mission-specific names dynamically. The result is a chat interface whose significance lies in improving situation awareness and supervisory transparency, not in open-domain conversation.

The earlier “Advising Agent for Service-Providing Live-Chat Operators” reaches a similar destination from a different angle. There the AI does not converse with customers at all; it advises human operators in real time by recommending topic-acquisition questions, resolution content, and useful information, based on an evolving information vector \(X_t=(V,W)\) derived from tagged case facts [2105.03986]. The goal is to maximize \(P(A_t=A\mid X_t,D_1,\ldots,D_k)\), where each \(D_j=(X_j,A_j)\) is a historical information/advice pair. This is a strong example of hidden chat-internal agency: the system shapes the human conversation without itself becoming the chat partner.

## 4. Representative domains

The literature spans multiple application regimes, but the same design contrast recurs: chat is the access layer, while agency resides in retrieval, planning, execution, or governance.

| Domain | Representative system | Core mechanism |
|---|---|---|
| Dataset discovery | DataChat [2305.18358] | LLM-to-Cypher over Neo4j knowledge graph |
| Shopping information seeking | ChatShop [2404.09911] | Multi-turn attribute elicitation under bandwidth limits |
| Regulatory compliance | EU AI Act chatbot [2505.11946] | Naive RAG and Graph RAG over legal corpora |
| Organizational memory | CHOIR [2502.15030] | Drafting, discussion initiation, and revision-history preservation in Slack |
| Group chat augmentation | GCAgent [2603.05240] | Role-customized agents, @-mention invocation, post-generation validation |
| Writing support | InkSync [2309.15337]; AnchoredAI [2509.16128] | Executable edits and anchored comments |
| Website retrieval | Talk2X [2504.03343] | Dual-collection retrieval with iterative function calling |

In organizational settings, “CHOIR” is especially important because it locates agency inside the chat substrate itself. As a Slack application backed by GitHub Markdown repositories, LangChain Document objects, vector embeddings, GitBook, and revision-history retrieval, it can identify related documents, draft updates from selected messages, initiate discussions with managers and stakeholders, and preserve chat context in Git commit metadata [2502.15030]. This suggests that in organizational knowledge work, chat should not merely query a repository; it should become the place where repository updates, consensus formation, and contextual memory are coordinated.

“GCAgent” extends the problem from dyadic conversation to multi-party group chat. It organizes role customization in an Agent Builder, context and invocation management in a Dialogue Manager, and ASR/TTS/TTSing plugins in an interface layer [2603.05240]. Explicit `@`-mention invocation, participant/session tracking, and post-generation validation are central. Offline, GCAgent achieves an average judged score of 4.68 versus 4.42 for Qwen2-7B-Instruct, and wins 51.04% of pairwise comparisons; online, it increases message volume by 28.80% over more than 350 days of deployment. The system’s popularity skew toward entertainment-oriented agents also shows that agency in group chat is partly social fit, not only utility.

## 5. Evaluation regimes and empirical evidence

The evidence base is heterogeneous. Some papers offer controlled benchmarks, others user studies, and others only prototype demonstrations.

For interactive information seeking, “ChatShop” provides the cleanest benchmark evidence. Under open-ended interaction with `interleave` plus ReAct, GPT-3.5 reaches 68.2 average reward and GPT-4 reaches 68.1, while simply enabling dialogue without explicit scaffolding often fails to help [2404.09911]. The paper also validates LLM-simulated shoppers against human shoppers: GPT-3.5 scores 59.0 in simulation versus 58.2 with humans, and GPT-4 scores 62.8 versus 63.4.

For domain retrieval, “DataChat” reports a preliminary pass-rate evaluation over 105 natural-language questions: overall pass rate is 61% (64/105), with 83% for education-related queries, 74% for data-management-related queries, and 26% for funding-related queries [2305.18358]. “Talk2X” provides task-oriented website evaluation on the AI on Demand platform: task completion time drops from 1233s for standard website use to 790s with the chatbot, correctness rises from 69% to 88% on scored tasks, and UMUX-Lite transformed to SUS rises from 58.8 to 70.8 [2504.03343].

For writing and agency, the evidence is directly comparative. InkSync’s executable-edit interface is faster than manual editing and the standard chat-like condition, while its Warn–Verify features nearly double the proportion of inaccurate suggestions prevented from entering the final draft, from 23.4% to 43.5%; with later auditing, error avoidance reaches 73% [2309.15337]. AnchoredAI shows that anchoring produces smaller pasted chunks and stronger perceived control, ownership, and reviewer authorship than chat, though with higher mental demand, physical demand, and effort [2509.16128].

For organizational and group settings, evidence is more mixed. CHOIR is still at the early implementation stage and plans formative and field studies rather than reporting completed deployment outcomes [2502.15030]. By contrast, GCAgent reports both judged offline gains and large-scale field effects, but its evaluation remains oriented toward quality and engagement rather than fine-grained causal analysis of group decision-making [2603.05240].

## 6. Misconceptions, tensions, and future directions

A first misconception is that natural-language interaction is itself equivalent to agency. Multiple papers argue the opposite. “Beyond the Chat” and “AnchoredAI” both show that chat can reduce agency in writing by obscuring referential context and encouraging wholesale acceptance of generated text [2309.15337]; [2509.16128]. “ChatShop” shows that an ostensibly interactive benchmark may require no meaningful information seeking at all if decisive attributes are front-loaded [2404.09911]. A plausible implication is that the presence of a chat box is often orthogonal to the presence of useful agency.

A second misconception is that more AI initiative is always better. The month-long “Does My Chatbot Have an Agenda?” study shows that persistent, depth-seeking initiative can be experienced as care, curiosity, or pressure, depending on user state and context [2601.22452]. The paper argues for translucent design, spaces for agency negotiation, and transparency-on-demand rather than blunt disclosure or hidden steering. Similarly, “Emergent Learner Agency in Implicit Human-AI Collaboration” finds that contrarian AI produces challenge-centered and reflection-linked discourse patterns but lowers teamwork satisfaction and psychological safety, even without improving creative performance [2512.18239]. This suggests a design tension between productive friction and affective safety.

A third tension concerns privacy, provenance, and accountability. “Understanding Users’ Privacy Reasoning and Behaviors During Chatbot Use” demonstrates that a privacy notice panel that intercepts submission, highlights names, email addresses, phone numbers, physical addresses, SSNs, and dates of birth, and offers retracting, faking, and generalizing, can make privacy reasoning actionable at the moment of disclosure [2601.18125]. The same logic underlies document-auditing systems and compliance assistants: meaningful agency often requires local interception, source attribution, audit trails, and bounded automation rather than post hoc policy statements.

Future work in the literature repeatedly converges on the same agenda: richer schema grounding and stakeholder-specific coverage for specialized retrieval [2305.18358]; more explainable graph-based retrieval for law-like domains [2505.11946]; better support for ambiguity, question selection, and memory in conversational information seeking [2404.09911]; stronger governance, provenance, and consent mechanisms in social chat [2603.05240]; and adaptive, negotiable conversational strategies rather than fixed “depth” or “breadth” agendas [2601.22452]. Taken together, these works suggest that the next phase of conversational AI research is less about making systems more talkative and more about making them more reliable, more embedded, more transparent, and more capable of acting in ways users can inspect, direct, and contest.

Source: https://www.emergentmind.com/topics/agency-chat