Chain-of-Query (CoQ): Multi-Agent Reasoning

• Chain-of-Query (CoQ) is a framework that decomposes complex tasks into explicit sub-questions, enabling multi-agent collaboration for SQL, IR, and multimodal applications.
• It uses techniques like natural-language schema representations and clause-by-clause SQL generation to enhance table comprehension and reduce query errors.
• Empirical evaluations show improved accuracy (from 61.11% to 74.77%) and reduced invalid SQL rates (from 9.48% to 3.34%), demonstrating robust performance across diverse benchmarks.

Chain-of-Query (CoQ) is a query-centric reasoning formulation used in several recent large-language-model systems to externalize intermediate information needs as explicit sub-questions, query–answer nodes, or question chains. In the 2025 work "Chain-of-Query: Unleashing the Power of LLMs in SQL-Aided Table Understanding via Multi-Agent Collaboration," CoQ denotes a novel multi-agent framework for SQL-aided table understanding, motivated by the observation that table understanding requires structured, multi-step reasoning and that LLMs struggle with the structural complexity of tabular data (Sui et al., 14 Aug 2025). The same acronym also appears in "Search-in-the-Chain: Interactively Enhancing LLMs with Search for Knowledge-intensive Tasks," where the LLM generates a reasoning chain named Chain-of-Query for interaction with information retrieval (Xu et al., 2023), and is occasionally associated with the multimodal "Chain of Questions" framework (Iji et al., 6 Aug 2025). This suggests that CoQ is not a single canonical architecture, but a family of explicit query-decomposition schemes applied to different inference substrates.

1. Problem setting and motivating rationale

The SQL-oriented CoQ framework is introduced against a specific failure mode of LLM-based table understanding: existing multi-agent frameworks for SQL generation have shown promise, but often suffer from the inability to comprehend table structure for reliable SQL generation, error propagation that results in invalid queries, and over-reliance on execution correctness (Sui et al., 14 Aug 2025). In this formulation, the central difficulty is not merely language generation, but structured reasoning over schemas, rows, columns, and compositional operations that must be converted into executable SQL.

A related but broader motivation appears in the retrieval-grounded CoQ literature. "Search-in-the-Chain" argues that making content generated by LLMs accurate, credible, and traceable is crucial for complex knowledge-intensive tasks that require multi-step reasoning and where each step needs knowledge to solve (Xu et al., 2023). In that setting, previous retrieval-augmented methods are described as vulnerable to two failures: wrong knowledge retrieved by IR can mislead the LLM, and interaction between IR and LLM can break the reasoning chain.

A third motivation, in multimodal reasoning, is that improvements from chain-of-thought and explicit step-by-step explanations have not yet fully transitioned to settings where models must proactively decide which sensory modalities to engage (Iji et al., 6 Aug 2025). There, CoQ is framed as a curiosity-driven question sequence for modality selection rather than SQL generation or IR verification. Across these variants, the common design pressure is the same: multi-step reasoning becomes more controllable when intermediate information requests are made explicit.

2. CoQ for SQL-aided table understanding

In the SQL-aided table-understanding sense, CoQ is a novel multi-agent framework that uses SQL as an auxiliary reasoning substrate rather than treating SQL generation as an end in itself (Sui et al., 14 Aug 2025). Its first stated component is the use of natural-language-style representations of table schemas to abstract away structural noise and enhance understanding. This directly targets the structural complexity of tabular data by reformulating schema information in a representation intended to be more congenial to LLM parsing.

Its second stated component is a clause-by-clause SQL generation strategy to improve query quality. The phrase is important: the method does not merely generate SQL, but decomposes SQL synthesis at clause granularity. A plausible implication is that the framework attempts to localize errors within specific SQL clauses rather than allowing mistakes to propagate through an entire monolithic query, but the supplied summary specifies the strategy only at the level of clause-by-clause generation.

Its third stated component is a hybrid reasoning division that separates SQL-based mechanical reasoning from LLM-based logical inference, thereby reducing reliance on execution outcomes. This separation defines the framework’s conceptual boundary: SQL handles operations that are mechanical in character, while the LLM is reserved for logical inference. In the supplied description, this division is the main mechanism by which CoQ is said to move beyond execution correctness as the dominant signal.

3. Architectural principles and error-control mechanisms

The SQL-oriented CoQ framework presents three architectural principles as its main response to prior limitations (Sui et al., 14 Aug 2025). First, natural-language-style schema representations address comprehension of table structure. Second, clause-by-clause SQL generation addresses invalid or low-quality queries. Third, hybrid reasoning division addresses over-reliance on execution correctness.

These principles can be read as a staged noise-reduction strategy. Structural noise is addressed at the schema interface, syntactic error propagation is addressed during SQL construction, and epistemic over-dependence on execution is addressed at the reasoning interface between SQL and the LLM. This suggests that CoQ does not treat tabular reasoning failure as a single bottleneck; instead, it partitions failure into representation, generation, and reasoning-control layers.

The title’s emphasis on multi-agent collaboration indicates that these operations are distributed across a multi-agent framework rather than collapsed into a single prompt (Sui et al., 14 Aug 2025). However, the supplied description does not enumerate individual agents or provide a full protocol for their interaction. What is explicit is that CoQ is positioned as a multi-agent system in which SQL generation and logical inference are deliberately differentiated rather than conflated.

4. Empirical performance in table understanding

The reported empirical results for the SQL-oriented CoQ framework are given at the level of aggregate benchmark performance (Sui et al., 14 Aug 2025). Experiments with four models, including both closed- and open-source systems, across five widely used benchmarks show that Chain-of-Query improves accuracy from 61.11% to 74.77% and reduces the invalid SQL rate from 9.48% to 3.34%. The paper characterizes these results as demonstrating superior effectiveness in table understanding.

Two metrics are especially salient. The improvement in accuracy targets end-task quality, while the reduction in invalid SQL rate targets syntactic or executable well-formedness. The pairing of these metrics matters because a method that only improves execution validity without improving answer correctness would not resolve the underlying reasoning problem, and a method that improves answer correctness while still emitting many invalid queries would remain operationally fragile. The reported gains therefore position CoQ as addressing both semantic and syntactic failure modes.

The evaluation spans five widely used benchmarks and four models, which indicates cross-benchmark and cross-model testing rather than a single-dataset or single-backbone demonstration (Sui et al., 14 Aug 2025). A plausible implication is that the method is intended as a framework-level intervention rather than a benchmark-specific heuristic.

5. Retrieval-grounded Chain-of-Query in Search-in-the-Chain

In "Search-in-the-Chain," CoQ has a different but structurally related meaning: the LLM generates a reasoning chain named Chain-of-Query in which each node consists of an IR-oriented query–answer pair (Xu et al., 2023). Formally, for an original complex question $Q$, the chain is

$$\mathrm{CoQ} = (q_1, a_1) \rightarrow (q_2, a_2) \rightarrow \cdots \rightarrow (q_n, a_n),$$

where $q_i \in \mathcal{Q}$ are IR-oriented sub-questions, $a_i \in \mathcal{A}$ are the answers currently assigned by the LLM, and unanswered nodes are marked as $[ \mathrm{Unsolved\ Query} ]$. Each node can be represented as $(q_i, a_i, flag_i)$, with the flag indicating whether external knowledge is needed.

The operational loop interleaves LLM generation and IR passes. The LLM is prompted to construct a global reasoning chain for $Q$. For each node, the system retrieves the single top-ranked document, using ColBERTv2 in the described implementation, and then applies a Reader model for verification or completion. If a node is already solved, the Reader checks consistency between the retrieved answer candidate and the LLM answer; for short answers, the test is whether the candidate is a substring of the answer, and for long-form outputs, the criterion is $\mathrm{ROUGE\mbox{-}L}(g,a_i) \ge \alpha$ with $\alpha = 0.35$. If inconsistency is detected and the Reader confidence exceeds $\theta = 1.5$, IR issues corrective feedback. If a node is unsolved, IR supplies the answer and the chain continues.

This interaction turns the reasoning process into a tree rather than a simple chain. When IR corrects or completes a node, the LLM regenerates a new CoQ branch rooted at the revised node; the paper describes this as a node-identify depth-first search on a Tree-of-Reasoning. The resulting design is intended to preserve a global reasoning plan while still allowing local correction, completion, and citation of each sub-claim. The authors highlight benefits including global planning of the entire reasoning chain, mitigation of hallucinations by letting IR intervene only where the LLM is uncertain or wrong, dynamic backtracking, and fully traceable output with citations. They also note limitations: multiple back-and-forth rounds between LLM and IR, dependence on single top-1 retrieval, the need to tune $$\mathrm{CoQ} = (q_1, a_1) \rightarrow (q_2, a_2) \rightarrow \cdots \rightarrow (q_n, a_n),$$0 and $$\mathrm{CoQ} = (q_1, a_1) \rightarrow (q_2, a_2) \rightarrow \cdots \rightarrow (q_n, a_n),$$1 on held-out QA data, and prompting templates that are more complex than vanilla CoT.

6. Multimodal “Chain of Questions” and the extension of CoQ

A further extension appears in "Chain of Questions: Guiding Multimodal Curiosity in LLMs," where CoQ is a curiosity-driven multimodal reasoning framework that encourages multimodal LLMs to dynamically generate targeted questions regarding their surroundings (Iji et al., 6 Aug 2025). Here the chain lies between the initial prompt $$\mathrm{CoQ} = (q_1, a_1) \rightarrow (q_2, a_2) \rightarrow \cdots \rightarrow (q_n, a_n),$$2 and final answer $$\mathrm{CoQ} = (q_1, a_1) \rightarrow (q_2, a_2) \rightarrow \cdots \rightarrow (q_n, a_n),$$3, with intermediate queries $$\mathrm{CoQ} = (q_1, a_1) \rightarrow (q_2, a_2) \rightarrow \cdots \rightarrow (q_n, a_n),$$4. Each query is mapped by a task-selection function $$\mathrm{CoQ} = (q_1, a_1) \rightarrow (q_2, a_2) \rightarrow \cdots \rightarrow (q_n, a_n),$$5 to an operation and by a sensor-assignment function $$\mathrm{CoQ} = (q_1, a_1) \rightarrow (q_2, a_2) \rightarrow \cdots \rightarrow (q_n, a_n),$$6 to the corresponding modality, such as vision, audio, or spatial perception. The formal objective is to choose the next question by maximizing expected information gain with respect to the final answer.

The framework specifies a multimodal fusion architecture in which sensor outputs are embedded by modality-specific encoders: a ResNet- or ViT-based visual encoder for vision, a 1D-CNN or pretrained audio transformer for audio, and a graph-network or point-cloud network for spatial data. These embeddings are fused with text-token embeddings using cross-modal attention layers, with both early-fusion and late-fusion variants described. The benchmark is assembled by integrating WebGPT, ScienceQA, AVSD, and ScanQA datasets, and the paper reports Match % and Ask % as modality-targeting diagnostics. On held-out test data, FLAN-T5 xl achieves 76.2% Match and 80.0% Asked, and the improvement over both LLaMA-7B and FLAN-T5 large in Match % is reported as significant at $$\mathrm{CoQ} = (q_1, a_1) \rightarrow (q_2, a_2) \rightarrow \cdots \rightarrow (q_n, a_n),$$7 by a two-tailed $$\mathrm{CoQ} = (q_1, a_1) \rightarrow (q_2, a_2) \rightarrow \cdots \rightarrow (q_n, a_n),$$8 test. The paper also states that final answer accuracy rose by 8–12 points over the no-question baseline across all modalities.

Although this work is titled "Chain of Questions" rather than "Chain-of-Query," the supplied description notes that it is occasionally referred to as Chain-of-Query. This terminological overlap is important because it extends CoQ beyond text-and-SQL reasoning into active modality selection and multimodal evidence acquisition.

7. Comparative interpretation, scope, and misconceptions

The literature represented here uses CoQ for three distinct targets: SQL-aided table understanding, IR-grounded knowledge-intensive reasoning, and curiosity-driven multimodal question asking (Sui et al., 14 Aug 2025). In the SQL setting, CoQ addresses table structure comprehension, invalid SQL, and reasoning partition between SQL and the LLM. In the retrieval setting, CoQ is an explicit sequence of IR-oriented query–answer nodes verified and completed by search. In the multimodal setting, CoQ is a sequence of questions used to decide which sensors and perception modules to activate.

A common misconception is to treat CoQ as a standardized method with a fixed algorithmic signature. The published usage instead shows a shared abstraction with different execution substrates. A plausible unifying interpretation is that CoQ names an explicit intermediate-query layer inserted between an initial problem statement and a final answer. What varies across papers is the substrate to which those intermediate queries are delegated: SQL clauses, IR documents and readers, or multimodal sensing operations.

From that perspective, the importance of the SQL-oriented CoQ framework lies in showing that the same query-centric decomposition principle can be applied to table understanding, where the intermediate substrate is executable SQL and the key concern is reliable structured reasoning over tabular data (Sui et al., 14 Aug 2025). The broader CoQ literature indicates that explicit query chains can also be used to improve credibility, traceability, backtracking, interpretability, and modality control, but those properties arise through different mechanisms in different instantiations.