---
title: Sub-Question Decomposition
url: https://www.emergentmind.com/topics/sub-question-decomposition
type: topic
---

# Sub-Question Decomposition

Sub-question decomposition is the process of transforming a complex, multi-faceted, or compositional question into a set of simpler sub-questions whose answers collectively address the original information need. This technique has become central in modern machine learning and information retrieval pipelines, particularly for tasks—such as multi-hop question answering (QA), video-language understanding, fact verification, retrieval-augmented generation (RAG), numerical and multi-table reasoning, and knowledge-based question answering—where direct, holistic reasoning is infeasible for current models due to limitations in token capacity, data coverage, transparency, or reasoning depth.

## 1. Formal Definitions and Algorithmic Foundations

Sub-question decomposition formalizes the mapping from a complex query $Q$ to a set (or sequence, or graph) of sub-questions $\{q_1, ..., q_n\}$, each designed to isolate a distinct facet, reasoning step, or piece of evidence. The process can be captured algebraically as a function $D$,
\[
D(Q) = \{q_i\}_{i=1}^n
\]
where each $q_i$ is ideally “self-contained” and collectively, the $q_i$’s are sufficient to answer $Q$ [2510.08818][2410.15531][2507.00355].

Algorithmic approaches include:
- **Prompting LLMs**: Using few-shot or tailored templates to invoke a frozen LLM (e.g., GPT-4, GPT-3.5-turbo) to produce sub-questions by direct generation [2510.08818][2410.15531][2510.07718].
- **Supervised Sequence-to-Sequence Models**: Training T5/BART models to autoregressively predict sub-questions given the original question (optionally with additional context or schema) [2211.03277][2305.14901][2205.06938][2310.13575].
- **Unsupervised or Semi-supervised Methods**: Leveraging denoising/back-translation over large question corpora (e.g., ONUS) [2002.09758], or mining pseudo-decompositions for either direct model training or as weak supervision.
- **Structural Parsing**: Mapping $Q$ to semantic structures (e.g., Abstract Meaning Representation graphs, QPL sequential operators, or explicit trees) whose segmentation yields sub-questions aligned with programmatic or logical reasoning steps [2206.08486][2306.07597][2310.13575][2204.07190].
- **Taxonomies and Typing**: Classifying sub-questions into roles—such as “core,” “background,” or “follow-up” in RAG evaluation [2410.15531]; or “literal” vs. “implied” in fact verification [2205.06938]—to guide downstream processing and evaluation.

## 2. Integration into Downstream Inference Pipelines

Once decomposed, sub-questions are incorporated into various pipeline architectures:
- **Sequential Answering & Aggregation**: Each $q_i$ is answered by a base model (single-hop QA, VLM, retriever, etc.), and their answers $\{a_i\}$ are recombined—either directly or via another LLM call that conditions on $Q$ plus the sub-answers [2510.08818][2507.00355][2305.14901][2307.11768].
- **Parallel or DAG Execution**: In the presence of compositional structure (e.g., DAGs in AGQA-Decomp [2204.07190] or QDTrees [2306.07597]), sub-questions may be answered in parallel or following dependency constraints, with answers passed according to the task’s composition rules.
- **Retrieval-Augmentation**: RAG and Graph-RAG approaches retrieve evidence not just for $Q$ but for each $q_i$, assembling complementary document (or triple) sets which are then reranked, merged, and used as input to answer synthesis [2410.15531][2507.00355][2510.07718][2603.07950].
- **Program Synthesis and Execution**: In numerical QA and text-to-SQL, sub-questions correspond to intermediate steps (e.g., specific joins, aggregates), forming interpretable programs that are incrementally constructed and executed [2310.13575][2603.07950].

The architectural interfaces vary. Some maintain a strict API- or prompt/LLM-driven separation (e.g., D-CoDe has zero architectural modification, using only prompt engineering [2510.08818]), while others opt for parameter sharing or explicit hard-EM/training objectives tying decomposition and answer generation [2305.14901][2211.03277].

## 3. Empirical Benefits and Evaluation

Across domains, sub-question decomposition yields significant gains:
- **Multi-hop QA (text, knowledge base, video)**: Explicit decomposition narrows errors in reasoning chains, exposes model shortcuts, and enables more robust multi-step inference. Empirical results on HotpotQA show +6–12 F1 points for decomposed pipelines over one-shot baselines; similar results hold for DROP, ComplexWebQuestions, AGQA, and KBQA tasks [2211.03277][2305.14901][2310.13575][2306.07597][2206.08486][2204.07190][2507.00355].
- **Retrieval Coverage and Precision**: In RAG, decomposing queries enables higher evidence recall (MultiHop-RAG MRR@10: +36.7%) and answer accuracy (F1: +11.6%) [2507.00355]. Classification of sub-questions by type refines evaluation—and direct optimization of “core” sub-question coverage increases win-rate by 74% over naive RAG [2410.15531].
- **Numerical Multi-table Reasoning**: Table-aligned decomposition in MTQA drives +24% recall and +55% answer gains over leading baselines [2603.07950].
- **Multimodal Reasoning**: In video and image + language domains, both deterministic and prompt-driven decomposition mitigates perceptual bottlenecks and token overload, yielding +6.2 points (EgoSchema) [2510.08818], and improves VQA accuracy by +5–23 points upon targeted finetuning [2409.19339][2204.00879].
- **Faithfulness and Interpretability**: By forcing explicit sub-question answering, decomposition increases sensitivity to step corruption, truncation, or bias, enhancing “faithfulness” metrics by 10–20 points over Chain-of-Thought or one-shot reasoning [2307.11768].

## 4. Representational and Structural Variants

Decomposition can yield:
- **Flat Sequences**: Ordered or unordered lists of sub-questions, suitable for pipeline or batch answering [2211.03277][2507.00355][2510.08818][2305.14901].
- **Trees/DAGs**: Structures which directly encode dependencies (QDTrees, AMR graphs, compositional DAGs in AGQA), enabling principled flow of information and explicit mapping to logical, programmatic, or KB queries [2306.07597][2206.08486][2204.07190].
- **Typed or Annotated Collections**: Taxonomically classifying sub-questions for downstream metric weighting, selective retrieval, or prioritization [2410.15531][2205.06938].

The decomposition can be deterministic (e.g., AMR-QDAMR graph segmentation), heuristic (hand-written splitting and templating [1906.02916][2002.09919]), or learned (via LLMs, step-wise prompting, or neural sequence models).

## 5. Practical Limitations and Challenges

Despite empirical robustness, sub-question decomposition introduces several costs and open problems:
- **Latency and Cost**: Each sub-question entails a separate model (often LLM) invocation, leading to inference time and compute increases of up to 5–6x vs. baseline [2510.08818][2507.00355][2410.15531].
- **Over-decomposition and Noise**: Generation can yield trivial, redundant, or off-topic sub-questions; noisy decompositions can propagate errors, particularly in open-ended or static queries [2510.08818][2410.15531].
- **Prompt and Model Sensitivity**: Performance and compositional coverage are heavily influenced by prompt design, LLM choice/hallucinations, and the absence of explicit loss functions or coverage objectives in many frameworks [2507.00355][2510.08818].
- **Ambiguity in Facet Coverage**: Determining when decomposition is necessary, how many sub-questions to generate, and weighting their importance remains challenging. Equal weighting may not reflect user or task priorities [2410.15531][2409.19339].
- **Domain Adaptation and Supervision**: Gold decompositions (for training or evaluation) are expensive; cross-domain generalization, especially for implied sub-questions or schema-sensitive decompositions, is imperfect without substantial annotated corpora [2205.06938][2211.03277][2310.13575].
- **End-to-end Trainability**: Many architectures remain non-differentiable or are not amenable to explicit backpropagation of downstream losses through decomposition—limiting the capacity to jointly optimize sub-question quality relative to final performance [2510.08818][2305.14901][2211.03277].

## 6. Theoretical Guarantees and Open Problems

Formally, for any composite task that admits a polynomial-depth, constant-fanin decomposition, intermediate supervision—i.e., training with sub-task labels—renders otherwise unlearnable problems tractable for standard sequence models (polynomial-time SGD convergence) [2204.02892]. This provides theoretical grounding for the empirical success of chain-of-thought and decomposed reasoning approaches, demonstrating why tasks that are end-to-end unlearnable (e.g., random parities, deep composition) become learnable when decomposed and supervised at finer granularity.

However, these results rely on full sub-task annotation, teacher forcing at training time, and the existence of low-degree decompositions. Robustness to noisy, incomplete, or model-generated sub-questions at test time, and extension to problems without such efficient decompositions, are active areas of research.

## 7. Domain-Specific Adaptations and Successes

Sub-question decomposition exhibits broad applicability:
- **Multimodal QA**: In video (D-CoDe, AGQA-Decomp) and multimodal LLMs (Co-VQA, DecoVQA+), structured decomposition mitigates perception bottlenecks and supports variable-length, adaptive reasoning chains [2510.08818][2204.07190][2409.19339][2204.00879].
- **Fact Verification & Evidence Aggregation**: Generating explicit, minimal yes/no sub-questions, both literal and implied, enhances diagnosticity, evidence retrieval, and interpretability in complex claim verification (ClaimDecomp) [2205.06938].
- **Program Synthesis and Table QA**: Schema-sensitive, operator-based decomposition (QPL in text-to-SQL, DMRAL for multi-table QA) provides pipeline modularity, interpretability, and controllable complexity, with concomitant accuracy and robustness gains [2310.13575][2603.07950].
- **Knowledge Base QA**: Tree-based or computation-graph-based decompositions (QDT, RL ordering with full compositional trees) enable tractable, explainable multi-hop inference even over large KGs [2306.07597][1911.04065].

In all domains, decomposition is both a tool for engineering more robust models and a critical axis for evaluating compositional generalization, logical faithfulness, and coverage. The future trajectory will likely feature more integrated, learnable, and domain-adaptive decomposition modules, new evaluation metrics favoring compositional transparency, and hybrid architectures balancing explicit structure with the fluency of large generative models.

Source: https://www.emergentmind.com/topics/sub-question-decomposition