--- title: Question Decomposition Strategy url: https://www.emergentmind.com/topics/question-decomposition-strategy type: topic --- # Question Decomposition Strategy A question decomposition strategy is a set of algorithmic and architectural techniques that divides a complex, multi-step, or compositional question into simpler sub-units—typically simpler sub-questions or structured fields—whose answers or representations can be more efficiently and reliably processed, and then recombined to obtain the final answer. Such strategies play a pivotal role in question answering (QA) systems that must resolve reasoning paths across unstructured text, structured knowledge bases, or multimodal data. Question decomposition enables these systems to address compositionality, improve interpretability, boost faithfulness of explanation, and in certain data regimes, substantially improve accuracy. ## 1. Principles and Formal Definitions A question decomposition strategy always involves (at least) two mappings: - **Decomposition**: A function (or sequence transducer) that takes a complex input question $Q$ and outputs a sequence or graph-structured set of simpler units (e.g., $D(Q) = \{q_1,\ldots, q_n\}$ for subquestions, or a tree/graph for hierarchical decompositions). - **Composition**: A second function $C$ that aggregates the answers (or intermediate representations) from sub-units back into a final answer (typically $A = C(a_1, \ldots, a_n)$). Depending on the setting and modality, decomposition may be explicit (human annotation, model output as text, or structural representation) or implicit (latent fields, operator tokens, attention distributions). Some representative formalizations include: - Explicit decomposition: $D: Q \mapsto \langle q_1, ..., q_k\rangle$, with $a_i = \operatorname{QA}(q_i, \cdot), \quad \hat{a} = C(a_1, ..., a_k)$ [2212.10019]. - Field-based structural decomposition: Each question and document is represented as $n$ aligned fields $q = \{q_1, ..., q_n\}$, $d^t = \{d_1^t, ..., d_n^t\}$, scored via per-field retrieval and weighted sum aggregation [1604.00938]. - Hierarchical/graph/tree-based decompositions: E.g., question decomposition trees (QDTs) [2306.07597], hierarchical question decomposition trees (HQDTs) [2305.15056]. - Mixed-modality extensions: Visual question decomposition with sub-question generation and selective composition [2409.19339], [2310.17050]. ## 2. Algorithmic and Model Strategies Contemporary decomposition methodologies can be categorized broadly as follows: **A. Structural/Linguistic Decomposition** Deploying NLP techniques (POS, dependency, SRL, coreference) to map questions into fields or mini-documents that are then indexed, scored, and recomposed using statistical or search-based frameworks [1604.00938]. Each field corresponds to a bag-of-terms (lexical, syntactic, semantic), and retrieval incorporates per-field TF-IDF scoring, weighted by learnable coefficients. **B. Learned Neural Decomposition** Trainable neural modules (pointer networks, seq2seq LMs, BART/T5/Flan-T5-based decomposers) generate either explicit sub-questions (as text) or intermediate operator chains. Learning may be fully supervised [2211.03277, 1906.02916], weakly supervised via hard-EM or unsupervised round-trip objectives [2002.09758], or involve distant/pretrain signal mining from comparable texts [2210.16865]. **C. Graph/AMR-Based Decomposition** Mapping questions to semantic (AMR) or logical (SPARQL, SQL, QPL) graphs, then partitioned via graph segmentation or operator mapping to produce sub-questions tied to entities or compositional structures. Subsequent sub-question generation uses graph-to-text models [2206.08486, 2310.13575]. **D. Hierarchical and Tree-Based Methods** Modeling decompositions as trees (e.g., QDT, HQDT) that organize sub-questions via composition, conjunction, and constraints. Two-stage neural architectures (“Clue-Decipher,” BART-based decomposers) first hypothesize separator placements and then align to the question, ensuring coverage of all original tokens and compositional types [2306.07597, 2305.15056]. **E. Prompt-based, In-Context and Retrieval-Augmented Decomposition** Zero- or few-shot in-context learning to induce decomposition ability from existing datasets or tasks (e.g., ICAT via Frechet Term-Embedding Distance for exemplar selection [2310.18371]). Retrieval-augmented pipelines intertwine LLM-driven decomposition with document or passage retrieval, followed by answer composition after cross-encoder reranking [2507.00355]. **F. Reward-Shaped and Selective Decomposition** Fine-tuning LLMs to pose only those sub-questions that provably improve downstream QA performance, with reward functions based on answer flipping and preservation [2510.01526]. Selective decomposition mechanisms decide, per question, whether decomposition is needed, achieving substantial improvements in multimodal and medical VQA [2409.19339, 2310.17050]. ## 3. Applications Across QA Modalities The decomposition paradigm is highly modular, enabling its application across textual, tabular, visual, and knowledge-base QA. - **Open-Domain and Multi-Hop Textual QA**: Decomposition recasts multi-hop queries into factored chains, generally producing large gains in supporting-fact retrieval, multi-hop reasoning, and end-to-end F1 under low- and medium-data regimes [1906.02916, 2212.10019, 2211.03277, 2002.09758]. - **Knowledge Base QA (KBQA)**: Tree-based and QPL decomposition strategies map complex compositions into sequences of SPARQL or SQL program steps, often yielding performance boosts and improved interpretability [2306.07597, 2310.13575]. - **Quantitative Reasoning / Financial QA**: Reward-pruned, domain-specific decomposition can focus LLM attention on a single crucial supporting sub-question (average 1.2 per QA) and surpass long CoT chains, both in accuracy and inference efficiency [2510.01526]. - **Multimodal / Visual QA**: Visual QA models benefit from decomposing high-level, semantic questions into a blend of low-level perceptual sub-questions and conceptual reasoning steps; selective decomposition policies further increase SOTA accuracy, especially in domain-specialized and low-data regimes [2409.19339, 2310.17050]. ## 4. Effectiveness, Empirical Results, and Limitations Question decomposition can dramatically improve several key metrics, often dependent on the regime: | Setting/Strategy | Typical Gain | Notes | |----------------------------------|------------------|-----------------------------------------| | Structural fields (bAbI, all λ) | +>40 pp MAP | Multi-field, learned weights [1604.00938]| | Hard compositional RC (DROP, DecompRC) | +6–7 F1 | Supervised pointer-based [1906.02916] | | Weakly/unsupervised multi-hop (ONUS, QDAMR) | +2–8 F1 | No gold decompositions [2002.09758, 2206.08486] | | RAG multi-hop retrieval (QD+RR) | +11.6% F1 / +36% MRR | Zero-shot, passage-level [2507.00355] | | Financial QA (EQD) | +0.6–10.5% EM | Only 1–1.2 sub-questions generated [2510.01526]| | Zero-shot VQA, selective-decomp. | +6–26 pp acc. | Largest gains in medical/art tasks [2310.17050]| Major considerations and limitations identified: - Decomposition improves performance most in low-data or high-compositionality regimes. As labeled data increases, end-to-end seq2seq models can implicitly learn decompositions through attention and hidden representations [2212.10019]. - Over-decomposition (producing many sub-questions when a single supporting fact suffices) may harm performance due to error propagation or retrieval overhead [2510.01526, 2507.00355]. - Faithfulness improves substantially: factored decomposition can force LLMs to actually condition the answer on sub-answers, reducing “post-hoc rationalization” and hidden sycophancy compared to Chain-of-Thought [2307.11768]. - Quality of sub-question generation is critical; annotation or data refinement on repetition, groundedness, and relevance is central in multimodal settings [2409.19339]. - Error propagation and ill-posed sub-questions remain a concern in explicit explicit or unsupervised chains [2212.10019, 2205.12538]. - For knowledge-integration tasks, hierarchical structures (trees) provide more robust fusion of heterogeneous sources than flat lists [2305.15056]. ## 5. Interpretability, Faithfulness, and Explainability Decomposition strategies provide intrinsic interpretability—each sub-question constitutes an explicit, human-auditable reasoning step. This property underpins both user trust and fault localization in complex QA systems. - Factored and field-based decompositions expose intermediate structures (fields, trees, graphs) that clarify decision provenance [1604.00938, 2306.07597, 2305.15056]. - Explicit answer composition footsteps (e.g., in RoHT/HQDT) allow probabilistic merging of KB-supported, text-supported, and recursively derived answers, with global likelihoods available for each branch [2305.15056]. - Faithfulness metrics—including truncation sensitivity, corruption sensitivity, and bias-induced accuracy drop—objectively quantify how tightly final predictions depend on generated reasoning, highlighting the superiority of decomposition over CoT on these axes [2307.11768]. ## 6. Practical Guidance and Future Directions - **When Decomposition Helps**: Most beneficial in few-shot, zero-shot, or cross-domain adaptation settings; for difficult, compositional, or multi-hop questions; and for retrieval-augmented or knowledge-integration tasks [2212.10019, 2507.00355]. - **Modularity and Transfer**: Decomposers can be trained or selected via in-context transfer (e.g., ICAT), compositional data augmentation, or reward shaping, with plug-and-play architectures in most modern LLM or RAG pipelines [2310.18371, 2510.01526, 2507.00355]. - **Extension to New Modalities**: Finetuned multimodal models (VQD) and multimodal knowledge integration (HQDT, selective decomposition) represent state-of-the-art modular designs for explainable VQA and XQA [2409.19339, 2305.15056]. - **Limitations and Cautions**: Careful calibration is required to avoid over-decomposition and error cascades; joint training and dynamic decomposition length selection are open areas for research. Sub-question generation quality, particularly in highly compositional or domain-specific settings, is pivotal. A plausible implication is that as LLM reasoning abilities improve and implicit decomposition becomes more salient in deep models, the value of explicit decomposition may further shift toward interpretability, reliability, and modularity, rather than pure accuracy, especially in mission-critical or high-risk domains. --- **References**: - [1604.00938] Multi-Field Structural Decomposition for Question Answering - [2510.01526] One More Question is Enough, Expert Question Decomposition (EQD) Model for Domain Quantitative Reasoning - [2212.10019] When Do Decompositions Help for Machine Reading? - [2002.09758] Unsupervised Question Decomposition for Question Answering - [2211.03277] Complex Reading Comprehension Through Question Decomposition - [2306.07597] Question Decomposition Tree for Answering Complex Questions over Knowledge Bases - [2305.15056] Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering - [2507.00355] Question Decomposition for Retrieval-Augmented Generation - [2310.13575] Semantic Decomposition of Question and SQL for Text-to-SQL Parsing - [2409.19339] Visual Question Decomposition on Multimodal Large Language Models - [2310.17050] Exploring Question Decomposition for Zero-Shot VQA - [2210.16865] Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts - [2310.18371] In-Context Ability Transfer for Question Decomposition in Complex QA - [1906.02916] Multi-hop Reading Comprehension through Question Decomposition and Rescoring - [2307.11768] Question Decomposition Improves the Faithfulness of Model-Generated Reasoning - [2205.12538] Is a Question Decomposition Unit All We Need? - [2206.08486] Interpretable AMR-Based Question Decomposition for Multi-hop Question Answering - [2101.05988] Coarse-grained decomposition and fine-grained interaction for multi-hop question answering Source: https://www.emergentmind.com/topics/question-decomposition-strategy