Composite Reasoning in AI

• Composite Reasoning is a type of problem-solving that composes atomic elements to infer reliable conclusions across vision and language tasks.
• It integrates multiple strategies—including deductive, inductive, abductive, causal, and decompositional reasoning—to address complex compositional challenges.
• Research shows that despite progress in benchmarks and training protocols, CR still faces significant performance drops and interpretability challenges.

Composite Reasoning (CR) denotes a family of reasoning problems in which success depends on composing atomic elements rather than relying on isolated recognition, memorized associations, or a single fixed inference pattern. In vision–language settings, compositional reasoning entails grasping the significance of attributes, relations, and word order, so that a model must prefer a true caption or answer over perturbed alternatives that replace, swap, or permute atomic concepts (Huang et al., 2024). In language-only settings, complementary or composite reasoning has been defined as answering questions that require both internal parametric knowledge and external contextual information (Cheng et al., 1 Dec 2025), while another line treats CR as a meta-reasoning strategy that combines deductive, inductive, abductive, causal, and decompositional styles within a single trajectory (Ahmad et al., 26 Sep 2025). A mechanistic formulation, compositional relational reasoning, further casts CR as chaining elementary relations such as $r_{\mathrm{retrieve}}\circ r_{\mathrm{lookup}}$ to infer a novel conclusion (Ni et al., 2024).

1. Definitions and conceptual scope

In the ConMe formulation for modern vision–LLMs (VLMs), compositional reasoning is the ability to identify atomic visual concepts, attend to their attributes, understand pairwise or higher-order relations, and respect word-order constraints. The canonical examples are sensitivity to attribute substitutions, sensitivity to relational flips, and insensitivity to mere word co-occurrence. The formal criterion is that, for an image $I$ and text $T$, a model succeeds when it correctly prefers $T$ over a perturbed $T'$ in which one or more atomic concepts are replaced, swapped, or permuted (Huang et al., 2024).

In Cheng et al.’s complementary or composite reasoning setting, the decomposition is explicit: Parametric Reasoning relies on knowledge stored in model parameters, Contextual Reasoning depends on facts provided in the context window, and Complementary Reasoning requires both. A question is generated from a relational path whose hops are labeled “Mem” or “Ctx,” and a competent model must traverse the path step by step, retrieving parametric facts when required and reading contextual facts when required (Cheng et al., 1 Dec 2025).

In the “Thinking in Many Modes” formulation, Composite Reasoning is a meta-reasoning strategy over a set of reasoning styles $$S=\{\text{deductive, inductive, abductive, causal, decompositional}\}$$. A reasoning trajectory is generated under a possibly changing style assignment, and the prompt explicitly instructs the model to “think in many modes” by interleaving hypothesis generation, generalization, rule application, causal analysis, and decomposition (Ahmad et al., 26 Sep 2025).

Compositional relational reasoning provides a still narrower definition. In GAR, the task is a two-hop relation composition problem: a model must use one semantic relation to locate an intermediate entity and a second relation to retrieve its attribute, yielding $$A=(r_{\mathrm{retrieve}}\circ r_{\mathrm{lookup}})(Q)$$. This isolates composition itself from broader language competence and makes the task suitable for mechanistic interpretability study (Ni et al., 2024).

This suggests that CR is not a single standardized task class. Rather, the current literature uses the label for several technically distinct but structurally related problems in which the central requirement is reliable composition of primitives.

2. Benchmarking regimes and task constructions

Current CR research uses both naturalistic and synthetic benchmarks. Naturalistic benchmarks emphasize realistic inputs and difficult negatives, whereas synthetic benchmarks enforce strict control over relations, paths, and leakage.

Formulation	Benchmark or data	Defining characteristics
ConMe (Huang et al., 2024)	919 MS-COCO val images; 24,347 QA pairs	replace-att, replace-obj, replace-rel; 1,000-item Manual Subset
GAR (Ni et al., 2024)	384 tasks; 4,608 examples	two-hop lookup–retrieve; generation and classification
Synthetic biographies (Cheng et al., 1 Dec 2025)	39 relations; 10,000 Mem and 10,000 Ctx biographies	2–5-hop paths; I.I.D., Composition, Zero-shot
Sparkle (Tang et al., 2024)	34 K synthetic samples	train on direction, distance, localization; test on SPP and TSP

ConMe was introduced in response to the claim that existing CR benchmarks such as Winoground, ARO, CREPE, and SugarCrepe rely on LLM-only negative generation. The stated failure modes are negatives that lie outside the decoder’s learned natural-language distribution or negatives that are linguistically plausible yet implausible in the image context. ConMe therefore uses VLMs conversing with each other to generate, evaluate, and select hard CR Q&A, and it further reports a manually filtered 1,000-item subset after finding approximately 20% minor errors in a random sample of 1,000 items (Huang et al., 2024).

GAR was designed to be “challenging enough for existing LLMs” while also being “easy enough for systematic MI study.” Its task space is controlled by the number of distractor key–value pairs, semantic variations such as negation and generation-to-classification conversion, and syntactic variations such as swapQA and swapKV. The benchmark defines a compositionality gap $$G=\mathrm{Acc}_{\mathrm{sub}}/\mathrm{Acc}_{\mathrm{comp}}$$, where large $G>1$ indicates that subproblems are solved in isolation but their composition fails (Ni et al., 2024).

The synthetic biography benchmark in Cheng et al. is expressly constructed to avoid pretraining contamination. It uses a controlled relational knowledge graph with 39 relations, two disjoint relation partitions for Mem and Ctx, and three evaluation regimes: I.I.D. Generalization, Compositional Generalization, and Zero-shot Structural Generalization. This design makes it possible to distinguish in-distribution competence from genuine out-of-distribution composition (Cheng et al., 1 Dec 2025).

Sparkle applies the same benchmark philosophy in multimodal spatial reasoning. It disentangles direction comprehension, distance estimation, and localization, then evaluates whether training only on these basic spatial capabilities transfers to composite tasks such as the Shortest Path Problem and the Traveling Salesman Problem (Tang et al., 2024).

3. Algorithmic frameworks and training protocols

ConMe’s core algorithm is a multi-stage generator–responder pipeline. GPT-4V acts as generator, while LLaVA-1.5, LLaVA-1.6, InstructBLIP-Flan-T5, and InstructBLIP-Vicuna-7b serve as responders and filters. The pipeline first collects GPT-4V and responder descriptions of each image, then generates an initial batch of questions, filters out items that all responder models answer correctly, gathers open-ended answers from the responders, and performs a second round of QA generation with the instruction to “make them harder.” A candidate’s difficulty can be written as one minus the average responder accuracy, and items with zero difficulty are removed (Huang et al., 2024).

Cheng et al. study a two-stage post-training pipeline on Qwen-2.5-1.5B. Supervised fine-tuning is applied either to atomic tasks only, $\mathrm{SFT}_{\mathrm{Mem+Ctx}}$, or directly to complementary tasks, $I$0. Reinforcement learning then initializes a policy from a frozen SFT checkpoint and optimizes exact-match final-answer reward with Group Relative Policy Optimization (GRPO; Shao et al., 2024). The central experimental question is whether RL merely amplifies existing behaviors or synthesizes new composite strategies (Cheng et al., 1 Dec 2025).

In the many-modes formulation of Composite Reasoning, the base model is Qwen-2.5-7B-Instruct augmented with LoRA adapters. The adapters have rank $I$1 and $I$2, and they are inserted into $I$3, $I$4, $I$5, $I$6, $I$7, $I$8, and $I$9. Training proceeds in three phases: generation of CR trajectories from a prompt template, supervised fine-tuning on those trajectories, and outcome-based GRPO tuning with binary exact-match reward (Ahmad et al., 26 Sep 2025).

Two adjacent uses of the acronym are methodologically relevant because they also operationalize composition over intermediate reasons. Combinatorial Reasoning is a fully automated prompting framework that samples candidate reasons from an LLM, embeds them in a 768-dim sentence-transformer space, deduplicates them with cosine threshold $T$0, maps selection into a Quadratic Unconstrained Binary Optimization problem, and uses an Ising-style solver before constructing a weighted Chain-of-Thought prompt (Esencan et al., 2024). Cumulative Reasoning defines an iterative procedure in which a Proposer, Verifier(s), and Reporter build a directed acyclic graph of validated propositions, with the validated set growing monotonically until a goal condition is met (Zhang et al., 2023).

4. Empirical performance and generalization behavior

ConMe reports a substantial drop in performance relative to prior compositional reasoning benchmarks. Across seven VLMs, the mean accuracy falls from 88.6 on SugarCrepe to 65.3 on ConMe and 64.8 on the Manual Subset. The four models seen during curation—LLaVA 1.5-7b, LLaVA 1.6-7b, InstructBLIP Flan-T5, and InstructBLIP Vicuna-7b—each suffer a 29–33% drop. The two unseen models, InternLM-XComposer2-7b and Idefics2-8b, show 12–15% drop, and GPT-4V itself drops 11.2%. Paired $T$1-tests yield $T$2 for all seen models (Huang et al., 2024).

The complementary reasoning study reports a different failure mode. $T$3 SFT alone achieves 35.18 on I.I.D., 28.20 on Compositional, and 24.07 on Zero-shot evaluation of the complementary task. Direct $T$4 SFT reaches 90.30 on I.I.D. and 76.25 on Compositional evaluation but collapses to 18.41 on Zero-shot. This behavior is named the “SFT Generalization Paradox.” RL initialized from $T$5 yields large absolute improvements, especially in Zero-shot settings, whereas RL on top of $T$6 yields negligible gains (Cheng et al., 1 Dec 2025).

The many-modes Composite Reasoning framework reports consistent gains under the combined SFT+GRPO setting. On ARC-Complex, MedMCQA, and MedXpertQA, CR reaches 94.99%, 56.30%, and 15.90%, respectively. The same study states that CR prompt-only performance is 83.10% on ARC-Complex, 54.62% on MedMCQA, and 7.8% on MedXpertQA, and that CR adaptively emphasizes abductive and deductive reasoning for medical QA while prioritizing causal, deductive, and inductive reasoning for scientific QA (Ahmad et al., 26 Sep 2025).

Sparkle provides a curriculum-based result in multimodal reasoning. After fine-tuning only on direction, distance, and localization, InternVL2-8B improves on direction from 35.0 to 83.0, on distance from 44.5 to 84.0, and on localization from 60.5 to 73.0. On composite tasks, the same model improves on SPP 4×4 from 16.5 to 36.5, on SPP 5×5 from 13.5 to 40.0, on TSP 4 points from 17.5 to 20.0, and on TSP 5 points from 11.5 to 14.5. The reported out-of-distribution spatial benchmarks improve by up to +140% on hard 2-object queries (Tang et al., 2024).

Cumulative Reasoning reports broad gains across logic, arithmetic search, and mathematics. On FOLIO-wiki with GPT-4, CR reaches 87.45% test accuracy, and on the curated subset it reaches 98.04%. On the Game of 24, CR reaches 98% accuracy with 14.9 visited states versus 74% and 61.7 visited states for Tree of Thoughts at the same breadth. On MATH with code, CR reaches 72.2%, exceeding PAL at 51.8 and ToRA at 61.6 (Zhang et al., 2023). Combinatorial Reasoning, in turn, improves BIG-Bench-Hard average accuracy from 47.7% zero-shot to 59.9% with CR, although 3-shot CoT remains higher at 74.2% (Esencan et al., 2024).

5. Theoretical limits and mechanistic accounts

A formal complexity-theoretic treatment appears in the study of Compositional Reasoning Questions (CRQ). There, CRQs are tree-structured multi-step reasoning problems that capture tasks such as Boolean formula evaluation and multi-step word problems. Under the assumption $T$7, no constant-depth transformer with polynomial size and $T$8 bit precision can solve all CRQs of size $T$9. For RNNs, an adversarial input ordering yields an $T$0 hidden-memory lower bound, whereas a carefully chosen memory-rank sort permits a 5-layer RNN with hidden dimension $T$1 to solve all CRQs on $T$2-node binary trees. For transformers with chain-of-thought, a 2-layer transformer with embedding dimension $T$3 and $T$4 CoT tokens can solve every CRQ of size $T$5 (Yehudai et al., 3 Mar 2025).

These results characterize a resource trade-off rather than a single dominant architecture. Transformers trade depth for parallelism, RNNs trade parallelism for compact memory, and shallow transformers with CoT trade token efficiency for architectural simplicity. The paper’s conclusion is explicit: none of the architectures is strictly better than the others, and each requires at least one resource to grow with problem size (Yehudai et al., 3 Mar 2025).

Mechanistic interpretability work on GAR identifies specific attention heads associated with successful relational composition. In Vicuna-33B, attribution patching reveals a truthfulness sub-circuit with True heads $T$6 and False heads $T$7. These heads write a binary signal into the residual stream at the answer token and their activations separate true from false statements. Intervention experiments show that enforcing correct attention patterns in pre-predicting and predicting heads boosts Vicuna-33B’s accuracy by up to +20 percentage points on the hardest tasks, while patching True/False heads improves Vicuna-7B by +17 points, Vicuna-13B by +14 points, and Vicuna-33B by +6 points (Ni et al., 2024).

This mechanistic evidence supports a circuit-level view of CR: relating heads detect semantic matches, higher-order heads collapse them into a truth-value representation, induction-style heads carry demonstration signals in in-context learning, and predicting heads emit the final attribute (Ni et al., 2024).

6. Limitations, debates, and research directions

A central debate concerns whether apparent CR competence reflects genuine composition or benchmark weakness. ConMe argues that prior VLM benchmarks had become too easy because their negatives were produced by an LLM-only pipeline and therefore were either linguistic outliers or image-context outliers. Its reported “up to 33%” decrease in performance is presented as evidence that the challenge had not been solved but partially obscured by benchmark design (Huang et al., 2024).

A second debate concerns the role of reinforcement learning. Cheng et al. explicitly frame the issue as “reasoning synthesizer” versus “probability amplifier.” Their conclusion is that RL can synthesize complex strategies, but only after a strict atomic prerequisite has been satisfied through supervised fine-tuning of both Parametric and Contextual reasoning. Baselines with Mem-only SFT, Ctx-only SFT, or shallow composite SFT do not benefit from the same RL stage (Cheng et al., 1 Dec 2025).

Several studies also delimit the scope of current gains. The many-modes CR framework relies on high-quality CR-style exemplars and does not explicitly model how to weight each reasoning style or when to switch between them (Ahmad et al., 26 Sep 2025). Sparkle is restricted to toy 2D diagrams, and its gains are smaller on NP-hard TSP instances than on shortest-path tasks (Tang et al., 2024). Cumulative Reasoning incurs latency and prompt-engineering costs because each iteration invokes multiple role-specific model calls (Zhang et al., 2023).

A more radical response to CR failure is to change the model’s representational geometry rather than only its search procedure. Recursive Concept Evolution argues that token-level search methods such as chain-of-thought prompting, self-consistency, or reinforcement learning leave the latent space fixed and therefore fail when the needed abstraction is not already encoded. RCE introduces dynamically generated low-rank concept subspaces, selected through a minimum description length criterion, merged when synergistic, and consolidated through constrained optimization. Integrated with Mistral-7B, it reports 12–18 point gains on ARC-AGI-2, 8–14 point improvements on GPQA and BBH, and a flat depth-induced error curve on MATH out to 15 steps, with less than 5% FLOPs overhead (Chaudhry, 17 Feb 2026). A plausible implication is that some CR failures are representational rather than purely inferential.

Across these lines of work, the most stable conclusion is that composite reasoning remains an open problem. The current literature converges on several recurring requirements: hard and image-aware or path-aware benchmarks, explicit decomposition into atomic skills, training protocols that separate atomic mastery from composition, and analyses that probe not only accuracy but also the internal mechanisms by which composition is implemented (Huang et al., 2024).