Compositional Complexity: Theory & Practice

• Compositional Complexity is the study of constructing global phenomena from nested, interacting components, characterized by layered processing, interface constraints, and hierarchical structure.
• It spans diverse domains such as decision-tree analysis, optimization, machine learning architectures, and materials science, illustrating practical system decomposition.
• The framework informs efficient design and analysis by quantifying trade-offs between compositional and monolithic approaches in both algorithmic and physical systems.

Compositional complexity denotes the complexity induced by constructing a global object from interacting parts, but the literature uses the term in several technically distinct senses. In decision-tree complexity it refers to exact laws for composites of the form $h=g(f^1,\ldots,f^n)$ (Montanaro, 2013); in Boolean lower bounds it measures how many $k$-local summaries are needed to realize majority (Lecomte et al., 2022); in optimization it names objectives such as $h(g(x))$ and finite-sum analogues (Lin et al., 2018); in contemporary machine learning it can refer to hierarchical computation structure, multilevel novelty, or the number of atomic capabilities required by a task (Ram et al., 2024, Li et al., 2024, Wu et al., 30 Apr 2025); and in materials science it describes multicomponent chemistry, defect compensation, and near-atomic-scale heterogeneity (Krämer et al., 21 Jan 2026, Li et al., 26 Mar 2026). Taken together, these uses suggest a common theme: compositional complexity concerns the costs, constraints, and structural consequences of mediating a global phenomenon through intermediate components rather than treating it monolithically.

1. Exact composition theorems in query and Boolean complexity

In deterministic decision-tree complexity, compositional complexity is characterized exactly for a broad class of boolean-valued inner relations. If $g \subseteq \{0,1\}^n \times Z$, each $f^i \subseteq X^{m_i}\times\{0,1\}$ is non-trivial, and $h=g\circ(f^1,\ldots,f^n)$, then

$D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$

where $\bar g$ is obtained by substituting any constant inner relations into $g$ (Montanaro, 2013). This is an exact formula, not merely an upper or lower bound, and it shows that in the boolean-inner regime the composite problem is completely characterized by weighted outer complexity. The same paper derives $D(g\circ(f,\ldots,f))=D(f)D(g)$, $k$0, and a deterministic direct-sum theorem $k$1 for $k$2 (Montanaro, 2013).

The restriction to boolean-valued inner objects is essential. The paper gives counterexamples showing that nonboolean inner outputs can expose richer structure than a single interface bit, so naive multiplication or black-box substitution fails (Montanaro, 2013). A second line of work makes the same point from the opposite direction: in the local-function composition model

$k$3

where each $k$4 is $k$5-local and $k$6 is arbitrary, majority provably requires a superconstant composition overhead. The optimal lower bound is

$k$7

for $k$8, with the more general form

$k$9

and a matching upper bound in the same regime (Lecomte et al., 2022). Here the ideal baseline $h(g(x))$0 is unattainable: majority needs $h(g(x))$1 extra local summaries per block. The proof proceeds through sharp lower bounds for Hamming weight, information-loss estimates through the inner functions $h(g(x))$2, and a bootstrap from a multi-output function to a single-output one (Lecomte et al., 2022).

These results formalize two complementary facts. In some models the cost of composition collapses to a clean weighted outer problem; in others, even an unrestricted outer combiner cannot recover information discarded by too-aggressive local compression. This suggests that compositional complexity is often controlled by the informational sufficiency of the interface between inner and outer levels.

2. Compositional optimization and oracle complexity

In optimization, compositional complexity refers to objectives whose gradient structure is harder than that of ordinary stochastic optimization because the gradient of a composition requires both the inner map and its Jacobian. A canonical finite-sum problem is

$h(g(x))$3

for which

$h(g(x))$4

The difficulty is that one must estimate the inner average $h(g(x))$5, the inner Jacobian $h(g(x))$6, and the outer gradient simultaneously (Lin et al., 2018). For the convex finite-sum regime, SCVRG achieves

$h(g(x))$7

sample complexity, improving the dependence on $h(g(x))$8 up to a logarithmic factor (Lin et al., 2018).

For non-convex finite-sum compositions

$h(g(x))$9

STORM-Compositional tracks three recursive estimators—$g \subseteq \{0,1\}^n \times Z$0 for $g \subseteq \{0,1\}^n \times Z$1, $g \subseteq \{0,1\}^n \times Z$2 for $g \subseteq \{0,1\}^n \times Z$3, and $g \subseteq \{0,1\}^n \times Z$4 for the compositional gradient—and obtains IFO complexity

$g \subseteq \{0,1\}^n \times Z$5

for producing $g \subseteq \{0,1\}^n \times Z$6 with $g \subseteq \{0,1\}^n \times Z$7 (Yuan et al., 2020). Its contribution is not a better asymptotic exponent than the best prior result, but a single-loop recursive-momentum scheme that avoids checkpointing and large/small batch alternation (Yuan et al., 2020).

A distinct deterministic line studies non-smooth, non-convex compositional objectives with additional structure. For

$g \subseteq \{0,1\}^n \times Z$8

where $g \subseteq \{0,1\}^n \times Z$9 is smooth and $f^i \subseteq X^{m_i}\times\{0,1\}$0 is Lipschitz, non-smooth, non-convex, and proximable, a smoothing compositional gradient method finds a chain-rule $f^i \subseteq X^{m_i}\times\{0,1\}$1-stationary point in

$f^i \subseteq X^{m_i}\times\{0,1\}$2

iterations (Yao et al., 2024). For the DC-structured class

$f^i \subseteq X^{m_i}\times\{0,1\}$3

with convex Lipschitz $f^i \subseteq X^{m_i}\times\{0,1\}$4 and smooth $f^i \subseteq X^{m_i}\times\{0,1\}$5, a prox-linear method finds a nearly $f^i \subseteq X^{m_i}\times\{0,1\}$6-critical point in

$f^i \subseteq X^{m_i}\times\{0,1\}$7

iterations (Yao et al., 2024). In this literature, compositional complexity is therefore primarily algorithmic: it is the extra burden imposed by nested structure on gradient estimation, stationarity notions, and oracle complexity.

3. Hierarchical computation and compositional generalization

A theoretical account of compositional functions for sequence models defines

$f^i \subseteq X^{m_i}\times\{0,1\}$8

where $f^i \subseteq X^{m_i}\times\{0,1\}$9 is a token encoder, $h=g\circ(f^1,\ldots,f^n)$0 is a computation DAG, $h=g\circ(f^1,\ldots,f^n)$1 is a recursively reused span processor, and $h=g\circ(f^1,\ldots,f^n)$2 is a final read-out (Ram et al., 2024). The corresponding compositional complexity is measured through the locus of influence: $h=g\circ(f^1,\ldots,f^n)$3 where $h=g\circ(f^1,\ldots,f^n)$4 is the set of all source-to-sink paths from token $h=g\circ(f^1,\ldots,f^n)$5 (Ram et al., 2024). This yields architecture-dependent complexity profiles. A balanced binary-tree recurrence has

$h=g\circ(f^1,\ldots,f^n)$6

whereas a dense transformer with $h=g\circ(f^1,\ldots,f^n)$7 blocks has

$h=g\circ(f^1,\ldots,f^n)$8

(Ram et al., 2024). The same framework proves sensitivity bounds, an input-agnostic versus input-dependent expressivity gap, and systematic generalization guarantees whose dependence on $h=g\circ(f^1,\ldots,f^n)$9 is explicit (Ram et al., 2024).

A separate benchmark-oriented literature treats compositional complexity as hierarchical novelty. One proposal distinguishes word-word, phrase-word, and phrase-phrase levels, and defines “consistency” as simultaneous generalization across a phrase-phrase composition and the derived phrase-word and word-word compositions (Li et al., 2024). The associated framework partitions training data by a longest-phrase-length proxy, introduces multiple meta-weight-nets, and optimizes them sequentially from simple to complex in a multilevel optimization procedure (Li et al., 2024). On the GQA-CCG dataset, this improves not only levelwise accuracies but also the stricter triplet consistency score, supporting the claim that compositional complexity is structured internally rather than being a single undifferentiated OOD phenomenon (Li et al., 2024).

This body of work reframes compositional complexity as a property of computation graphs and generalization regimes. The central issue is not only whether a model composes, but whether its induced composition structure matches the latent structure of the task.

4. Capability-count, architectural, and solution-complexity views in modern machine learning

Several recent empirical papers treat compositional complexity as an explicitly controllable design variable. In COMPACT, a visual-language task $D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$0 is assigned a capability set $D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$1, and the number $D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$2 of required atomic visual capabilities is defined as its compositional complexity (Wu et al., 30 Apr 2025). The paper introduces 10 atomic capabilities, balances training data across $D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$3, and evaluates on benchmark questions whose inferred complexity extends to $D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$4 and $D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$5 (Wu et al., 30 Apr 2025). The reported finding is that balancing training by compositional complexity, rather than relying on scale alone, improves performance especially on multi-capability questions; for example, at $D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$6, COMPACT reports 57.5 versus 32.5 on MM-Vet and 64.7 versus 35.3 on MMStar relative to full LLaVA-665K VIT (Wu et al., 30 Apr 2025).

In language-conditioned reinforcement learning, CERLLA treats compositional complexity as an architectural and empirical problem rather than a formal theorem. Tasks are generated from colors, object types, and Boolean operators, yielding 162 unique tasks in an augmented BabyAI/MiniGrid environment (Cohen et al., 21 Jan 2025). CERLLA pretrains nine basis World Value Functions, one for each attribute, and composes them with Boolean operations: $D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$7 while an LLM-based parser maps instructions to Boolean expressions over abstract symbols (Cohen et al., 21 Jan 2025). The method reaches 92% success, matching the oracle upper bound, whereas the non-compositional baseline reaches about 80% under the same total environment-step budget (Cohen et al., 21 Jan 2025). The paper explicitly presents this as a reduction in sample complexity through reusable policy structure.

A third strand studies the complexity of the learned solution itself. On synthetic compositional tasks, smaller initialization scale and larger weight decay bias Transformers toward reasoning-based solutions rather than memory-based solutions (Zhang et al., 15 Jan 2025). The reported indicators of lower solution complexity include lower stable rank,

$D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$8

stronger neuron condensation in first-layer query weights, and internal masked-circuit behavior consistent with computing primitive anchor-level rules rather than memorizing composite mappings (Zhang et al., 15 Jan 2025). This suggests that compositional complexity in model behavior can refer not only to task structure but also to the effective complexity of the representation learned by optimization.

5. Structured decomposition in verification, representation, and cognition

In structured model checking, compositional complexity is controlled by decomposing a global concurrent system into components with typed interfaces. Petri Nets with Boundaries assign each component a two-labelled transition-system semantics, and sequential composition is governed by

$D(h)=D(\bar g,[D(f^1),\ldots,D(f^n)]),$9

Weak language equivalence is a congruence for the composition operators, so intermediate automata can be minimized safely during bottom-up reachability checking (Sobociński, 2016). On the $\bar g$0-bit counter family, whose global automaton has $\bar g$1 states and shortest accepting path length $\bar g$2, the paper states that the compositional procedure solves the family in linear time (Sobociński, 2016). Here compositional complexity is not a new complexity class; it is the practical reduction of state explosion through interface-aware decomposition and equivalence-based minimization.

A related representation-theoretic viewpoint appears in hierarchical compositional object models with part sharing. For serial inference with shared dictionaries, the cost is

$\bar g$3

whereas without sharing the multiple-object cost is

$\bar g$4

In favorable dictionary-scaling regimes, part sharing yields serial inference linear in the number of levels for an exponential number of objects; in less favorable regimes it offers little serial advantage but still supports linear-time parallel inference at the cost of many nodes (Yuille et al., 2013). The explicit lesson is that compositional complexity is reduced only when the number of distinct reusable parts is much smaller than the number of part instances.

At a broader conceptual level, a radically compositional approach to cognitive concepts treats composition as primitive via applied category theory. The paper develops categories, monoidal categories, string diagrams, compositional probability via $\bar g$5, dynamical systems via $\bar g$6, and a functor

$\bar g$7

to relate compositional probabilistic models to neural circuits (Smithe, 2019). This work does not define computational complexity quantitatively; rather, it argues that complexity in cognition and neuroscience is managed by preserving typed compositional structure across representational levels (Smithe, 2019).

6. Message-like structure and non-trivial semantics

In emergent communication, compositionality is often formalized by the condition

$\bar g$8

where derivations $\bar g$9 are recursively structured and $g$0 composes representations (Korbak et al., 2020). This paper distinguishes trivial compositionality, where semantic composition is set intersection, from non-trivial compositionality (NTC), where composition may be order-sensitive, context-sensitive, entangled, or operator-like, as in negation or “biggest apple” (Korbak et al., 2020). Its central empirical result is that most popular metrics—generalization, positional disentanglement, bag-of-words disentanglement, context independence, topographic similarity, and conflict count—fail to detect NTC, whereas tree reconstruction error (TRE) succeeds because it is explicitly grounded in recursive derivation structure (Korbak et al., 2020). The point is not that NTC is non-compositional, but that many metrics encode only a narrow, slot-wise notion of composition.

A different communicative notion appears in Local Compositional Complexity (LCC), which targets data that could support a human-readable message. LCC splits the shortest exact description into a structured part and an unstructured residual, and for discrete data defines

$g$1

for the codebook-model $g$2 and index $g$3 in the minimizing exact encoding $g$4 (Mahon, 7 Jan 2025). The paper proposes local compositionality—tree-like reuse of nearby parts—as the relevant structure, and implements it with substring aliasing for text and recursive local GMM clustering for continuous data (Mahon, 7 Jan 2025). Reported experiments show that the method distinguishes meaningful signals from noise and repetitive signals in text, images, and audio, and that the Arecibo message receives its highest score at the correct $g$5 aspect ratio (Mahon, 7 Jan 2025). In this setting, compositional complexity measures the size of reusable hierarchical structure rather than unpredictability or raw Kolmogorov-style incompressibility.

These works widen the concept substantially. Compositional complexity can refer not only to how a system computes, but also to how a representation supports semantics, interpretation, and message-like organization.

7. Compositional complexity in materials science

In materials science, compositional complexity denotes chemically heterogeneous composition and the local structural or functional consequences of that heterogeneity. A 2D transition-metal oxide nominally written as $g$6 is shown by atom probe tomography to exhibit several coupled forms of compositional complexity: oxygen deficiency relative to the precursor, retained alkali content, intrinsic Ti-sublattice vacancies, and near-atomic-scale spatial variation, especially for potassium (Krämer et al., 21 Jan 2026). The measured residual concentrations in the final nanosheets are $g$7 at.% Li, $g$8 at.% K, and $g$9 at.% C, and frequency-distribution analysis gives $D(g\circ(f,\ldots,f))=D(f)D(g)$0 and $D(g\circ(f,\ldots,f))=D(f)D(g)$1, supporting homogeneous Li but clustered K (Krämer et al., 21 Jan 2026). The paper interprets these observations as evidence for a reconstruction mechanism involving oxygen vacancies and residual alkali cations that compensates negatively charged Ti-sublattice vacancies (Krämer et al., 21 Jan 2026).

In medium-entropy MXenes, compositional complexity refers to multimetal occupation of the carbide lattice, as in TiVNbMoC$D(g\circ(f,\ldots,f))=D(f)D(g)$2 and TiVCrMoC$D(g\circ(f,\ldots,f))=D(f)D(g)$3, rather than single-metal Ti-based MXenes (Li et al., 26 Mar 2026). The work reports that annealing at $D(g\circ(f,\ldots,f))=D(f)D(g)$4C converts OH terminations to O terminations, reduces adhesion and friction, and, together with higher out-of-plane bending stiffness in the thicker multimetal sheets, leads to ultralow friction. The minimum reported coefficient of friction is 0.0022 for annealed TiVCrMoC$D(g\circ(f,\ldots,f))=D(f)D(g)$5, below the superlubricity threshold $D(g\circ(f,\ldots,f))=D(f)D(g)$6 and lower than graphene, MoSe$D(g\circ(f,\ldots,f))=D(f)D(g)$7, and other MXenes measured by the same method (Li et al., 26 Mar 2026). Here compositional complexity is a design lever coupling chemistry, surface termination, mechanics, and tribology.

A compositionally complex double perovskite,

$D(g\circ(f,\ldots,f))=D(f)D(g)$8

illustrates a related but distinct phenomenon (Bhattacharya et al., 4 Nov 2025). The rare-earth-site cation-size variance is reported as

$D(g\circ(f,\ldots,f))=D(f)D(g)$9

yet the material still shows robust ferromagnetic ordering with $k$00 K, close to the expectation from the average rare-earth ionic radius and consistent with a mean-field picture at high temperature (Bhattacharya et al., 4 Nov 2025). Raman spin-phonon renormalization follows the mean-field scaling $k$01 below $k$02, while at lower temperature the system exhibits reentrant spin-glass-like behavior with $k$03 K and a magnetic memory effect (Bhattacharya et al., 4 Nov 2025). This combination of self-averaging long-range order and disorder-driven low-temperature frustration is a characteristic materials-science manifestation of compositional complexity.

Taken together, these materials papers suggest a usage quite different from the algorithmic one. Compositional complexity is not primarily the cost of nested computation, but the presence of multicomponent chemistry, defect-rich reconstruction, and spatially heterogeneous local environments whose effects may preserve average macroscopic behavior while reshaping microscopic physics.