Compute-Law Compositionality

• Compute-law compositionality is a framework that formalizes how meanings of complex expressions are systematically built from their parts using computable functions.
• It integrates classical semantic theories, probabilistic algebras, and neural architectures, employing metrics like the modulus of continuity and variance-decline score to quantify composition.
• The framework bridges linguistic, cognitive, and computational models, advancing our understanding of systematic generalization in both human cognition and artificial intelligence.

Compute-law compositionality concerns the existence, formalization, and empirical manifestation of explicit, quantitative principles—‘computable laws’—governing how the meanings of complex structures are determined by their parts and composition rules. This domain encompasses classical semantic theories, categorical models, symbolic and subsymbolic computational architectures, as well as neural and probabilistic systems. The study of compute-law compositionality aims to bridge linguistic, cognitive, and algorithmic models by making precise the mappings that underlie systematic generalization and emergent combinatorial capacity in both human cognition and artificial intelligence.

1. Formal Foundations of Compositional Law

Classical compositionality, in the Fregean tradition, asserts that for any complex expression $E$ constructed from sub-expressions $A$, $B$ by combination rule $C$, the meaning $\llbracket E \rrbracket$ is entirely determined by the meanings of $A$, $B$, and the operation $C$: $$\llbracket E \rrbracket = F\left(\llbracket A \rrbracket,\, \llbracket B \rrbracket,\, C\right)$$ where $F$ is a computable semantic-composition function. Category-theoretic generalizations formalize grammar as a category $\mathcal{C}$ with objects (syntactic types) and morphisms (composition rules), with semantics as a functor $\mathbb{M}:\mathcal{C}\to \mathcal{S}$ into a semantic category $\mathcal{S}$, enforcing

$$\mathbb{M}(f \circ g) = \mathbb{M}(f) \circ \mathbb{M}(g)$$

thereby ensuring that syntactic and semantic compositions commute (Russin et al., 2024).

In formal computational modeling, compositionality is specified by inductively defined sets and functions. For natural language, let $L_0$ (lexicon) and $S$ (strings) be as follows:

• $S \supseteq L_0$
• If $w, v \in S$, then $(w + v) \in S$

Constituent structures $L_i$ are defined by recursive Merge operations. At each level $i$, a (partial) semantic function $f_i$ computes higher-level structure meaning, preserving the meanings of subcomponents (Kaushik et al., 2020).

In probabilistic process algebras, the compositionality of an operator is quantified via its modulus of continuity $z$: $$d_0(f(t_1,\ldots,t_n), f(t'_1,\ldots,t'_n)) \leq z(d_0(t_1, t'_1), \ldots, d_0(t_n, t'_n))$$ where $d_0$ is a bisimulation pseudometric and $z$ is determined by the fixed-point on rule multiplicities (Gebler et al., 2014).

2. Core Set-theoretic and Functional Laws

Compute-law compositionality is captured by explicit set-theoretic and functional lemmas:

• Non-isomorphism of domains: The linear domain of strings ($S$) is not isomorphic to constituent-time patterns ($T_i^*$), ensuring that meaning is not directly recoverable from the string alone. Each semantic function $f_i$ constructs higher-level meanings via partial, incremental application along structurally defined domains, culminating in a final interpretation $h$: $$m(e) = h_n \circ f_n(e, f_{n-1}(e, \ldots, f_1(e)))$$ for $e$ with $n$ levels of merge (Kaushik et al., 2020).
• Weak Compositionality: Because each $f_i$ is partial, meaning is built in stages, ensuring that only complete structures (those with $\theta=1$ at the root) project uniquely to valid syntax trees.
• Probabilistic Modulus of Continuity: For algebraic operators, the impact on process distance is bounded above by a law $z_x$ computed from rule multiplicities via a fixed-point: $$z_x(\varepsilon_1,\ldots,\varepsilon_n) = \min\left\{ \sum_i M_f(i)\varepsilon_i, 1 \right\}$$ where $M_f$ is the least solution to $M = \sup_{r \in R_f} H_r(M)$ (with $H_r$ defined from SOS rules) (Gebler et al., 2014).

3. Cognitive and Neural Constraints

Cognitive-plausible models impose architectural requirements and computational constraints stemming from these laws:

• Constituents must be represented incrementally, indexed by string positions or time.
• Semantic functions $f_i$ operate in a strict bottom-up (and optionally top-down) manner, with operations applied in an online, left-to-right fashion.
• Partial constituency representations are maintained; only at the root is full structure recovered.
• Structure-dependent transformations are computed over constituent domains ($T_i^*$), whereas rigid (structure-independent) operations can act only on strings ($S$).

In the DORA model (a symbolic-connectionist role–filler binding architecture), these requirements are met via layered dynamical systems:

• Four layers (Propositions, Role-Binder, Predicate/Object, Semantics)
• Timing-based role–filler binding—temporal asynchrony encodes relations
• Hebbian mapping and competitive inhibition realize incremental structure mapping
• Empirical evaluations with word embeddings demonstrate that structural constraints prune mapping errors and optimize relational learning (Kaushik et al., 2020).

4. Compositionality in Deep Neural and Probabilistic Systems

Recent advances have produced explicit compute-law characterizations in deep learning and probabilistic scene parsing:

• Neural Models: Compositionality may be “built in” using architectural inductive biases (role–filler separation, tensor-product binding, relational bottlenecks) or “learned” through meta-learning regimes. Metalearning discovers compositional algorithms through exposure to multiple tasks, resulting in an inner-loop implementation that generalizes systematically (Russin et al., 2024).
• CLAP (Compositional Law Parsing): In visual reasoning, each concept $c$ is mapped by a latent random function $f^c:{\mathcal X}\rightarrow\mathbb{R}^{d_z}$, instantiated as a Neural Process. The compositional law of a scene bundles $f^1,\ldots,f^K$. Laws may be composed across scenes by recombining latent variables $g^c$ corresponding to each concept, enabling exchange and novel law formation. Compositionality is diagnosed by the variance-decline score, quantifying the match between learned latents and ground-truth sub-laws (Shi et al., 2022).

Method	Law Formalization	Empirical Domain
Symbolic (Kaushik)	Partial semantics via $f_i,s$	Natural language syntax/semantics
Probabilistic (Desharnais et al.)	Modulus of continuity $z$ via fixed points	Probabilistic processes
Deep learning (CLAP)	Latent random functions + variance-decline	Scene/physics parsing
DNNs, LLMs	Token-level composition + ICL	NLP, code, reasoning

5. Quantitative Evaluation and Empirical Metrics

Compute-law compositionality enables quantitative assessment:

• Geometric compositionality score: For phrase $p$ in context, embedding $v_p$ aligns with context subspace $S_c$; compositionality is measured by

$$C(p,\,\text{context}) = \cos(v_p,\,\mathrm{Proj}_{S_c}(v_p))$$

Values near 1 indicate literal (compositional) usage; scores near 0 identify idiomatic or metaphorical usage (Gong et al., 2016).

• Variance-decline score (CLAP): For each concept–law, the score $s_{l,c}$ compares variance in $g^c$ with and without fixing law $l$: $$s_{l,c} = \frac{1}{N_B}\sum_{b=1}^{N_B} \frac{\bar v^c - v^{c, b}_l}{\bar v^c}$$ $s_{l,c} \approx 1$ implies that $g^c$ encodes law $l$ (Shi et al., 2022).
• Precision in mapping structure: In DORA, predicted mapping matrices are compared to ground-truth to compute precision (TP/(TP+FP)); performance increases when embedding structure better reflects syntax (Kaushik et al., 2020).

6. Emergent Laws in Large-scale Neural Systems

Empirical work demonstrates robust, quantitative “computable laws” connecting architecture, learning strategy, and compositional generalization:

• Out-of-distribution generalization error decays exponentially in training examples under strong inductive biases: $$\mathrm{Err}_{o.o.d.}(n) \leq c(B) \exp(-\lambda(B)n)$$ where $\lambda(B)$ is the compositional generalization rate for bias $B$.
• Under metalearning, the inner-loop error on new composition tasks scales as $O(K^{-\alpha} + M^{-\beta})$ in prompt-length $K$ and meta-training set size $M$.
• Increasing model scale and corpus size in pre-trained DNNs leads to strengthening 'emergent' compositionality (Russin et al., 2024).

Instruction-tuned, large pre-trained models display compositional behaviors across NLP and medical reasoning, and code generation benchmarks. These findings position compute-law compositionality as the theoretical bedrock and practical instrument for both measuring and inducing systematic generalization in artificial and biological systems.

7. Worked Examples in Formal Compositionality

The application of these laws is exemplified in both symbolic and neural systems:

• Natural language syntax: For $s=\text{“Big dogs bite men”}$,
  1. $f_1(s) =$ assigns categories and position indices
  2. $f_2(s,f_1(s)) =$ constructs NP, VP constituents
  3. $f_3(s,...) =$ merges to full S

  $m(s) = h_3( f_3( s, f_2(s), f_1(s) ) )$

  $m(s)$ is thus a composition of meanings from atomic terms and three merges (Kaushik et al., 2020).

• Probabilistic operators: For non-deterministic choice $g(x, y) = x + y$, the modulus $z_g$ is

  $$z_g(\varepsilon_1, \varepsilon_2) = \max\{\varepsilon_1, \varepsilon_2\}$$

  and for probabilistic prefix $f(x)=a.([q]x \oplus [1-q]0)$, $z_f(\varepsilon)=q\varepsilon$ (Gebler et al., 2014).

• CLAP law composition: By swapping global latent variables $g^c$ between sample scenes A and B for concept sets $C_A, C_B$, new composed laws are instantiated, and decoded to generate scenes with combined sub-law behaviors (Shi et al., 2022).

These formal examples and empirical systems collectively instantiate compute-law compositionality as a rigorous, quantitative, and implementable principle across symbolic, probabilistic, and connectionist models.