Compositional Function Networks (CFNs)

• CFNs are interpretable models that represent functions as explicit compositions of elementary mathematical units, reflecting the hierarchical structure of their target domains.
• They utilize sequential, parallel, and conditional composition patterns, enabling dynamic function assembly and parameterization through gradient descent.
• Empirical evaluations across tasks like image classification and tabular prediction demonstrate that CFNs can improve robustness and generalization, albeit with increased design complexity.

Searching arXiv for the cited CFN-related papers to ground the article. I’ll look up the relevant arXiv entries now. Compositional Function Networks (CFNs) denote a family of models in which the primary computational object is an explicit composition of lower-level functions, parts, or modules, rather than a uniform stack of generic neurons. In the explicit 2025 formulation, a CFN represents $F(\mathbf{x}; \mathbf{\Theta})$ as a composition of elementary mathematical functions with clear semantics, combined through sequential, parallel, and conditional layers and trained by standard gradient descent (Li, 28 Jul 2025). Closely related work instantiates the same principle in several forms: directed-acyclic-graph compositions of low-dimensional local maps with norm-based approximation and excess-risk guarantees (Huang et al., 25 May 2026), tree models whose node-specific composition rules are dynamically generated by a meta-network (Liu et al., 2017), generative compositional models over CNN feature maps for occlusion robustness (Kortylewski et al., 2019), parse-assembled grounded language modules with shared compositional interfaces (Kuo et al., 2020), and relation-aware tabular architectures that recover closed-form feature laws (Li, 27 Jan 2026). This suggests that CFNs are best understood as a design principle: the compositional structure of the network is made to reflect the compositional structure of the target domain.

1. Core formalism and compositional operators

In the 2025 framework, a CFN is a parametric model that represents a function $F(\mathbf{x}; \mathbf{\Theta})$ as a composition of elementary mathematical functions, called function nodes, combined through composition layers (Li, 28 Jul 2025). The training objective is

$$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$

where $\mathcal{L}$ is task loss and $R$ is a regularizer. The built-in node library includes Linear, Gaussian, Sigmoid along a direction, Polynomial along a direction, Sinusoidal, and ReLU nodes; the PyTorch implementation also adds Fourier, Gabor, and SharedPatch nodes. The stated motivation is to preserve the hierarchical expressiveness and end-to-end trainability of deep learning while replacing opaque units with well-understood mathematical functions whose parameters have clear semantics.

Three composition patterns are central. Sequential composition has the feedforward form

$$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$

Parallel composition applies multiple nodes to the same input and combines them by sum, weighted sum, product, or concatenation. Conditional composition implements a gated mixture of experts: $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$ with nonnegative scalar gates $c_i(\mathbf{x})$ and compatible expert outputs $g_i(\mathbf{x})$. In this formulation, the network is a directed acyclic graph of differentiable function nodes and composition layers. Unlike additive interpretable models restricted to univariate or pairwise terms, this design explicitly supports sequential, parallel, and conditional compositional patterns.

The differentiability claim is structural rather than auxiliary: all built-in nodes are smooth or piecewise smooth, and the composition operators are differentiable as well. As a result, backpropagation applies directly through sequential chains, parallel branches, and normalized conditional gates. The same paper treats this as the distinctive feature that separates CFNs from purely search-based symbolic regression and from post-hoc explanation methods such as LIME, SHAP, and Grad-CAM, because the explanation is intended to be identical with the model’s own internal computation rather than an approximation layered on top of it (Li, 28 Jul 2025).

2. Context-dependent composition in trees and grounded language

A major line of work replaces the fixed compositional function used at every internal node of a tree with a dynamically generated, context-dependent function. In Dynamic Compositional Neural Networks over Tree Structure, the motivating claim is that standard TreeNNs underfit because they recursively reuse the same shared compositional function throughout the whole compositional process and therefore cannot capture the richness of compositionality (Liu et al., 2017). The proposed remedy is a meta TreeNN that computes a low-dimensional control vector $\mathbf{z}_j$ for each node $F(\mathbf{x}; \mathbf{\Theta})$0, from which the basic network’s parameters are generated: $F(\mathbf{x}; \mathbf{\Theta})$1 For DC-RecNN and DC-TreeLSTM, the dynamic parameters are factorized through low-rank forms such as $F(\mathbf{x}; \mathbf{\Theta})$2, so the space of possible compositional matrices is spanned by shared basis components while $F(\mathbf{x}; \mathbf{\Theta})$3 acts as continuous coefficients. In the paper’s own interpretation, this is “a network over compositional functions”: each node receives its own composition function, instantiated from a learned manifold of functions.

The grounded-language architecture on gSCAN applies a different but related mechanism: for each command, the model parses the sentence, replaces every node in the parse with a recurrent module specific to that word, and composes the resulting modules according to the parse DAG (Kuo et al., 2020). Each word module is a learned function

$F(\mathbf{x}; \mathbf{\Theta})$4

and attention maps are the only communication channel between modules. The parse therefore determines a command-specific network assembled from a shared function library. The paper treats this as the core architectural reason for systematic generalization: the network’s compositional structure reflects the compositional structure of the problem domain, while the module parameters remain trainable end-to-end.

These two strands share a common functional view of composition. In DC-TreeNN, context changes the parameters of the local composition rule. In grounded language, the parse changes the computation graph assembled from reusable modules. A plausible implication is that CFNs encompass both “dynamic function selection” and “dynamic function assembly,” provided that the operative unit is an explicitly compositional function rather than a homogeneous neuron.

3. Generative compositional models in vision

Compositional Convolutional Networks, or CompositionalNets, provide a vision-oriented CFN in which the higher-level representation is an explicit generative composition of lower-level parts over a CNN feature map (Kortylewski et al., 2019). At a fixed layer with spatial feature vectors $F(\mathbf{x}; \mathbf{\Theta})$5, the original model encodes each feature vector using a dictionary $F(\mathbf{x}; \mathbf{\Theta})$6, producing sparse binary activations $F(\mathbf{x}; \mathbf{\Theta})$7 via cosine-similarity thresholding. For class $F(\mathbf{x}; \mathbf{\Theta})$8, the binary activation tensor is modeled with a factorized Bernoulli distribution,

$F(\mathbf{x}; \mathbf{\Theta})$9

with a mixture over viewpoints or spatial configurations. The compositionality is explicit: the object-level likelihood is a product over spatial positions of local part probabilities.

To handle occlusion, the model introduces a position-wise foreground/background switch $$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$0, with a learned background model $$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$1. This creates an occlusion-aware likelihood in which each position is explained either by the foreground object model or by a background model. The paper’s central finding is that the original model is not able to localize occluders well. Two limitations are identified: binarization of features discards information, and the thresholding operation is non-differentiable, preventing end-to-end training of the CNN and compositional model together.

The proposed extension replaces the Bernoulli model over binary codes with a continuous mixture of von Mises–Fisher distributions over $$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$2-normalized feature vectors: $$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$3 The background model is defined in the same family,

$$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$4

Occluder localization then uses the log-likelihood ratio

$$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$5

The paper explicitly presents this as a fully generative, differentiable compositional model over real-valued CNN features. In CFN terms, the local functions are mixtures of vMF units, while the global log-likelihood remains a sum over positions, preserving the part-based compositional structure.

4. StructuralCFN and relation-aware tabular discovery

Structural Compositional Function Networks extend the CFN idea to tabular data by imposing a structural prior over features rather than treating features as independent entities (Li, 27 Jan 2026). The architecture has two stages. In the Dependency Layer, each feature $$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$6 is modeled as a context function of the remaining features,

$$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$7

Each $$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$8 is implemented as a masked hybrid CFN node combining a polynomial basis and a sinusoidal basis,

$$\mathbf{\Theta}^* = \arg\min_{\mathbf{\Theta}} \mathbb{E}_{(\mathbf{x}, y)}\big[ \mathcal{L}(y, F(\mathbf{x}; \mathbf{\Theta})) \big] + \lambda R(\mathbf{\Theta}),$$9

$\mathcal{L}$0

followed by LayerNorm and a Differentiable Adaptive Gate,

$\mathcal{L}$1

The gate is intended to discover whether a feature relation behaves more like attention-style filtering or inhibitory polarity.

In the Aggregation Layer, the model forms a residual structural context $\mathcal{L}$2 and predicts

$\mathcal{L}$3

The paper characterizes this as a Hybrid Functional Committee: a linear bypass plus additional CFN basis heads, each with an explicit analytic form. The design supports Structured Knowledge Integration by allowing masks or regularization on dependency projections, and it encourages sparse interaction maps through an $\mathcal{L}$4 penalty,

$\mathcal{L}$5

A distinguishing claim of StructuralCFN is Intrinsic Symbolic Interpretability. Because each dependency node and each prediction head is already a parametric analytic function, the trained network can be exported as closed-form equations rather than explained by a surrogate. The paper gives an illustrative extracted law for the Diabetes case study,

$\mathcal{L}$6

interpreted as a threshold-type effect plus a residual nonlinear correction. It also defines a normalized Dependency Matrix

$\mathcal{L}$7

which summarizes the directed contribution of feature $\mathcal{L}$8 to feature $\mathcal{L}$9’s context.

5. Approximation theory and norm-constrained learning

The theoretical literature treats CFNs as sparse compositional functions whose structure is encoded by a directed acyclic graph of low-dimensional local maps. In the 2019 approximation-theoretic treatment, a target function is represented by a DAG $R$0 whose nodes compute local constituent functions of low arity, and deep networks are regarded as advantageous precisely when their computational graph mirrors that compositional structure (Mhaskar et al., 2019). The central technical statement is “good propagation of errors”: if local node functions are Lipschitz, then the error at a parent node is bounded by its own local approximation error plus a Lipschitz term involving the errors of its children. In the spherical setting this yields additive control of node-wise uniform norms across the DAG. The paper’s summary claim is that deep networks are better than shallow networks at approximating functions that can be expressed as a composition of functions described by a directed acyclic graph, because the deep networks can be designed to have the same compositional structure, while a shallow network cannot exploit this knowledge; this is the sense in which the blessing of compositionality mitigates the curse of dimensionality.

The 2026 norm-constrained theory makes this DAG view explicit for overparameterized ReLU networks (Huang et al., 25 May 2026). A sparse compositional function has the levelwise form

$R$1

where each node function $R$2 is Hölder-smooth and depends only on a small number $R$3 of parents. The learning class is a norm-constrained deep ReLU network $R$4 with multiplicative Frobenius complexity

$R$5

and

$R$6

Within this framework, approximation and excess-risk bounds depend on the local dimensions and effective Hölder indices along critical DAG paths, not on the ambient input dimension.

The paper specializes those results to representative CFN architectures. For multi-index models $R$7, the approximation rate is

$R$8

independent of the ambient dimension $R$9, and the excess-risk rate is

$$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$0

For binary tree architectures with $$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$1 and local input dimension $$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$2, the rates depend on $$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$3 rather than on $$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$4 itself. The paper explicitly interprets these results as showing that deep networks can exploit the compositional structure of the target functions, effectively avoiding the curse of dimensionality through hierarchical representations. A plausible implication is that the theoretical and architectural uses of “CFN” are converging on the same object: a network whose complexity is governed by local compositional structure rather than raw ambient dimension.

6. Empirical behavior, interpretability, and limitations

Across the cited literature, CFNs are evaluated in symbolic regression, tabular prediction, tree-structured NLP, grounded language understanding, occlusion-robust vision, and image classification (Li, 28 Jul 2025, Liu et al., 2017, Kuo et al., 2020, Kortylewski et al., 2019, Li, 27 Jan 2026). The reported results are heterogeneous because the architectures and tasks differ, but the recurring empirical claim is that explicit compositional structure improves robustness, generalization, or interpretability without requiring post-hoc explanation.

Setting	Model	Reported result
CIFAR-10	Hybrid DeepCFN	96.24% accuracy
OccludedVehicles mean column	CompNet-Full	92.6%
SICK test accuracy	DC-TreeLSTM	80.2
gSCAN, train length $$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$5, test length $$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$6	semantic compositional model	89.05 ± 2.60% exact match
Blood Transfusion	StructuralCFN	$$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$7
Breast Cancer	CFN (PyTorch)	$$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$8 accuracy

In the explicit 2025 CFN framework, the empirical range runs from symbolic regression to image classification. A single Sinusoidal node recovers the simple harmonic-motion parameters $$F(\mathbf{x}) = f_n(f_{n-1}(\dots f_1(\mathbf{x}) \dots)).$$9, $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$0, and $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$1; a conditional-composition CFN for spatially varying regression reports $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$2; and Hybrid DeepCFN reaches 96.24% accuracy on CIFAR-10 (Li, 28 Jul 2025). On standard tabular benchmarks, CFN (PyTorch) reports $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$3 accuracy and $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$4 AUC on Breast Cancer, while CFN (NumPy) reports $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$5 RMSE on Diabetes. The same paper explicitly notes an interpretability–performance trade-off on very complex tasks: the highest-performing CIFAR-10 model introduces BatchNorm, SE blocks, stochastic depth, Mixup, cosine learning-rate scheduling, and widened channels, which slightly reduce fine-grained interpretability relative to the simpler CFN core.

The domain-specific variants report similarly targeted gains. In DC-TreeNN, DC-TreeLSTM outperforms Tree-LSTM on idiom-rich and entailment-sensitive tasks, with IE increasing from 56.0 to 60.2 and SICK test accuracy increasing from 77.5 to 80.2 (Liu et al., 2017). In grounded language, the compositional gSCAN model largely matches i.i.d. performance while substantially improving several systematic splits; for example, on split I with train length $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$6 and test length $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$7, seq2seq reports 19.32 ± 0.02% exact match while the compositional model reports 89.05 ± 2.60%, and on split G with $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$8 adverb examples the same contrast is 2.04 ± 0.95% versus 36.91 ± 5.13% (Kuo et al., 2020). In occlusion-robust vision, CompNet-Full reports a mean classification accuracy of 92.6% on OccludedVehicles, compared with 89.5% for CompNet-Dict and 78.9% for VGG, while the vMF-based extension also substantially improves ROC curves for occluder localization across white, noise, texture, and real-object occluders (Kortylewski et al., 2019). StructuralCFN reports statistically significant improvements $$F(\mathbf{x}) = \sum_{i=1}^N \frac{c_i(\mathbf{x})}{\sum_{j=1}^N c_j(\mathbf{x}) + \epsilon} \cdot g_i(\mathbf{x}), \qquad \epsilon = 10^{-10},$$9 on several scientific and clinical datasets, including Blood Transfusion $c_i(\mathbf{x})$0 versus $c_i(\mathbf{x})$1, Ozone $c_i(\mathbf{x})$2 versus $c_i(\mathbf{x})$3, and WDBC $c_i(\mathbf{x})$4 versus $c_i(\mathbf{x})$5, while using 300–2,500 parameters and a footprint 10x–20x smaller than standard deep baselines (Li, 27 Jan 2026).

The interpretability claims are equally varied. In the 2025 CFN framework, node parameters are themselves the explanation: Gaussian centers and widths, sinusoidal amplitudes and frequencies, polynomial coefficients, and Gabor orientations are directly inspectable (Li, 28 Jul 2025). In grounded language, attention maps constitute semantic denotations for individual words and phrases, and the model’s use of attention-only communication is reported to yield better systematic recombination than state vectors or mixed channels (Kuo et al., 2020). In StructuralCFN, the Dependency Matrix and extracted feature laws provide global and local symbolic structure (Li, 27 Jan 2026). In CompositionalNets, positive occlusion scores produce heatmaps that highlight regions more likely to be background or occluder than foreground object (Kortylewski et al., 2019). In DC-TreeNN, neurons in the control vector $c_i(\mathbf{x})$6 correlate with interpretable semantic and syntactic patterns such as lexical sentiment, idiomatic phrases, noun phrases involving colors, and verb phrases with light verbs (Liu et al., 2017).

Several limitations recur across the literature. The 2025 CFN framework states that architecture and node selection effort remains high and that the available node library does not yet cover all domains (Li, 28 Jul 2025). DC-TreeNN incurs computational overhead because two TreeNNs are run over the same tree and depends on parse quality (Liu et al., 2017). The grounded-LLM still finds novel-direction generalization difficult and relies on fixed parsers (Kuo et al., 2020). CompositionalNets are evaluated on pool4 features of a pretrained VGG and on synthetically occluded vehicles, so multi-layer compositional modeling and more realistic occlusions remain open directions (Kortylewski et al., 2019). StructuralCFN assumes features lie on a smooth relational manifold, scales as $c_i(\mathbf{x})$7 in dense form, and underperforms GBDTs on high-entropy or discontinuous datasets such as Wine Quality and California Housing (Li, 27 Jan 2026). The norm-constrained theory assumes the DAG structure is known and does not address structure discovery (Huang et al., 25 May 2026). Taken together, these limitations indicate that CFNs are not a solved architecture class; they are a compositional modeling program whose practical success depends on matching the node library, structural prior, and training regime to the domain’s latent functional organization.