Homological Neural Networks

• Homological Neural Networks (HNNs) are neural architectures that leverage simplicial complexes and algebraic topology to construct sparse, interpretable models for complex, high-dimensional data.
• They extract topological features through data-driven dependency estimation and clique complex construction, enforcing fixed wiring and high sparsity in neural connections.
• Empirical results show that HNNs achieve competitive performance with state-of-the-art methods while ensuring model stability, interpretability, and lower computational cost.

Homological Neural Networks (HNNs) are a class of neural architectures that leverage combinatorial topology—particularly the structure of simplicial complexes and their associated algebraic invariants—to encode sparse, interpretable, and compositionally structured models for high-dimensional data. Rooted in the intersection of topological data analysis and neural computation, HNNs systematically incorporate higher-order interactions via topological filtrations, incidence structures, and explicit homological constructions. Their design is motivated by the need to reconcile data complexity and sample-efficiency with the tractability, interpretability, and stability of learned models, especially in the regime of multivariate tabular data or compositional generative tasks.

1. Formal Foundations: Simplicial Complexes and Homology

The mathematical core of HNNs is the simplicial complex. For a set of $p$ variables $V=\{v_1,\ldots,v_p\}$, a simplicial complex $K$ is a set of subsets (“simplices”) of $V$, closed under subset inclusion: if $\sigma\in K$, then any $\tau\subseteq\sigma$ also lies in $K$ (Wang et al., 2023, Lin et al., 14 May 2026). A $d$-simplex is defined as a set of $d+1$ vertices, and the entire hierarchy spans from vertices (0-simplices), edges (1-simplices), faces (2-simplices), and so on.

Given a simplicial complex $K$, one assigns to each dimension $V=\{v_1,\ldots,v_p\}$0 a vector space of $V=\{v_1,\ldots,v_p\}$1-chains $V=\{v_1,\ldots,v_p\}$2. The boundary operator $V=\{v_1,\ldots,v_p\}$3 is linearly defined on basis elements via

$V=\{v_1,\ldots,v_p\}$4

where the hat denotes omission of the $V=\{v_1,\ldots,v_p\}$5-th vertex. The $V=\{v_1,\ldots,v_p\}$6-th homology group $V=\{v_1,\ldots,v_p\}$7 encodes $V=\{v_1,\ldots,v_p\}$8-dimensional topological holes, with dimension $V=\{v_1,\ldots,v_p\}$9 (Betti number) quantifying their count.

HNN architectures are built upon these algebraic and combinatorial structures: neurons and connections correspond—respectively—to simplices and their face/containment relations, directly mirroring the topology of the data relationships (Wang et al., 2023, Lin et al., 14 May 2026).

2. Architectural Construction: From Data to Homological Wiring

The HNN pipeline incorporates data-driven topology extraction and strict wiring constraints (Wang et al., 2023, Lin et al., 14 May 2026):

1. Dependency Estimation: Pairwise (or higher-order) dependencies (e.g., Pearson correlations, mutual information) are computed on the variables, forming a weighted dependency matrix $K$0.
2. Sparse Graph Construction: Algorithms such as the Triangulated Maximally Filtered Graph (TMFG) or Maximally Filtered Clique Forest (MFCF) produce sparse, chordal graphs consistent with data dependencies. TMFG, for instance, iteratively leverages four-vertex cliques (tetrahedra) to maximize locality of association.
3. Clique Complex Extraction: Maximal cliques in the sparse graphical model are identified and closed under inclusion to obtain the clique complex, which defines a simplicial complex structure over the variable set (Lin et al., 14 May 2026).
4. Layered Homological Representation: Neurons are instantiated for each simplex in each dimension. Connections only exist between each $K$1-simplex and its $K$2-dimensional faces; the corresponding weight matrices $K$3 are strictly masked by unsigned incidence matrices $K$4, enforcing that

$K$5

This results in a fixed-wiring, highly sparse DNN, where typical sparsity exceeds 90% (Lin et al., 14 May 2026, Wang et al., 2023).

1. Forward Propagation: Activations propagate hierarchically, with each unit aggregating (weighted) signals only from its combinatorial faces. Nonlinearities (ReLU, LeakyReLU) follow each aggregation. All-layer readouts enable explicit decomposition by interaction order.

3. Theoretical Motivation: Compositional Sparsity and Inductive Biases

HNNs are motivated by the principle of compositional sparsity: high-dimensional functions of the form

$K$6

with each $K$7 of bounded size, admit sample-efficient learning if architecture and parameterization exploit this compositional structure (Lin et al., 14 May 2026). By mapping maximal cliques to simplices and restricting neuron aggregation to faces, HNNs instantiate an inductive bias mirroring the hierarchical composition of generative functions.

The architecture is completely determined from the data, and structural inference (variable dependency, clique extraction) is entirely decoupled from parameter learning (“fixed wiring”) (Lin et al., 14 May 2026). This results in orders-of-magnitude reductions in trainable parameters relative to comparably expressive dense MLPs. Empirical results demonstrate stability as model dimension grows and significantly lower sensitivity to hyperparameter choices.

4. Relation to Neural Representations and Persistent Homology

Beyond architectural design, homological methods have been employed to analyze and interpret neural representations. ReLU networks induce a partition of input space into convex polyhedral regions $K$8, within which the map is affine (Beshkov, 3 Feb 2025). For a dataset manifold $K$9, the neural representation $V$0 has homology groups isomorphic to the relative homology $V$1, where the “overlap decomposition” $V$2 detects input equivalence across glued polyhedra (Beshkov, 3 Feb 2025). This opens the door to purely topological (rather than geometric) feature analysis and clarifies the effect of neural composition on topological invariants.

Persistent homology features—such as barcodes and persistence diagrams—have also been used as supervision targets or as neural inputs/outputs. For instance, neural networks (CNNs on images, GNNs on complexes) can be trained to predict or approximate persistence-based features, including stable tropical coordinates and filtered persistence images, directly from data (Montúfar et al., 2020).

5. Algorithmic Recipes and Practical Implementation

HNN Construction and Training Pipeline

A typical end-to-end HNN training workflow includes:

1. Compute variable dependencies (correlation or information-based) on training data.
2. Construct the sparse chordal graph (TMFG or MFCF).
3. Extract the clique/simplicial complex.
4. Generate incidence matrices $V$3 for all $V$4.
5. Instantiate and initialize masked weight matrices $V$5 and readout layers.
6. Train weights using Adam or similar optimizer, with early stopping and layer normalization (Wang et al., 2023, Lin et al., 14 May 2026).

Sparsity is enforced structurally—no pruning or $V$6 regularization is required. Layer readouts preserve interpretability by decomposition order.

Model Configurations

Model Type	Typical Parameter Count	Sparsity
Tabular HNN	$V$7M (vs. $V$8M dense)	$V$9
LSTM-HNN (solar)	$\sigma\in K$0M (vs. $\sigma\in K$1M)	$\sigma\in K$2
HNN (OpenML-CTR23)	$\sigma\in K$3k–$\sigma\in K$4k (MLP: 17–115k)	90–99%

6. Empirical Results, Interpretability, and Applications

Benchmarking and Sample Complexity

HNNs have been empirically evaluated on synthetic tasks (Erdős–Rényi precision matrices, nonlinear clique sum generative models) and real-world tabular and time-series datasets (Wang et al., 2023, Lin et al., 14 May 2026). Key findings include:

• HNNs sustain high $\sigma\in K$5 in low-sample or high-dimensional regimes where dense MLPs collapse.
• On real datasets, HNNs are consistently competitive with state-of-the-art tree ensembles (e.g., XGBoost, LightGBM) while requiring far fewer parameters.
• In time-series (solar-energy, exchange rate) forecasting, LSTM-HNN hybrids attain RSE and correlation on par with the best spatio-temporal models.

Interpretability

Due to the one-to-one mapping between neurons, simplices, and variables, HNNs provide explicit structural interpretability. Betti-number statistics of activations can be tracked to detect topological features and to understand the emergence of global “holes” (cycles, voids) in the feature composition (Wang et al., 2023, Lin et al., 14 May 2026). Learned weights can be visualized on TMFG or clique-forest structures to highlight critical dependencies and higher-order interactions.

Efficiency

HNNs achieve $\sigma\in K$6 per-sample computational complexity for sparse architectures ($\sigma\in K$7=number of variables), contrasting with $\sigma\in K$8 in fully connected nets. Measurements indicate %%%%49$K$50%%%% speedup and $\tau\subseteq\sigma$135% lower energy consumption per epoch compared to dense MLP baselines (Wang et al., 2023).

7. Extensions, Limitations, and Open Problems

Extensions include the learning of graph topology via differentiable optimization, integrating GNN-style message passing into the homological backbone, and leveraging higher-dimensional simplices for domains naturally supporting larger variable interactions (Wang et al., 2023). Tracking the evolution of the structural topology in streaming or time-varying data is another active direction.

A limitation is that HNNs, in their standard form, do not explicitly compute homology groups or Betti numbers during inference; instead, they rely on the combinatorial and hierarchical inductive structure of the simplicial complex. A plausible implication is that future architectures may combine explicit relative homology computation (e.g., as in (Beshkov, 3 Feb 2025)) with neural learning objectives, enabling models with “features as homology classes” rather than as static summary statistics or diagram-based features.

Homological Neural Networks represent a principled synthesis of algebraic topology and deep learning, instantiating compositional inductive biases via the language of simplices, faces, and incidence, and enabling interpretable, efficient, and scalable models for high-dimensional and structured data (Wang et al., 2023, Lin et al., 14 May 2026, Montúfar et al., 2020, Beshkov, 3 Feb 2025).