---
title: Persistent Homology on Activations
url: https://www.emergentmind.com/topics/persistent-homology-on-activations
type: topic
---

# Persistent Homology on Activations

Persistent homology on activations is a methodology that applies tools from algebraic topology—specifically, persistent homology—to the study of activation patterns, pathways, and intermediate representations in neural networks. It characterizes the multiscale topological structure of activations, providing a rigorous approach for understanding distributed representations, layerwise transformations, polyhedral decompositions, and their implications for robustness and representation learning. This perspective supplements classical geometric and information-theoretic analyses, revealing how neural architectures transform the topology of data and highlighting the organizational principles and vulnerabilities in deep learning.

## 1. Formalism: Activation Graphs, Polyhedra, and Simplicial Complexes

Three principal activation-to-topology correspondences underpin persistent homology on activations:

1. **Activation Graphs (Layerwise Graphical Structures):**  
   For a fixed input $\mathcal{I}$ and feed-forward architecture with $L$ layers, one induces the activation graph $G^\mathcal{I} = (V, E, \varphi)$, where:
   - $V = V_0 \cup V_1 \cup \ldots \cup V_{L-1}$, the neurons across all layers;
   - $E = \{(u,v) : u \in V_l, v \in V_{l+1}\}$, connecting neurons across adjacent layers;
   - $\varphi: E \to \mathbb{R}_+$, edge-weights given by $\varphi(u,v) = |w_{u \rightarrow v} h_u|$, with $h_u$ the activation of $u$ and $w_{u \rightarrow v}$ its outgoing weight.
   The adjacency matrix $A_{ij}$ records $\varphi(v_i,v_j)$ or $0$ as appropriate [1901.09496].

2. **Polyhedral Decompositions by Piecewise-Linear Activations (e.g., ReLU):**  
   ReLU networks induce hyperplane arrangements partitioning input space into convex polyhedra $G_J$, each associated with a distinct global activation code. On each $G_J$, the network is affine; the collection forms a polyhedral decomposition. The dual graph $G$ of this decomposition has vertices indexed by codewords, with edges joining codewords that differ in only one neuron and induce adjacent polyhedra [2306.17418, 2502.01360].

3. **Activation Point Clouds for Vietoris–Rips Complexes:**  
   For a batch or data cloud $X$ at any given layer, the corresponding activations $a_i \subset \mathbb{R}^{d_i}$ (where $d_i$ is layer width) are embedded as a point cloud. Pairwise (often Euclidean or intrinsic) distances define metric filtrations—typically Vietoris–Rips complexes [2110.10136, 2004.06093, 2208.06438].

## 2. Persistent Homology Pipeline: Filtration, Computation, and Summarization

The persistent homology pipeline for activation analysis comprises:

1. **Filtration Construction:**  
   - **Activation Graphs:**  
     Edges are sorted by $\varphi$ in nonincreasing order, giving a superlevel-set filtration $G^{\mathcal{I}}_k$ (edges $\geq \omega_k$). Each $G^{\mathcal{I}}_k$ is a simplicial complex (vertices and edges) [1901.09496].
   - **Polyhedral Dual Graphs:**  
     For data points $x_i$, compute corresponding binary codewords $s(x_i)$; define pairwise Hamming, graph-geodesic, or other combinatorial metrics to build filtration complexes—Vietoris–Rips, cliques, or order complexes [2306.17418].
   - **Activation Point Clouds:**  
     Apply standard distance-based filtrations, typically Euclidean Vietoris–Rips, to the activation vectors of each layer [2110.10136, 2004.06093, 2208.06438].

2. **Persistent Homology Computation:**  
   - Apply standard matrix-reduction or union-find for $H_0$ (connected components) in $O(N\alpha(N))$ (inverse Ackermann), or use PH software (Ripser, Perseus, PHAT) for higher homology [1901.09496, 2306.17418].
   - Each homology class is tracked by its birth and death indices, producing persistence diagrams $Dgm_k$, barcodes, and Betti curves:
     \[
     \beta_k(\epsilon) = \operatorname{rank} H_k(K_\epsilon)
     \]
   - Summaries include Wasserstein/bottleneck distances, landscapes ($\Lambda$), images, and silhouettes [1901.09496, 2110.10136].

## 3. Layerwise, Pathwise, and Relative Topological Signatures

Multiple representations can be analyzed:

1. **Persistent Subgraphs (0-cycles) as “Information Pathways”:**  
   Long-lived $H_0$ classes correspond to globally relevant signal pathways from input pixels to output neurons; their lifetimes quantify pathway globality [1901.09496].

2. **Layerwise Evolution of Betti Numbers (Topological Simplification):**  
   - In both simulated and real data, Betti numbers $\beta_k$—especially $\beta_1$ and $\beta_2$—fall sharply across layers, particularly for ReLU activation, indicating "topological thinning" as data manifolds are unfolded and collapsed to linearly separable representatives [2004.06093, 2208.06438].
   - With bijective nonlinearities (e.g., Tanh), topological features (e.g., loops) persist longer through more layers; non-bijective ReLU mappings rapidly destroy higher homology, consistent with their nonhomeomorphic behavior [2208.06438].

3. **Relative Homology via Polyhedral Overlap Decomposition:**  
   - The quotient of the input manifold $M$ by the network-induced "overlap" decomposition $O_\Phi$ yields relative-homology groups $H_k(\Phi(M)) \cong H_k(M,O_\Phi)$, allowing metric-independent computation of topological invariants under contractibility assumptions on the polyhedral cells. These capture purely topological (not geometric) deviations in representation as training progresses [2502.01360].

## 4. Empirical Findings: Robustness, Adversarial Examples, and Representation Analysis

1. **Adversarial Example Anatomy:**  
   - Persistent homology of activation graphs reveals that adversarial perturbations, such as PGD or CW $L_2$ attacks, do not create new topological features associated with target classes but instead subtly reroute dominant activation pathways, resulting in class misclassification even as activation subgraph topology remains similar [1901.09496].
   - For a range of architectures, subgraph-based SVMs trained on persistence-weighted templates robustly recover the correct class for adversarial instances (70–83% accuracy), even where the base network fails on 100% of such inputs. Wasserstein distances between diagram pairs correlate with noise/perturbation magnitude [1901.09496].

2. **Manifold Hypothesis and Layerwise Homeomorphism:**  
   - In early layers, network maps act as approximate homeomorphisms, preserving the input manifold’s Betti numbers, while deeper layers collapse non-essential topology for classification [2208.06438, 2004.06093].
   - Deep architectures distribute topological changes more evenly across layers; shallow ones concentrate topological alterations in later stages [2004.06093].

3. **Polyhedral and Graph-based Persistence is Stable and Scalable:**  
   - Graph-based persistence using combinatorial metrics (Hamming, graph-geodesic) on activation codes converges (in bottleneck distance) to the true manifold homology with increased sampling, providing practical and robust large-scale analysis [2306.17418].

## 5. Comparative Methodologies, Limitations, and Extensions

| Approach                              | Object                | Metric/Complex       | Uniqueness                          |
|----------------------------------------|-----------------------|---------------------|-------------------------------------|
| Activation Graph PH [1901.09496]       | Graph $(V,E,\varphi)$ | Edge-weight         | Encodes explicit pathway structure  |
| Polyhedral Dual Graph PH [2306.17418]  | Bit-code graph $G$    | Hamming/geodesic    | Encodes activation cell transitions |
| Point-cloud VR PH [2110.10136, 2004.06093] | Activation vectors    | Euclidean           | Layerwise geometric distortion      |
| Relative Homology [2502.01360]         | Overlap classes $O_\Phi$ | None             | Purely topological equivalence      |

- **Choice of filtration and metric influences sensitivity**: Graph-based methods avoid geometric artifacts, while Euclidean VR-PH is sensitive to sampling density and curvature [2502.01360].
- **Limitations**: Polyhedral explosion (exponential in neurons/layers), convexity requirements for relative homology, and computational costs for large-scale data constrain feasible domains [2502.01360, 2306.17418].
- **Extensions**: Multimetric and multiparameter filtrations across resolutions; local, per-neuron, or stratified homology to probe finer representational features; integration with classical interpretability approaches [2110.10136, 2306.17418].

## 6. Significance for Neural Representation Theory and Robustness

Persistent homology on activations offers principled, multiscale measurement of neural architectures’ effect on data topology, with ramifications for robustness, interpretability, and the structure of representation spaces. Topological simplification induced by architecture and activation nonlinearity may explain observed differences in network generalization. The framework exposes how class-relevant “information pathways” can be rerouted or destabilized by small input perturbations, accounting for adversarial vulnerability not as the introduction of novel class features but as subtle deformation within the existing subgraph topology [1901.09496]. Quantifying these effects via persistent homology enables more targeted architectural design, informed robustness regularization, and in-depth comparison of learned representations across models and tasks.

Source: https://www.emergentmind.com/topics/persistent-homology-on-activations