---
title: Capsule Routing in Neural Networks
url: https://www.emergentmind.com/topics/capsule-routing
type: topic
---

# Capsule Routing in Neural Networks

Capsule routing refers to a family of algorithms for dynamically assigning outputs (“votes”) from lower-layer groups of neurons—capsules—to upper-layer capsules in neural networks. Unlike static connections in conventional deep networks, capsule routing explicitly models part–whole relationships between entities (“capsules”) and selects how much each lower-level entity should contribute to candidate higher-level entities. Routing is central to the function and distinct representational properties of Capsule Networks (CapsNets).

## 1. Mathematical Formulation and Dynamic Routing by Agreement

The canonical capsule routing mechanism is the dynamic routing-by-agreement algorithm introduced by Sabour et al., which iteratively refines couplings between lower-layer and upper-layer capsules based on the degree of agreement among their predictions. The key steps can be summarized as follows:

Let layer $l$ contain $M$ input capsules $u_i \in \mathbb{R}^{D_1}$ and the next layer $l+1$ contain $N$ output capsules $v_j \in \mathbb{R}^{D_2}$. For each pair $(i, j)$, a trainable transformation $W_{ji} \in \mathbb{R}^{D_2 \times D_1}$ maps the input capsule to a “vote”:
\[
\hat{u}_{j|i} = W_{ji} u_i
\]
The routing iteratively updates coupling coefficients $c_{ij}$:
1. Initialize $b_{ij} = 0$ and set $c_{ij} = \operatorname{softmax}_j(b_{ij})$.
2. Compute each output capsule’s input: $s_j = \sum_{i=1}^M c_{ij} \hat{u}_{j|i}$.
3. Apply a squashing nonlinearity to ensure output vectors have length in $(0,1)$:
\[
v_j = \frac{\|s_j\|^2}{1 + \|s_j\|^2} \frac{s_j}{\|s_j\|}
\]
4. Update $b_{ij} \leftarrow b_{ij} + v_j \cdot \hat{u}_{j|i}$.
This sequence is repeated for a fixed number of iterations (commonly $r=3$) and constitutes one routing “pass” [1806.04699].

In capsule networks for sound event detection, Zhao et al. implemented this procedure in a time-slice–wise fashion, inserting it into a temporal architecture to model both part–whole coherence and attention to salient events [1806.04699]. By running routing independently at each time-slice and combining outputs with an auxiliary temporal attention mechanism, the architecture achieved state-of-the-art performance on polyphonic sound event detection.

## 2. Non-Iterative and Fast Routing Variants

The computational overhead of iterative routing has led to significant interest in non-iterative or single-pass routing schemes. Notable approaches include:

- **Cluster Routing**: Each lower-level capsule produces a *cluster* of votes via multiple learned matrices acting on a spatial neighborhood. Variance within a cluster quantifies self-consistency, and centroids of low-variance clusters obtain higher routing weights. The next-layer capsule input is a weighted sum of centroids, with weights given by an elementwise softmax of $- \log \sigma_i$, where $\sigma_i$ is the cluster’s variance. The entire routing is performed in a single pass, avoiding any recurrent coefficient refinement [2109.09213].

- **Self-Attention Routing**: The routing-by-agreement step is replaced by an attention mechanism. Each lower-level capsule predicts every higher-level capsule’s pose, and similarity (e.g., scaled dot-product) between votes defines the “agreement.” Coupling coefficients are computed by a single softmax, and capsule outputs are obtained via attention-weighted aggregation, with a final nonlinearity [2101.12491, 2112.00775].

- **Prototype-Based (ProtoCaps) Routing**: Each lower-layer capsule is projected into a shared subspace; similarity to a set of learned prototypes determines soft clustering coefficients (via softmax), which in turn weight contributions to the next layer. No per-iteration voting or agreement update is performed [2307.09944].

These non-iterative algorithms provide significant reductions in memory, FLOPs, and wall-clock runtime, while achieving competitive or superior accuracy to traditional dynamic/EM routing on standard vision and sequence datasets [2109.09213, 2101.12491, 2307.09944].

## 3. Extensions: Probabilistic and Geometric Routing

Probabilistic formulations of capsule routing generalize the dynamic routing approach:

- **EM Routing**: Treats each higher-level capsule as a Gaussian mixture component, with votes from lower-level capsules as samples. Routing alternates E-steps (computing responsibilities based on Gaussian likelihood) and M-steps (fitting the mean, variance, and activation of each capsule), with activation computed as a logistic function of the sum of log-variances and mixture assignment weights [1905.08744].

- **Variational Bayes Routing**: Places explicit priors over mixture weights, means, and precisions of the Gaussian clusters, and infers posteriors via coordinate ascent. This regularization prunes degenerate solutions (e.g., variance collapse), helps prevent overfitting, and enables interpretation of capsules as VAE components. Bayesian routing maintains or improves accuracy while increasing stability and generalization to novel transformations [1905.11455].

- **Pseudo-Riemannian Capsule Routing**: Classical routing operates in Euclidean or constant-curvature spaces. PR-CapsNet extends capsule routing to pseudo-Riemannian manifolds with adaptive curvature, decomposing capsule representation spaces into spherical-temporal and Euclidean-spatial components. Adaptive curvature routing learns per-capsule mixtures over curvature perspectives and aggregates them via geometry-aware attention, achieving robust hierarchical and cluster structure modeling in graph domains [2512.08218].

## 4. Complexity, Scaling, and Speed

A recurrent bottleneck for capsule routing has been computational and memory cost:

| Method                   | Time per layer (big-O)           | Memory per layer        |
|--------------------------|-----------------------------------|------------------------|
| Dynamic routing ($r$ iters) | $O(r\,m\,n\,d)$                 | $O(m\,n\,d)$           |
| EM routing               | $O(r\,m\,n\,d^2)$                 | $O(m\,n\,d + m\,n)$    |
| Cluster/self-attn/ProtoCaps| $O(m\,n\,d)$ (1 pass)            | $O(m\,n + (m+n)d)$     |

Empirically, iterative approaches can require up to an order of magnitude more operations than non-iterative schemes (e.g., 0.40 G vs. 0.06 G ops per batch for standard vs. Efficient-CapsNet on MNIST- or smallNORB-sized inputs [2101.12491]). Memory requirements for iterative routing preclude training on ImageNet-scale tasks unless reduced-memory approximations—such as ProtoCaps or accumulated (master) routing coefficients—are employed [2307.09944, 1904.07304].

Optimizations for sequential or temporal tasks include *sequential dynamic routing* (accumulating coupling updates across time steps) to maintain linear decoding time in sequence length [2007.11747], and blockwise or shortcut routing to enable deep capsule stacks without exacerbating routing cost [2307.10212, 2410.03987].

## 5. Empirical Performance and Representational Properties

Routing enables capsule networks to exhibit viewpoint equivariance, part–whole compositionality, and disentangled generative factors even with shallow architectures and relatively few parameters. Notable experimental outcomes include:

- On Fashion-MNIST, non-iterative cluster routing achieves 5.17% error with 146K parameters, outperforming earlier CapsNets [2109.09213].
- On smallNORB and MNIST-to-affNIST transfer, non-iterative and Bayesian methods exhibit better generalization to novel transforms than standard CNNs or dynamic routing [1905.11455, 2109.09213, 1806.04699].
- In sequence and multimodal tasks, capsule routing enables parameter sharing over variable-length sequences or parallel streams, avoids parameter blow-up, and improves sample efficiency [2007.11747, 2112.00775].
- Object-centric and hierarchical representations learned via routing are more robust to adversarial and out-of-distribution scenarios than their statically-aggregated equivalents [2508.16922].

However, practical benefits in classification accuracy have sometimes been marginal relative to parameter- and computation-matched CNNs, and some studies report that, in standard settings, non-learned or even random routing may achieve similar results, prompting reconsideration of routing objectives and regularization [1907.13327].

## 6. Limitations, Pathologies, and Theoretical Insights

Despite its promise, capsule routing exhibits several critical limitations:

- **Expressivity limitations**: Peer et al. proved that both dynamic and EM routing are restricted to symmetric functions $f(x)=f(-x)$, failing to be universal approximators unless explicit bias is introduced per capsule. Without these modifications, the networks cannot learn anti-symmetric or sign-sensitive mappings—even in principle. Adding small learned biases corrects this limitation and restores universality [1905.08744].
- **Over-polarization**: Empirical studies found that repeated routing iterations can drive coupling coefficients to degenerate one-hot assignments, losing the soft, distributed representations capsule networks are meant to provide. This polarization renders routing behavior similar to max-pooling and undermines uncertainty modeling [1907.13327].
- **Limited impact on decision boundary**: On standard datasets, the vast majority of input–class assignments are unchanged by the routing step—most of the classification power arises from the initial (unrouted) assignments. Only a small fraction of examples experience a change in the final decision due to routing [1907.13327].

Remedial techniques include soft/entropy regularization, adaptive temperature in softmax, curvature-adaptive geometric routing, and explicit Bayesian uncertainty modeling to prevent collapse and maintain distributed part–whole credit assignment [2512.08218, 1905.11455].

## 7. Application Domains and Advanced Routing Architectures

Capsule routing algorithms have been adapted across vision, audio, natural language, graph representation learning, and multimodal tasks:

- Sound event detection architectures use time-slice–wise routing, combined with temporal attention, to achieve robust performance and reduce overfitting on noisy or small track datasets [1806.04699].
- In visual recognition, multi-scale routing mechanisms such as cross-agreement routing (CAR) efficiently combine features from multiple spatial resolutions, focusing on their maximally-agreeing cross-scale capsules in one pass, boosting both parameter efficiency and adversarial robustness [2508.16922].
- Mamba capsule routing compresses the set of pixel-wise capsules with a selective state-space model, performs EM routing at the “type-level,” and employs Capsule-to-Spatial Detail Retrieval (CSDR) for improved segmentation efficiency [2410.03987].
- Graph learning has benefited from adaptive curvature pseudo-Riemannian capsule routing, integrating complex geometric inductive biases for modeling both hierarchical and cyclic graph structures [2512.08218].

These architectural innovations underscore the flexibility of the capsule routing paradigm, especially when adapted to domain-specific data structure and inductive bias requirements.

---

In sum, capsule routing constitutes the distinguishing ingredient of Capsule Networks, enforcing dynamic, data-dependent part–whole relationships between higher- and lower-order entities. The field has progressed from iterative dynamic and EM routing to a diversity of non-iterative, probabilistic, geometric, and attention-inspired schemes. Ongoing challenges include improving expressivity, stability, computational efficiency, and reliably demonstrating routing’s value beyond what is obtained from static aggregation or trivial assignments. The next phase of research will likely focus on routings that integrate principled uncertainty, robustness to degeneracy, and geometric adaptation to data and task structure [1806.04699, 2109.09213, 1905.08744, 2508.16922, 2512.08218, 2307.09944, 2410.03987, 1907.13327].

Source: https://www.emergentmind.com/topics/capsule-routing