Dynamic Routing in Capsules

• Dynamic routing in capsules is an iterative routing-by-agreement mechanism that adjusts coupling coefficients to preserve part-whole relationships and encode instantiation parameters.
• It addresses scalability challenges by reformulating routing as a feature extractor or non-iterative surrogate, enabling efficient use in high-dimensional and many-class problems.
• Empirical and theoretical studies highlight trade-offs between compositional accuracy and computational cost, with varied interpretations including optimization and energy-based models.

Dynamic routing in capsules is the iterative routing-by-agreement mechanism used in capsule networks to determine how lower-level vector-valued capsules contribute to higher-level capsules. In the canonical formulation, a capsule encodes the probability that an entity exists in the length of its activity vector and its instantiation parameters in the vector orientation; routing then repeatedly adjusts inter-capsule coupling coefficients so that predictions that agree are reinforced and incompatible routes are weakened (Sabour et al., 2017). Subsequent work has treated dynamic routing not only as a classification heuristic but also as a feature extractor for scalable capsule architectures, an optimization procedure under constraints, a component of energy-based and product-of-experts models, and a target of substantial critique and redesign (Mandal et al., 2019).

1. Canonical routing-by-agreement formulation

The original capsule formulation defines a lower-layer capsule output $\mathbf u_i$, a learned transformation matrix $\mathbf W_{ij}$, and a prediction vector

$\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$

For each lower capsule $i$, routing coefficients are obtained by a softmax over routing logits:

$c_{ij}=\frac{\exp(b_{ij})}{\sum_k \exp(b_{ik})},$

so that $\sum_j c_{ij}=1$. Higher-level capsule $j$ then receives

$\mathbf s_j=\sum_i c_{ij}\hat{\mathbf u}_{j|i},$

followed by the squash nonlinearity

$$\mathbf v_j=\frac{\|\mathbf s_j\|^2}{1+\|\mathbf s_j\|^2}\frac{\mathbf s_j}{\|\mathbf s_j\|}.$$

Agreement is measured by the scalar product, and routing logits are updated as

$$b_{ij}\leftarrow b_{ij}+\hat{\mathbf u}_{j|i}\cdot \mathbf v_j.$$

This iterative procedure is the standard dynamic routing loop introduced by Sabour, Frosst, and Hinton (Sabour et al., 2017).

Within that formulation, the length $\mathbf W_{ij}$0 represents the probability that the entity encoded by capsule $\mathbf W_{ij}$1 exists, while the vector orientation carries pose-like or instantiation information such as position, size, orientation, deformation, velocity, albedo, hue, and texture (Sabour et al., 2017). In the basic MNIST architecture, lower convolutional features are reshaped into PrimaryCapsules and routed into DigitCaps, one output capsule per class. The original paper reports that 3 routing iterations were the best practical choice in that setting, and couples the routing mechanism with a margin loss

$\mathbf W_{ij}$2

using $\mathbf W_{ij}$3, $\mathbf W_{ij}$4, and $\mathbf W_{ij}$5, plus a reconstruction regularizer scaled by $\mathbf W_{ij}$6 (Sabour et al., 2017).

Historically, dynamic routing was proposed as a replacement for max-pooling and fixed aggregation in order to preserve part-whole relationships and pose information. The original experiments associated this mechanism with 0.25% ± 0.005 test error on shifted MNIST, 79% accuracy on affNIST versus 66% for a comparable CNN, and 5.2% error on MultiMNIST with 3 routing iterations and reconstruction, compared with 8.1% for a baseline CNN (Sabour et al., 2017). These results established routing-by-agreement as the defining computational primitive of early capsule networks.

2. Scalability pressures and architectural reformulations

A central limitation of the canonical design is that routing cost scales with the number of candidate parent capsules. In the standard class-capsule architecture, each output capsule corresponds to one class, so the number of routing weights $\mathbf W_{ij}$7 is directly proportional to the number of classes. The paper "Using dynamic routing to extract intermediate features for developing scalable capsule networks" argues that this makes training slow and memory-intensive and that performance degrades on many-class problems because the model must resolve many more interrelationships and tends to overfit (Mandal et al., 2019).

That work keeps dynamic routing but changes what it routes to. Instead of routing primary capsules directly to class-specific output capsules, it routes them to an intermediate feature capsule layer producing $\mathbf W_{ij}$8 feature capsules rather than $\mathbf W_{ij}$9 class capsules. These feature capsules are flattened and passed to a fully connected classifier:

$\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$0

The reconstruction decoder is driven by the full feature capsule set rather than masked class capsules, so no masking by class label is used. On Bangla Basic Character, baseline CapsNet achieved 90.6% accuracy, whereas the proposed model reached 93.23% at $\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$1; on Bangla Compound Character, baseline CapsNet achieved 79.34%, whereas the proposed model reached 87.44% at $\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$2. Training time per epoch also dropped from 120.61 s to 14.15 s on Bangla Basic and from 1206.99 s to 36.92 s on Bangla Compound when $\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$3 (Mandal et al., 2019).

Other work has pursued efficiency by altering routing itself or by restructuring the network around it. "Fast Dynamic Routing Based on Weighted Kernel Density Estimation" reformulates routing as weighted KDE and proposes FRMS and FREM, reporting routing time efficiency improved by nearly 40% with negligible performance degradation and a practical hybrid CNN-capsule architecture for $\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$4 inputs (Zhang et al., 2018). "PDR-CapsNet: an Energy-Efficient Parallel Approach to Dynamic Routing in Capsule Networks" keeps standard dynamic routing but parallelizes the capsule-production pipeline and reduces the number of capsules entering routing; on CIFAR-10 it reports 83.55% accuracy with 87.26% fewer parameters, 32.27% fewer MACs, 47.40% fewer FLOPs, 3x faster inference, and 7.29J less energy consumption than CapsNet on a 2080Ti GPU with 11GB VRAM (Javadinia et al., 2023).

A more radical line replaces iterative routing with one-pass surrogates. "Efficient-CapsNet: Capsule Network with Self-Attention Routing" introduces a non-iterative, highly parallelizable self-attention routing algorithm, reports an architecture with about 161K parameters, and states an 84.96% improvement in operations over the original CapsNet while remaining competitive on MNIST, smallNORB, and MultiMNIST (Mazzia et al., 2021). "Attention routing between capsules" replaces dynamic routing and squash with an attention module and capsule activation, reporting training-time reductions to x0.19 of CapsuleNet on MNIST and x0.35 on CIFAR-10, with fewer parameters and higher accuracy on affNIST and CIFAR-10 (Choi et al., 2019). These results collectively indicate that the main computational burden lies in the iterative all-to-all agreement refinement rather than in vector-valued capsules per se.

3. Optimization, energy-based, and probabilistic interpretations

Dynamic routing has also been reinterpreted as an inference procedure inside probabilistic models. "Training capsules as a routing-weighted product of expert neurons" models capsule networks as a routing-weighted product of experts, with pairwise energy

$\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$5

and total energy

$\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$6

In that view, dynamic routing both discovers subnetworks in the dense capsule graph and mixes the model distribution during contrastive-divergence training (Hauser, 2019).

"Training products of expert capsules with mixing by dynamic routing" makes the probabilistic interpretation more explicit by treating the squashed capsule magnitude as a Bernoulli-like hidden variable:

$\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$7

It constructs an energy

$\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$8

marginalizes hidden magnitudes to obtain a product-of-experts visible distribution, and uses dynamic routing as the mixing operator in an approximate MCMC / contrastive-divergence procedure. The paper reports realistic MNIST and Fashion-MNIST generations and interpretable specialization of hidden capsules over stroke thickness, angles, lengths, sleeve length, and shoe height (Hauser, 2019).

A distinct reformulation makes routing explicitly discriminative. "Routing Towards Discriminative Power of Class Capsules" argues that standard routing-by-agreement is misaligned with classification because couplings are constrained to be positive by softmax and because routing assumes output capsules are already correctly activated. It replaces agreement refinement with a regularized quadratic program that directly maximizes the true-class capsule length and suppresses the others, with both $\hat{\mathbf u}_{j|i}=\mathbf W_{ij}\mathbf u_i.$9- and $i$0-regularized variants. On Fashion-MNIST it reports 7.21% error for dynamic routing, 7.01% for the $i$1 variant, and 6.76% for the $i$2 variant; on CIFAR-10 it reports 15.3% for dynamic routing, 14.52% for $i$3/FC, and 14.04% for $i$4/FC, while also claiming each training step can save at least 20% runtime (Yang et al., 2021).

The strongest mathematical statement to date is given by "The Convergence of Dynamic Routing between Capsules", which shows that dynamic routing can be written as a nonlinear gradient method solving a concave optimization problem with row-simplex constraints. The paper identifies the objective

$i$5

subject to $i$6 and $i$7, and proves the monotone decrease

$i$8

This establishes that the standard routing loop is not purely heuristic, although the same paper also reports that repeated routing tends to polarize coupling strengths (Ye et al., 8 Jan 2025).

4. Empirical disputes, polarization, and the role of entropy

A major controversy concerns what routing actually contributes in practice. "Capsule Networks Need an Improved Routing Algorithm" compares five routing algorithms—CapsNet, EMCaps, OptimCaps, GroupCaps, and AttnCaps—against uniform and random assignment baselines and reports that routing often performs no better than, or worse than, these simple baselines. On CIFAR-10, for example, CapsNet yielded 90.86 with uniform routing, 91.09 with random routing, 88.41 with 3 iterations, and 86.49 with 10 iterations; AttnCaps degraded from 93.20 with uniform routing to 70.27 with 3 iterations and 42.57 with 10 iterations. The paper further reports that routing usually does not change the classification result but instead polarizes link strengths, and that continued iteration makes the polarization extreme (Paik et al., 2019).

The convergence analysis of 2025 reaches a compatible empirical conclusion from a different direction: routing converges, but the converged state often sharpens or filters couplings rather than substantially changing the decision boundary (Ye et al., 8 Jan 2025). This convergence result does not contradict the critique; rather, it indicates that the procedure may be mathematically well defined while still failing to provide the intended representational benefit under many training regimes.

A counterposition is given by "Learning Compositional Structures for Deep Learning: Why Routing-by-agreement is Necessary". That paper argues that routing is necessary not primarily for top-1 accuracy on standard benchmarks, but for parse-tree-like compositional structure. It treats routing coefficients as OR-rule likelihoods in an AND-OR grammar and argues that the entropy of routing weights controls whether lower-level capsules make sharp derivation choices. The experimental claim is that as routing entropy increases, the ability to detect changes in compositionality decreases, and that capsule models without routing behave similarly to CNNs on compositionality tests even if classification accuracy remains comparable (Venkatraman et al., 2020). This suggests that disagreement in the literature partly reflects different evaluation criteria: classification accuracy, optimization stability, and compositional structure need not align.

A further empirical study, "Grouping Capsules Based Different Types", emphasizes that routing behavior is sensitive to how capsules are grouped and initialized. It reports that changing the effective initialization scale of coupling coefficients can significantly alter outcomes, and that grouping PrimaryCaps by type before routing improves MNIST, F-MNIST, K-MNIST, SVHN, and CIFAR-10 but degrades SmallNORB and AFFNIST in some settings (Ren, 2019). The combined picture is that routing is highly configuration-dependent: iteration count, normalization direction, initialization scale, and group structure all materially affect its behavior.

5. Adaptations beyond small-image classification

Dynamic routing has been adapted far beyond the original digit-classification setting. In dense prediction, "Capsules for Object Segmentation" introduces SegCaps, which replaces global all-to-all routing with locally connected routing over spatial windows and adds deconvolutional capsules for upsampling. SegCaps handles $i$9 inputs, uses 95.4% fewer parameters than U-Net, and reports an average dice coefficient of 98.479%, compared with 98.449% for U-Net and 98.410% for Tiramisu on the LUNA16 subset of LIDC-IDRI (LaLonde et al., 2018). Here dynamic routing is preserved, but only within local neighborhoods and with spatially shared transformations.

In NLP, the mechanism has been used both as a classifier and as a generic aggregation operator. "Investigating Capsule Networks with Dynamic Routing for Text Classification" adapts routing to text with three stabilizers—orphan category, leaky-softmax, and coefficient amendment—to reduce interference from noisy or background capsules, reports best results on 4 out of 6 text benchmarks, and further reports strong transfer from single-label training to multi-label prediction on Reuters splits (Zhao et al., 2018). "Information Aggregation via Dynamic Routing for Sequence Encoding" treats word representations as input capsules and output sentence vectors as output capsules, proposing standard and reversed DR-AGG policies; both outperform max pooling, average pooling, and self-attention on five text classification tasks, with standard DR-AGG performing best and performance peaking around 3 routing iterations (Gong et al., 2018).

At the same time, text has motivated arguments against full dynamic routing. "Text Classification using Capsules" proposes static routing,

$c_{ij}=\frac{\exp(b_{ij})}{\sum_k \exp(b_{ik})},$0

followed by squashing, and argues that text is more flexible than images, so iterative agreement may be unnecessary or harmful. On seven benchmarks, static routing outperforms dynamic routing on most datasets, including 87.17 versus 86.45 on 20news, 87.52 versus 86.72 on Reuters10, 89.6 versus 88.1 on MR (2004), and 94.84 versus 93.80 on TREC-QA (Kim et al., 2018). This establishes an important domain-specific caveat: the usefulness of routing may depend on whether the task genuinely benefits from rigid part-whole assignment.

Speech recognition introduces a different scaling issue: sequence length. "Sequential Routing Framework: Fully Capsule Network-based Speech Recognition" proposes slice-by-slice sequential dynamic routing with weight sharing across time windows, so parameter count depends on window size rather than utterance length. The paper states that the method can operate in a non-iterative manner without dropping accuracy, reports 16.9% WER on Wall Street Journal and 17.5% PER on TIMIT, and shows about 1% lower PER for sequential dynamic routing than standard dynamic routing in a small TIMIT setup (Lee et al., 2020). In this setting, routing is no longer a global layerwise procedure but a temporally initialized local inference step.

6. Design principles and open problems

Several recurrent design principles emerge from the literature. One is to decouple routing from class count. The feature-capsule architecture of 2019 explicitly proposes using dynamic routing as an intermediate feature-learning mechanism rather than as the final class-routing mechanism, with the broader implication that capsules may be most effective when they learn equivariant intermediate representations and defer final discrimination to a lighter classifier (Mandal et al., 2019). Another is to reduce or remove the dependence of gradient flow on tiny coupling coefficients. "Adaptive Routing Between Capsules" argues that many learned $c_{ij}=\frac{\exp(b_{ij})}{\sum_k \exp(b_{ik})},$1 values are around 0.1 or smaller, which suppresses gradients and makes deep stacking difficult, and proposes adaptive routing with

$c_{ij}=\frac{\exp(b_{ij})}{\sum_k \exp(b_{ik})},$2

to support multiple capsule layers. It reports better results than dynamic routing on CIFAR10, Fashion-MNIST, SVHN, and MNIST, but also shows that very small $c_{ij}=\frac{\exp(b_{ij})}{\sum_k \exp(b_{ik})},$3 values cause near-random collapse, especially in deeper models (Ren et al., 2019).

A second principle is that non-iterative approximations are attractive when hardware efficiency matters. Self-attention routing, attention routing, and parallel branch routing all trade exact iterative refinement for matrix-parallel operations or reduced capsule counts (Mazzia et al., 2021, Choi et al., 2019, Javadinia et al., 2023). A third principle is that routing behavior is strongly shaped by entropy and polarization. Low-entropy routing may be necessary for parse-tree-like compositionality (Venkatraman et al., 2020), yet excessive iteration can push the system toward extreme winner-take-all assignments (Paik et al., 2019, Ye et al., 8 Jan 2025). This tension remains unresolved.

The open problems identified by the papers are correspondingly specific. The scalable feature-routing paper notes that further analysis is needed on how feature capsules relate to equivariance and acknowledges that reconstruction from the full feature capsule set makes class-specific reconstruction impossible (Mandal et al., 2019). The convergence work clarifies what objective standard routing optimizes but does not imply that the resulting optimum is the most useful one for recognition (Ye et al., 8 Jan 2025). The discriminative-routing literature argues that agreement alone is not aligned with classification (Yang et al., 2021), while the compositionality literature argues that routing remains essential if the aim is structured parse-tree behavior rather than merely competitive accuracy (Venkatraman et al., 2020). As a result, dynamic routing in capsules remains both a foundational idea and an active design space: one in which agreement-based iteration, constrained optimization, generative inference, entropy control, and domain-specific approximations are all viable but differently motivated interpretations of the same underlying capsule-to-capsule assignment problem.