Feedforward Linear Networks Overview

• Feedforward Linear Networks (FFNs) are defined as directed acyclic graphs where each node (except the root) has a unique predecessor, ensuring a clear causal order.
• They employ information-theoretic measures, using forward and backward Shannon entropies, to compute a hierarchical index that quantifies the balance between causal richness and predictability.
• The analysis differentiates hierarchical, anti-hierarchical, and non-hierarchical configurations, offering insights for designing robust network architectures and managing connectivity trade-offs.

Feedforward Linear Networks (FFN), as investigated in the context of graph-theoretical and information-theoretic analysis, are networks whose topology can be abstracted as a directed acyclic graph (DAG) with a clearly defined causal ordering. In contrast to generic ordered systems, an FFN is said to be hierarchical if it satisfies specific structural properties—most notably the combination of definiteness, predictability, and a pyramidal, multi-scale organization. The quantification of hierarchy in these networks, along with the identification of anti-hierarchical and non-hierarchical forms, is central to understanding the topological and functional diversity of systems that can be abstracted in this manner (Corominas-Murtra et al., 2010).

1. Structural Definition and Hierarchical Criteria in FFNs

A feedforward linear network is defined as a connected directed acyclic graph (DAG) in which the directionality of arcs encodes causal precedence. The notion of "hierarchy" in such networks is refined beyond mere acyclicity or topological order. The conditions that must be satisfied for a structure to be maximally hierarchical are:

1. Order (Definiteness): Every node, except the unique maximal node ("root"), has exactly one immediate predecessor, ensuring a single causal pathway for every downstream node.
2. Predictability: The causal flow must be reversible in a deterministic fashion: retracing the path from any terminal ("leaf") node to the unique root must incur zero uncertainty. This is quantified as minimum entropy in the reverse flow.
3. Pyramidal Structure: The network must possess layers (or strata) such that the number of nodes increases monotonically from the unique root towards the leaves: $|ω_1| < |ω_2| < \ldots < |ω_{|W|}|$, where $ω_i$ denotes the set of nodes at layer $i$.

Networks that satisfy all three are termed perfectly hierarchical. This architecture enforces an informational and topological asymmetry: a "fan-out" as information propagates forward, and a uniquely traceable, contractionary structure backward.

2. Information-Theoretic Hierarchical Index

To formalize hierarchy, two Shannon entropies are introduced:

• Forward Entropy $H(G|M)$: Measures uncertainty in information pathways from the maximal node $M$ (root) to the set of minimal nodes $\mu$ (leaves). Defined via path probabilities, where for a path $\pi_k$ originating at $v_i$:

$$P(\pi_k \mid v_i) = \prod_{j \in v(\pi_k)} \frac{1}{k_{in}(v_j)}$$

The entropy sums the uncertainty introduced at each choice-point along all possible forward paths.

• Backward Entropy $H(G|\mu)$: Quantifies uncertainty when reversing the causal flow from leaves to root, again using path probabilities but reflecting the in-degree at each node.

The two entropies are combined in a normalized functional:

$$f(G) = \frac{H(G|M) - H(G|\mu)}{\max\{H(G|M), H(G|\mu)\}}$$

$f(G)$ attains $+1$ for a maximally hierarchical (forward-rich, backward-predictable) structure and $-1$ for an anti-hierarchical (reverse pyramid, backward-rich) structure.

To obtain a robust, scale-invariant hierarchical measure, the network is recursively dissected using layer-wise leaf removal algorithms, yielding successively smaller subgraphs. The global hierarchical index $\nu(G)$ is the average of $f(G)$ across the original graph and all subgraphs induced by iterative leaf (or root) removal:

$$\nu(G) = \frac{f(G) + \sum_{i < |W|-1} \left[f(G_i) + f(\tilde{G}_i)\right]}{2|W| - 3}$$

where $|W|$ is the number of layers, $G_i$ are top-down subgraphs, and $\tilde{G}_i$ are bottom-up subgraphs.

A symmetrized version replaces the denominator with the $\log$ of the total number of paths, motivated by Jensen's inequality:

$$g(G) = \frac{H(G|M) - H(G|\mu)}{\log |\Pi_{M\mu}(G)|}$$

and

$$\nu_s(G) = \frac{g(G) + \sum_{i < |W|-1} \left[g(G_i) + g(\tilde{G}_i)\right]}{2|W| - 3}$$

This hierarchical index is a rigorous, information-theoretic order parameter for hierarchy in any feedforward causal DAG.

3. Classification: Hierarchical, Anti-hierarchical, and Non-hierarchical Networks

Three qualitative regimes, characterized by their index values and structural features, are identified:

Regime	Structure Description	Hierarchical Index ($\nu(G)$)
Hierarchical	Feedforward tree: unique root, unique parent per node,	$+1$ (maximal)
	number of nodes increases with distance from root
Anti-hierarchical	Inverted tree: multiple roots converge to one leaf	$-1$ (minimal)
Non-hierarchical	Linear chain, fully connected DAG, or cliques	$0$ (null)

• Hierarchical: Maximal information richness in the forward direction; completely predictable reversibility (unique parent for every downstream node); pyramidal expansion.
• Anti-hierarchical: Information converges rather than fans out; maximal richness and predictability in the reverse direction.
• Non-hierarchical: Forward diversity and backward uncertainty are equal and thus cancel, as in linear chains or fully connected cliques.

Numerical and example analyses in the paper demonstrate that hierarchical deviations (e.g., adding extra arcs to a tree) monotonically reduce $|\nu(G)|$ towards zero.

4. Network Examples and Numerical Illustrations

The distinction between network types is grounded in concrete models and numerical index calculations:

• Hierarchical Trees (Fig. 2a): A perfectly symmetrical tree yields $\nu(G)=+1$. Asymmetrical trees (Fig. 2b) remain hierarchical, but the index value is slightly reduced, reflecting deviation from perfect symmetry.
• Linear Chains (Fig. 2c, 2e): Both forward and backward entropies vanish ($H(G|M) = H(G|\mu) = 0$), so $\nu(G)=0$.
• Fully Connected DAGs ("cliques"): Forward path richness is negated by complete unpredictability in backward traversal; $\nu(G)=0$.
• Inverted Trees and Star Graphs (Fig. 2d, 2f): Induce strong anti-hierarchicality ($\nu(G)<0$).

Numerical studies confirm that successive addition of redundant arcs drives the hierarchical index towards zero, signaling a loss of hierarchical structure.

5. Interpretation and Design Implications

This formalism has substantive implications for theoretical modeling and practical network design:

• Quantitative Assessment: The hierarchical index provides a basis for ranking and comparing the organizational quality of systems, whether in biology, engineered systems, or information processing networks, where feedforward causality is a central organizing principle.
• Optimal Architectures: Classical fan-out trees (binary or $k$-ary) are optimal if diversity in onward causal pathways and reversibility are priorities. Conversely, excessive connections or cycles (even if causal order persists) degrade hierarchy.
• Connectivity Trade-offs: The index quantifies the trade-off between redundancy for robustness and loss of predictable signal flow. Excessive redundancy increases backward uncertainty, reducing $\nu(G)$.
• Generality: The concepts extend to any domain where DAG-based causal graphs are an appropriate abstraction, such as social or regulatory networks.

6. Broader Context and Extensions

The theoretical apparatus developed for FFNs in this context demonstrates a rigorous way to distinguish hierarchy from pure order, using entropy-based measures that balance richness of causal propagation with reversibility. This framework bridges graph theory, information theory, and network science, establishing a foundation for the systematic paper of causality and organization in complex directed networks. Extensions to non-feedforward or cyclic systems require modified formalisms, as this index is fundamentally defined for DAGs.

7. Summary Table: Core Properties of Hierarchical Index in FFNs

Property	Formalization / Criterion
Order (Definiteness)	Unique parent per node, root has no parents
Predictability	Zero entropy in backward traversal
Pyramidal structure	$|ω_1| < |ω_2| < ... < |ω_{|W|}|$
Hierarchical Index $f(G)$	$$f(G) = \frac{H(G|M) - H(G|\mu)}{\max\{H(G|M), H(G|\mu)\}}$$
Global Index $\nu(G)$	Averaged $f(G)$ across all layers/subgraphs
Regime boundaries	$+1$ (hierarchical), $0$ (non-hierarchical), $-1$ (anti-hierarchical)

In conclusion, the detailed hierarchy index and its mathematical basis provide a robust metric for the design, analysis, and optimization of feedforward linear networks, allowing for precise characterization of their structural and functional properties (Corominas-Murtra et al., 2010).