Codes on Graphs & System-Theoretic Realizations

• Codes on graphs are subspaces defined by linking local constraint codes at vertices with state spaces on edges.
• System-theoretic realizations map concepts like observability, controllability, and duality onto graph structures, impacting decoding complexity.
• This framework unifies coding theory with algebraic topology and linear systems, enabling efficient design of homological and expander codes.

A code on a graph is a subgroup (or subspace, in the linear case) of a product space whose structure is naturally described by associating local constraint codes with the vertices and state spaces with the edges of a graph. System-theoretic realizations provide a framework in which codes can be represented as interconnections of local constraints, and in which classical concepts such as observability, controllability, minimality, and duality are precisely characterized in terms of the underlying graph structure and algebraic properties of the local codes. This unifies coding theory with graphical models, algebraic topology, and linear system theory, and has implications for the complexity and performance of decoding algorithms associated with such realizations.

1. Graphical Realization of Codes

A graphical realization of a linear code $\mathcal{C} \subset \mathbb{F}^I$ on a graph $G=(V, E)$ comprises an assignment of coordinates to vertices $\omega: I \to V$, state spaces $\mathcal{S}_e$ for each edge $e \in E$, and local constraint codes $$C_v \subset \mathbb{F}^{\omega^{-1}(v)} \oplus \bigoplus_{e \ni v} \mathcal{S}_e$$ for each vertex $v$ (0805.2199). The global behavior $$\mathcal{B} \subset \mathbb{F}^I \oplus \bigoplus_{e \in E}\mathcal{S}_e$$ consists of all assignments that satisfy all local constraints, and the code realized is the projection of $\mathcal{B}$ onto the symbol coordinates.

A realization is called essential if projections onto local variables are exactly the specified constraint codes, and a normal realization is one where each symbol variable has degree one and each state variable has degree two. Any realization can be transformed into a normal one by variable replication and equality constraints (Jr, 2013).

2. Constraint Complexity and Graph Structure

The $\kappa$-complexity of a graphical realization is defined as $$\kappa(\Gamma) := \max_{v \in V} \dim_\mathbb{F}(C_v)$$. This serves as an estimate of the decoding complexity for message-passing (sum-product) algorithms, with the computation per vertex scaling exponentially in $\dim C_v$ (0805.2199).

Lower bounds on $\kappa$-complexity are obtained via the Vertex-Cut Bound: for a star partition $(V_0, V_1, ..., V_d)$ and $J_i = \omega^{-1}(V_i)$,

$$\sum_{v \in V_0} \dim C_v \geq \dim \mathcal{C} - \sum_{i=1}^d \dim (\mathcal{C}|_{J_i}),$$

where $\mathcal{C}|_{J_i}$ is the projection or quotient by the complementary coordinates (0805.2199). The minimal constraint complexity over all cycles or paths in $G$ is related to the graph’s vc-treewidth $\operatorname{tw}_{vc}(G)$, a generalization of treewidth suited for vertex-cuts. Explicit bounds are given:

$$\kappa(\mathcal{C}; G) \geq \kappa_{\text{tree}}(\mathcal{C}) / \operatorname{tw}_{vc}(G),$$

and similarly for pathwidth. Good error-correcting codes (with large rate and distance) require realizations on graphs with large vc-treewidth to admit low-complexity decoders—thereby excluding certain families of bounded-treewidth graphs from hosting efficient realizations of asymptotically good codes.

The ratio of best trellis (path) to tree realization complexity grows at most logarithmically in code length:

$$\kappa_{\text{path}}(\mathcal{C}) \leq O(\log n) \cdot \kappa_{\text{tree}}(\mathcal{C}),$$

and this rate is tight for some code families.

3. Minimality, Trimness, and Properness

Minimality of realizations is characterized by the trimness and properness of constraint codes. A constraint $C_i$ is trim at $S_j$ if the projection onto $S_j$ is surjective; it is proper if no codeword is supported on $S_j$ alone (Jr, 2013, Jr. et al., 2012, Jr et al., 2012). These are dual properties: the dual constraint $C_i^\perp$ is proper at $S_j$ iff $C_i$ is trim at $S_j$.

For cycle-free graphs:

• Minimal realization ⇔ All constraints are both trim and proper.
• Each state space $S_j$ is isomorphic to an explicit quotient of projected codes from the induced tree subgraphs.

Local reducibility is present if any constraint fails to be both trim and proper, or equivalently if the realization is unobservable or uncontrollable. Trimming or merging state spaces strictly reduces complexity without altering the realized code.

4. Observability, Controllability, and Duality

A realization is observable if every codeword corresponds to exactly one internal state trajectory; uncontrollability refers to redundancy among constraints. There is perfect duality: observability of a realization is equivalent to controllability of its dual, and vice versa (Jr, 2013, Jr. et al., 2012, Jr et al., 2012).

A controllability dimension-count is given by:

$$\dim \mathscr{B} = \dim A + \dim S - \sum_{i \in V}\dim C_i.$$

Parity-check realizations are always observable; they are controllable if and only if the parity checks are linearly independent. In tail-biting trellises, uncontrollability leads to disconnected subbehaviors, but in general graphs, support of unobservable configurations forms generalized cycles (i.e., subgraphs where all vertex degrees are at least two).

5. System-Theoretic and Topological Perspectives

Codes on graphs are naturally interpreted via algebraic topology. Chain complexes and cochain complexes are constructed for the graph, with boundary and coboundary operators corresponding to generator and parity-check matrices (Jr, 2017). Homology and cohomology spaces—$\ker \delta^0$, $\operatorname{im}\delta^0$, $\ker\partial_1$, $\operatorname{im}\partial_1$—are realized via normal factor graphs using repetition and zero-sum nodes. All these spaces are systematic $(n,k)$ group codes.

Belief propagation and other decoding algorithms fit into this system-theoretic description, where codewords are unobservable states and syndrome computations are viewed as outputs of linear systems. Minimal I/O realizations correspond to systematic encoders defined over chosen information sets.

The duality of factor graphs exchanges kernel and image, cuts and cycles, and repetition and sum constraints. Hybrid primal-dual models further allow complexity reduction by splitting parts of a code realization into primal and dual domains, connected via Fourier transforms.

6. Homological Codes and Expander Constructions

Expander codes, notably those of Sipser–Spielman, can be realized as the first homology of graphs with twisted coefficient systems (Meshulam, 2018). Each vertex hosts a local code, and the code as a whole is defined as the kernel of the associated boundary map in the chain complex. This homological interpretation provides explicit expressions for block length, rate, and minimum distance, coinciding with the classical expander-code theorems:

• Rate $r(C) \geq 2r_0 - 1$
• Relative distance $\delta(C) \geq \delta_0 (\delta_0 - \alpha/d)$

Here, $r_0$ and $\delta_0$ are the rate and relative distance of local codes, and $\alpha$ is the expansion parameter of the graph.

The system-theoretic language extends naturally: observability and controllability are directly linked to the rank properties of the boundary map and thus to code parameters and decoding behavior.

7. Fragmentation, 2-Core Structure, and Reductions

Any realization on a cyclic graph can be decomposed into a 2-core (the maximal subgraph with minimum degree at least two) and attached cycle-free leaf fragments (Jr, 2013). The core contains the nontrivial interactions; leaf fragments are internally trim and proper and function as effective input nodes. Reductions proceed by pruning leaf fragments and merging improper/trim constraint codes.

Message-passing algorithms operate primarily on the 2-core, significantly reducing computational overhead for codes with sparse graphical realizations.

---

Key papers: "Constraint Complexity of Realizations of Linear Codes on Arbitrary Graphs" (0805.2199), "Codes on Graphs: Fundamentals" (Jr, 2013), "Observability, Controllability and Local Reducibility of Linear Codes on Graphs" (Jr. et al., 2012), "Codes on Graphs: Observability, Controllability and Local Reducibility" (Jr et al., 2012), "Graph codes and local systems" (Meshulam, 2018), "Codes on graphs: Models for elementary algebraic topology and statistical physics" (Jr, 2017).