BP-RNN Diversity for LDPC Decoding

• The paper introduces BP-RNN decoders specialized for error-inducing absorbing sets, significantly enhancing decoding performance for short LDPC codes.
• It employs recurrent unrolling of belief propagation with trainable weights, ensuring improved reliability and controlled latency in the decoding process.
• Ensemble architectures integrate diverse specialized decoders with light-weight OSD post‐processing to efficiently approximate maximum-likelihood decoding.

Neural BP-RNN Diversity Architectures describe a class of neural decoders for short low-density parity-check (LDPC) codes that leverage recurrent neural network (RNN) unrolling of belief-propagation (BP) alongside architectural and training-driven diversity. Specialization of RNN-based BP decoders to classes of error-inducing absorbing sets, followed by ensemble and reliability-driven post-processing, brings significant advances in decoding performance for short blocklengths, nearly approaching maximum-likelihood (ML) decoding performance with controlled complexity and latency (Rosseel et al., 2022).

1. Fundamentals: Belief-Propagation as RNN

The core of BP-RNN diversity architectures is the unrolling of the BP algorithm into an RNN framework over the Tanner graph 𝒢 of the LDPC code. For an $(N, K)$ code with variable nodes $n\in\{1,\ldots,N\}$ and check nodes $m\in\{1,\ldots,M\}$, messages along edges are denoted $m_{m\to n}^{(t)}$ (check-to-variable at iteration $t$) and $m_{n\to m}^{(t)}$ (variable-to-check). The channel log-likelihood ratios (LLRs) are $L_{ch,n}$. The classical sum-product updates are:

• Check-to-variable:

$$m_{m\to n}^{(t)} = 2\ \text{atanh} \left( \prod_{n'\in\mathcal{N}(m)\setminus n} \tanh\left( \frac{m_{n'\to m}^{(t-1)}}{2} \right) \right)$$

• Variable-to-check:

$$m_{n\to m}^{(t)} = L_{ch,n} + \sum_{m'\in\mathcal{M}(n)\setminus m} m_{m'\to n}^{(t)}$$

After $T$ iterations, the a posteriori LLRs are:

$$\tilde L_n = L_{ch,n} + \sum_{m\in\mathcal{M}(n)} m_{m\to n}^{(T)}$$

The BP-RNN introduces learnable weights per pass edge:

$$m_{n\to m}^{(t)} = L_{ch,n} + w_{n\to m} \sum_{m'\in\mathcal{M}(n)\setminus m} m_{m'\to n}^{(t)}$$

All learnable weights $w,\ \tilde w$ are shared across time-steps (iterations), with training based on the binary cross-entropy loss: $$L = -\frac{1}{N}\sum_{n=1}^N \log \sigma(\tilde L_n), \qquad \sigma(x) = (1+e^{-x})^{-1}$$ This recurrent unrolled structure enables trainable flexibility while retaining BP interpretability (Rosseel et al., 2022).

2. Absorbing-Set Specialization and Decoder Diversity

BP failures at short blocklength are dominated by small absorbing sets. A subset $A\subset\{1,\ldots,N\}$ is an absorbing set if each $v\in A$ in the induced subgraph has more even-degree (all-zeros-satisfied) than odd-degree (error-detecting) neighboring checks. Each is classified by type $\nu-(\omega,\epsilon,P_c)$, denoting set size, counts of odd/even checks, and degree profile. Specialized BP-RNN decoders are trained per absorbing set type, with datasets generated by sampling vectors $y_n=+1+z_n$, $z_n$ drawn from truncated Gaussians so that $E(y) = \{n: y_n \leq 0\}$ matches $A$. Stochastic gradient descent (SGD) is performed over these specialized error patterns. This targeted training yields decoders that efficiently correct errors associated with specific structural failures (Rosseel et al., 2022).

3. Ensemble Architectures and Diversity Selection

Let $J$ denote all available specialized decoders $\{D_j\}$. To optimize diversity, a greedy selection constructs a subset $\mathcal{D}_Z = [D_{j_1},\ldots,D_{j_Z}]$ of size $Z\ll J$ that provides complementary failure coverage on a reference validation set. Two ensemble architectures are proposed:

• Parallel: All $Z$ decoders process the received word, valid codewords are pooled, and the best is selected (argmin over syndrome-valid candidates).
• Serial: Decoders are run sequentially, accepting output from the first valid codeword encountered.

If none of the ensemble outputs are valid after $I_{test}$ iterations (typically 25), reliability-driven ordered statistics decoding (OSD) of weight $w\leq 1$ is performed on each output, again filtering valid codewords and using minimum metric selection for the final estimate (Rosseel et al., 2022).

Step	Serial Ensemble	Parallel Ensemble
Run BP-RNNs	Sequentially, stop once	All in parallel
Codeword selection	First valid	ML codeword from valid set
OSD post-processing	On failures only	On pooled invalid outputs

4. Training and Inference Protocols

Training workflow:

1. Enumerate all absorbing sets up to target size $\nu_{max}$.
2. Classify and group by type, yielding $J$ distinct error classes.
3. For each, generate training data by truncated-Gaussian noise injection; train corresponding BP-RNNs with $I_{train}=10$ iterations, RMSProp ($\eta=10^{-3}$), batch size 8192, 10 epochs.
4. Optionally, train one unspecialized BP-RNN on randomly generated noise.

For inference, channel LLRs are computed and each decoder in $\mathcal{D}_Z$ is applied (parallel or serially) up to $I_{test}=25$ iterations. If unsuccessful, OSD-0 or OSD-1 post-processing is invoked per decoder, yielding $Z$ candidate codewords. The final output is the ML choice among valid candidates (Rosseel et al., 2022).

5. Performance and Complexity Characteristics

On two representative codes (Code-1: $N=64$, left-degree 3; Code-2: $N=128$, mixed left-degrees), the following key results are observed:

• Single BP-RNN vs BP at $I=25$ iterations: $\approx 0.2$ dB gain.
• Diversity ensemble $\mathcal{D}_{10}$: $\approx 0.4$ dB improvement over non-specialized BP-RNN at same iterations.
• With OSD post-processing: $\mathcal{D}_{10}$-OSD-1 achieves $\approx 0.4$ dB gain over BP-OSD-1, reaching within $0.1$ dB of ML performance for Code-1; $\mathcal{D}_{10}$-OSD-2 closes within $0.2$ dB of ML for Code-2 and outperforms BP-OSD-2 by $\approx 1.0$ dB.
• Ensemble (serial, $I_{test}=25$) matches the per-word BP(25) computational complexity while achieving higher accuracy.
• OSD invocation is reserved for rare failure cases, limiting additional complexity (Rosseel et al., 2022).

6. Distinctiveness from Generic RNN Diversity in Neural Modeling

Generic RNN diversity in neuroscientific modeling (e.g., vanilla RNN, GRU, LSTM, and UGRNN) emphasizes differences in representational geometry (SVCCA, principal angles) and sensitivity to architecture, but reports a universal topology of fixed-point and dynamical structure across architectures. This universality at the topological level contrasts with the operational diversity sought in BP-RNN decoders, where explicit specialization to error-inducing substructures (absorbing sets) is leveraged for ensemble decoding enhancement (Maheswaranathan et al., 2019). A plausible implication is that in communication decoding, unlike in neuroscience modeling, architectural and training diversity can be concretely harnessed to approach optimality for short codes by addressing failure modes non-universally distributed in the state space.

7. Practical Implications and Best Practices

Specialized BP-RNN ensemble architectures with absorbing-set-driven diversity, when paired with reliability-based OSD post-processing, efficiently bridge the gap to ML decoding for short LDPC codes without increasing worst-case latency. For code designers and practitioners, the recommended approach entails:

• Enumerating critical absorbing sets and training corresponding BP-RNN decoders.
• Selecting a compact yet diverse ensemble for runtime.
• Incorporating light-weight OSD post-processing only on ensemble failures.

This architecture delivers substantial performance improvements in the waterfall region (e.g., $0.4$–$1.0$ dB gain over standard BP) with negligible additional complexity under typical operating conditions (Rosseel et al., 2022).