Bitwise Over-Parameterized Neural Polar Decoding: A Theoretical Performance Analysis

Abstract: This paper proposes a bitwise over-parameterized neural network (ONN) decoder for polar-coded transmission and develops a tractable theoretical performance analysis framework. By modeling each synthesized message channel as an individual supervised regression task, the proposed decoder preserves the successive structure of polar decoding while enabling a communication-oriented integration of neural-network learning theory and polar-code reliability analysis. Under over-parameterization, we first characterize the empirical convergence behavior of each bitwise ONN and show that the training trajectory remains close to the random initialization. By expressing the empirical MSE convergence in the dB domain, the result further reveals a per-iteration training gain determined by the learning rate, the bit-channel Gram spectrum, and the training-set size. Upon this observation, we then derive a population mean squared error (MSE) bound via local generalization analysis and convert it into a bitwise decoding error bound through the posterior-margin structure of the bitwise maximum a posteriori (MAP) target. Under additive white Gaussian noise (AWGN) channels, a Gaussian approximation (GA)-based characterization of the low-margin probability is further established, which leads to explicit bounds for the bit error rate (BER) and block error rate (BLER). The analysis clarifies how the hidden-layer width affects optimization, generalization, and the final decoding performance, thereby providing theoretical guidance for network-scale selection. Numerical results validate the main theoretical findings and show that increasing the network width consistently improves both oracle-aided and sequential decoding performance.

• The paper demonstrates that over-parameterized neural networks can closely approximate the MAP rule for polar-coded systems by modeling each bit channel as an independent regression task.
• It establishes linear convergence and robust generalization bounds tied to network width, with empirical results confirming the theoretical predictions.
• The work derives explicit BER/BLER error bounds based on activation patterns and channel reliability, offering actionable design guidance for neural decoders.

Bitwise Over-Parameterized Neural Polar Decoding: A Theoretical Performance Analysis

Problem Formulation and Motivation

The paper investigates a bitwise over-parameterized neural network (ONN) architecture for polar-coded communication systems, where each synthesized message channel is modeled as a supervised regression task within a deep learning framework. Conventional polar code decoders, including SC, SCL, and SCS, are fundamentally limited by linear recursive operations and their performance in finite-length regimes is highly dependent on how efficiently the synthesized channel reliability is exploited. Introducing nonlinear function approximation via neural networks allows the decoder to more closely approximate the optimal MAP rule without sacrificing the bit-channel structure.

The over-parameterization regime is leveraged to enable tractable analysis of the neural decoder's optimization and generalization behavior. Recent theoretical advances on ONNs, which describe their convergence properties and local generalization behavior, are combined with polar code reliability characterization to yield performance guarantees for decoding accuracy. The core question addressed is how neural architecture width directly affects optimization, generalization, and decoding reliability—beyond empirical simulation results.

System Architecture and Bitwise ONN Decoder

The end-to-end system consists of a polar-coded transmitter using BPSK modulation over an AWGN channel. Each message bit is decoded at the receiver by a dedicated ONN whose input comprises the received signal and preceding bit decisions, maintaining the successive structure required by polar codes.

Figure 1: End-to-end system model of the proposed bitwise ONN polar decoder.

The ONNs are configured as two-layer fully connected ReLU networks. The training objective utilizes the soft bitwise MAP target (posterior mean) instead of hard labels, aligning the learning process with the Bayes optimal regressor under the mean square error (MSE) metric. The thresholded ONN output serves as the hard bit decision, so the decoder matches the MAP rule as long as the ONN output's sign aligns with the target's sign.

Theoretical Analysis: Optimization, Generalization, and Decoding Performance

Empirical Convergence

The analysis proves that ONN training error decays at a linear rate in the number of gradient descent iterations, provided the hidden-layer width is sufficiently large relative to the sample size and the Gram matrix's spectral properties. The empirical MSE convergence is conveniently expressed in the dB domain, providing an interpretable per-iteration training gain, which depends on the step size, Gram spectrum, and data cardinality.

Figure 2: Training loss on the most reliable synthesized message channel for different hidden-layer widths $B$.

Generalization Bounds

A local generalization framework is adopted: the population MSE is bounded by the sum of empirical MSE, a local Rademacher complexity term, a linearization remainder due to activation-pattern changes, and classical concentration terms. As network width increases, the parameter neighborhood around initialization shrinks, making the local expansion more accurate and improving the generalization bound.

Figure 3: Validation MSE on the most reliable synthesized message channel for different hidden-layer widths $B$.

The theory also establishes that parameter updates remain within an $O(B^{-1/2})$-radius of initialization, a property that is empirically confirmed.

Figure 4: Maximum neuron-wise deviation of the first-layer weights from their initialization on the most reliable synthesized message channel for different hidden-layer widths $B$.

Decoding Error Bounds

The conversion from population MSE to bitwise error probability leverages the posterior-margin structure of the MAP rule: ONN decisions deviate from MAP only when regression error causes sign flips within the margin region. The bitwise error probability of the ONN comprises three components:

• The intrinsic bitwise MAP error (channel reliability limit)
• The low-margin probability (posterior mass near decision boundary)
• A scaled regression-error term

This decomposition is rigorously validated with empirical results.

Figure 5: Empirical verification of the bitwise error decomposition on the synthesized message channels for the widest ONN used in the experiment at $\rho=0.6$ and ${\rm{E_b/N}_0=0$ dB.

Low-Margin Probability and Gaussian Approximation

The low-margin probability, central to the decoding bound, is expressed via the synthesized channel's LLR distribution; under the AWGN model, the Gaussian approximation (GA) yields closed-form probability in terms of channel reliability. The empirical low-margin probabilities align tightly with GA-based predictions.

Figure 6: Empirical low-margin probability versus its GA-based prediction for the synthesized message channels at ${\rm{E_b/N}_0=0$ dB.

BER and BLER Performance

The overall BER and BLER bounds are obtained by aggregating the per-channel contributions. Numerical results show decoding performance reliably improves with increasing hidden-layer width for both oracle-aided and sequential decoding modes.

Figure 7: BER performance of the proposed bitwise ONN decoder in the oracle-aided mode for different hidden-layer widths $B$.

Figure 8: BLER performance of the proposed bitwise ONN decoder in the oracle-aided mode for different hidden-layer widths $B$.

Figure 9: BER performance of the proposed bitwise ONN decoder in the sequential mode for different hidden-layer widths $B$.

Figure 10: BLER performance of the proposed bitwise ONN decoder in the sequential mode for different hidden-layer widths $B$0.

Numerical Verification

Experimental results strongly validate the theoretical predictions. The empirical training loss, validation MSE, parameter drift, and low-margin characterization all follow the behavior forecast in the analytical framework. The BER/BLER curves demonstrate consistent improvement with increased network width, reflecting the role of over-parameterization in both optimization and generalization. Oracle-aided and sequential modes both benefit from wider ONNs, although error propagation in the sequential mode produces a moderate penalty.

Implications and Future Directions

The study establishes a formally grounded link between ONN optimization theory and polar-code reliability analysis, generating explicit, interpretable bounds for bitwise and block error rates. By quantifying how network width interacts with statistical learning and channel reliability, the analysis provides actionable guidance for network-scale design in learning-based decoding systems.

The methodology—modeling each synthesized channel as an independent regression task and leveraging over-parameterization for theoretical tractability—can be generalized to other communication-oriented learning settings. One implication is that principled network scaling, informed by channel metrics and population error bounds, can supplant heuristic hyperparameter tuning in practical deployment of learning-assisted decoders. Future research may extend this framework to multi-layer architectures, joint bitwise modeling, and broader classes of codes/channels, as well as investigate robustness to non-AWGN impairments.

Conclusion

Bitwise over-parameterized neural polar decoding achieves nonlinear function approximation of MAP decoding within the successive bit-channel structure, enabling theoretical guarantees on optimization, generalization, and decoding reliability. The analysis integrates ONN theory and polar-code reliability characterization, yielding explicit BER/BLER bounds where network width plays a central role. Empirical results confirm theoretical claims, and the framework provides practical guidance for neural decoder design. The study advances the formal analysis of learning-based channel decoding and sets the stage for further performance-driven architectures in coded communication systems.