Lightweight MLP Decoder in PDD Framework

• The paper introduces a lightweight MLP decoder that replaces iterative projection in LP decoding with a trained MLP-based approximation.
• It achieves significant efficiency gains by halving iteration counts and reducing multiplications and additions by nearly 45%.
• The design incorporates parameter sharing, power-of-two quantization, and custom activations to meet strict on-device resource constraints.

A lightweight MLP decoder is a neural module that leverages multi-layer perceptrons (MLPs), often in hybrid or novel architectural configurations, to efficiently translate representations (e.g., from encoder backbones or noisy channel observations) into accurate task outputs, while minimizing parameter count, computation, and latency. These designs are widely adopted for applications demanding high throughput, real-time response, or strict on-device resource constraints, such as low-complexity channel decoding, semantic segmentation, time-series analysis, and more.

1. Theoretical Motivation and Core Design Patterns

Lightweight MLP decoders are motivated by the universal approximation power of MLPs and the need to overcome the inefficiencies of traditional decoders—often dominated by iterative algorithms (e.g., LP decoders for codes, transformer self-attention in vision/language), parameter-heavy convolutional blocks, or memory-bound skip connection architectures.

Key design strategies include:

• Parameter Sharing and Quantization: Reducing parameter count through careful dimensionality choices, weight tying, and quantization (e.g., to powers-of-two for hardware efficiency).
• Nonlinear Function Approximation: Instead of iterative optimization for projection or reconstruction, an MLP is trained to approximate the critical nonlinear mapping directly (e.g., the polytope projection parameter in maximum-likelihood code decoding).
• Custom Activation Functions: Use of specialized nonlinearities (e.g., 'SinAct', lookup-table implementable activations) to boost expressivity with minimal computational penalty.

These qualities are exemplified by the CPP-net ("Neural Check Polytope Projection") MLP in the PDD framework for binary linear code decoding (Wei et al., 2020). Here, the MLP replaces a prohibitive projection step in the inner loop, delivering a one-shot or near-instant estimate that catalyzes algorithmic speed-ups:

$$\mathbf{y}^h = \mathrm{SinAct}(\mathbf{W}_a \mathbf{v} + \mathbf{b}_a), \quad \tilde{s} = \mathrm{SinAct}(\mathbf{w}_b^\top \mathbf{y}^h + b_b)$$

Where $\mathbf{v}$ is the input vector (e.g., a local codeword realization), and $\tilde{s}$ is a learned scalar projection parameter.

2. Algorithmic Integration: The PDD–MLP Hybrid Decoder

The principal use of a lightweight MLP decoder in (Wei et al., 2020) is within the Penalty Dual Decomposition (PDD) framework for LP decoding. The most computationally intensive subroutine is the projection onto the parity polytope, typically realized via iterative convex solvers. By training an MLP (CPP-net) to regress the projection hyperparameter $s$ given the current decoding state, the update for block variable $\mathbf{z}_j$ becomes:

$$\mathbf{z}_j^{k+1} = \Pi_{\mathbb{PP}_{d_j}} \left( \mathbf{P}_j \mathbf{x}^k + \frac{\mathbf{y}_j}{\mu_m} \right)$$

Replacing $\Pi_{\mathbb{PP}_{d_j}}$ with the output of the trained MLP enables the decoder to initialize the projection extremely close to the optimal solution, leading in practice to convergence within zero or a single post-MLP refinement steps.

The lightweight nature is further enhanced by quantizing weights to power-of-two domains, so that all matrix multiplications may be replaced by binary shift and add operations, removing multiplication overhead—this is directly tailored to FPGA and embedded ASIC deployment.

3. Empirical Performance, Latency, and Resource Scaling

Empirical results (see Table I and II in (Wei et al., 2020)) demonstrate that the neural decoder achieves:

Algorithm	Avg. Iterations	Worst Iterations	Multiplications	Additions
ICPP	20.4	72	366.6	244.4
NCPP	11.1	61	201.2	139.8

Here, NCPP (the MLP-based approach) halves the iteration count and reduces arithmetic operations by ~45%. Importantly, no significant degradation in error correction performance is observed versus the LP baseline.

Latency benefits stem from the one-shot nature of the MLP output; since projection calculations dominate the overall runtime, these savings directly translate to system throughput improvements—critical for communication system applications requiring real-time or near real-time decoding.

4. Training Protocols and Implementation Considerations

Training the lightweight MLP decoder involves:

• Data Pipeline: Collecting input vectors $\mathbf{v}$ and 'true' $s$ parameters from classical decoders operating at moderate-to-high SNR (to ensure an informative target distribution).
• Sample Curation: Excluding trivial easy cases (where the iterative projection converges at step one) to focus capacity on harder projections.
• Loss Function: Employing a composite objective to enforce slight overestimation (for robust convergence) and minimize mean-square-error:

$$\mathcal{L}_\mathrm{CPP-net} = \frac{1}{P} \sum_{p=1}^P (\hat{s}_p - \tilde{s}_p + \kappa \|\hat{s}_p - \tilde{s}_p\|_2^2)$$

Hyperparameter $\kappa$ controls regularization, and a positive bias is beneficial for rapid convergence in the residual projection steps.

Deployments often freeze MLP weights post-training; quantization is applied offline, with further potential for LUT-based activation function implementation (SinAct), further reducing inference cost.

5. Applicability Beyond Channel Decoding

The analysis and practical results for lightweight MLP decoders in (Wei et al., 2020) have implications for broader domains:

• Generic Convex Projection Tasks: The same neural parameter regression approach is applicable to other optimization-driven projection bottlenecks in signal processing and control.
• Encoder–Decoder Distillation: For neural or hybrid systems where non-parametric steps limit efficiency, MLP-based regression of update parameters or latent representations is a principled, scalable way to amortize cost.
• Embedded AI and Edge Deployment: The quantization and small-parameter approach taken here is directly portable to AI accelerators in communications, sensing, and time-sensitive control.

A plausible implication is that, by viewing expensive algorithmic subroutines as function approximators, lightweight MLP decoders can systemically replace slow inner loops across many classes of optimization-backed tasks, provided a relevant regression target and sufficient training coverage.

6. Comparative Analysis: Trade-offs and Limitations

The MLP-based decoder achieves superior efficiency compared to both traditional LP decoders and message-passing algorithms. However, several trade-offs are inherent:

• Expressivity vs. Size: Too small an MLP may fail to capture the nonlinear geometry of the projection task, resulting in weak estimates and loss of error correction performance.
• Hardware-optimized Quantization: Aggressive quantization is beneficial for speed but may require careful calibration to avoid accuracy loss.
• Domain Generality: While the approach is robust across LDPC code classes, re-training is needed for codes of different lengths/degree profiles.

These trade-offs must be balanced based on application domain constraints and error performance requirements; in practice, the efficiency gains of lightweight MLP decoders dominate for many real-time, high-throughput systems governed by strict resource envelopes.

7. Summary Table: Lightweight MLP Decoder in PDD Framework (Wei et al., 2020)

Attribute	Traditional ICPP	MLP-based NCPP (CPP-net)
Avg. Iterations	20–28	11–16
Multiplications/Code	366–530	201–310
Latency	High	Low
Error Correction Loss	None	Negligible
Deployment Target	Software, ASIC	FPGA/ASIC/Software

This configuration—compact, quantized, and hardware-friendly—encapsulates the defining properties and operational advantages of lightweight MLP decoders as substantiated in this work.