Differentiable Syndrome Loss in Decoding

• Differentiable Syndrome Loss is a parity-check-based training objective that replaces a discrete syndrome test with a backpropagation-friendly soft surrogate for neural decoders.
• It integrates code structure into training by penalizing outputs that violate parity constraints, thereby supporting unsupervised learning and online adaptation under varying channel conditions.
• Variants such as frozen-bit and CRC-enabled syndrome loss extend its application to polar decoding, yielding measurable improvements in frame and block error rates.

Searching arXiv for the primary papers on differentiable syndrome loss and its polar-code adaptations. Differentiable syndrome loss function denotes a parity-check-based training objective for neural error-correcting decoders in which the discrete syndrome test is replaced by a soft surrogate that is compatible with backpropagation. The method was introduced as a relaxation of the syndrome for binary linear codes and later adapted to polar decoding through frozen-bit and CRC-derived constraints, thereby enabling unsupervised learning when transmitted labels are unavailable and supporting online adaptation to changing channel conditions (Lugosch et al., 2018, Teng et al., 2019, Teng et al., 2020).

1. Origin and conceptual basis

The central motivation for differentiable syndrome loss is that ordinary bitwise objectives such as binary cross-entropy treat decoding as standard binary classification, whereas coded communication imposes dependencies among output bits through code structure. In this view, error correction is a structured prediction problem: a decoder output must not only be locally plausible at each bit position, but also globally consistent with the code constraints. Syndrome loss injects that structure directly into training by penalizing outputs that do not correspond to valid codewords (Lugosch et al., 2018).

For a binary linear code with parity-check matrix $H$, a candidate codeword is valid if its syndrome is zero over $GF(2)$. The original syndrome-loss formulation replaces the need for the transmitted codeword with a constraint-based objective computed from the decoder output and the code structure alone. This makes the loss attractive not only as a regularizer during supervised training but also as a mechanism for unsupervised learning and potential online adaptation. The loss is used only during training, so the decoder architecture at inference time is unchanged and there is no additional inference cost (Lugosch et al., 2018).

A recurrent theme in subsequent work is that the usefulness of syndrome loss depends on whether the decoder exposes outputs and structural constraints in a form compatible with parity checking. This is straightforward for BCH and LDPC decoders with explicit parity-check matrices, but it is nontrivial for polar decoders, whose standard formulations are organized around factor-graph message passing and source-bit estimates rather than direct soft codeword outputs (Teng et al., 2019).

2. Mathematical formulation and differentiability

For a binary linear code with parity-check matrix $\bm{H} \in GF(2)^{(n-k)\times n}$ and decoder soft output $\bm{s} \in \mathbb{R}^n$, the hard decision is written as

$$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$

The hard syndrome is then determined by the parity-check constraints; in bipolar form, the $i$-th check is

$$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$

where $\mathcal{M}(i)$ is the set of variable-node indices participating in the $i$-th parity-check equation (Lugosch et al., 2018).

Because the hard syndrome is unsuitable for gradient-based learning, the differentiable surrogate is the soft syndrome

$$\operatorname{softsynd}(\bm{s})_i=\min_{j\in\mathcal{M}(i)} |s_j| \prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j).$$

The associated syndrome loss is

$GF(2)$0

This quantity is small when each parity check is confidently satisfied and increases when checks are violated or weakly supported (Lugosch et al., 2018).

The construction is directly related to iterative decoding: the soft syndrome is exactly the check-node equation from min-sum decoding. That correspondence is important because it gives the surrogate a non-trivial gradient while preserving the logic of parity satisfaction. Strictly speaking, the $GF(2)$1 and $GF(2)$2 operations are not smooth everywhere, but the loss is a soft, backpropagation-friendly surrogate for hard parity checking. In that sense it is suitable for differentiable training (Lugosch et al., 2018, Teng et al., 2019).

The relaxed syndrome also preserves information that the discrete syndrome discards. In the $GF(2)$3-Hamming example given in the original work, the hard syndrome is

$GF(2)$4

whereas the soft syndrome is

$GF(2)$5

The violated checks are therefore not only identified but quantified as weakly violated rather than strongly contradicted (Lugosch et al., 2018).

3. Relation to supervised objectives

The standard supervised alternative in this literature is binary cross-entropy. For the neural BP polar decoder, the supervised loss is written as

$GF(2)$6

where $GF(2)$7 (Teng et al., 2020).

The distinction between the two objectives is structural. Binary cross-entropy requires the true transmitted codeword or message, whereas syndrome loss does not require labels and depends only on the decoder output and the code constraints. The original formulation also combines the two in a composite objective,

$GF(2)$8

with $GF(2)$9 corresponding to standard supervised training, $\bm{H} \in GF(2)^{(n-k)\times n}$0 to supervised training with syndrome regularization, and $\bm{H} \in GF(2)^{(n-k)\times n}$1 to purely unsupervised syndrome-loss training (Lugosch et al., 2018).

This formulation supports two distinct uses. First, syndrome loss can regularize supervised training by forcing the network toward outputs that satisfy parity constraints. Second, it can serve as the sole objective when transmitted labels are unavailable. The latter possibility is the basis for unsupervised decoder adaptation under channel variation. At the same time, syndrome loss alone does not guarantee meaningful decoding: a decoder could output any valid codeword unrelated to the received signal and still incur zero syndrome loss. The original work therefore emphasizes that some prior information about decoding must already be present, for example through a neural belief-propagation architecture whose graph structure reflects the code (Lugosch et al., 2018).

4. Polar-code reformulation via frozen-bit constraints

Standard syndrome loss cannot be applied to polar decoding directly because polar decoders in this setting output source bits or message bits $\bm{H} \in GF(2)^{(n-k)\times n}$2, not a soft codeword estimate with a conventional parity-check matrix. The obstacle is therefore twofold: there is no natural parity-check matrix in the usual LDPC/BCH sense, and the decoder output is not naturally a soft codeword suitable for direct check-node syndrome computation (Teng et al., 2019, Teng et al., 2020).

The polar-code modification exploits the defining property of frozen bits. A polar code of length $\bm{H} \in GF(2)^{(n-k)\times n}$3 and dimension $\bm{H} \in GF(2)^{(n-k)\times n}$4 is formed by

$\bm{H} \in GF(2)^{(n-k)\times n}$5

with

$\bm{H} \in GF(2)^{(n-k)\times n}$6

Because $\bm{H} \in GF(2)^{(n-k)\times n}$7 and $\bm{H} \in GF(2)^{(n-k)\times n}$8, the relation can be rewritten as

$\bm{H} \in GF(2)^{(n-k)\times n}$9

If $\bm{s} \in \mathbb{R}^n$0 is the information-bit index set and $\bm{s} \in \mathbb{R}^n$1 is the frozen set, then frozen bits satisfy

$\bm{s} \in \mathbb{R}^n$2

This yields a parity-check-like matrix for syndrome checking,

$\bm{s} \in \mathbb{R}^n$3

or, in the later notation, $\bm{s} \in \mathbb{R}^n$4 (Teng et al., 2019, Teng et al., 2020).

The neural decoder remains a belief-propagation decoder on the polar factor graph. Its soft output after $\bm{s} \in \mathbb{R}^n$5 iterations is formed from the final left and right messages:

$\bm{s} \in \mathbb{R}^n$6

or equivalently

$\bm{s} \in \mathbb{R}^n$7

This soft output is then inserted into the same soft-syndrome expression used for linear block codes, but now with the polar-specific matrix derived from frozen-bit positions. The resulting frozen-bit syndrome loss is therefore not the classical code syndrome; it is a differentiable check of whether the estimated soft output obeys the frozen-bit-induced constraints (Teng et al., 2019, Teng et al., 2020).

5. Variants, architectures, and training procedures

The main variants described in the literature are summarized below.

Variant	Constraint source	Principal use
Syndrome loss	Parity-check matrix $\bm{s} \in \mathbb{R}^n$8 of a linear block code	BCH, LDPC, short-block-code neural decoders
Frozen-bit syndrome loss	Rows of the polar generator matrix indexed by frozen positions	Neural BP or RNN-based polar decoder
CRC-enabled syndrome loss	CRC parity-check matrix $\bm{s} \in \mathbb{R}^n$9 on the information-bit set	CRC-polar decoding and syndrome-enabled blind equalization

In the polar setting, the decoder is a neural-network-based belief-propagation decoder with trainable scaling weights. The iterative updates use trainable parameters $$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$0 and $$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$1 to scale right-to-left and left-to-right messages in the factor graph, and training proceeds by stochastic gradient descent:

$$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$2

For the unsupervised polar-decoder experiments, the practical setup is explicitly specified as polar code $$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$3, BPSK modulation, SNRs from $$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$4 to $$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$5 dB, $$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$6 training codewords per SNR, $$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$7 testing codewords per SNR, mini-batch size $$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$8, learning rate $$\hat{\bm{x}}=\operatorname{sign}(\bm{s}), \qquad \hat{\bm{c}}=0.5-0.5\hat{\bm{x}}.$$9, and SGD optimizer (Teng et al., 2019).

A later extension introduces multi-loss across all BP iterations. Instead of computing the soft output only at the final iteration, the formulation includes

$i$0

and averages the frozen-bit syndrome loss over iterations. The same paper also defines a CRC-enabled syndrome loss for CRC-polar concatenated codes. The CRC soft output is computed only on the information-bit set,

$i$1

and the loss averages $i$2 over CRC checks and iterations (Teng et al., 2020).

The same CRC-enabled and frozen-bit objectives are also used beyond decoder training. In the syndrome-enabled blind equalizer, the received signal is equalized by a filter $i$3, converted to LLRs, and decoded by a pretrained, frozen NN-BP polar decoder. The equalizer alone is adapted by SGD on the syndrome loss, so optimization is performed against decoder-side failure rather than mean-squared error or a known training sequence (Teng et al., 2020).

6. Empirical behavior, limitations, and related surrogate-loss perspective

The empirical record is mixed but consistent in its main pattern. In the original syndrome-loss paper, supervised experiments on neural normalized min-sum decoders for $i$4 BCH, $i$5 LDPC, $i$6 polar, and $i$7 LDPC used Adam with learning rate $i$8, $i$9 minibatches, $$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$0 codewords per minibatch, noise drawn uniformly from all SNRs, the all-zeros codeword during training, and random codewords during testing. With $$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$1, syndrome regularization yielded a small but consistent FER improvement across all SNRs relative to pure cross-entropy training. At SNR $$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$2, the $$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$3 BCH FERs were $$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$4 for no learning, $$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$5 for $$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$6, and $$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$7 for $$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$8; for the $$\operatorname{synd}(\bm{s})_i=\prod_{j\in\mathcal{M}(i)} \operatorname{sign}(s_j),$$9 polar code, the corresponding values were $\mathcal{M}(i)$0, $\mathcal{M}(i)$1, and $\mathcal{M}(i)$2 (Lugosch et al., 2018).

For unsupervised polar decoding, the frozen-bit formulation converges stably. The validation-loss curves over the first $\mathcal{M}(i)$3 epochs show convergence at a rate similar to BCE, with stable training and small variance across seeds. In FER, however, supervised BCE training performs best, while the proposed unsupervised frozen-bit syndrome loss gives slight improvement over conventional BP and remains worse than supervised training. The explanation given is that the frozen-bit constraint is only partial and does not constrain the whole polar factor graph as comprehensively as full supervision (Teng et al., 2019).

The later CRC-enabled extension strengthens the constraint from frozen positions to block-level CRC consistency. That change alters the empirical picture. The paper reports that the CRC-enabled syndrome loss has steeper loss and BLER improvement curves than BCE, that frozen-bit syndrome loss works but only modestly improves over conventional BP, and that CRC-enabled syndrome loss can outperform BCE in BLER. In the blind equalizer, the proposed method matches or exceeds MMSE-based methods in BLER, gains about $\mathcal{M}(i)$4 dB over MMSE with RNN-BP decoder in the time-invariant case, and under block fading outperforms MMSE with training sequence by about $\mathcal{M}(i)$5 dB when $\mathcal{M}(i)$6, with about $\mathcal{M}(i)$7 dB and $\mathcal{M}(i)$8 dB gains over online label recovery for $\mathcal{M}(i)$9 and $i$0, respectively (Teng et al., 2020).

Several limitations recur across the literature. First, syndrome loss can be minimized by a valid but wrong codeword, so architectural or training priors remain necessary. Second, purely unsupervised training can fail if the model is trained only on the all-zeros codeword; in the original study this could drive FER near $i$1. Two remedies were identified there: constraining weights to be positive or training on random codewords instead of only the all-zeros codeword. Using random codewords with $i$2, better FER than the no-learning baseline was achieved on a $i$3 BCH code (Lugosch et al., 2018). Third, improvement is more consistent for frame or block error rate than for bit error rate; the original paper notes that BER is sometimes better and sometimes worse, plausibly because the decoder is encouraged to output a valid codeword even if that requires changing more bits (Lugosch et al., 2018).

A related but distinct line of research addresses other non-differentiable objectives through surrogate losses. In binary classification with imbalanced data, a differentiable surrogate $i$4 loss was designed so that its gradient condition matches the large-sample $i$5 gradient condition, thereby approximating the gradient paths of the target performance measure (Lee et al., 2021). This suggests a broader methodological pattern: discrete evaluation criteria can be replaced by surrogate objectives whose gradient geometry is better aligned with the desired end metric. In the communication setting, differentiable syndrome loss instantiates that pattern through parity-check, frozen-bit, or CRC consistency rather than class-imbalance metrics.