Semantic Error Correction and Decoding for Short Block Codes

Abstract: This paper presents a semantic-enhanced receiver framework for transmitting natural language sentences over noisy wireless channels using multiple short block codes. After ASCII encoding, the sentence is divided into segments, each independently encoded with a short block code and transmitted over an AWGN channel. At the receiver, segments are decoded in parallel, followed by a semantic error correction (SEC) model, which reconstructs corrupted segments using LLM context. We further propose the semantic list decoding (SLD), which generates multiple candidate reconstructions and selects the best one via weighted Hamming distance, and a semantic confidence-guided HARQ (SHARQ) mechanism that replaces CRC-based error detection with a confidence score, enabling selective segment retransmission without CRC overhead. All modules are designed and trained using bidirectional and auto-regressive transformers (BART). Simulation results demonstrate that the proposed scheme significantly outperforms conventional capacity-approaching short codes and long codes at the same rate. Specifically, SEC provides approximately 0.4 dB BLER gain over plain short-code transmission, while SLD extends this to 0.8 dB. Compared to transmitting the entire sentence as a single long 5G LDPC codeword, our approach significantly improves semantic fidelity and reduces decoding latency by up to 90\%. SHARQ further provides an additional 1.5 dB gain over conventional HARQ.

• The paper introduces a novel framework that integrates semantic error correction into short block codes to address reliability and latency challenges in URLLC.
• It employs a fine-tuned BART model for semantic reconstruction and list decoding, achieving up to 0.8 dB BLER improvement over traditional methods.
• The approach eliminates CRC overhead and reduces latency by 76–90%, while maintaining high semantic fidelity as measured by BLEU and ROUGE.

Semantic Error Correction and Decoding for Short Block Codes: An Expert Review

Introduction and Motivation

Ultra-reliable and low-latency communications (URLLC) in 5G—and extending to future 6G—demand both stringent reliability and minimal transmission latency. Conventional block codes with long blocklengths (e.g., LDPC) offer near-Shannon-capacity performance but introduce prohibitive latencies, which is incompatible with short control packets in URLLC and IoT. By contrast, short block codes yield lower latency but suffer steep reliability penalties in the finite blocklength regime. This work investigates the integration of semantic intelligence, via pretrained LLMs, into the physical-layer error correction process for segmented natural-language payloads, aiming to bridge this gap in reliability and latency.

System Architecture: Parallel Short Block Codes and Semantic Modules

The proposed architecture is built around the Multiple Short Code (MSC) framework, where natural language sentences are partitioned into fixed-size segments. Each is independently encoded by a short block code and transmitted over a memoryless AWGN channel. At the receiver, segments are decoded in parallel. Failure of a subset of segments is likely, but the corresponding semantic context (surviving from correctly decoded segments) is retained, enabling downstream semantic reconstruction.

Figure 1: The MSC framework with parallel segmentation, per-segment channel coding, and downstream Semantic Error Correction (SEC) module.

After base-level OSD decoding, a fine-tuned BART (Bidirectional and Auto-Regressive Transformer) decoder operates as the Semantic Error Correction (SEC) engine, directly reconstructing corrupted segments from intact contextual segments using bidirectional self-attention. This SEC step is followed by a Semantic List Decoding (SLD) module. SLD generates multiple plausible candidate reconstructions per segment (via BART’s diverse beam search), re-encodes them, and selects the candidate minimizing weighted Hamming distance (WHD) to the soft channel observations—thereby aligning high-level semantic inference with low-level channel reliability.

Figure 2: SLD processing: BART generates candidate reconstructions which are re-encoded and ranked by WHD against the channel output.

Semantic List Decoding: Candidate Generation and Bit-level Selection

SLD error identification is performed by re-encoding SEC outputs and measuring their WHD against the received symbols. Low-confidence segments (below a probabilistic confidence threshold) are flagged. For each such segment, all others are masked; BART generates a candidate list, and WHD ranking over the list achieves probabilistically optimal selection under the error pattern.

Extraction of the reconstructed content from each candidate is non-trivial due to variable candidate lengths, but robust anchoring in surrounding correct segments enables deterministic extraction, including handling consecutive segment errors.

Figure 3: Extraction of candidate reconstructed segments is enabled by anchoring on correct segments, even for consecutive errors.

The list-based search is illustrated concretely by ranking samples within the candidate pool for a given segment and choosing the minimum WHD reconstruction.

Figure 4: WHD-based selection in SLD: only the closest candidate (e.g., “powerful”) is retained for reassembly.

Analytical Characterization

The MSC-SEC/SLD architecture is analyzed under two central assumptions:

1. Error events across segments are independent and parametrized by the number of erroneous segments;
2. Recovery probability for SEC/SLD is conditional on the total number of erroneous segments, leveraging semantic dependency among segments.

Explicit information-theoretic interpretation is provided: correctly decoded segments reduce the residual conditional entropy on the target segment, thereby improving its Fano-bound lower error probability. This quantifies the semantic side information advantage when transmission is segmented.

Numerical Results: Reliability, Semantic Fidelity, and Latency

BLER and semantic fidelity (BLEU, ROUGE) are measured as functions of SNR and segmentation. Key findings include:

• Error correction tradeoff: Shorter codes ($q$ large) yield higher semantic fidelity at low SNR; longer codes (smaller $q$) yield lower BLER and improved fidelity at high SNR. The segmentation tradeoff is governed by competing finite blocklength coding loss and semantic gain.

  Figure 5: Sentence-level BLER, BLEU, and ROUGE for (128,64) MSC, SEC/SLD, and LC baselines.

  

  Figure 6: BLER, BLEU, and ROUGE scalability for SEC across different code lengths/segmentations.

  

  Figure 7: SLD further pushes BLER lower and enhances semantic metrics, especially for highly segmented transmissions.

• List decoding benefit: SLD provides $0.8\,\mathrm{dB}$ BLER gain over baseline MSC (double the improvement of SEC-only) at the same spectral efficiency. Approximately $99\%$ of $(32,16)$ segment errors are corrected in context, with negligible semantic distortion.
• Latency reduction: Parallel segment decoding yields $76$–$90\%$ latency reduction relative to long LDPC block decoding, even after accounting for BART postprocessing (1630 ms for LC, 90–160 ms for MSC, plus 63 ms SEC or 230 ms SLD).

Semantic Confidence-Guided Hybrid ARQ (SHARQ)

Unlike conventional CRC-based HARQ (where segment-level CRCs impose crippling rate penalties on short codes), the proposed SHARQ mechanism replaces explicit CRC with a semantic-channel confidence metric derived from WHD and prior probabilities.

Retransmission is performed only for segments below the semantic confidence threshold, and, when retransmission budget is limited, segments with lowest confidence are prioritized. This strategy yields both higher error correction efficiency and near-elimination of CRC overhead.

Figure 8: BLER and ROUGE for MSC-HARQ, SHARQ, and LC-HARQ. SHARQ achieves additional $1.5\,\mathrm{dB}$ gain and almost-perfect semantic fidelity at low SNR.

Figure 9: Confidence-guided retransmission outperforms random segment selection, with larger gains as codeword length increases.

Role of Fine-Tuning and Model Architecture

Fine-tuning BART on channel-corrupted natural language is essential for substantial BLER reduction and high-fidelity recovery. Direct application of off-the-shelf pretrained models brings negligible improvements, as channel error patterns diverge substantially from generic language denoising objectives.

Figure 10: Without fine-tuning, BART adds almost no error correction capability over MSC alone; fine-tuning is essential for significant gain.

Implications for Practical Communications

The proposed architecture, built atop classical source–channel separation, offers a viable migration path for URLLC and edge scenarios demanding ultra-low latency and moderate reliability. Semantic postprocessing compensates for the finite-blocklength penalty, providing both robust error correction and graceful semantic degradation. Importantly, the approach requires no changes to the encoder/transmitter or physical-layer signaling—a critical feature for backward compatibility.

Future work can address:

• Extending semantic correction frameworks to non-textual and multilingual payloads;
• Robustness to adversarial channel modeling and structured non-AWGN errors;
• Theoretical characterization of mutual information among semantic segments for complex sources.

Conclusion

This work demonstrates a rigorous integration of pretrained generative LLMs into short block code decoding, with joint bit-level and contextual error correction. SEC and SLD tightly couple structured unsupervised inference at the semantic level with rigorous channel-aware selection, while SHARQ removes the rate penalty of CRCs for short codes. The framework achieves near-long-code BLER, maintains exceptionally high semantic fidelity, and drastically outperforms both traditional and end-to-end neural baseline methods at comparable rates and latencies. Such architectures are strong candidates for modular, semantic-aware physical layers in future URLLC systems.

Figure 11: BLER as a function of $q$ (number of segments): semantic gain compensates for finite blocklength loss, especially as segmentation becomes finer.

Figure 12: Analytical BLER expressions tightly bound simulated performance for SEC/SLD pipelines and quantify impact of key modeling choices.