Unified Transformer for Wireless Signal Processing

• The paper introduces an integrated Transformer-based framework that replaces classical modular design with an attention-driven model for wireless signal processing.
• It leverages dynamic tokenization and multi-head self-attention to adapt to tasks like channel estimation, interpolation, and end-to-end reception, achieving efficient performance.
• The approach demonstrates robust, low-latency operation under diverse conditions, paving the way for simplified, software-defined 5G/6G receiver implementations.

A unified Transformer-based architecture for wireless signal processing denotes a paradigm in which the classical modular design of communication receivers—historically comprised of independent, hand-engineered subsystems such as synchronization, channel estimation, equalization, and demapping—is replaced by an integrated, attention-driven model. This model leverages the Transformer’s multi-head self-attention mechanism, dynamic tokenization of the resource grid, and an adaptable output head to achieve low-latency, data-driven, and robust signal processing that can scale across a variety of tasks and operating conditions. The approach enables direct operation on time–frequency–antenna resource grids, consistently adapts to diverse use cases, and achieves strong accuracy and efficiency compared to traditional methods (Kawai et al., 25 Aug 2025).

1. Unified Transformer Architecture: Structure and Operation

The core design starts with tokenization of the wireless resource grid: each resource element (RE)—representing a basic unit indexed by subcarrier, symbol, and possibly antenna—is mapped to a high-dimensional embedding via a Dense projection layer. Positional encodings are then applied (e.g., by summing a deterministic function of RE indices to each embedded token) to preserve the context required for attention computation.

A stack of Transformer encoder layers—typically shallow in deployment (1–4 layers)—sequentially processes the token sequence. Each layer implements a standard sequence: $$X' = f_\text{Transformer}(X) = \mathrm{LN}\left( \mathrm{MLP}\left( \mathrm{MHSA}(X + \text{PE}) \right) \right)$$ where $\mathrm{MHSA}$ denotes the multi-head self-attention module, $\mathrm{LN}$ is layer normalization, and $\mathrm{MLP}$ is a feedforward projection. The post-processing block typically includes a final normalization, MLP, and a Dense projection to match output modalities: $$Y = \mathrm{Dense}\left( \mathrm{MLP}\left( \mathrm{LN}(f_\text{Transformer}(X)) \right) \right)$$ where $Y$ is task-adaptive: it may represent soft bits (demapping), channel coefficients, or other signal processing outputs. Early normalization is omitted to preserve amplitude information; RE tokens encode amplitude and, if present, spatial identifiers (e.g., antenna index).

2. Task Reconfiguration and Output Adaptation

A key feature is dynamic adaptation to downstream wireless PHY tasks using a shared backbone and minor output head modification. Three principal use cases are demonstrated:

• End-to-End Receiver: From pilot symbols to bit-level soft output, with the output head mapping to soft bits using binary cross-entropy loss.
• Channel Frequency Interpolation: Given a sparse set of pilot observations, the model reconstructs the full-band channel coefficients using a mean squared error (MSE) criterion.
• Channel Estimation: The model emits the complete resource grid of channel coefficients across symbols and subcarriers, also optimized via MSE.

Adapting to a new task requires changing only the final Dense layer and selecting the appropriate loss function; thus, the architecture can be flexibly reused across receiver subsystems.

3. Performance Benchmarks and Robustness

Experimental results in both simulated and over-the-air (OTA) wireless environments show:

• End-to-End BLER: The architecture achieves block error rate curves approaching those with perfect channel knowledge, outperforming classical least-squares (LS)+MMSE and CNN-based baselines, and supporting reliable detection in both single-user and multi-user MIMO scenarios.
• Channel Interpolation (OTA, OAI+Aerial platform): The Transformer, even when configured with a single layer and single attention head, achieves higher uplink throughput and lower pipeline execution latency (on the order of a few hundred microseconds) than deeper CNN baselines.
• Channel Estimation: Across a range of SNRs, the architecture reconstructs full-band channel responses with lower error, consistently outperforming linear and nearest-neighbor interpolation.

Robustness is maintained with respect to variations in user count, pilot placement, modulation order (e.g., from QPSK to 256-QAM), and channel conditions, with latency constraints satisfied (sub-millisecond processing) suitable for practical 5G/6G requirements.

4. Design Implications and Real-World Integration

The unified Transformer architecture presents several operational benefits:

• Low Latency: The shallow, attention-based structure allows sub-millisecond inference times, meeting HARQ and scheduling deadlines in modern 5G systems.
• Software-Defined Flexibility: Direct support for diverse PHY tasks (as in Table 1, below) via minor modifications simplifies pipeline design and reduces maintenance compared to modular, hand-crafted approaches.

Task	Output Head	Loss Function	Application Area
End-to-End Receiver	Soft bit mapping	BCE	Baseband RX chain
Channel Interpolation	Channel coefficients	MSE	Pilot infill
Channel Estimation	Channel grid	MSE	Channel est.

• Generalization: Strong performance across varying modulation, pilot structure, and user multiplicity; the same TF backbone generalizes to new scenarios.

5. Impact Relative to Classical Modular Pipelines

Traditional receiver designs segment tasks (e.g., synchronization, channel estimation, interpolation, demapping) into distinct processing blocks, each designed and tuned separately—often with limited adaptability to unforeseen impairments or system reconfiguration. The unified Transformer-based receiver subsumes these into a single, data-driven module, learning to extract and fuse signal features end-to-end. Classical synchronization, interpolation, or hand-engineered feature extraction steps are eliminated as the network implicitly learns canonical representations and corrections during training. This integration yields:

• Enhanced robustness in non-ideal, dynamic, or pilot-sparse environments.
• Simplified receiver implementation and maintenance, aiding in deployment of adaptive and intelligent radio access networks.

6. Scalability, Optimization, and Future Directions

The architecture can be further optimized for edge deployment and real-time constraints:

• Model Compression: Early results with highly compact configurations (1 Transformer layer, 1 head) already surpass deeper CNNs; additional gains are feasible via quantization, pruning, and early-exit schemes.
• Cross-Layer Extension: While current applications center on Layer-1 (PHY) tasks, plausible future directions include cross-layer integration—e.g., combining MAC scheduling metadata or supporting scheduling and prediction as auxiliary heads.
• Enhanced Positional Encoding: Investigating encodings tailored to the time–frequency–antenna geometry may further improve performance, especially in complex MIMO contexts.
• Robustness Mechanisms: Improvements in context memory, temporal smoothing, and cross-grid attention may further boost performance in highly nonstationary or bursty interference regimes.
• Standardization Alignment: There is a trajectory toward aligning unified Transformer-based signal processing with 3GPP Release-18 initiatives for AI/ML-native NR air interface functionalities.

7. Summary and Broader Implications

The unified Transformer-based architecture for wireless signal processing provides a data-driven, low-latency, and reconfigurable alternative to traditional modular receivers. Its adaptability, robustness, and computational efficiency render it a promising candidate for deployment in AI-native, software-defined 5G/6G systems, reducing engineering complexity while enhancing performance across receiver subsystems. The architecture is extensible, enabling integration of further PHY and cross-layer tasks, and is aligned with ongoing trends toward software-defined, machine learning-augmented radio access network (RAN) designs (Kawai et al., 25 Aug 2025).