Soft Graph Transformer for MIMO Detection

• The paper introduces SGT, which integrates factor-graph-aware attention with learnable message passing to improve MIMO symbol detection accuracy and efficiency.
• SGT employs graph-aware tokenization by embedding channel state information and bit-level LLRs, facilitating robust soft-input and soft-output processing in IDD pipelines.
• SGT is extended in SGDiT to incorporate noise-level conditioning and stage-aware normalization, achieving near-ML performance and significant BLER improvements.

Soft Graph Transformer (SGT) denotes a soft-input–soft-output neural architecture for MIMO detection that marries factor-graph-aware attention with learnable message passing for symbol detection and seamless integration into iterative detection-decoding (IDD). In its original formulation, SGT operates directly on bit-level log-likelihood ratios (LLRs), embeds channel state information (CSI) into node features, and propagates information via directed cross-attention across the bipartite graph of observation and symbol tokens. In subsequent work, the same SGT serves as the neural backbone of the Soft Graph Diffusion Transformer (SGDiT), where it is conditioned on a denoising stage and used in a progressive noise-level-conditioned detection process (Hong et al., 16 Sep 2025).

1. Detection problem and factor-graph formulation

SGT is defined for MIMO symbol detection under the linear observation model

$$\mathbf{y}_{\mathbb{C}} = \mathbf{H}_{\mathbb{C}} \mathbf{x}_{\mathbb{C}} + \mathbf{n}_{\mathbb{C}},$$

with real-valued equivalent

$\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$

where $\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$, $\mathbf{x} \in \mathbb{R}^{2N_t}$, and $\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$. The noise is Gaussian and is often modeled as $$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$ with $$\boldsymbol{\Sigma} = \mathrm{diag}(\sigma_1^2, \ldots, \sigma_{2N_r}^2)$$. Within this setting, maximum-likelihood detection remains optimal in accuracy but has exponential complexity in the number of transmit symbols, while conventional message passing methods such as AMP, OAMP, and VAMP rely on large-system random matrix assumptions that deteriorate in practical small-to-medium MIMO settings. The stated motivation for SGT is therefore to retain graph-structured reasoning while remaining compatible with decoder-side soft information and practical IDD pipelines (Hong et al., 16 Sep 2025).

The architectural starting point is the dense bipartite factor graph induced by the MIMO system. Observation, or factor, nodes correspond to linear constraints from rows of $\mathbf{H}$, while variable nodes correspond to transmitted symbols or bits. This formulation is central because prior Transformer-based detectors are described as failing to incorporate the MIMO factor graph structure and as being unable to utilize decoder-side soft information. SGT addresses both issues by making the graph explicit in its tokenization and by preserving a soft-input–soft-output interface throughout the detection loop (Hong et al., 16 Sep 2025).

2. Graph-aware tokenization and soft-input representation

At iteration $l$, SGT constructs two token sets. The first consists of linear-constraint tokens, one per receive dimension,

$\tau_i^{(l)} = (y_i, \mathbf{h}_i, \sigma_i^2),$

where $\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$0 is the $\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$1-th row of $\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$2. The second consists of symbol tokens, one per transmit dimension, representing the current belief $\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$3. The full token set is written as

$\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$4

A key design choice is that CSI is absorbed into node features, so edges are treated uniformly and no explicit edge features are required, while the factor graph information is preserved in the observation tokens themselves (Hong et al., 16 Sep 2025).

The “soft” aspect of SGT has an explicit probabilistic meaning. It operates directly on bit-level a priori LLRs from the decoder. Let $\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$5 denote the number of bits per symbol and $\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$6 the total bits. The prior LLR vector $\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$7 is projected into token embeddings through

$\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$8

This preserves uncertainty in the LLR domain and aligns the representation with probabilistic inference. The associated detector-side and decoder-side LLRs are defined in the paper as

$\mathbf{y} = \mathbf{H} \mathbf{x} + \mathbf{n},$9

where $\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$0 denotes code constraints (Hong et al., 16 Sep 2025).

3. Attention as learnable message passing

The SGT backbone is a multi-layer Transformer that combines multi-head self-attention within each token type and directed multi-head cross-attention across token types. Self-attention captures intra-type dependencies, while cross-attention emulates message passing from observations to symbols and optionally in the reverse direction. The graph-aware cross-attention mechanism computes, for a query token $\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$1 and key-value token $\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$2,

$\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$3

Messages are then aggregated through the attention weights $\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$4 and value projections $\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$5. Because channel coefficients and noise variances are embedded into the observation tokens, the attention is described as naturally encoding reliability and coupling strength (Hong et al., 16 Sep 2025).

The paper explicitly interprets this mechanism as learned message passing on the MIMO bipartite graph. The attention score acts like a normalized edge weight modulated by token reliabilities, while the projected values carry messages analogous to means or logits. This suggests a close formal analogy to sum-product or expectation-propagation updates under Gaussian models, but with the weights and message transformations learned end-to-end rather than fixed by a hand-derived inference rule. The same paper also states that “soft” in SGT has two meanings: soft-input, because decoder-side a priori LLRs are embedded directly, and soft-output, because the model emits bit-wise posterior probabilities and LLRs suitable for downstream BP or Turbo decoding (Hong et al., 16 Sep 2025).

4. Soft-output generation and integration into IDD

After $\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$6 Transformer layers, the bit-token outputs $\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$7 are passed through a learnable projection and sigmoid to produce posterior probabilities and posterior LLRs:

$\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$8

When extrinsic information is required, the IDD interface uses

$\mathbf{y}, \mathbf{n} \in \mathbb{R}^{2N_r}$9

with $\mathbf{x} \in \mathbb{R}^{2N_t}$0 denoting a priori LLRs. The detector then feeds posterior or extrinsic LLRs to a belief-propagation decoder, and the resulting decoder output is re-embedded as the next detector prior through

$\mathbf{x} \in \mathbb{R}^{2N_t}$1

This alternating detector-decoder schedule is the operational definition of SGT-IDD in the paper (Hong et al., 16 Sep 2025).

The IDD loop alternates SGT detection and BP decoding for a fixed number of outer iterations $\mathbf{x} \in \mathbb{R}^{2N_t}$2, with $\mathbf{x} \in \mathbb{R}^{2N_t}$3 BP iterations per round. Damping, clipping, and trainable affine mixing between detector and decoder LLRs are used to maintain stability. The detector remains soft-input–soft-output and differentiable end-to-end. The reported integration result is that replacing MMSE-PIC in DUIDD with SGT, under the name “SGT-IDD,” and fine-tuning end-to-end delivers significant BLER improvements over DUIDD at comparable iteration budgets (Hong et al., 16 Sep 2025).

5. Training protocol, complexity, and empirical behavior

The original SGT detector is trained with binary cross-entropy on bit outputs. Training uses Perfect-CSI Rayleigh fading channels with QPSK modulation, and $\mathbf{x} \in \mathbb{R}^{2N_t}$4 is sampled uniformly in $\mathbf{x} \in \mathbb{R}^{2N_t}$5 dB. The evaluated MIMO sizes are $\mathbf{x} \in \mathbb{R}^{2N_t}$6, $\mathbf{x} \in \mathbb{R}^{2N_t}$7, and $\mathbf{x} \in \mathbb{R}^{2N_t}$8. The reported model dimension is $\mathbf{x} \in \mathbb{R}^{2N_t}$9, the detector uses 8 Transformer layers with alternating self- and cross-attention, and optimization uses AdamW with learning rate $\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$0. Batch sizes are 5000 for $\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$1 and 1000 for $\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$2 and $\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$3. The implementation is reported in Sionna v0.19.2 on Ubuntu 22.04 with an Intel i7-13700K and two RTX 4090 GPUs (Hong et al., 16 Sep 2025).

The stated inference complexity scales quadratically in node counts:

$\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$4

This is contrasted with $\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$5 for ML detection, $\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$6 for OAMP and OAMPNet, and a cubic trend for Transformer-based MIMO detectors that depend on QR preprocessing. The reported runtime per 1000 samples on an RTX 4090 is 0.09351s for $\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$7, 0.09464s for $\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$8, and 0.09498s for $\mathbf{H} \in \mathbb{R}^{2N_r \times 2N_t}$9. In the same table, the encoder-only Transformer baseline is faster but less accurate, while ML is orders of magnitude slower (Hong et al., 16 Sep 2025).

Empirically, the paper states that in Perfect-CSI Rayleigh fading with QPSK, SGT consistently outperforms deep-unfolded OAMPNet2 and the QR-preprocessed Transformer-based MIMO across $$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$0, $$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$1, and $$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$2 systems. The figures are described as showing near-ML accuracy on small systems and maintained advantages as dimensions grow. The training-dynamics analysis further states that graph-aware tokenization reduces loss on smaller systems, whereas cross-attention accelerates convergence and improves final performance as dimensions grow. These observations support the interpretation of SGT as a graph-structured alternative to both asymptotic message passing and structure-agnostic Transformer detection (Hong et al., 16 Sep 2025).

6. SGT as the backbone of the Soft Graph Diffusion Transformer

In "Soft Graph Diffusion Transformer for MIMO Detection," SGT is reused as the neural module inside SGDiT rather than as a standalone fixed-depth detector. SGDiT reformulates detection as a noise-level-conditioned denoising process defined along the interpolation path

$$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$3

Detection becomes conditional denoising: for each $$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$4, the model predicts the clean $$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$5 from $$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$6 given $$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$7. The SGT module remains a soft-graph message-passing Transformer with symbol-domain nodes and observation-domain nodes, but the soft graph is now explicitly described as having edges that are not hard-wired; instead, SGT learns a soft adjacency via attention, with weights that adapt to the channel and the current stage $$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$8 (Jiang et al., 1 May 2026).

SGDiT introduces stage-aware conditioning through adaptive layer normalization. If $$\mathbf{n} \sim \mathcal{N}(\mathbf{0}, \boldsymbol{\Sigma})$$9 denotes the input token representations to a block, the conditioned block computes

$$\boldsymbol{\Sigma} = \mathrm{diag}(\sigma_1^2, \ldots, \sigma_{2N_r}^2)$$0

$$\boldsymbol{\Sigma} = \mathrm{diag}(\sigma_1^2, \ldots, \sigma_{2N_r}^2)$$1

Training uses signal-space prediction with bit-wise cross-entropy rather than regression-based losses. The paper states that the proposed BCE-based signal-space formulation achieves the lowest BER for both $$\boldsymbol{\Sigma} = \mathrm{diag}(\sigma_1^2, \ldots, \sigma_{2N_r}^2)$$2 and $$\boldsymbol{\Sigma} = \mathrm{diag}(\sigma_1^2, \ldots, \sigma_{2N_r}^2)$$3, and that SGDiT achieves the best BER across the SNR range for $$\boldsymbol{\Sigma} = \mathrm{diag}(\sigma_1^2, \ldots, \sigma_{2N_r}^2)$$4 while approaching ML performance. For $$\boldsymbol{\Sigma} = \mathrm{diag}(\sigma_1^2, \ldots, \sigma_{2N_r}^2)$$5, performance is described as comparable to SGT, with SGT slightly better in some SNR regions. This suggests that SGDiT extends SGT by adding a controllable denoising schedule and stage conditioning, rather than replacing the underlying graph-aware detection mechanism (Jiang et al., 1 May 2026).

7. Nomenclature and common confusions

The acronym SGT is heavily overloaded in the literature, and several papers use it for models unrelated to Soft Graph Transformer. This is not a minor terminological issue, because one of the cited papers explicitly states that its SGT is not “Soft Graph Transformer” (He et al., 2019).

Paper	Expansion of SGT	Relation to the term “Soft Graph Transformer”
(He et al., 2019)	Spectral Graph Transformer	Explicitly stated to be not “Soft Graph Transformer”
(Gao et al., 2022)	Simplified Graph Transformer	Protein design encoder inside ADesign
(Zhang et al., 2021)	Syntax-guided Graph Transformer	Temporal relation extraction model
(Yang et al., 2022)	Scene Graph Transformer	Image outpainting model
(Lin et al., 1 Oct 2025)	Structure-injecting Graph Transformer	Graph-aware CBM component

A separate but related source of confusion is conceptual rather than lexical. "Graph Transformer Networks" introduces a Graph Transformer layer that performs soft selection over relation types and composes meta-path adjacencies, and its description characterizes that layer as a soft graph transformer in the sense of soft and differentiable graph transformation, but the paper does not use the term “Soft Graph Transformer” as a model name (Yun et al., 2019). By contrast, "SoftGPT" uses SGT to refer to the Generative Pre-trained Heterogeneous Graph Transformer inside a soft-object manipulation world model, again with a different expansion and application domain (Liu et al., 2023). The most precise usage of “Soft Graph Transformer” in the present corpus is therefore the MIMO detector introduced in 2025 and the backbone module inherited by SGDiT in 2026 (Hong et al., 16 Sep 2025).