Phase Tracking Partial Response DFE

• The paper demonstrates a 0.9 dB optical power margin improvement in amplifier-less coherent links using FTN-16QAM with PT-PRDFE and turbo equalization.
• It integrates partial-response channel equalization, decision-directed phase tracking via a digital PLL, and iterative turbo equalization to jointly mitigate ISI and carrier-phase noise.
• The methodology employs adaptive LMS tap updating and BCJR-based detection, enhancing spectral efficiency and robustness in high-speed coherent communication systems.

Phase Tracking Partial Response Decision-Feedback Equalizer (PT-PRDFE) is an advanced signal processing architecture that jointly addresses intersymbol interference (ISI) and carrier-phase noise in high-speed coherent communication systems. It integrates partial-response channel equalization, decision-directed phase tracking via digital phase-locked loop (PLL), and iterative turbo equalization with soft-input soft-output decoders. The PT-PRDFE demonstrates measurable performance benefits when applied to Faster-Than-Nyquist (FTN) 16QAM signaling over fiber links, notably achieving a 0.9 dB optical power margin improvement over multi-band probabilistically shaped 64QAM (MB-PCS-64QAM) when used in amplifier-less coherent links with turbo-FEC (Zou et al., 25 Jan 2026).

1. System Model and Partial-Response Targeting

The communication system under consideration employs FTN-16QAM transmission, with each in-phase (I) and quadrature (Q) channel filtered by a square-root-raised-cosine (SRRC) pulse shape whose roll-off (α=0.8) is matched to the FTN baud-rate compression. The received signal model post-coherent detection, chromatic dispersion compensation, timing recovery, and frequency offset estimation (FOE), is

$$y[n] = \sum_{k=0}^{2} h[k] \, a[n-k] \, e^{j\phi[n]} + w[n]$$

where:

• $a[n]$: transmitted 16QAM symbol sequence (FTN-compressed)
• $h[0]=h[1]=h[2]=1$, $h[k]=0$, $k\geq3$: 2nd-order partial-response impulse
• $\phi[n]$: cumulative phase noise plus residual carrier frequency offset
• $w[n]$: complex additive white Gaussian noise (AWGN), combining amplifier spontaneous emission (ASE) and receiver thermal noise

Amplifier-less short-reach links present unique equalization demands: the deliberate partial-response shaping ($H(z)=1+z^{-1}+z^{-2}$) forces controlled ISI, sharpening spectral efficiency at the cost of post-compensation complexity.

2. Joint Phase Tracking Algorithm

PT-PRDFE embeds carrier-phase extraction within the equalization loop. The block diagram is as follows:

• The input sample $y[n]$ passes through an adaptive feedforward filter ($c_0 \ldots c_M$).
• The filtered symbol is phase-rotated using the latest phase estimate $e^{-j\hat{\phi}[n]}$.
• Hard decision via a minimum-distance slicer yields $\bar{a}[n]$ (nearest 16QAM constellation point).
• Feedback filtering ($b_1 \ldots b_N$) subtracts estimated ISI, generating equalized output $y_{\mathrm{eq}}[n]$.

Phase error is sensed by

$$\epsilon[n] = \Im \left\{ y_{\mathrm{eq}}[n] \bar{a}[n]^* \right\}$$

The PLL uses a second-order digital loop filter:

$$\hat{\phi}[n] = \hat{\phi}[n-1] + \alpha\epsilon[n] + \beta\epsilon[n-1]$$

where $\alpha,\beta$ set bandwidth and damping. Phase correction is then applied:

$y_c[n] = y_{\mathrm{eq}}[n] e^{-j\hat{\phi}[n]}$

This approach leverages decision-directed tracking, assuming upstream FOE and timing recovery have resolved coarse synchronization.

3. Decision-Feedback Equalizer (DFE) Architecture

Equalization attacks channel ISI by combining feedforward filtering, feedback subtraction based on past decisions, and phase compensation. The canonical equations are:

• Feedforward:

$u[n] = \sum_{i=0}^{M} c_i[n] \, y[n-i]$

• Hard decision:

$\bar{a}[n] = Q(u[n] e^{-j\hat{\phi}[n]})$

• Feedback subtraction:

$$v[n] = u[n] - \sum_{j=1}^{N} b_j[n] \, \bar{a}[n-j]$$

• Output:

$y_{\mathrm{eq}}[n] = v[n] e^{-j\hat{\phi}[n]}$

Adaptive tap updating employs the decision-directed least-mean-square (LMS) procedure:

• Error signal:

$$e_{\mathrm{DFE}}[n] = y_{\mathrm{eq}}[n] - \bar{a}[n]$$

• Feedforward update:

$$c_i[n+1] = c_i[n] - \mu_c \, e_{\mathrm{DFE}}[n] \, y[n-i]^*$$

• Feedback update:

$$b_j[n+1] = b_j[n] + \mu_b \, e_{\mathrm{DFE}}[n] \, \bar{a}[n-j]^*$$

This LMS-driven adaptation rapidly converges given pre-equalization by pilot-tone FOE.

4. Outer Turbo Equalization Loop

To further suppress residual nonlinearities and error propagation, PT-PRDFE interfaces to an iterative turbo equalizer. The sequence is:

• PT-PRDFE processed samples are whitened by a post-filter $h_{\mathrm{pF}}$.
• Symbol log-likelihood ratios (LLRs) are forwarded to a BCJR sequence detector, exploiting channel memory.
• The BCJR output passes to a Forward Error Correction (FEC) decoder (soft-decision).
• The decoder’s extrinsic information recycles as a priori LLRs for the next BCJR iteration.

At bit index $n$,

$$L_{\mathrm{app}}(n) = L_c(n) + L_a^E(n), \qquad L_e(n) = L_{\mathrm{app}}(n) - L_a^E(n)$$

where $L_c(n)$ is the channel reliability, $L_a^E(n)$ is extrinsic input, and $L_e(n)$ is extrinsic output to FEC. This iterative turbo equalization loop typically yields steep “waterfall” BER performance as observed in the referenced work.

5. Performance Quantification and Comparative Results

Measured BER versus received optical power margin is the primary metric. For 16QAM under partial-response equalization (disregarding phase noise), the AWGN benchmark is:

$$BER \approx \frac{3}{2(M-1)} \, \operatorname{erfc} \left(\sqrt{0.1\,\gamma\,/\, [1 + \text{ISI power}]} \right)$$

where $\gamma$ is received SNR and ISI power is post-equalization energy in $h[1],h[2]$.

Power margin advantage is summarized in the following excerpt (Zou et al., 25 Jan 2026):

Format	BER = 2e–2 crossing (dB PM)
MB-PCS-64QAM	8.3 dB
FTN-16QAM w/ turbo	7.4 dB (≈0.9 dB gain)

At a hard-decision FEC threshold of $2\times 10^{-2}$ (7% overhead), FTN-16QAM with PT-PRDFE/turbo equalization achieves a $0.9\,\mathrm{dB}$ optical power margin improvement compared to MB-PCS-64QAM.

6. Implementation, Complexity, and Assumptions

Complexity per symbol comprises:

• $(M+N+1)$ complex multiplies (feedforward + feedback taps)
• One complex multiply plus argument or imaginary extraction for phase-error sensing
• Two real multiplies and add for PLL loop filter

The BCJR turbo equalizer’s state complexity grows exponentially with channel memory but remains manageable for 16QAM under memory truncation.

Key system-level assumptions:

• Perfect timing recovery and FOE upstream of PT-PRDFE
• Fast convergence of LMS adaptation after pilot-tone FOE
• Residual phase noise tracked with second-order digital PLL, analysis assumes small-phase error regime
• Turbo equalizer approximates post-filtered partial-response channel as full-response for BCJR trellis simplification

This approach yields practical implementation feasibility in amplifier-less coherent links, enabling improved spectral efficiency and reduced optical power requirements (Zou et al., 25 Jan 2026).

7. Broader Context and Implications

The integration of PT-PRDFE with turbo equalization provides a comprehensive methodology for countering ISI and phase noise in FTN transmission systems, notably where amplifier-less architectures restrict link budget. The documented 0.9 dB improvement in power margin for FTN-16QAM reflects the effectiveness of this collaborative equalization strategy. A plausible implication is that such architectures may facilitate next-generation short-reach coherent systems without recourse to power-hungry optical amplification or excessively high-order modulation under spectral constraints.