---
title: OTFS-Modulated FTN Transmission
url: https://www.emergentmind.com/topics/otfs-modulated-faster-than-nyquist-ftn-transmission
type: topic
---

# OTFS-Modulated FTN Transmission

Orthogonal time frequency space (OTFS)-modulated Faster-Than-Nyquist (FTN) transmission is a hybrid signaling paradigm that integrates OTFS—an effective solution for reliable communication in doubly dispersive channels—with intentional violation of the time-frequency orthogonality constraints imposed by the Nyquist criterion via FTN signaling. This combination aims to substantially boost spectral efficiency and information rate in high-mobility and spectrum-constrained wireless environments. The fundamental principle is to deliberately introduce controlled inter-symbol interference (ISI) and inter-carrier interference (ICI) by compressing the time and frequency grid beyond the Nyquist limit, while exploiting advanced precoding and detection strategies in the delay-Doppler (DD) domain to suppress resulting interference and maximize mutual information. The following sections present a comprehensive technical account of system models, signal constructions, transceiver algorithms, analytical performance, and implementation trade-offs for OTFS-modulated FTN systems, including recent multi-carrier, MIMO, and RIS-empowered extensions.

## 1. OTFS and FTN Signaling: System and Signal Model

In Nyquist OTFS, an $M\times N$ block of DD-domain QAM symbols $x[l,k]$ ($l=0,\ldots,M-1,\;k=0,\ldots,N-1$) is mapped to the time-frequency domain via the inverse symplectic finite Fourier transform (ISFFT):
\[
X[m,n]\;=\;\frac{1}{\sqrt{MN}}\sum_{k=0}^{N-1}\sum_{l=0}^{M-1}x[l,k]e^{j2\pi(\frac{nk}{N}-\frac{ml}{M})}.
\]
Each $X[m,n]$ is pulse-shaped (typically with a Nyquist or root-raised-cosine (RRC) pulse) and transmitted with symbol interval $T_0$ and subcarrier spacing $\Delta f_0$ such that $T_0\Delta f_0=1$, maintaining strict orthogonality.

In FTN signaling, symbol time and/or frequency grid is compressed by factors $\alpha\leq1$ (time) and $\beta\leq1$ (frequency): 
\[
T = \alpha T_0, \quad \Delta f = \beta \Delta f_0,\quad T_0\Delta f_0=1.
\]
Non-orthogonal (e.g., RRC with roll-off $\theta$) pulses are used on this compressed grid. The transmit signal then becomes:
\[
s(t) = \sqrt{\alpha\beta E_0} \sum_{m=0}^{M-1} \sum_{n=0}^{N-1} X_P[m,n]\, g_{tx}(t-n\alpha T_0)\, e^{j2\pi m\beta\Delta f_0 (t-n\alpha T_0)},
\]
where $X_P[m,n]$ may include DD-domain precoding. ISI and ICI are now present due to loss of orthogonality.

Upon passing through a delay-Doppler sparse channel $h(\tau,\nu) = \sum_{i=1}^L h_i \delta(\tau-\tau_i)\delta(\nu-\nu_i)$, and after matched filtering and sampling, the time-frequency (TF) domain input-output is:
\[
Y[m,n] = \sum_{m',n'} H_{m,n}[m',n'] X_P[m',n'] + Z[m,n],
\]
with $H_{m,n}[m',n']$ capturing all ISI/ICI contributions via ambiguity integrals over transmit/receive pulse shapes.

The equivalent DD-domain input-output after SFFT is:
\[
y^{\mathrm{DD}} = H^{\mathrm{DD}} x_P^{\mathrm{DD}} + z^{\mathrm{DD}}, \quad x_P^{\mathrm{DD}} = P x^{\mathrm{DD}},
\]
where $H^{\mathrm{DD}}\in\mathbb C^{MN\times MN}$ encapsulates ISI/ICI from both FTN grid compression and the physical channel, and $z^{\mathrm{DD}}$ is colored noise [2501.06750][2411.01244][2512.20332][2601.11869].

## 2. Precoding, Equalization, and Power Allocation Strategies

To counter FTN-induced interference and correlated noise, advanced precoding and detection are implemented in the DD-domain.

### SISO Case:
Mutual information under Gaussian DD-domain inputs $x^{\mathrm{DD}}\sim\mathcal{CN}(0, \sigma_x^2 I)$ is given as:
\[
I(x^{\mathrm{DD}} ; y^{\mathrm{DD}}) = \log_2 \det \left( I + \frac{\sigma_x^2}{N_0} P^H H^{\mathrm{DD},H} G^{-1} H^{\mathrm{DD}} P \right ),
\]
where $G$ is the TF-domain ISI/ICI matrix. By whitening and eigen-decomposition (EVD),
\[
C^{\mathrm{SISO}} = \sum_{k=0}^{MN-1} \log_2 \left(1 + \frac{\sigma_x^2}{N_0} \lambda_{P,k} \lambda_{D,k}\right).
\]
Optimal power allocation per sub-channel is given by water-filling:
\[
\lambda_{P,k} = \max \left\{ \frac{1}{\xi\phi_k \ln 2} - \frac{N_0}{\sigma_x^2 \lambda_{D,k}}, 0 \right\}.
\]

EVD-based precoding diagonalizes the effective channel and enables symbol-by-symbol detection. This two-stage process (noise whitening and EVD) removes both inter-symbol correlation and channel mixing [2501.06750][2411.01244].

### MIMO Case and SIC Precoding:
For $N_T$ transmit and $N_R$ receive antennas, the input-output relation is (with stacked DD streams):
\[
y^{\mathrm{DD}}_{\rm MIMO} = H^{\mathrm{DD}}_{\rm MIMO} P_{\rm MIMO} x^{\mathrm{DD}}_{\rm MIMO} + z^{\mathrm{DD}}_{\rm MIMO}.
\]
MIMO capacity and power allocation can be computed via full EVD (complexity $O((MNN_T)^3)$) or Successive Interference Cancellation (SIC)-based decomposition, which splits the high-dimensional optimization into $N_T$ subproblems of size $MN\times MN$ each, reducing complexity to $O(N_T (MN)^3)$ while incurring marginal SNR loss [2501.06750].

## 3. Channel Estimation and Detection in OTFS-FTN

OTFS-FTN with RRC pulses and compressed symbol interval $T_f = \alpha T_0$ requires specialized channel estimation and detection to handle FTN-induced ISI:

- The DD-domain input-output for FTN-OTFS, after matched filtering, is characterized by a dense kernel where ISI bandwidth is limited to $2c+1$ diagonals, with $c \approx \beta/(2\alpha)$, exploiting the decay of $g(nT_f)$.
  
- Channel estimation uses FTN-interval pilots with Doppler and delay guards, detected in the DD grid after whitening, and supports joint estimation of fractional Doppler and path gain per tap.

- Reduced-complexity LMMSE equalization leverages the banded ISI matrix for $O(MN\,c^2) + O(MN\log_2 N)$ complexity, replacing the $\mathcal{O}((MN)^3)$ cost of full matrix inversion [2601.11869].

## 4. Transceiver Algorithmic Flows

A typical FTN-OTFS transceiver consists of the following operations [2411.01244]:

**Transmitter:**
1. Map input bits to DD-symbols $X[k,l]$.
2. Perform EVD-based DD-domain precoding: $x_P = U^H x$, where $U$ is from EVD of noise-whitened effective channel.
3. ISFFT to map to TF domain: $X_{\rm TF} = F_M X F_N^H$.
4. IFFT to generate time-domain vector.
5. FTN sequence generation: $s(t) = \sum_n s_n g(t-nT_f)$.
6. Transmission.

**Receiver:**
1. Matched filtering and FFT/SFFT.
2. Noise whitening using EVD of the colored noise covariance.
3. Channel diagonalization, yielding independent scalar subchannels.
4. Log-likelihood ratio computation and FEC decoding.

## 5. Performance and Trade-Offs

Simulation studies consistently demonstrate that OTFS-modulated FTN transmission provides tangible improvements in both spectral efficiency and robustness under high Doppler, at manageable complexity and BER cost:

- **Spectral Efficiency:** Normalized capacity increases by $\sim$10–20% (even up to 30% in RIS-assisted settings) with moderate FTN factors (e.g., $\alpha=0.8,0.9$). Spectral efficiency grows approximately as $1/\alpha$, with $\alpha$ the time compression parameter [2501.06750][2411.01244][2512.20332][2601.11869].

- **BER/FER:** The BER penalty for moderate FTN ($\alpha \approx 0.8$) is minor—$< 0.5$ dB at $10^{-4}$ BER for multi-carrier FTN-OTFS even with MMSE equalization and LDPC coding. Performance remains comparable or even superior to Nyquist-OTFS when joint DD-domain processing and/or advanced coding are used [2501.06750][2411.01244][2512.20332][2601.11869][2508.02950].

- **Complexity:** EVD and full water-filling realize maximum achievable capacity at cubic complexity, but SIC-based schemes in MIMO settings retain near-optimal rates with linear complexity in $N_T$.

- **PAPR/IBO:** FTN packing leads to negligible PAPR increase, and RIS beamforming can compensate average SNR loss, increasing available input back-off [2512.20332].

- **Robustness:** FTN-OTFS with DD-domain estimation consistently outperforms conventional FTN (including DMFTN or EVD-FTN) and matches or exceeds Nyquist-OTFS in high-mobility channels [2601.11869][2508.02950].

## 6. Advanced Architectures: MC-FTN, RIS-OTFS-FTN, and Zak-OTFS-FTN

- **MC-FTN-OTFS** generalizes FTN to both time and frequency compression (parameters $\alpha$, $\beta$). It leverages DD-domain EVD or SIC-based precoding for high-capacity MIMO with efficient resource utilization [2501.06750].

- **RIS-OTFS-FTN** incorporates passive beamforming with a finite-phase RIS, maximizing DD-domain channel taps to further boost spectral efficiency and reliability. Practical closed-form quantized phase design algorithms maximize channel gain. FTN-induced ISI and RIS beamforming jointly govern the rate-reliability trade-off [2512.20332].

- **Zak-OTFS-FTN** enables FTN by symbol superposition in the DD domain using two mutually unbiased bases. A channel-aware MMSE precoder transforms the channel to effectively identity, and the receiver exploits Gaussianity of inter-frame interference, greatly simplifying detection. With trellis-coded modulation, Zak-OTFS-FTN achieves superior performance at high SNR and can surpass Nyquist-based schemes [2508.02950].

## 7. Limitations and Practical Considerations

- Aggressive FTN compression ($\alpha, \beta \ll 1$) can render the ISI/ICI matrix ill-conditioned or singular, necessitating subcarrier deactivation or elaborate equalization. 
- BER degradation under strong compression mandates stronger channel coding (e.g., LDPC, TCM) or iterative DD-domain detection.
- All advanced transceiver methods presuppose accurate DD-domain CSI; pilot design for FTN intervals and high-mobility scenarios remains an active research area [2601.11869].
- Real-time approximations (random-beam precoding, message-passing) are essential to bring EVD/SVD complexity down for practical 6G deployment.
- Implementation of large RIS, joint optimization of FTN factor $\alpha$, and quantization of RIS phases, as well as careful selection of DD grid size, provide further performance/complexity trade-offs [2512.20332].

---

In conclusion, OTFS-modulated FTN transmission and its variants systematically exploit time-frequency resources and DD-domain channel sparsity, enabling a substantive increase in capacity and information rate in doubly-selective, high-mobility environments. These systems, rigorously developed and analyzed across several recent works [2501.06750][2411.01244][2512.20332][2601.11869][2508.02950], delineate the technical path toward high-performance, spectrum-efficient 6G waveform design.

Source: https://www.emergentmind.com/topics/otfs-modulated-faster-than-nyquist-ftn-transmission