On the Design of Channel Shortening Demodulators for Iterative Receivers in MIMO and ISI Channels (1506.07331v4)

Abstract: We consider the problem of designing demodulators for linear vector channels with memory that use reduced-size trellis descriptions for the received signal. We assume an overall iterative receiver, and for the parts of the signal not covered by the trellis description, we use interference cancelation based on the soft information provided by the outer decoder. In order to reach a trellis description, a linear filter is applied as front-end to compress the signal structure into a small trellis. This process requires three parameters to be designed: (i) the front-end filter, (ii) the feedback filter through which the interference cancelation is done, and (iii) a target response which specifies the trellis. Demodulators of this form have been studied before under then name channel shortening (CS), but the interplay between CS and interference cancelation has not been adequately addressed in the literature. In this paper, we analyze two types of CS demodulators that are based on the Forney and Ungerboeck detection models, respectively. The parameters are jointly optimized based on a generalized mutual information (GMI) function. We also introduce a third type of CS demodulator that is in general suboptimal but has closed form solutions for all parameters. Moreover, signal to noise ratio (SNR) asymptotic properties are analyzed and we show that the third CS demodulator asymptotically converges to the optimal CS demodulator in the sense of maximizing the GMI.