Asymptotic Comparison of ML and MAP Detectors for Multidimensional Constellations (1607.01818v3)

Abstract: A classical problem in digital communications is to evaluate the symbol error probability (SEP) and bit error probability (BEP) of a multidimensional constellation over an additive white Gaussian noise channel. In this paper, we revisit this problem for nonequally likely symbols and study the asymptotic behavior of the optimal maximum a posteriori (MAP) detector. Exact closed-form asymptotic expressions for SEP and BEP for arbitrary constellations and input distributions are presented. The well-known union bound is proven to be asymptotically tight under general conditions. The performance of the practically relevant maximum likelihood (ML) detector is also analyzed. Although the decision regions with MAP detection converge to the ML regions at high signal-to-noise ratios, the ratio between the MAP and ML detector in terms of both SEP and BEP approach a constant, which depends on the constellation and a priori probabilities. Necessary and sufficient conditions for asymptotic equivalence between the MAP and ML detectors are also presented.