Performance of QAM Schemes with Dual-Hop DF Relaying Systems over Mixed $η$-$μ$ and $κ$-$μ$ Fading Channels

Abstract: Performance of quadrature amplitude modulation (QAM) schemes is analyzed with dual-hop decode-and-forward (DF) relaying systems over mixed $\eta$-$\mu$ and $\kappa$-$\mu$ fading channels. Closed-form expressions are obtained for the average symbol error rate (ASER) for general order rectangular QAM and cross QAM schemes using moment generating function based approach. Derived expressions are in the form of Lauricella's $(F_D^{(n)}(\cdot), \Phi_1^{(n)}(\cdot))$ hypergeometric functions which can be numerically evaluated using either integral or series representation. The obtained ASER expressions include other mixed fading channel cases addressed in the literature as special cases such as mixed Hoyt, and Rice fading, mixed Nakagami-$m$, and Rice fading. We further obtain a simple expression for the asymptotic ASER, which is useful to determine a factor governing the system performance at high SNRs, i.e., the diversity order. Additionally, we analyze the optimal power allocation, which provides a practical design rule to optimally distribute the total transmission power between the source and the relay to minimize the ASER. Extensive numerical and computer simulation results are presented that confirm the accuracy of presented mathematical analysis.