Joint ML Detection Rule

Updated 13 January 2026

Joint ML detection rule is an optimal statistical framework that minimizes symbol error probability by maximizing likelihood functions using known channel models.
It underpins diverse applications including MIMO, spatial modulation, noncoherent SIMO, and cooperative detection, ensuring full diversity and error-floor removal.
Advanced methods such as sphere decoding, tree search algorithms, and quantum optimization reduce complexity while retaining near-optimal performance.

Joint Maximum Likelihood (ML) Detection Rule

The joint maximum likelihood (ML) detection rule serves as a pivotal paradigm for optimal symbol and data sequence detection across numerous multiuser, multiantenna, and signal processing frameworks. It establishes the statistically optimal estimator (in the sense of minimizing symbol or sequence error probability) whenever the received observations are governed by a parametric likelihood model with unknown discrete-valued or finite-alphabet (vector) parameters. The rule is especially central to MIMO communications, non-orthogonal multiple access (NOMA), spatial modulation, noncoherent massive SIMO, and various cooperative molecular- or radar-detection scenarios.

1. General Principle and Problem Formulation

The joint ML detection rule selects the transmit symbol vector $\mathbf{x}$ that maximizes the likelihood $p(\mathbf{y}|\mathbf{x})$ of observing the receive vector (or matrix) $\mathbf{y}$ under the relevant statistical channel/noise model, with all channel or ancillary nuisance parameters held constant (or, for noncoherent scenarios, marginalized via concentration or ML estimation). For complex AWGN channels, this reduces to a minimization over a finite set:

$\hat{\mathbf{x}}_{\text{ML}} = \arg\max_{\mathbf{x}\in\mathcal{A}^n} p(\mathbf{y}|\mathbf{x}) = \arg\min_{\mathbf{x}\in\mathcal{A}^n} \| \mathbf{y} - H\mathbf{x}\|^2$

where $\mathbf{x}$ is the transmit symbol vector sampled from constellation $\mathcal{A}$ , $H$ is the known (or estimated) channel, and $\mathbf{y}$ is the received vector. This formulation underpins MIMO ML detection (Hassibi et al., 2013), spatial modulation (Rajashekar et al., 2012), NOMA uplink (Semira et al., 2021), and quantum ML detection (Cui et al., 2021).

2. Canonical Applications and System Models

2.1 MIMO and Spatial Modulation

In MIMO, the received signal model $y = Hx + v$ (with $x \in \mathcal{A}^n$ , $v \sim \mathcal{N}(0, I)$ ) yields the multivariate ML problem over $\mathcal{A}^n$ (Hassibi et al., 2013). Spatial Modulation (SM) generalizes this to joint estimation of the active antenna index (among $N_t$ transmit antennas, only one active per channel use) and symbol $s\in\mathcal{S}$ , leading to

$(\hat i, \hat s) = \arg\min_{i,s} \| y - H_i s \|^2$

with optimality holding under Gaussian noise (Rajashekar et al., 2012).

2.2 NOMA Uplink and Massive IoT

In uplink NOMA with $K$ single-antenna devices, receivers observe

$y = \sum_{k=1}^K h_k x_k + w$

where $x_k \in \chi_k$ are M-PSK symbols, $w$ is AWGN, and $H=[h_1, \dots, h_K]$ is the composite channel. The joint ML rule is

$(\hat{x}_1, \ldots, \hat{x}_K) = \arg\min_{(s_1, \ldots, s_K) \in \prod_k \chi_k} \| y - Hs \|^2$

guaranteeing error-floor-free performance and full receive diversity (Semira et al., 2021).

2.3 Noncoherent SIMO and Massive MIMO

In noncoherent block-fading SIMO/Massive MIMO, the received matrix $X = h s^T + W$ (unknown $h$ , $s$ in length- $T$ block, $W$ i.i.d. noise) leads to the joint ML formulation

$(\hat h, \hat s) = \arg\min_{h,\, s} \| X - h s^T \|_F^2$

which upon elimination of $h$ reduces to a quadratic form search in $s$ :

$\hat s = \arg\max_{s \in \Omega^T} \frac{ s^* X^* X s }{ \|s\|^2 }$

(Alshamary et al., 2015, Alshamary et al., 2014).

2.4 Cooperative Molecular and Radar Communication

In molecular communication, the joint ML rule may be symbol-by-symbol, fusing noisy RX or reporting channel counts under Poisson assumptions to maximize the probability of the observed molecule arrival pattern for each symbol hypothesis (Fang et al., 2018, Fang et al., 2017). Radar detection leverages a binary joint hypothesis test (e.g., target absent/present), stacking the signal and interference likelihood, often via an EM-Bayesian hybrid for latent variable systems (Yin et al., 4 Mar 2025).

3. Theoretical Properties and Optimality

The joint ML detection rule, whenever the system model and parameters are exactly known, is optimal in the sense of minimizing the symbol or sequence error probability—the probability of misdetecting $\mathbf{x}$ based on the observed $\mathbf{y}$ and the channel model. In multiuser environments (e.g., NOMA uplink), exhaustive joint ML detection removes the error floor observed with SIC detectors and achieves the device-wise full receive diversity:

The BER decay is $\sim$ SNR $^{-N_r}$ , with diversity order $N_r$ regardless of $K$ or $M_k$ (Semira et al., 2021).
Analysis using union bounds, pairwise error probabilities, and moment-generating functions delivers closed-form BER upper bounds in terms of the system dimensions, SNRs, and modulation orders.

In noncoherent block fading, joint ML over channel and data achieves the detection performance unattainable by separate or iterative channel estimation, closing the gap to coherent schemes (Alshamary et al., 2015, Alshamary et al., 2014).

4. Computational Methods and Complexity-Reduction Techniques

The combinatorial nature of joint ML rules (search over $|\mathcal{A}|^n$ , $|\mathcal{S}|N_t$ , or $\prod_k M_k$ hypotheses) motivates several algorithmic strategies:

Sphere Decoding: Used for ML search in spatial modulation, massive SIMO, and MIMO integer least squares. Sphere decoders (and hard-limiting variants) dramatically reduce average search complexity from exponential to linear (e.g., $O(N_t)$ for SM with QAM, $O(N T^2 + T^3 + |Ω| T^2)$ for massive SIMO at large $N$ ) while retaining ML performance (Rajashekar et al., 2012, Alshamary et al., 2015, Alshamary et al., 2014).
Tree Search Algorithms: Best-first tree search (TSA) guarantees exact ML solutions with node-visits bounded by those of the sphere decoder (Alshamary et al., 2015).
Markov Chain Monte Carlo (MCMC): Optimized MCMC samplers (with temperature $\alpha \sim \sqrt{\textrm{SNR}/\ln N}$ ) achieve rapid mixing and polynomially large ML solution weights in systems with high dimensions, providing near-ML detection at reduced cost—empirically matching ML BER in moderate iterations for $N$ up to $50$ (Hassibi et al., 2013).
Quantum Optimization (QAOA): The ML detection problem can be mapped onto an Ising-type Hamiltonian, with QAOA circuits providing a framework for polynomial-time (in depth and size) quantum ML detection; the classical simulation remains exponential, but prototype quantum devices can potentially provide an advantage for moderate dimensions (Cui et al., 2021).
Alternating Minimization: Relaxations that reformulate the detection problem as a sum of convex functions over both discrete and continuous auxiliary variables enable fast, matrix-inversion-free iterative implementations achieving near-ML performance—especially effective for large-scale MIMO (Elgabli et al., 2018).

5. Extensions: Channel Estimation, Robustness, and Non-Gaussian Models

Noncoherent and Joint Channel/Data Estimation: In block-fading and unknown-channel settings, ML frameworks eliminate nuisance parameters via concentration (inner minimization over channel coefficients), yielding quadratic-form searches and ML cost surfaces depending only on data symbols (Alshamary et al., 2015, Alshamary et al., 2014).
Robustness to Model Errors: For channels with unknown or uncertain statistics (e.g., channel estimation errors in MIMO, or unknown clutter in radar), alternative joint ML rules integrate error covariance modeling, model marginalization, or a Bayesian-ML hybrid structure solved via expectation-maximization. The resulting detector achieves performance gains (in Pd and error probability) and constant false alarm rate (CFAR) behavior under covariance uncertainty (Yin et al., 4 Mar 2025).
Phase Noise: In the presence of oscillator phase noise, joint ML rules marginalize over the phase error, yielding likelihoods expressed as weighted sums of central moments of the phase error PDF; Gaussian approximations and finite-moment truncations yield practical metrics for symbol detection and soft decoding (Krishnan et al., 2013).

6. Performance Insights and Application-Specific Considerations

Error Probability and Diversity: In all settings, joint ML detection achieves the minimum error floor and full diversity predicted by the physical layer model (e.g., NOMA's error floor is removed by joint ML in contrast to SIC) (Semira et al., 2021).
Complexity–Performance Tradeoff: While ML is optimal, suboptimal or structured search (SM, sphere/hard-limiting decoders, best-first/TSA, MCMC) often yield performance indistinguishable from ML at orders of magnitude lower complexity, provided the constellation or antenna dimension is appropriately exploited (Rajashekar et al., 2012, Alshamary et al., 2015, Hassibi et al., 2013).
Cooperative Detection and Fusion: In molecular or distributed sensing settings, symbol-by-symbol joint ML at the fusion center achieves the performance lower bound; simplified deterministic fusion rules (e.g., majority) may approach ML performance when the reporting channels are sufficiently noisy or symmetric (Fang et al., 2018, Fang et al., 2017).
Resource Allocation and Optimization: In molecular communication, molecule allocation across RXs to minimize joint ML error is a constrained optimization that, under channel symmetry and equal priors, is convex and solved by equal allocation. Asymmetry in channel gain or prior probabilities leads to non-equal optimal allocations (Fang et al., 2018).

7. Summary Table: Representative Detection Models

Application Area	ML Detection Rule Form	Notable Complexity/Performance Features
MIMO (AWGN)	$\arg\min_{x\in\mathcal{A}^n}\\|y - Hx\\|^2$	Sphere decoding, MCMC, QAOA, AltMin (Hassibi et al., 2013, Cui et al., 2021, Elgabli et al., 2018)
Spatial Modulation	$\arg\min_{i,s}\\|y - H_i s\\|^2$	Constellation-size independence for QAM (Rajashekar et al., 2012)
NOMA Uplink	$\arg\min_{s}\\|y - Hs\\|^2$ , $s\in\prod_k\chi_k$	Error floor removal, full diversity (Semira et al., 2021)
Noncoherent massive SIMO	ML over channel and sequence: quadratic form	Sphere/Tree search, polynomial scaling (Alshamary et al., 2015, Alshamary et al., 2014)
Molecular Comm.	ML over reporting patterns and counts	Mixture-based, resource optimization (Fang et al., 2018, Fang et al., 2017)
Radar (non-Gaussian)	EM-Bayesian hybrid for latent binary label	CFAR, iterative EM, outperforms GLRT (Yin et al., 4 Mar 2025)

The joint ML detection rule constitutes the foundational optimality reference for statistical decision and estimation in contemporary communication systems, multiuser detection, and distributed/integrated sensing paradigms. Its widespread adoption—alongside growing research into scalable algorithmic relaxations and complexity-reduction techniques—makes it central in both theoretical analysis and practical transceiver design.