Complex Kernel LMS Overview

• CKLMS is a nonlinear adaptive filtering algorithm that maps complex-valued signals into an RKHS using complex kernels for enhanced learning of nonlinear input–output relationships.
• The algorithm leverages an extension of Wirtinger calculus and augmented filtering to efficiently process both circular and noncircular (improper) signals, achieving lower mean-squared error and faster convergence.
• Sparsification strategies and pseudo-kernel design in CKLMS control computational complexity while enabling practical implementations in tasks such as channel equalization and system identification.

A Complex Kernel Least-Mean-Square (CKLMS) algorithm is a nonlinear adaptive filtering method that extends the classical LMS algorithm to complex-valued signals and nonlinear input–output relationships via a reproducing kernel Hilbert space (RKHS) framework. The CKLMS leverages either complex kernels or the complexification of real RKHSs, exploits an extension of Wirtinger calculus for infinite-dimensional settings, and allows for both standard ("pure") and augmented ("widely linear" or "augmented CKLMS", ACKLMS) variants. The augmented form brings significant improvements in regression and equalization tasks involving improper (noncircular) complex data by modeling both covariance and pseudo-covariance structures.

1. Mathematical Framework and Formulation

Let $\{(z(n),d(n))\}_{n=1}^N$ denote a training sequence with $z(n)\in\mathbb{C}^\nu$ and $d(n)\in\mathbb{C}$. Data is mapped to a complex RKHS $\mathcal{H}$ by a positive-definite kernel $\kappa$, typically the complex Gaussian kernel

$$\kappa(z, w) = \exp\left(-\frac{\|z-w^*\|^2}{\sigma^2}\right)$$

with the associated feature map $\phi(z) = \kappa(\cdot, z)$ (Bouboulis et al., 2010).

For adaptive filtering, the estimator in CKLMS is

$$\hat{d}(n) = \langle \phi(z(n)), w\rangle_{\mathcal{H}}$$

where $w \in \mathcal{H}$ is the unknown representer. In the augmented CKLMS (ACKLMS), two elements $w, v \in \mathcal{H}$ are used to implement a widely linear (augmented) filter:

$$\tilde{d}(n) = \langle \phi(z(n)), w \rangle_\mathcal{H} + \langle \phi(z(n))^*, v \rangle_\mathcal{H}$$

where $\phi(z)^*$ denotes the complex-conjugate feature (Bouboulis et al., 2011).

The cost function is the instantaneous squared error:

$$L_n(w, v) = |d(n) - \langle \phi(z(n)), w \rangle_\mathcal{H} - \langle \phi(z(n))^*, v \rangle_\mathcal{H}|^2$$

2. Kernel Construction and Complexification

Two principal ways exist to define the RKHS:

• Native Complex Kernel: Employ kernels defined directly on $\mathbb{C}^\nu \times \mathbb{C}^\nu$, such as the complex Gaussian kernel.
• Complexification of Real RKHS: Given a real kernel $\kappa_R$ on $\mathbb{R}^{2\nu}$, rewrite $z = x + i y$ and define $\hat{\phi}(z) = \Phi(x, y) + i \Phi(x, y)$ with $\Phi$ the real-feature map for $\kappa_R$. The inner product structure persists: $$\langle \phi(z), \phi(w) \rangle_\mathcal{H} = \kappa(z, w)$$ (Bouboulis et al., 2010, Bouboulis et al., 2010).

The complexification approach allows the re-use of well-established real kernels for complex-valued learning, while native complex kernels can exploit signal geometry tailored to complex domains.

3. Algorithmic Derivation and Wirtinger Calculus

The functional gradient of the cost function $L_n(w,v)$ in $\mathcal{H}$ is calculated using the extension of Wirtinger calculus to infinite-dimensional settings. For $w, v \in \mathcal{H}$, treating variables and their conjugates independently, the derivatives are

$$\frac{\partial L_n}{\partial w^*} = -\phi(z(n)) \cdot e(n)$$

$$\frac{\partial L_n}{\partial v^*} = -\phi(z(n))^* \cdot e(n)$$

where $$e(n) = d(n) - \langle \phi(z(n)), w \rangle_\mathcal{H} - \langle \phi(z(n))^*, v \rangle_\mathcal{H}$$. The update rules, with step-size $\mu>0$, are

$w(n) = w(n-1) + \mu \phi(z(n)) e(n)^*$

$v(n) = v(n-1) + \mu \phi(z(n))^* e(n)^*$

For standard CKLMS, only the $w$ update is present (Bouboulis et al., 2011, Bouboulis et al., 2010).

4. Relation to Linear and Nonlinear Filtering

The increase in model expressivity over standard complex LMS arises through the kernel mapping, which enables nonlinear regression via the "kernel trick" in RKHSs. The standard (non-augmented) CKLMS recovers many properties of classic LMS, including convergence and misadjustment, but offers substantially lower mean-squared error and faster convergence in nonlinear scenarios (Bouboulis et al., 2010, Bouboulis et al., 2010, Bouboulis et al., 2010).

The widely linear (augmented) CKLMS exploits "pseudo-covariance" and complementary information present in improper (noncircular) complex signals. For circular signals, standard and augmented CKLMS yield identical results (Bouboulis et al., 2011).

Table: Comparison of CKLMS Variants

Variant	Model Formulation	Key Advantage
CKLMS	$\hat{d}(n) = \langle \phi(z(n)),w\rangle$	Standard nonlinear adaptive estimation for complex signals
ACKLMS	$$\tilde{d}(n) = \langle \phi(z(n)),w\rangle + \langle \phi(z(n))^*,v\rangle$$	Enhanced modeling for noncircular signals

5. Sparsification, Complexity, and Implementation

The CKLMS and ACKLMS algorithms possess an inherent computational growth, as each time step potentially increases the expansion in the RKHS. To control complexity, a sparsification strategy (novelty criterion) is typically employed:

• Accept a new center $z(n)$ into the dictionary only if the distance $\|\phi(z(n)) - \phi(z_k)\|_\mathcal{H}$ for all $k$ exceeds a threshold $\delta_1$, and $|e(n)|>\delta_2$.
• Otherwise, discard $z(n)$ as a non-novel or sufficiently modeled sample.

This results in a manageable dictionary size (often hundreds of centers for several thousand samples), making real-time operation feasible. Each iteration then requires $O(M)$ kernel and inner product operations, where $M$ is the size of the current dictionary (Bouboulis et al., 2010, Bouboulis et al., 2010).

6. Performance Characteristics and Empirical Results

Empirical studies in nonlinear channel equalization and identification tasks demonstrate the benefits of CKLMS and its augmented extension:

• For noncircular inputs ($\rho \approx 0.1$), ACKLMS achieves approximately $2$ dB lower steady-state MSE than standard CKLMS.
• For circular inputs, both kernel methods yield virtually identical performance.
• Both CKLMS and ACKLMS achieve faster convergence and significantly reduced error floors compared to classical LMS, widely linear LMS, and other nonlinear algorithms such as MLP or nonlinear gradient descent (Bouboulis et al., 2011, Bouboulis et al., 2010, Bouboulis et al., 2010).
• The generalized Complex Kernel LMS (gCKLMS) further demonstrates superior performance by including a pseudo-kernel, providing better convergence and steady-state MSE especially when real and imaginary signal components display different statistics or are correlated (Boloix-Tortosa et al., 2019).

7. Extensions and Kernel Design Considerations

The generalized framework developed in "The Generalized Complex Kernel Least-Mean-Square Algorithm" (Boloix-Tortosa et al., 2019) provides a kernel and pseudo-kernel approach rooted in widely linear RKHS theory. Design rules dictate:

• Use only the kernel $k$ if $\Re\{f\}$ and $\Im\{f\}$ are independent and share the same statistics; set the pseudo-kernel $\tilde{k}=0$.
• Employ distinct kernel bandwidths or pseudo-kernel cross-terms if real and imaginary parts differ in smoothness or are statistically dependent.
• Pseudo-kernel introduction is critical for learning tasks with complex-valued signals where real and imaginary parts are not independent or have distinct properties.

The learning rate $\mu$ and kernel parameters should be selected to satisfy stability bounds and adjusted based on validation data. Dictionary management is essential for computational tractability in long-running applications.

• "The Augmented Complex Kernel LMS" (Bouboulis et al., 2011)
• "Extension of Wirtinger's Calculus to Reproducing Kernel Hilbert Spaces and the Complex Kernel LMS" (Bouboulis et al., 2010)
• "The Complex Gaussian Kernel LMS algorithm" (Bouboulis et al., 2010)
• "Extension of Wirtinger Calculus in RKH Spaces and the Complex Kernel LMS" (Bouboulis et al., 2010)
• "The Generalized Complex Kernel Least-Mean-Square Algorithm" (Boloix-Tortosa et al., 2019)