MM-BCD-SCA Algorithm in IVA Optimization

• The MM-BCD-SCA algorithm is a method for optimizing IVA via majorization-minimization and block coordinate descent that alternates updates of separation matrices and auxiliary variables.
• It integrates iterative projection (IP) and iterative source steering (ISS) strategies to update the separation matrix with closed-form solutions for small block sizes.
• The framework transforms the cost into a quadratic form using surrogate functions, enabling practical convergence and posing challenges for extending to higher block sizes (d ≥ 3).

The MM-BCD-SCA algorithm refers to a class of methods for majorization-minimization (MM)-based optimization in independent vector analysis (IVA), employing block coordinate descent (BCD) strategies. Within this context, the algorithmic family leverages surrogates for negative log-likelihood, alternating coordinate minimization of the separation matrix $W \in \mathbb{C}^{m \times m}$ and auxiliary variables, using iterative block updates. MM-BCD-SCA (Majorization-Minimization, Block Coordinate Descent, Source Column Algorithm) specifically encompasses approaches such as iterative projection (IP) and iterative source steering (ISS), including their higher-block generalizations such as ISS$_2$.

1. Surrogate Function and the Majorization-Minimization Principle

In MM-based IVA, optimization is carried out on a separation matrix $$W = [w_1, \ldots, w_m]^h \in \mathbb{C}^{m \times m}$$ by minimizing a surrogate objective given by:

$$\mathcal{L}(W) = \sum_{i=1}^m w_i^H V_i w_i - \log |\det W|^2,$$

where each $V_i \in \mathbb{C}^{m \times m}$ is a positive definite matrix computed at each MM iteration from the current separated signals. These matrices are constructed as $$V_i = (1/2n) X \operatorname{diag}(\Lambda_{i, \cdot}) X^H$$, with $\Lambda \in \mathbb{R}_+^{m \times n}$ an auxiliary weight matrix updated at each outer iteration. The surrogate $\mathcal{L}(W)$ is obtained via the super-Gaussian majorization $$\phi(r) = -\log p(y) \leq (\phi'(\alpha)/2\alpha) r^2 + \text{const}$$ (tight at $\alpha = r$), rendering the cost function quadratic in the variables of $W$ at each iteration (Ikeshita et al., 2022). This allows for analytical updates in block coordinates.

2. Block Coordinate Descent Methodologies

Block coordinate descent (BCD) reduces $\mathcal{L}(W)$ by partitioning $W$ (or its inverse $A = W^{-1}$) and optimally updating blocks in closed form. Two principal strategies are prominent:

• Iterative Projection (IP): Operates directly on $W$, updating one (IP$_1$) or two (IP$_2$) rows at a time by solving low-dimensional subproblems, each yielding closed-form updates. For IP$_1$, each update computes $w \leftarrow u/\sqrt{u^H V_\ell u}$ with $u = (W V_\ell)^{-1} e_\ell$. IP$_2$ solves a $2 \times 2$ subproblem (Ikeshita et al., 2022).
• Iterative Source Steering (ISS): Operates on $A = W^{-1}$, updating its columns with the constraint $W A = I$. ISS$_1$ updates one column at a time, and ISS$_2$ extends this to updating two columns simultaneously.

For $m \geq 3$, global minimization of $\mathcal{L}(W)$ remains intractable; therefore, BCD strategies offer practical convergence and reliable performance empirically.

3. Closed-Form Update Formulas for ISS$_d$

Let $d$ be the block size (e.g., $d=1$ for ISS$_1$, $d=2$ for ISS$_2$). The update proceeds by partitioning $A$ into $L = m/d$ blocks of $d$ columns. In each BCD step, the update is represented as $W \leftarrow T W$ with $T \in D_{\mathrm{ISS}_d}$, a block-matrix characterized by $$T = \begin{bmatrix} P & 0 \ Q & I_{m-d} \end{bmatrix}$$, $P \in \mathbb{C}^{d \times d}$, $Q \in \mathbb{C}^{(m-d) \times d}$.

The cost decomposes as:

$$\mathcal{L}(T W) = \sum_{i=1}^d p_i^H G_i p_i - \log |\det P|^2 + \sum_{i=d+1}^m \begin{bmatrix} q_i \ 1 \end{bmatrix}^H \begin{bmatrix} G_i & g_i \ g_i^H & c_i \end{bmatrix} \begin{bmatrix} q_i \ 1 \end{bmatrix} + \text{const}$$

where $G_i = W_{1:d, \cdot} V_i W_{1:d, \cdot}^H$, $g_i = W_{1:d, \cdot} V_i W_{i, \cdot}^H$.

The Q-step (for $i = d+1,\ldots,m$) involves $q_i = - G_i^{-1} g_i$, followed by an update to $$Y_{i,\cdot} \leftarrow Y_{i,\cdot} + q_i^H Y_{1:d,\cdot}$$. The P-step depends on the block size:

• For $d=1$, $p_1 = G_1^{-1/2}$.
• For $d=2$, the update uses eigendecomposition: set $H = G_1^{-1} G_2$, then compute roots $\theta_1, \theta_2$, and associated eigenvectors $u_1$, $u_2$, normalized as $p_i = u_i / \sqrt{u_i^H G_i u_i}$.

For $d \geq 3$, a closed-form solution for the P-step is not available, which remains an open problem and direction for potential further generalizations.

4. Unified Framework for ISS$_d$ Methods

The MM-BCD-SCA framework allows systematic development of block-update source steering methods for general block sizes $d$ dividing $m$. The unified recipe is as follows (Ikeshita et al., 2022):

1. Partition the mixing matrix $A$ into $L = m/d$ blocks of $d$ columns.
2. For each block, optionally permute $W, Y, \Lambda$ so the active block is leading.
3. For each block, update $T \in D_{\mathrm{ISS}_d}$ by first solving the Q-step (closed-form for any $d$), then the P-step (analytical for $d=1,2$, open for $d \geq 3$).
4. Update the separated signals $Y$ and auxiliary matrix $\Lambda$ accordingly.
5. Iterate to convergence of the MM surrogate.

ISS$_1$ and ISS$_2$ can be directly obtained via this systematic process, and the framework is extensible, conditional on analytical tractability of the P-step for higher $d$.

5. Algorithmic Workflow and Pseudocode

A typical majorization-minimization cycle for ISS$_2$ proceeds as follows:

1. Initialize $W^{[k]}$ (optionally via data whitening), set $Y^{[k]} = W^{[k]} X^{[k]}$ for each dataset $k$.
2. Outer MM loop: update $$\Lambda_{ij} \leftarrow \phi'(\|y_{ij}\|+\epsilon)/(\|y_{ij}\|+\epsilon)$$.
3. For each block $\ell = 1,...,m/2$ (inner BCD cycle):
   • For each $k$ and $i=3,...,m$, compute $G_i^{[k]}$ and $g_i^{[k]}$, update $Y_{i,\cdot}^{[k]}$ by subtracting $(g_i^{[k]})^H (G_i^{[k]})^{-1} Y_{1:2,\cdot}^{[k]}$.
   • Compute $G_1^{[k]}, G_2^{[k]}$, build $H^{[k]}$, eigenvalues $\theta_1, \theta_2$, eigenvectors $u_1, u_2$, set $P^{[k]} \leftarrow [p_1, p_2]^H$.
   • Update $$Y_{1:2,\cdot}^{[k]} \leftarrow P^{[k]} Y_{1:2,\cdot}^{[k]}$$.
   • Permute rows of $Y$ and $\Lambda$ by the $2 \times 2$ block-cycle permutation $\Pi_2$. This cycle is iterated to MM convergence, and outputs the separated $Y^{[k]}$.

6. Computational Complexity and Empirical Performance

The computational overhead per MM iteration for ISS$_d$ methods depends on block size $d$:

• ISS$_1$ (one column): $O(K m^2 n)$
• ISS$_2$ (two columns): $O(K m^2 n)$
• IP$_1$/IP$_2$ (one/two rows): $O(K m^3 n + K m^4)$

Here, $K$ is the number of datasets, $m$ is the number of sensors, $n$ the signal length. The dominant cost for ISS methods is the computation of $G_i, g_i$ and $Y$ updates.

Numerical experiments on reverberant speech mixtures (for $m=4,6,8,10$) indicate that IP$_2$ and ISS$_2$ converge in approximately $20$ to $30$ MM iterations to a target source-to-distortion ratio (SDR), while ISS$_1$ and IP$_1$ require more iterations ($\approx 50\text{–}80$). ISS$_2$ achieves the convergence rate of IP$_2$ while retaining the $1/m$ computational complexity advantage characteristic of ISS-type updates (Ikeshita et al., 2022).

7. Extensions and Open Problems

The ISS$_d$ framework, as outlined, immediately yields practical and efficient algorithms for $d=1$ and $d=2$. For $d \geq 3$, the P-step does not admit an analytical solution, which delineates a notable open problem for extending the family to higher block-sizes. This suggests future research directions aimed at either characterizing closed-form solutions for $d \geq 3$ or developing efficient numerical subroutines for high-dimensional block updates. The MM-BCD-SCA paradigm offers a systematic path, unifying disparate algorithms under a common MM and BCD scaffold and facilitating informed tradeoffs between convergence speed and per-iteration cost (Ikeshita et al., 2022).