Block-Lower-Triangular Procrustes Problem

• Block-lower-triangular Procrustes problem is a constrained matrix approximation challenge that imposes block-zero constraints to align datasets.
• The methodology employs semidefinite programming reformulation with convex relaxations to manage structural and rank constraints efficiently.
• Its applications include multitask factor models and structured multivariate data alignment, making it vital for advanced optimization tasks.

The block-lower-triangular Procrustes problem is a constrained matrix approximation problem wherein the variable transformation matrix is restricted to the subspace of block-lower-triangular matrices. The fundamental objective is to minimize a least-squares discrepancy, typically with respect to the Frobenius norm, between a transformed dataset and a target dataset, subject to the structural requirement that certain off-diagonal blocks in the transformation vanish. This constraint arises in diverse settings, including multitask factor models and structured multivariate data alignment, and it can be addressed using a rank-constrained semidefinite programming (SDP) relaxation that incorporates both the least-squares objective and the block-structural constraints (Fulová et al., 2023).

1. Formulation of the Block-Lower-Triangular Procrustes Problem

The general problem seeks $X \in \mathcal{X}$, where $\mathcal{X}$ is the linear space of block-lower-triangular matrices, that minimizes the Frobenius norm objective

$\min_X \| A X - B \|_F^2,$

where $A \in \mathbb{R}^{m \times r}$, $X \in \mathbb{R}^{r \times n}$, and $B \in \mathbb{R}^{m \times n}$. The block-lower-triangular constraint fixes a partitioning of $X$ into $p \times p$ blocks $X_{ij} \in \mathbb{R}^{d_i \times d_j}$, with $\sum_i d_i = r$ and $\sum_j d_j = n$, and imposes

$X_{ij} = 0 \quad \text{for all} \;\; i < j.$

Componentwise, this yields $\left( X_{ij} \right)_{\alpha\beta} = 0$ for all $i < j$, $1 \le \alpha \le d_i$, $1 \le \beta \le d_j$. Each such zero-block constraint is a linear equality in the entries of $X$ (Fulová et al., 2023).

2. Semidefinite Programming Reformulation

The Procrustes objective $\| AX - B \|_F^2$ can be equivalently expressed via a Schur complement as an SDP. Introducing an auxiliary matrix variable $Z \in \mathrm{Sym}_+^m$, the objective is rewritten as

$$\min_{X, Z} \; \operatorname{tr} Z \qquad \text{subject to} \qquad \begin{pmatrix} I_n & (A X - B)^T \ A X - B & Z \end{pmatrix} \succeq 0,\qquad Z \succeq 0.$$

This formulation is convex with respect to $(X, Z)$ except for the structural subspace constraint on $X$, which can itself be encoded as linear equalities (Fulová et al., 2023).

To facilitate dualization and convex relaxations, one employs a “lifted” variable $\widetilde{W} \in \mathrm{Sym}_+^{r+n}$: $$\widetilde{W} = \begin{pmatrix} I_r & X^T \ X & Y \end{pmatrix},$$ where $Y = X X^T$. The block-lower-triangular requirements on $X$ correspond to specific affine constraints on the entries of $\widetilde{W}$: $$\widetilde{W}_{r+\sum_{k<i} d_k + \alpha,\; \sum_{k<j} d_k + \beta} = 0 \qquad \forall i<j,\, \forall \alpha \le d_i,\, \forall \beta \le d_j.$$ Each such linear constraint can be written in the form $$\langle E^{(i, j)}_{\alpha\beta},\,\widetilde W\rangle =0$$ for selector matrices $E$.

3. Rank-Constraint and Convex Relaxations

By construction, $\widetilde{W}$ is rank $r$: $\widetilde{W}$ arises from the dyadic expansion of the original variables. Enforcing $\operatorname{rank}(\widetilde{W}) = r$ renders the SDP nonconvex. However, convex relaxations proceed by (i) dropping the rank constraint, (ii) encouraging low rank via trace or log-det penalty heuristics, e.g., adding a term $-\gamma \log \det(\widetilde{W} + \epsilon I)$ to the objective, or (iii) employing “convex-iteration” schemes [Dattorro ’10] that iteratively reduce the rank of $\widetilde{W}$ until the target is achieved (Fulová et al., 2023).

A plausible implication is that, while removing the rank constraint simplifies the optimization landscape, post-processing via eigen-decomposition or further projection is often necessary to extract a suitable low-rank, block-lower-triangular $X$.

4. Final SDP Model and Structural Constraints

The comprehensive SDP encoding is as follows. For $r = \sum_i d_i$, $n = \sum_j d_j$, one introduces variables $\widetilde W \in \mathrm{Sym}_+^{r+n}$ and $Z \in \mathrm{Sym}_+^m$: $$\begin{array}{rl} \min_{\widetilde{W}, Z} & \operatorname{tr}(A Y A^T) - 2 \operatorname{tr}(B^T A X) + \operatorname{tr}(B B^T) \ \text{s.t.} & \widetilde W = \begin{pmatrix} I_r & X^T \ X & Y \end{pmatrix} \succeq 0, \ & \begin{pmatrix} I_n & (AX - B)^T \ AX - B & Z \end{pmatrix} \succeq 0, \ & Z \succeq 0, \ & (\widetilde W)_{r+\sum_{k<i} d_k + \alpha,\;\sum_{k<j} d_k + \beta} = 0 \quad \forall\,i<j,\,\alpha \le d_i,\,\beta \le d_j. \end{array}$$ The block-zero constraints add $O(p^2 \max_i d_i^2)$ affine equality rows to the SDP.

5. Recovery of the Block-Lower-Triangular Solution

Upon solution of the SDP, the upper-right block of $\widetilde{W}$ yields the approximation to $X$. If the relaxed solution achieves $\operatorname{rank}(\widetilde{W}) = r$, $X^*$ is readily extracted. Otherwise, extraction via the top $r$ eigenpairs is required, followed by re-projection onto the space of block-lower-triangular matrices. This suggests an inherent trade-off: relaxation enables tractability, while subsequent post-processing is essential for exactness (Fulová et al., 2023).

6. Computational and Practical Considerations

The SDP can be addressed using standard solvers (SeDuMi, SDPT3, MOSEK); for $(r+n)$ up to a few hundred, such solvers are practical, with per-iteration complexity $O((r+n)^6)$. For larger problems or to exploit sparsity induced by block-zero constraints, first-order conic solvers (SCS, ProxSDP) may be preferable. Empirically, block-zero constraints are no more demanding than coordinate zero equalities. Convex-iteration or log-det regularization commonly yield the minimal rank solutions necessary for extraction of block-lower-triangular $X^*$ (Fulová et al., 2023).

7. Broader Context and Impact

The block-lower-triangular Procrustes formulation, within the conic programming framework, permits seamless incorporation of additional structure—linear, quadratic, or semi-definite constraints—beyond block triangularity. This generality considerably extends the scope of classical Procrustes analysis, enabling applications in partially specified target structures, oblique or two-sided alignment, and weighted variants. The architectural decomposition by blocks exploits and preserves inherent problem symmetries, allowing efficient imposition of operational and interpretive constraints in applied domains such as computer vision, factor analysis, and finance (Fulová et al., 2023).