Robust Multi-Task Learning for Principal Component Analysis

Abstract: Principal component analysis (PCA) is a fundamental tool for learning low-dimensional structure from high-dimensional data. When data are collected from multiple sources, the underlying task distributions may exhibit unknown degrees of similarity, with some tasks potentially arising from arbitrary distributions. We propose new multi-task PCA procedures that exploit similarity structure across tasks to improve eigenspace estimation while remaining robust to outlier tasks. We establish non-asymptotic convergence rates and show that the proposed procedures attain minimax optimal rates in a range of regimes. One of the procedures builds on the matrix-depth notion of Chen, Gao, and Ren (2018) and can achieve the optimal dependence of the estimation error on the proportion of outlier tasks, addressing a key challenge in robust multi-task learning. Extensive simulations and real-data analyses demonstrate the effectiveness of the proposed methods.