Reduced-Rank Tensor-on-Tensor Regression and Tensor-variate Analysis of Variance (2012.10249v5)

Abstract: Fitting regression models with many multivariate responses and covariates can be challenging, but such responses and covariates sometimes have tensor-variate structure. We extend the classical multivariate regression model to exploit such structure in two ways: first, we impose four types of low-rank tensor formats on the regression coefficients. Second, we model the errors using the tensor-variate normal distribution that imposes a Kronecker separable format on the covariance matrix. We obtain maximum likelihood estimators via block-relaxation algorithms and derive their computational complexity and asymptotic distributions. Our regression framework enables us to formulate tensor-variate analysis of variance (TANOVA) methodology. This methodology, when applied in a one-way TANOVA layout, enables us to identify cerebral regions significantly associated with the interaction of suicide attempters or non-attemptor ideators and positive-, negative- or death-connoting words in a functional Magnetic Resonance Imaging study. Another application uses three-way TANOVA on the Labeled Faces in the Wild image dataset to distinguish facial characteristics related to ethnic origin, age group and gender.

• The paper introduces extended tensor regression models using four low-rank formats to drastically reduce parameter dimensionality.
• It employs a tensor-variate normal error model with Kronecker-separable covariance, enabling independent mode error handling.
• Extensive simulations and applications in neuroimaging and face recognition demonstrate improved estimation accuracy and practical scalability.

Insights into Tensor-on-Tensor Regression and Tensor-variate Methods

This paper by Llosa-Vite and Maitra contributes notable advancements in the field of high-dimensional data analysis, specifically focusing on tensor-on-tensor regression (ToTR) and tensor-variate analysis of variance (TANOVA). The authors extend traditional multivariate regression techniques to efficiently handle and exploit the structure in tensor-variate data. The models presented in the research allow computationally feasible estimation in complex datasets understood within this domain, such as neuroimaging (fMRI data) and large image datasets.

The paper constructs upon the classical multivariate multiple linear regression (MVMLR) models by extending them to accommodate tensor-valued responses and covariates, proposing four low-rank tensor formats: Tucker, CANDECOMP/PARAFAC (CP), Tensor Ring (TR), and the outer product (OP) model. These formats are pivotal because they significantly reduce the problem's dimensionality, transforming infeasible computation due to millions of parameters into manageable, structured problems.

To improve modeling, the authors implement a tensor-variate normal distribution for errors by assuming Kronecker-separable covariance matrices. This approach splits the covariance into a product affecting different modes independently, facilitating separate correlation structures for various contexts like spatial, temporal, etc., in tensor data. Such a setup significantly reduces the parameter space from $\prod_{i=1}^p m_i$ to a manageable $\sum_{i=1}^p m_i$ form.

Performance of the proposed methodology is elucidated through intensive simulation studies confirming consistency and enhanced accuracy of estimations when compared with traditional vectorized approaches. A variety of low-rank formats demonstrate adaptability across different problem sizes and arrangements.

Substantive applications enrich this research, highlighting its practicality. For instance, in the context of neuroimaging, the ability to delineate cerebral regions related to suicidal thought processing using TANOVA showcases the adaptability of tensor methods to real-world medical datasets. The imaging applications, particularly in face recognition tasks like Labeled Faces in the Wild (LFW) dataset, demonstrate ToTR's potential in distinguishing nuanced attributes such as ethnicity, age, and gender.

Notably, the computations involved are adequately scalable due to the hierarchical structure provided by the chosen tensor formats, as detailed in the computational complexity results. As for statistical inference, the paper derives asymptotic distributions enlightening robust hypothesis evaluation capabilities within this tensor framework.

In conclusion, this research advances the frontier of tensor data modeling by robustly addressing the challenges associated with high-dimensional datasets. The methodology not only optimizes estimation accuracy but also yields critical insights for practical applications in fields requiring intricate data structures. Future works can consider incorporating regularization techniques for further dimension reduction and broadening applicability in non-Gaussian contexts, thus expanding tensor-based methods' repertoire in modern data analysis.