Consensus-based In-Network Computation of the PARAFAC Decomposition

Abstract: In this work, we present a new approach for the distributed computation of the PARAFAC decomposition of a third-order tensor across a network of collaborating nodes. We are interested in the case where the overall data gathered across the network can be modeled as a data tensor admitting an essentially unique PARAFAC decomposition, while each node only observes a sub-tensor with not necessarily enough diversity so that identifiability conditions are not locally fulfilled at each node. In this situation, conventional (centralized) tensor based methods cannot be applied individually at each node. By allowing collaboration between neighboring nodes of the network, we propose distributed versions of the alternating least squares (ALS) and Levenberg-Marquardt (LM) algorithms for the in-network estimation of the factor matrices of a third-order tensor. We assume that one of the factor matrices contains parameters that are local to each node, while the two remaining factor matrices contain global parameters that are common to the whole network. The proposed algorithms combine the estimation of the local factors with an in-network computation of the global factors of the PARAFAC decomposition using average consensus over graphs. They emulate their centralized counterparts in the case of ideal data exchange and ideal consensus computations. The performance of the proposed algorithms are evaluated in both ideal and imperfect cases.