Semi-MapReduce Meets Congested Clique (1802.10297v2)

Abstract: Graph problems are troublesome when it comes to MapReduce. Typically, to be able to design algorithms that make use of the advantages of MapReduce, assumptions beyond what the model imposes, such as the density of the input graph, are required. In a recent shift, a simple and robust model of MapReduce for graph problems, where the space per machine is set to be O(|V|), has attracted considerable attention. We term this model semi-MapReduce, or in short, semiMPC, and focus on its computational power. We show through a set of simulation methods that semiMPC is, perhaps surprisingly, equivalent to the congested clique model of distributed computing. However, semiMPC, in addition to round complexity, incorporates another practically important dimension to optimize: the number of machines. Furthermore, we show that algorithms in other distributed computing models, such as CONGEST, can be simulated to run in the same number of rounds of semiMPC while also using an optimal number of machines. We later show the implications of these simulation methods by obtaining improved algorithms for these models using the recent algorithms that have been developed.