Rate-limited Shuffling for Distributed Computing (2403.01296v2)
Abstract: This paper studies the shuffling phase in a distributed computing model with rate-limited links between nodes. Each node is connected to all other nodes via a noiseless broadcast link with a finite capacity. For this network, the shuffling phase is described as a distributed index-coding problem to extend an outer bound for the latter to the distributed computing problem. An inner bound on the capacity region is also established by using the distributed composite-coding scheme introduced for the distributed index-coding problem. We consider some special cases of the distributed computing problem through two examples for which we prove that the inner and outer bounds agree, thereby establishing the capacity regions. We, then, generalize the special cases to any number of nodes and computation loads under certain constraints.
- J. Dean and S. Ghemawat, “Mapreduce: Simplified data processing on large clusters," in ACM Commun., vol. 51, no. 1, pp. 107-113, Jan. 2008.
- S. Li, M. A. Maddah-Ali, Q. Yu and A. S. Avestimehr, “A Fundamental Tradeoff Between Computation and Communication in Distributed Computing," in IEEE Trans. Inf. Theory, vol. 64, no. 1, pp. 109-128, Jan. 2018.
- Q. Yan, S. Yang and M. Wigger, “Storage computation and communication: A fundamental tradeoff in distributed computing", in Proc. IEEE Inf. Theory Workshop, pp. 1-5, Guangzhou, China, Sept. 2018.
- L. Song, C. Fragouli and T. Zhao, “A pliable index coding approach to data shuffling", in IEEE Trans. Inf. Theory, vol. 66, no. 3, pp. 1333-1353, Mar. 2020.
- Y. Yang, P. Grover and S. Kar, “Coded distributed computing for inverse problems", in Proc. Adv. Neural Inf. Process. Syst., pp. 709-719, Long Beach, CA, Dec. 2017.
- H. Yang and J. Lee, “Secure distributed computing with straggling servers using polynomial codes", in IEEE Trans. Inf. Forensics Security, vol. 14, no. 1, pp. 141-150, Jan. 2019.
- S. Dhakal, S. Prakash, Y. Yona, S. Talwar and N. Himayat, “Coded computing for distributed machine learning in wireless edge network", in Proc. IEEE Veh. Technol. Conf., Honolulu, HI, Sept. 2019, pp. 1-6.
- N. Woolsey, R.-R. Chen and M. Ji, “A combinatorial design for cascaded coded distributed computing on general networks", in IEEE Trans. Commun., vol. 69, no. 9, pp. 5686-5700, Sept. 2021.
- M. Dai, Z. Zheng, S. Zhang, H. Wang and X. Lin, “SAZD: A low computational load coded distributed computing framework for IoT systems", in IEEE Internet Things J., vol. 7, no. 4, pp. 3640-3649, Apr. 2020.
- S. Li, M. A. Maddah-Ali and A. S. Avestimehr, “Coding for distributed fog computing", in IEEE Commun. Mag., vol. 55, no. 4, pp. 34-40, Apr. 2017.
- H. Park, K. Lee, J. Sohn, C. Suh and J. Moon, “Hierarchical coding for distributed computing", in Proc. IEEE Int. Symp. Inf. Theory, Vail, CO, June 2018, pp. 1630-1634.
- Y. Sun, J. Zhao, S. Zhou and D. Gunduz, “Heterogeneous coded computation across heterogeneous workers", in Proc. IEEE Global Commun. Conf., pp. 1-6, Waikoloa, HI, Dec. 2019.
- N. Woolsey, R. Chen and M. Ji, “A new combinatorial design of coded distributed computing", in Proc. IEEE Int. Symp. Inf. Theory, Vail, CO, June 2018, pp. 726-730.
- J. Jiang and L. Qu, “Cascaded coded distributed computing schemes based on placement delivery arrays", in IEEE Access, vol. 8, pp. 221385-221395, Dec. 2020.
- Y. Liu, P. Sadeghi, F. Arbabjolfaei and Y. -H. Kim, “Capacity Theorems for Distributed Index Coding," in IEEE Trans. Inf. Theory, vol. 66, no. 8, pp. 4653-4680, Aug. 2020.
- M. Li, L. Ong and S. J. Johnson, “Cooperative Multi-Sender Index Coding,” in IEEE Trans. Inf. Theory, vol. 65, no. 3, pp. 1725-1739, March 2019.
- L. Ong, C. K. Ho, and F. Lim, “The single-uniprior index-coding problem: The single-sender case and the multi-sender extension," IEEE Trans. Inf. Theory, vol. 62, no. 6, pp. 3165-3182, Jun. 2016.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.