Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Efficient Approximation Algorithms for Scheduling Coflows with Precedence Constraints in Identical Parallel Networks to Minimize Weighted Completion Time (2307.04107v1)

Published 9 Jul 2023 in cs.DS

Abstract: This paper focuses on the problem of coflow scheduling with precedence constraints in identical parallel networks, which is a well-known $\mathcal{NP}$-hard problem. Coflow is a relatively new network abstraction used to characterize communication patterns in data centers. Both flow-level scheduling and coflow-level scheduling problems are examined, with the key distinction being the scheduling granularity. The proposed algorithm effectively determines the scheduling order of coflows by employing the primal-dual method. When considering workload sizes and weights that are dependent on the network topology in the input instances, our proposed algorithm for the flow-level scheduling problem achieves an approximation ratio of $O(\chi)$ where $\chi$ is the coflow number of the longest path in the directed acyclic graph (DAG). Additionally, when taking into account workload sizes that are topology-dependent, the algorithm achieves an approximation ratio of $O(R\chi)$, where $R$ represents the ratio of maximum weight to minimum weight. For the coflow-level scheduling problem, the proposed algorithm achieves an approximation ratio of $O(m\chi)$, where $m$ is the number of network cores, when considering workload sizes and weights that are topology-dependent. Moreover, when considering workload sizes that are topology-dependent, the algorithm achieves an approximation ratio of $O(Rm\chi)$. In the coflows of multi-stage job scheduling problem, the proposed algorithm achieves an approximation ratio of $O(\chi)$. Although our theoretical results are based on a limited set of input instances, experimental findings show that the results for general input instances outperform the theoretical results, thereby demonstrating the effectiveness and practicality of the proposed algorithm.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. S. Agarwal, S. Rajakrishnan, A. Narayan, R. Agarwal, D. Shmoys, and A. Vahdat, “Sincronia: Near-optimal network design for coflows,” in Proceedings of the 2018 ACM Conference on SIGCOMM, ser. SIGCOMM ’18.   New York, NY, USA: Association for Computing Machinery, 2018, p. 16–29.
  2. S. Ahmadi, S. Khuller, M. Purohit, and S. Yang, “On scheduling coflows,” Algorithmica, vol. 82, no. 12, pp. 3604–3629, 2020.
  3. M. Al-Fares, A. Loukissas, and A. Vahdat, “A scalable, commodity data center network architecture,” ACM SIGCOMM computer communication review, vol. 38, no. 4, pp. 63–74, 2008.
  4. N. Bansal and S. Khot, “Inapproximability of hypergraph vertex cover and applications to scheduling problems,” in Automata, Languages and Programming, S. Abramsky, C. Gavoille, C. Kirchner, F. Meyer auf der Heide, and P. G. Spirakis, Eds.   Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, pp. 250–261.
  5. D. Borthakur, “The hadoop distributed file system: Architecture and design,” Hadoop Project Website, vol. 11, no. 2007, p. 21, 2007.
  6. M. Chowdhury and I. Stoica, “Coflow: A networking abstraction for cluster applications,” in Proceedings of the 11th ACM Workshop on Hot Topics in Networks, ser. HotNets-XI.   New York, NY, USA: Association for Computing Machinery, 2012, p. 31–36.
  7. ——, “Efficient coflow scheduling without prior knowledge,” in Proceedings of the 2015 ACM Conference on SIGCOMM, ser. SIGCOMM ’15.   New York, NY, USA: Association for Computing Machinery, 2015, p. 393–406.
  8. M. Chowdhury, M. Zaharia, J. Ma, M. I. Jordan, and I. Stoica, “Managing data transfers in computer clusters with orchestra,” ACM SIGCOMM computer communication review, vol. 41, no. 4, pp. 98–109, 2011.
  9. M. Chowdhury, Y. Zhong, and I. Stoica, “Efficient coflow scheduling with varys,” in Proceedings of the 2014 ACM Conference on SIGCOMM, ser. SIGCOMM ’14.   New York, NY, USA: Association for Computing Machinery, 2014, p. 443–454.
  10. M. I. Daoud and N. Kharma, “A high performance algorithm for static task scheduling in heterogeneous distributed computing systems,” Journal of Parallel and Distributed Computing, vol. 68, no. 4, pp. 399 – 409, 2008.
  11. J. M. Davis, R. Gandhi, and V. H. Kothari, “Combinatorial algorithms for minimizing the weighted sum of completion times on a single machine,” Operations Research Letters, vol. 41, no. 2, pp. 121–125, 2013.
  12. J. Dean and S. Ghemawat, “Mapreduce: Simplified data processing on large clusters,” Communications of the ACM, vol. 51, no. 1, p. 107–113, jan 2008.
  13. F. R. Dogar, T. Karagiannis, H. Ballani, and A. Rowstron, “Decentralized task-aware scheduling for data center networks,” ACM SIGCOMM Computer Communication Review, vol. 44, no. 4, pp. 431–442, 2014.
  14. A. Greenberg, J. R. Hamilton, N. Jain, S. Kandula, C. Kim, P. Lahiri, D. A. Maltz, P. Patel, and S. Sengupta, “Vl2: A scalable and flexible data center network,” in Proceedings of the ACM SIGCOMM 2009 conference on Data communication, 2009, pp. 51–62.
  15. X. S. Huang, X. S. Sun, and T. E. Ng, “Sunflow: Efficient optical circuit scheduling for coflows,” in Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies, 2016, pp. 297–311.
  16. X. S. Huang, Y. Xia, and T. S. E. Ng, “Weaver: Efficient coflow scheduling in heterogeneous parallel networks,” in 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2020, pp. 1071–1081.
  17. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, “Dryad: distributed data-parallel programs from sequential building blocks,” in Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, 2007, pp. 59–72.
  18. S. Khuller and M. Purohit, “Brief announcement: Improved approximation algorithms for scheduling co-flows,” in Proceedings of the 28th ACM Symposium on Parallelism in Algorithms and Architectures, 2016, pp. 239–240.
  19. Z. Qiu, C. Stein, and Y. Zhong, “Minimizing the total weighted completion time of coflows in datacenter networks,” in Proceedings of the 27th ACM Symposium on Parallelism in Algorithms and Architectures, ser. SPAA ’15.   New York, NY, USA: Association for Computing Machinery, 2015, p. 294–303.
  20. S. Sachdeva and R. Saket, “Optimal inapproximability for scheduling problems via structural hardness for hypergraph vertex cover,” in 2013 IEEE Conference on Computational Complexity, 2013, pp. 219–229.
  21. M. Shafiee and J. Ghaderi, “An improved bound for minimizing the total weighted completion time of coflows in datacenters,” IEEE/ACM Transactions on Networking, vol. 26, no. 4, pp. 1674–1687, 2018.
  22. ——, “Scheduling coflows with dependency graph,” IEEE/ACM Transactions on Networking, 2021.
  23. K. Shvachko, H. Kuang, S. Radia, and R. Chansler, “The hadoop distributed file system,” in 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), 2010, pp. 1–10.
  24. A. Singh, J. Ong, A. Agarwal, G. Anderson, A. Armistead, R. Bannon, S. Boving, G. Desai, B. Felderman, P. Germano, A. Kanagala, J. Provost, J. Simmons, E. Tanda, J. Wanderer, U. Hölzle, S. Stuart, and A. Vahdat, “Jupiter rising: A decade of clos topologies and centralized control in google’s datacenter network,” in Proceedings of the 2015ACM Conference on SIGCOMM, ser. SIGCOMM ’15.   New York, NY, USA: Association for Computing Machinery, 2015, p. 183–197.
  25. B. Tian, C. Tian, H. Dai, and B. Wang, “Scheduling coflows of multi-stage jobs to minimize the total weighted job completion time,” in IEEE INFOCOM 2018 - IEEE Conference on Computer Communications, 2018, pp. 864–872.
  26. H. Topcuoglu, S. Hariri, and M.-Y. Wu, “Performance-effective and low-complexity task scheduling for heterogeneous computing,” IEEE Transactions on Parallel and Distributed Systems, vol. 13, no. 3, pp. 260–274, Mar 2002.
  27. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica, “Spark: Cluster computing with working sets,” in 2nd USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 10), 2010.
  28. H. Zhang, L. Chen, B. Yi, K. Chen, M. Chowdhury, and Y. Geng, “Coda: Toward automatically identifying and scheduling coflows in the dark,” in Proceedings of the 2016 ACM Conference on SIGCOMM, ser. SIGCOMM ’16.   New York, NY, USA: Association for Computing Machinery, 2016, p. 160–173.
  29. Y. Zhao, K. Chen, W. Bai, M. Yu, C. Tian, Y. Geng, Y. Zhang, D. Li, and S. Wang, “Rapier: Integrating routing and scheduling for coflow-aware data center networks,” in 2015 IEEE Conference on Computer Communications (INFOCOM).   IEEE, 2015, pp. 424–432.
Citations (1)

Summary

We haven't generated a summary for this paper yet.