Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Efficient parallel implementation of the multiplicative weight update method for graph-based linear programs (2307.03307v3)

Published 6 Jul 2023 in cs.DC, cs.DM, and math.OC

Abstract: Positive linear programs (LPs) model many graph and operations research problems. One can solve for a $(1+\epsilon)$-approximation for positive LPs, for any selected $\epsilon$, in polylogarithmic depth and near-linear work via variations of the multiplicative weight update (MWU) method. Despite extensive theoretical work on these algorithms through the decades, their empirical performance is not well understood. In this work, we implement and test an efficient parallel algorithm for solving positive LP relaxations, and apply it to graph problems such as densest subgraph, bipartite matching, vertex cover and dominating set. We accelerate the algorithm via a new step size search heuristic. Our implementation uses sparse linear algebra optimization techniques such as fusion of vector operations and use of sparse format. Furthermore, we devise an implicit representation for graph incidence constraints. We demonstrate the parallel scalability with the use of threading OpenMP and MPI on the Stampede2 supercomputer. We compare this implementation with exact libraries and specialized libraries for the above problems in order to evaluate MWU's practical standing for both accuracy and performance among other methods. Our results show this implementation is faster than general purpose LP solvers (IBM CPLEX, Gurobi) in all of our experiments, and in some instances, outperforms state-of-the-art specialized parallel graph algorithms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Optimizing Sparse Matrix-Multiple Vectors Multiplication for Nuclear Configuration Interaction Calculations. In 2014 IEEE 28th International Parallel and Distributed Processing Symposium. 1213–1222. https://doi.org/10.1109/IPDPS.2014.125
  2. Zeyuan Allen-Zhu and Lorenzo Orecchia. 2014. Using optimization to break the epsilon barrier: A faster and simpler width-independent algorithm for solving positive linear programs in parallel. In Proceedings of the twenty-sixth annual ACM-SIAM symposium on Discrete algorithms. SIAM, 1439–1456.
  3. Zeyuan Allen-Zhu and Lorenzo Orecchia. 2016. Using Optimization to Solve Positive LPs Faster in Parallel. arXiv:1407.1925 [cs.DS]
  4. Zeyuan Allen-Zhu and Lorenzo Orecchia. 2019. Nearly linear-time packing and covering LP solvers. Mathematical Programming 175, 1-2 (2019), 307–353.
  5. Parallel approximate undirected shortest paths via low hop emulators. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing. 322–335.
  6. The multiplicative weights update method: a meta-algorithm and applications. Theory of Computing 8, 1 (2012), 121–164.
  7. Baruch Awerbuch and Rohit Khandekar. 2009. Stateless distributed gradient descent for positive linear programs. SIAM J. Comput. 38, 6 (2009), 2468–2486.
  8. Computing maximum cardinality matchings in parallel on bipartite graphs via tree-grafting. IEEE Transactions on Parallel and Distributed Systems 28, 1 (2016), 44–59.
  9. PETSc users manual. (2019).
  10. The Netflix prize. In Proceedings of KDD cup and workshop, Vol. 2007. New York, NY, USA., 35.
  11. Flowless: extracting densest subgraphs without flow computations. In Proceedings of The Web Conference 2020. 573–583.
  12. Parallel sparse matrix-vector and matrix-transpose-vector multiplication using compressed sparse blocks. In Proceedings of the twenty-first annual symposium on Parallelism in algorithms and architectures. 233–244.
  13. Aydın Buluç and John R Gilbert. 2011. The Combinatorial BLAS: Design, implementation, and applications. The International Journal of High Performance Computing Applications 25, 4 (2011), 496–509.
  14. Moses Charikar. 2000. Greedy approximation algorithms for finding dense components in a graph. In International Workshop on Approximation Algorithms for Combinatorial Optimization. Springer, 84–95.
  15. IBM ILOG Cplex. 2009. V12. 1: User’s Manual for CPLEX. International Business Machines Corporation 46, 53 (2009), 157.
  16. Timothy A. Davis and Yifan Hu. 2011. The University of Florida Sparse Matrix Collection. ACM Trans. Math. Softw. 38, 1, Article 1 (Dec. 2011), 25 pages. https://doi.org/10.1145/2049662.2049663
  17. Low-latency graph streaming using compressed purely-functional trees. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. 918–934.
  18. The graph based benchmark suite (gbbs). In Proceedings of the 3rd Joint International Workshop on Graph Data Management Experiences & Systems (GRADES) and Network Data Analytics (NDA). 1–8.
  19. The Yahoo! music dataset and KDD-cup’11. In Proceedings of KDD Cup 2011. PMLR, 3–18.
  20. Michael D. Grigoriadis and Leonid G. Khachiyan. 1994. Fast Approximation Schemes for Convex Programs with Many Blocks and Coupling Constraints. SIAM J. Optim. 4, 1 (1994), 86–107. https://doi.org/10.1137/0804004
  21. LLC Gurobi Optimization. 2021. Gurobi Optimizer Reference Manual. http://www.gurobi.com
  22. A fast parallel implementation of a PTAS for fractional packing and covering linear programs. International Journal of Parallel Programming 43, 5 (2015), 840–875.
  23. Richard M Karp and Vijaya Ramachandran. 1989. A survey of parallel algorithms for shared-memory machines. (1989).
  24. Richard M Karp and Michael Sipser. 1981. Maximum matching in sparse random graphs. In 22nd Annual symposium on foundations of computer science (sfcs 1981). IEEE, 364–375.
  25. Christos Koufogiannakis and Neal E Young. 2014. A nearly linear-time PTAS for explicit fractional packing and covering linear programs. Algorithmica 70, 4 (2014), 648–674.
  26. Jason Li. 2020. Faster parallel algorithm for approximate shortest path. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing. 308–321.
  27. A performance study of some approximation algorithms for minimum dominating set in a graph. arXiv preprint arXiv:2009.04636 (2020).
  28. Yang P Liu and Aaron Sidford. 2020. Faster energy maximization for faster maximum flow. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing. 803–814.
  29. Michael Luby and Noam Nisan. 1993. A parallel approximation algorithm for positive linear programming. In Proceedings of the twenty-fifth annual ACM symposium on Theory of computing. 448–457.
  30. Challenges in parallel graph processing. Parallel Processing Letters 17, 01 (2007), 5–20.
  31. Approximating the Solution to Mixed Packing and Covering LPs in Parallel O~⁢(ϵ−3)~𝑂superscriptitalic-ϵ3\tilde{O}(\epsilon^{-3})over~ start_ARG italic_O end_ARG ( italic_ϵ start_POSTSUPERSCRIPT - 3 end_POSTSUPERSCRIPT ) Time. In 43rd International Colloquium on Automata, Languages, and Programming (ICALP 2016). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik. Available at urlhttps://drops.dagstuhl.de/opus/volltexte/2016/6333/pdf/LIPIcs-ICALP-2016-52.pdf.
  32. A distributed algorithm for large-scale generalized matching. (2013).
  33. A Lightweight Infrastructure for Graph Analytics. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (Farminton, Pennsylvania) (SOSP’13). Association for Computing Machinery, New York, NY, USA, 456–471. https://doi.org/10.1145/2517349.2522739
  34. Jorge Nocedal and Stephen Wright. 2006. Numerical optimization. Springer Science & Business Media.
  35. Fast approximation algorithms for fractional packing and covering problems. Mathematics of Operations Research 20, 2 (1995), 257–301.
  36. Kent Quanrud. 2019. Fast Approximations for Combinatorial Optimization via Multiplicative Weight Updates. https://www.ideals.illinois.edu/bitstream/handle/2142/106153/QUANRUD-DISSERTATION-2019.pdf.
  37. Ankit Rohatgi. 2020. Webplotdigitizer: Version 4.4. https://automeris.io/WebPlotDigitizer
  38. Gerald Roth and Ken Kennedy. 1998. Loop Fusion in High Performance Fortran. In Proceedings of the 12th International Conference on Supercomputing (Melbourne, Australia) (ICS ’98). Association for Computing Machinery, New York, NY, USA, 125–132. https://doi.org/10.1145/277830.277857
  39. Jonah Sherman. 2013. Nearly maximum flows in nearly linear time. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science. IEEE, 263–269.
  40. Julian Shun and Guy E Blelloch. 2013. Ligra: a lightweight graph processing framework for shared memory. In Proceedings of the 18th ACM SIGPLAN symposium on Principles and practice of parallel programming. 135–146.
  41. Smaller and faster: Parallel processing of compressed graphs with Ligra+. In 2015 Data Compression Conference. IEEE, 403–412.
  42. Scaling betweenness centrality using communication-efficient sparse matrix multiplication. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 1–14.
  43. Parallel graph partitioning on multicore architectures. In International Workshop on Languages and Compilers for Parallel Computing. Springer, 246–260.
  44. Bipartite matching in nearly-linear time on moderately dense graphs. In 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS). IEEE, 919–930.
  45. Solving tall dense linear programs in nearly linear time. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing. 775–788.
  46. Vijay V Vazirani. 2013. Approximation algorithms. Springer Science & Business Media.
  47. Di Wang. 2017. Fast Approximation Algorithms for Positive Linear Programs. Ph.D. Dissertation. EECS Department, University of California, Berkeley. http://www2.eecs.berkeley.edu/Pubs/TechRpts/2017/EECS-2017-126.html
  48. David P Williamson and David B Shmoys. 2011. The design of approximation algorithms. Cambridge university press.
  49. Neal E Young. 2001. Sequential and parallel algorithms for mixed packing and covering. In Proceedings 42nd IEEE symposium on foundations of computer science. IEEE, 538–546.
  50. Neal E Young. 2014. Nearly linear-work algorithms for mixed packing/covering and facility-location linear programs. https://arxiv.org/abs/1407.3015. arXiv preprint arXiv:1407.3015 (2014).

Summary

We haven't generated a summary for this paper yet.