Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Sparse and Private Distributed Matrix Multiplication with Straggler Tolerance (2306.15134v1)

Published 27 Jun 2023 in cs.IT, cs.DC, and math.IT

Abstract: This paper considers the problem of outsourcing the multiplication of two private and sparse matrices to untrusted workers. Secret sharing schemes can be used to tolerate stragglers and guarantee information-theoretic privacy of the matrices. However, traditional secret sharing schemes destroy all sparsity in the offloaded computational tasks. Since exploiting the sparse nature of matrices was shown to speed up the multiplication process, preserving the sparsity of the input matrices in the computational tasks sent to the workers is desirable. It was recently shown that sparsity can be guaranteed at the expense of a weaker privacy guarantee. Sparse secret sharing schemes with only two output shares were constructed. In this work, we construct sparse secret sharing schemes that generalize Shamir's secret sharing schemes for a fixed threshold $t=2$ and an arbitrarily large number of shares. We design our schemes to provide the strongest privacy guarantee given a desired sparsity of the shares under some mild assumptions. We show that increasing the number of shares, i.e., increasing straggler tolerance, incurs a degradation of the privacy guarantee. However, this degradation is negligible when the number of shares is comparably small to the cardinality of the input alphabet.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (25)
  1. J. Dean and L. A. Barroso, “The tail at scale,” Communications of the ACM, vol. 56, pp. 74–80, 2013.
  2. J. S. Ng, W. Y. B. Lim, N. C. Luong, Z. Xiong, A. Asheralieva, D. Niyato, C. Leung, and C. Miao, “A comprehensive survey on coded distributed computing: Fundamentals, challenges, and networking applications,” IEEE Communications Surveys & Tutorials, vol. 23, no. 3, pp. 1800–1837, 2021.
  3. S. Ulukus, S. Avestimehr, M. Gastpar, S. Jafar, R. Tandon, and C. Tian, “Private retrieval, computing and learning: Recent progress and future challenges,” arXiv preprint arXiv:2108.00026, 2021.
  4. S. Li and S. Avestimehr, “Coded computing: Mitigating fundamental bottlenecks in large-scale distributed computing and machine learning,” Foundations and Trends® in Communications and Information Theory, vol. 17, no. 1, pp. 1–148, 2020.
  5. K. Lee, M. Lam, R. Pedarsani, D. Papailiopoulos, and K. Ramchandran, “Speeding up distributed machine learning using codes,” IEEE Transactions on Information Theory, vol. 64, no. 3, pp. 1514–1529, 2018.
  6. R. Tandon, Q. Lei, A. G. Dimakis, and N. Karampatziakis, “Gradient coding: Avoiding stragglers in distributed learning,” in International Conference on Machine Learning, vol. 70, Aug 2017, pp. 3368–3376.
  7. W.-T. Chang and R. Tandon, “On the capacity of secure distributed matrix multiplication,” in IEEE Global Communications Conference (GLOBECOM), 2018, pp. 1–6.
  8. R. Bitar, P. Parag, and S. El Rouayheb, “Minimizing latency for secure coded computing using secret sharing via staircase codes,” IEEE Transactions on Communications, vol. 68, no. 8, pp. 4609–4619, 2020.
  9. Q. Yu, S. Li, N. Raviv, S. M. M. Kalan, M. Soltanolkotabi, and S. A. Avestimehr, “Lagrange coded computing: Optimal design for resiliency, security, and privacy,” in International Conference on Artificial Intelligence and Statistics, vol. 89, Apr 2019, pp. 1215–1225.
  10. R. G. L. D’Oliveira, S. El Rouayheb, and D. Karpuk, “GASP Codes for Secure Distributed Matrix Multiplication,” IEEE Transactions on Information Theory, vol. 66, no. 7, pp. 4038–4050, 2020.
  11. M. Aliasgari, O. Simeone, and J. Kliewer, “Private and secure distributed matrix multiplication with flexible communication load,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 2722–2734, 2020.
  12. O. Makkonen and C. Hollanti, “Secure distributed gram matrix multiplication,” arXiv preprint arXiv:2211.14213, 2022.
  13. J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust face recognition via sparse representation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 2, 2008.
  14. J. Madarkar, P. Sharma, and R. P. Singh, “Sparse representation for face recognition: A review paper,” IET Image Processing, vol. 15, no. 9, pp. 1825–1844, 2021.
  15. P. Zardoshti, F. Khunjush, and H. Sarbazi-Azad, “Adaptive sparse matrix representation for efficient matrix–vector multiplication,” The Journal of Supercomputing, vol. 72, no. 9, p. 3366–3386, Sep 2016.
  16. N. Bell and M. Garland, “Implementing sparse matrix-vector multiplication on throughput-oriented processors,” in Conference on High Performance Computing Networking, Storage and Analysis, 2009.
  17. S. Wang, J. Liu, and N. Shroff, “Coded sparse matrix multiplication,” in International Conference on Machine Learning, 2018, pp. 5152–5160.
  18. M. Fahim and V. R. Cadambe, “Lagrange Coded Computing with Sparsity Constraints,” in Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2019, pp. 284–289.
  19. A. B. Das and A. Ramamoorthy, “Coded sparse matrix computation schemes that leverage partial stragglers,” IEEE Transactions on Information Theory, vol. 68, no. 6, pp. 4156–4181, 2022.
  20. M. Xhemrishi, R. Bitar, and A. Wachter-Zeh, “Distributed matrix-vector multiplication with sparsity and privacy guarantees,” in IEEE International Symposium on Information Theory (ISIT), 2022, pp. 1028–1033.
  21. M. Xhemrishi, M. Egger, and R. Bitar, “Efficient private storage of sparse machine learning data,” in IEEE Information Theory Workshop (ITW), 2022.
  22. A. Shamir, “How to share a secret,” Communications of the ACM, vol. 22, no. 11, pp. 612–613, 1979.
  23. G. R. Blakley, “Safeguarding cryptographic keys,” in National Computer Conference, vol. 48, 1979, pp. 313–317.
  24. R. T. Rockafellar, “Lagrange multipliers and optimality,” SIAM Review, vol. 35, no. 2, pp. 183–238, 1993.
  25. R. J. McEliece and D. V. Sarwate, “On sharing secrets and Reed-Solomon codes,” Communications of the ACM, vol. 24, no. 9, pp. 583–584, 1981.
Citations (3)

Summary

We haven't generated a summary for this paper yet.