Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Generalized regenerating codes and node repair on graphs (2405.11714v1)

Published 20 May 2024 in cs.IT and math.IT

Abstract: We consider regenerating codes in distributed storage systems where connections between the nodes are constrained by a graph. In this problem, the failed node downloads the information stored at a subset of vertices of the graph for the purpose of recovering the lost data. Compared to the standard setting, regenerating codes on graphs address two additional features. The repair information is moved across the network, and the cost of node repair is determined by the graphical distance from the helper nodes to the failed node. Accordingly, the helpers far away from the failed node may be expected to contribute less data for repair than the nodes in the neighborhood of that node. We analyze regenerating codes with nonuniform download for repair on graphs. Moreover, in the process of repair, the information moved from the helpers to the failed node may be combined through intermediate processing, reducing the repair bandwidth. We derive lower bounds for communication complexity of node repair on graphs, including repair schemes with nonuniform download and intermediate processing, and construct codes that attain these bounds. Additionally, some of the nodes may act as adversaries, introducing errors into the data moved in the network. For repair on graphs in the presence of adversarial nodes, we construct codes that support node repair and error correction in systematic nodes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. S. Akhlaghi, A. Kiani, and M. R. Ghanavati, “Cost-bandwidth tradeoff in distributed storage systems,” Computer Communications, vol. 33, no. 17, pp. 2105–2115, 2010.
  2. A. Beemer, A. Kılıç, and A. Ravagnani, “Network decoding,” IEEE Transactions on Information Theory, vol. 69, no. 6, pp. 3708–3730, 2023.
  3. È. L. Blokh and V. V. Zyablov, “Coding of generalized concatenated codes,” Problemy Peredachi Informatsii, vol. 10, no. 3, pp. 45–50, 1974.
  4. B. Bollobás, “The diameter of random graphs,” Trans. AMS, vol. 267, no. 1, pp. 41–52, 1981.
  5. Z. Chen and A. Barg, “Explicit constructions of MSR codes for clustered distributed storage: The rack-aware storage model,” IEEE Trans. Inf. Theory, vol. 66, no. 2, pp. 866–879, 2020.
  6. Z. Chen, “Rack-aware MSR codes with optimal access,” in 2022 IEEE Information Theory Workshop (ITW).   IEEE, 2022, pp. 19–24.
  7. F. Chung and L. Lu, “The diameter of sparse random graphs,” Advances in Applied Mathematics, vol. 26, no. 4, pp. 257–279, 2001.
  8. A. G. Dimakis, P. B. Godfrey, Y. Wu, M. J. Wainwright, and K. Ramchandran, “Network coding for distributed storage systems,” IEEE Trans. Inf. Theory, vol. 56, no. 9, pp. 4539–4551, 2010.
  9. I. M. Duursma and H.-P. Wang, “Multilinear algebra for minimum storage regenerating codes: a generalization of the product-matrix construction,” Applicable Algebra in Engineering, Communication and Computing, vol. 34, pp. 717–743, 2023.
  10. M. Gerami and M. Xiao, “Exact optimized-cost repair in multi-hop distributed storage networks,” in 2014 IEEE International Conference on Communications (ICC), 2014, pp. 4120–4124.
  11. S. Gupta, B. R. Devi, and V. Lalitha, “On rack-aware cooperative regenerating codes and epsilon-MSCR codes,” IEEE Journal on Selected Areas in Information Theory, vol. 3, no. 2, pp. 362–378, 2022.
  12. H. Hou, P. Lee, K. Shum, and Y. Hu, “Rack-aware regenerating codes for data centers,” IEEE Trans. Inf. Theory, vol. 65, no. 8, pp. 4730–4745, 2019.
  13. R. Koetter and F. R. Kschischang, “Coding for errors and erasures in random network coding,” IEEE Transactions on Information Theory, vol. 54, no. 8, pp. 3579–3591, 2008.
  14. O. Kosut, L. Tong, and D. N. C. Tse, “Polytope codes against adversaries in networks,” IEEE Transactions on Information Theory, vol. 60, no. 6, pp. 3308–3344, 2014.
  15. Z. Li, W. H. Mow, L. Deng, and T.-Y. Wu, “Optimal-repair-cost MDS array codes for a class of heterogeneous distributed storage systems,” in 2022 IEEE International Symposium on Information Theory (ISIT), 2022, pp. 2379–2384.
  16. Y. Liu, J. Li, and X. H. Tang, “Optimal repair/access MDS array codes with multiple repair degrees,” 2022, eprint arXiv:2205.13446.
  17. J. Lu, X. Guang, and F.-W. Fu, “Distributed storage over unidirectional ring networks,” in 2014 International Symposium on Information Theory and its Applications, 2014, pp. 368–372.
  18. A. Lubotzky, R. Phillips, and P. Sarnak, “Ramanujan graphs,” Combinatorica, vol. 8, no. 3, pp. 261–277, 1988.
  19. G. A. Margulis, “Explicit constructions of graphs without short cycles and low density codes,” Combinatorica, vol. 2, no. 1, pp. 71–78, 1982.
  20. A. Patra and A. Barg, “Interior-point regenerating codes on graphs,” in 2022 IEEE International Symposium on Information Theory (ISIT).   IEEE, 2022, pp. 1560–1565.
  21. ——, “Node repair on connected graphs,” IEEE Transactions on Information Theory, vol. 68, no. 5, pp. 3081–3095, 2022.
  22. ——, “Node repair on connected graphs, Part II,” 2022, eprint arXiv:2211.00797.
  23. ——, “Node repair for adversarial graphical networks,” in 2023 IEEE International Symposium on Information Theory (ISIT).   IEEE, 2023, pp. 2284–2289.
  24. N. Prakash, V. Abdrashitov, and M. Médard, “The storage versus repair-bandwidth trade-off for clustered storage systems,” IEEE Transactions on Information Theory, vol. 64, no. 8, pp. 5783–5805, 2018.
  25. V. Ramkumar, S. Balaji, B. Sasidharan, M. Vajha, M. N. Krishnan, and P. V. Kumar, “Codes for distributed storage,” Foundations and Trends in Communications and Information Theory, vol. 19, pp. 547–813, 2022.
  26. K. V. Rashmi, N. B. Shah, and P. V. Kumar, “Optimal exact-regenerating codes for distributed storage at the MSR and MBR points via a product-matrix construction,” IEEE Trans. Inf. Theory, vol. 57, no. 8, pp. 5227–5239, 2011.
  27. K. V. Rashmi, N. B. Shah, K. Ramchandran, and P. V. Kumar, “Regenerating codes for errors and erasures in distributed storage,” in Proc. IEEE International Symposium on Information Theory, Cambridge, MA, USA, 2012, pp. 1202–1206.
  28. N. B. Shah, K. V. Rashmi, P. V. Kumar, and K. Ramchandran, “Distributed storage codes with repair-by-transfer and nonachievability of interior points on the storage-bandwidth tradeoff,” IEEE Trans. Inf. Theory, vol. 58, no. 3, pp. 1837–1852, 2012.
  29. N. Silberstein, A. S. Rawat, and S. Vishwanath, “Error-correcting regenerating and locally repairable codes via rank-metric codes,” IEEE Trans. Inform. Theory, vol. 61, no. 11, pp. 5765–5778, 2015.
  30. J. Y. Sohn, B. Choi, S. W. Yoon, and J. Moon, “Capacity of clustered distributed storage,” IEEE Trans. Inf. Theory, vol. 65, no. 1, pp. 81–107, 2019.
  31. J. Wang and Z. Chen, “Rack-aware minimum-storage regenerating codes with optimal access,” arXiv preprint arXiv:2304.08747, 2023.
  32. J. Wang, D. Zheng, S. Li, and X. Tang, “Rack-aware MSR codes with error correction capability for multiple erasure tolerance,” IEEE Transactions on Information Theory, vol. 69, no. 10, pp. 6428–6442, 2023.
  33. J. Wang, Y. Luo, and K. W. Shum, “Storage and repair bandwidth tradeoff for heterogeneous cluster distributed storage systems,” Science China Information Sciences, vol. 63, pp. 1–15, 2020.
  34. Y. Wang, D. Wei, X. Yin, and X. Wang, “Heterogeneity-aware data regeneration in distributed storage systems,” in IEEE INFOCOM 2014-IEEE Conference on Computer Communications.   IEEE, 2014, pp. 1878–1886.
  35. S. Yang, A. Hareedy, R. Calderbank, and L. Dolecek, “Hierarchical coding for cloud storage: Topology-adaptivity, scalability, and flexibility,” IEEE Transactions on Information Theory, vol. 68, no. 6, pp. 3657–3680, 2022.
  36. M. Ye and A. Barg, “Explicit constructions of optimal-access MDS codes with nearly optimal sub-packetization,” IEEE Trans. Inf. Theory, vol. 63, no. 10, pp. 6307–6317, 2017.
  37. R. W. Yeung and N. Cai, “Network error correction, Part I: Basic concepts and upper bounds,” Communications in Information and Systems, vol. 6, no. 1, pp. 19 – 35, 2006.
  38. R. W. Yeung, S.-Y. R. Li, N. Cai, and Z. Zhang, “Network coding theory Part I: Single source,” Foundations and Trends® in Communications and Information Theory, vol. 2, no. 4, pp. 241–329, 2006.
  39. Q. Yu, K. W. Shum, and C. W. Sung, “Tradeoff between storage cost and repair cost in heterogeneous distributed storage systems,” Transactions on Emerging Telecommunications Technologies, vol. 26, no. 10, pp. 1201–1211, 2015.
  40. H. Zhang, H. Li, and S.-Y. R. Li, “Repair tree: fast repair for single failure in erasure-coded distributed storage systems,” IEEE Transactions on Parallel and Distributed Systems, vol. 28, no. 6, pp. 1728–1739, 2016.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now