On Low Field Size Constructions of Access-Optimal Convertible Codes (2405.09010v1)
Abstract: Most large-scale storage systems employ erasure coding to provide resilience against disk failures. Recent work has shown that tuning this redundancy to changes in disk failure rates leads to substantial storage savings. This process requires code conversion, wherein data encoded using an $[n{I\mskip-2mu},k{I\mskip-2mu}]$ initial code has to be transformed into data encoded using an $[n{F\mskip-2mu},k{F\mskip-2mu}]$ final code, a resource-intensive operation. Convertible codes are a class of codes that enable efficient code conversion while maintaining other desirable properties. In this paper, we focus on the access cost of conversion (total number of code symbols accessed in the conversion process) and on an important subclass of conversions known as the merge regime (combining multiple initial codewords into a single final codeword). In this setting, explicit constructions are known for systematic access-optimal Maximum Distance Separable (MDS) convertible codes for all parameters in the merge regime. However, the existing construction for a key subset of these parameters, which makes use of Vandermonde parity matrices, requires a large field size making it unsuitable for practical applications. In this paper, we provide (1) sharper bounds on the minimum field size requirement for such codes, and (2) explicit constructions for low field sizes for several parameter ranges. In doing so, we provide a proof of super-regularity of specially designed classes of Vandermonde matrices that could be of independent interest.
- I. Gohberg and V. Olshevsky, “Fast Algorithms with Preprocessing for Matrix-Vector Multiplication Problems,” Journal of Complexity, vol. 10, no. 4, pp. 411–427, 1994.
- J. Lacan and J. Fimes, “Systematic MDS erasure codes based on Vandermonde matrices,” IEEE Communications Letters, vol. 8, no. 9, pp. 570–572, 2004.
- Y. Hu, L. Cheng, Q. Yao, P. P. C. Lee, W. Wang, and W. Chen, “Exploiting Combined Locality for Wide-Stripe Erasure Coding in Distributed Storage,” in 19th USENIX Conference on File and Storage Technologies (FAST 21). USENIX Association, Feb. 2021, pp. 233–248.
- S. Kadekodi, S. Silas, D. Clausen, and A. Merchant, “Practical Design Considerations for Wide Locally Recoverable Codes (LRCs),” Association for Computing Machinery Transactions on Storage, vol. 19, no. 4, nov 2023.
- S. Kadekodi, K. V. Rashmi, and G. R. Ganger, “Cluster storage systems gotta have HeART: improving storage efficiency by exploiting disk-reliability heterogeneity,” in 17th USENIX Conference on File and Storage Technologies, FAST 2019, Boston, MA, February 25-28, 2019, A. Merchant and H. Weatherspoon, Eds. USENIX Association, 2019, pp. 345–358.
- F. Maturana and K. V. Rashmi, “Convertible codes: enabling efficient conversion of coded data in distributed storage,” IEEE Transactions on Information Theory, vol. 68, pp. 4392–4407, 2022.
- F. Maturana, V. S. C. Mukka, and K. V. Rashmi, “Access-optimal linear MDS convertible codes for all parameters,” in IEEE International Symposium on Information Theory, ISIT 2020, Los Angeles, California, USA, June 21-26, 2020, 2020.
- F. Maturana and K. V. Rashmi, “Bandwidth Cost of Code Conversions in the Split Regime,” in 2022 IEEE International Symposium on Information Theory (ISIT), 2022, pp. 3262–3267.
- ——, “Bandwidth Cost of Code Conversions in Distributed Storage: Fundamental Limits and Optimal Constructions,” IEEE Transactions on Information Theory, vol. 69, no. 8, pp. 4993–5008, 2023.
- I. E. Shparlinski, “On the singularity of generalised Vandermonde matrices over finite fields,” Finite Fields and Their Applications, vol. 11, no. 2, pp. 193–199, 2005.
- R. Roth and G. Seroussi, “On generator matrices of MDS codes (Corresp.),” IEEE Transactions on Information Theory, vol. 31, no. 6, pp. 826–830, 1985.
- F. Maturana and K. V. Rashmi, “Locally Repairable Convertible Codes: Erasure Codes for Efficient Repair and Conversion,” in 2023 IEEE International Symposium on Information Theory (ISIT), 2023, pp. 2033–2038.
- X. Kong, “Locally repairable convertible codes with optimal access costs,” ArXiv, vol. abs/2308.06802, 2023.
- Intel, “Corrupted fragment on decode · issue #10 · Intel/ISA-L.” [Online]. Available: https://github.com/intel/isa-l/issues/10
- R. Roth and A. Lempel, “On MDS codes via Cauchy matrices,” IEEE Transactions on Information Theory, vol. 35, no. 6, pp. 1314–1319, 1989.
- J.-J. Climent, D. Napp, C. Perea, and R. Pinto, “A construction of MDS 2D convolutional codes of rate 1/n1𝑛1/n1 / italic_n based on superregular matrices,” Linear Algebra and its Applications, vol. 437, no. 3, pp. 766–780, 2012.
- P. J. F. Almeida and D. N. Avelli, “Superregular matrices over small finite fields,” ArXiv, vol. abs/2008.00215, 2020.
- R. Hutchinson, R. Smarandache, and J. Trumpf, “On superregular matrices and MDP convolutional codes,” Linear Algebra and its Applications, vol. 428, no. 11, pp. 2585–2596, 2008.
- P. Almeida, D. Napp, and R. Pinto, “A new class of superregular matrices and MDP convolutional codes,” Linear Algebra and its Applications, vol. 439, no. 7, pp. 2145–2157, 2013.
- J. T. Schwartz, “Fast Probabilistic Algorithms for Verification of Polynomial Identities,” Journal of the Association for Computing Machinery, vol. 27, no. 4, p. 701–717, oct 1980.
- R. Zippel, “Probabilistic algorithms for sparse polynomials,” in Symbolic and Algebraic Computation, E. W. Ng, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, 1979, pp. 216–226.