Papers
Topics
Authors
Recent
Search
2000 character limit reached

Statistical Inference with Limited Memory: A Survey

Published 23 Dec 2023 in cs.LG, cs.IT, math.IT, and stat.ML | (2312.15225v3)

Abstract: The problem of statistical inference in its various forms has been the subject of decades-long extensive research. Most of the effort has been focused on characterizing the behavior as a function of the number of available samples, with far less attention given to the effect of memory limitations on performance. Recently, this latter topic has drawn much interest in the engineering and computer science literature. In this survey paper, we attempt to review the state-of-the-art of statistical inference under memory constraints in several canonical problems, including hypothesis testing, parameter estimation, and distribution property testing/estimation. We discuss the main results in this developing field, and by identifying recurrent themes, we extract some fundamental building blocks for algorithmic construction, as well as useful techniques for lower bound derivations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (139)
  1. H. Robbins, “A sequential decision problem with a finite memory,” Proceedings of the National Academy of Sciences of the United States of America, vol. 42, no. 12, p. 920, 1956.
  2. A. B. Tsybakov, “On nonparametric estimation of density level sets,” The Annals of Statistics, vol. 25, no. 3, pp. 948–969, 1997.
  3. L. Devroye and G. Lugosi, Combinatorial methods in density estimation. Springer Science & Business Media, 2001.
  4. E. L. Lehmann and G. Casella, Theory of point estimation. Springer Science & Business Media, 2006.
  5. O. Goldreich, “Property testing,” Lecture Notes in Comput. Sci, vol. 6390, 2010.
  6. T. T. Cai and M. G. Low, “Testing composite hypotheses, hermite polynomials and optimal estimation of a nonsmooth functional,” 2011.
  7. J. Jiao, K. Venkat, Y. Han, and T. Weissman, “Minimax estimation of functionals of discrete distributions,” IEEE Transactions on Information Theory, vol. 61, no. 5, pp. 2835–2885, 2015.
  8. Y. Wu and P. Yang, “Minimax rates of entropy estimation on large alphabets via best polynomial approximation,” IEEE Transactions on Information Theory, vol. 62, no. 6, pp. 3702–3720, 2016.
  9. T. M. Cover and J. A. Thomas, Elements of information theory. John Wiley & Sons, 2012.
  10. J. Neyman and E. S. Pearson, “On the problem of the most efficient tests of statistical hypotheses,” Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, vol. 231, no. 694-706, pp. 289–337, 1933.
  11. E. Tuncel, “On error exponents in hypothesis testing,” IEEE Transactions on Information Theory, vol. 51, no. 8, pp. 2945–2950, 2005.
  12. L. G. Valiant, “A theory of the learnable,” Communications of the ACM, vol. 27, no. 11, pp. 1134–1142, 1984.
  13. N. Alon, Y. Matias, and M. Szegedy, “The space complexity of approximating the frequency moments,” Journal of Computer and system sciences, vol. 58, no. 1, pp. 137–147, 1999.
  14. P. Indyk and D. Woodruff, “Optimal approximations of the frequency moments of data streams,” in Proceedings of the thirty-seventh annual ACM symposium on Theory of computing, pp. 202–208, 2005.
  15. A. Lall, V. Sekar, M. Ogihara, J. Xu, and H. Zhang, “Data streaming algorithms for estimating entropy of network traffic,” ACM SIGMETRICS Performance Evaluation Review, vol. 34, no. 1, pp. 145–156, 2006.
  16. A. Chakrabarti, K. Do Ba, and S. Muthukrishnan, “Estimating entropy and entropy norm on data streams,” Internet Mathematics, vol. 3, no. 1, pp. 63–78, 2006.
  17. A. Chakrabarti, G. Cormode, and A. McGregor, “A near-optimal algorithm for estimating the entropy of a stream,” ACM Transactions on Algorithms (TALG), vol. 6, no. 3, pp. 1–21, 2010.
  18. M. Charikar, K. Chen, and M. Farach-Colton, “Finding frequent items in data streams,” in International Colloquium on Automata, Languages, and Programming, pp. 693–703, Springer, 2002.
  19. G. Cormode and S. Muthukrishnan, “An improved data stream summary: the count-min sketch and its applications,” Journal of Algorithms, vol. 55, no. 1, pp. 58–75, 2005.
  20. A. Metwally, D. Agrawal, and A. El Abbadi, “Efficient computation of frequent and top-k elements in data streams,” in International conference on database theory, pp. 398–412, Springer, 2005.
  21. P. Flajolet, “Approximate counting: a detailed analysis,” BIT Numerical Mathematics, vol. 25, no. 1, pp. 113–134, 1985.
  22. M. Durand and P. Flajolet, “Loglog counting of large cardinalities,” in Algorithms-ESA 2003: 11th Annual European Symposium, Budapest, Hungary, September 16-19, 2003. Proceedings 11, pp. 605–617, Springer, 2003.
  23. P. Indyk and D. Woodruff, “Tight lower bounds for the distinct elements problem,” in 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings., pp. 283–288, IEEE, 2003.
  24. P. Flajolet, É. Fusy, O. Gandouet, and F. Meunier, “Hyperloglog: the analysis of a near-optimal cardinality estimation algorithm,” Discrete mathematics & theoretical computer science, no. Proceedings, 2007.
  25. D. M. Kane, J. Nelson, and D. P. Woodruff, “An optimal algorithm for the distinct elements problem,” in Proceedings of the twenty-ninth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pp. 41–52, 2010.
  26. S. Guha, A. McGregor, and S. Venkatasubramanian, “Sublinear estimation of entropy and information distances,” ACM Transactions on Algorithms (TALG), vol. 5, no. 4, pp. 1–16, 2009.
  27. A. Chakrabarti, T. Jayram, and M. Pǎtraşcu, “Tight lower bounds for selection in randomly ordered streams,” in Proceedings of the nineteenth annual ACM-SIAM Symposium on Discrete Algorithms, pp. 720–729, 2008.
  28. J. Acharya, S. Bhadane, P. Indyk, and Z. Sun, “Estimating entropy of distributions in constant space,” arXiv preprint arXiv:1911.07976, 2019.
  29. M. Aliakbarpour, A. McGregor, J. Nelson, and E. Waingarten, “Estimation of entropy in constant space with improved sample complexity,” arXiv preprint arXiv:2205.09804, 2022.
  30. D. P. Woodruff, “Optimal space lower bounds for all frequency moments,” in SODA, vol. 4, pp. 167–175, Citeseer, 2004.
  31. E. Meron and M. Feder, “Finite-memory universal prediction of individual sequences,” IEEE Transactions on Information Theory, vol. 50, no. 7, pp. 1506–1523, 2004.
  32. D. P. Woodruff, “The average-case complexity of counting distinct elements,” in Proceedings of the 12th International Conference on Database Theory, pp. 284–295, 2009.
  33. Z. Zhang and T. Berger, “Estimation via compressed information,” IEEE transactions on Information theory, vol. 34, no. 2, pp. 198–211, 1988.
  34. S.-i. Amari, “Fisher information under restriction of shannon information in multi-terminal situations,” Annals of the Institute of Statistical Mathematics, vol. 41, pp. 623–648, 1989.
  35. R. Ahlswede and M. V. Burnashev, “On minimax estimation in the presence of side information about remote data,” The Annals of Statistics, pp. 141–171, 1990.
  36. S. Amari et al., “Parameter estimation with multiterminal data compression,” IEEE transactions on Information Theory, vol. 41, no. 6, pp. 1802–1833, 1995.
  37. S. Amari et al., “Statistical inference under multiterminal data compression,” IEEE Transactions on Information Theory, vol. 44, no. 6, pp. 2300–2324, 1998.
  38. S.-i. Amari, “On optimal data compression in multiterminal statistical inference,” IEEE Transactions on Information Theory, vol. 57, no. 9, pp. 5577–5587, 2011.
  39. R. Ahlswede and I. Csiszár, “Hypothesis testing with communication constraints,” IEEE transactions on information theory, vol. 32, no. 4, pp. 533–542, 1986.
  40. S.-I. Amari and T. S. Han, “Statistical inference under multiterminal rate restrictions: A differential geometric approach,” IEEE Transactions on Information Theory, vol. 35, no. 2, pp. 217–227, 1989.
  41. H. M. Shalaby and A. Papamarcou, “Multiterminal detection with zero-rate data compression,” IEEE Transactions on Information Theory, vol. 38, no. 2, pp. 254–267, 1992.
  42. M. Wigger and R. Timo, “Testing against independence with multiple decision centers,” in 2016 International Conference on Signal Processing and Communications (SPCOM), pp. 1–5, IEEE, 2016.
  43. S. Watanabe, “Neyman–pearson test for zero-rate multiterminal hypothesis testing,” IEEE Transactions on Information Theory, vol. 64, no. 7, pp. 4923–4939, 2017.
  44. N. Weinberger and Y. Kochman, “On the reliability function of distributed hypothesis testing under optimal detection,” IEEE Transactions on Information Theory, vol. 65, no. 8, pp. 4940–4965, 2019.
  45. S. Sreekumar and D. Gündüz, “Distributed hypothesis testing over discrete memoryless channels,” IEEE Transactions on Information Theory, vol. 66, no. 4, pp. 2044–2066, 2019.
  46. S. Salehkalaibar and M. Wigger, “Distributed hypothesis testing with variable-length coding,” IEEE Journal on Selected Areas in Information Theory, vol. 1, no. 3, pp. 681–694, 2020.
  47. Y. Xiang and Y.-H. Kim, “Interactive hypothesis testing against independence,” in 2013 IEEE International Symposium on Information Theory, pp. 2840–2844, IEEE, 2013.
  48. K. Sahasranand and H. Tyagi, “Extra samples can reduce the communication for independence testing,” in 2018 IEEE International Symposium on Information Theory (ISIT), pp. 2316–2320, IEEE, 2018.
  49. U. Hadar, J. Liu, Y. Polyanskiy, and O. Shayevitz, “Communication complexity of estimating correlations,” in Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pp. 792–803, 2019.
  50. U. Hadar and O. Shayevitz, “Distributed estimation of gaussian correlations,” IEEE Transactions on Information Theory, vol. 65, no. 9, pp. 5323–5338, 2019.
  51. A. C.-C. Yao, “Some complexity questions related to distributive computing (preliminary report),” in Proceedings of the eleventh annual ACM symposium on Theory of computing, pp. 209–213, 1979.
  52. E. Kushilevitz, “Communication complexity,” in Advances in Computers, vol. 44, pp. 331–360, Elsevier, 1997.
  53. A. Rao and A. Yehudayoff, Communication Complexity: and Applications. Cambridge University Press, 2020.
  54. Y. Zhang, J. Duchi, M. I. Jordan, and M. J. Wainwright, “Information-theoretic lower bounds for distributed statistical estimation with communication constraints,” Advances in Neural Information Processing Systems, vol. 26, 2013.
  55. A. Garg, T. Ma, and H. Nguyen, “On communication cost of distributed statistical estimation and dimensionality,” Advances in Neural Information Processing Systems, vol. 27, 2014.
  56. M. Braverman, A. Garg, T. Ma, H. L. Nguyen, and D. P. Woodruff, “Communication lower bounds for statistical estimation problems via a distributed data processing inequality,” in Proceedings of the forty-eighth annual ACM symposium on Theory of Computing, pp. 1011–1020, 2016.
  57. A. Xu and M. Raginsky, “Information-theoretic lower bounds on bayes risk in decentralized estimation,” IEEE Transactions on Information Theory, vol. 63, no. 3, pp. 1580–1600, 2016.
  58. M. I. Jordan, J. D. Lee, and Y. Yang, “Communication-efficient distributed statistical inference,” Journal of the American Statistical Association, 2018.
  59. Y. Han, A. Özgür, and T. Weissman, “Geometric lower bounds for distributed parameter estimation under communication constraints,” in Conference On Learning Theory, pp. 3163–3188, PMLR, 2018.
  60. O. Fischer, U. Meir, and R. Oshman, “Distributed uniformity testing,” in Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing, pp. 455–464, 2018.
  61. J. Acharya, C. L. Canonne, and H. Tyagi, “Inference under information constraints: Lower bounds from chi-square contraction,” in Conference on Learning Theory, pp. 3–17, PMLR, 2019.
  62. J. Acharya, C. L. Canonne, and H. Tyagi, “Inference under information constraints ii: Communication constraints and shared randomness,” IEEE Transactions on Information Theory, vol. 66, no. 12, pp. 7856–7877, 2020.
  63. J. Acharya, C. L. Canonne, C. Freitag, Z. Sun, and H. Tyagi, “Inference under information constraints iii: Local privacy constraints,” IEEE Journal on Selected Areas in Information Theory, vol. 2, no. 1, pp. 253–267, 2021.
  64. A. Pensia, V. Jog, and P.-L. Loh, “Communication-constrained hypothesis testing: Optimality, robustness, and reverse data processing inequalities,” arXiv preprint arXiv:2206.02765, 2022.
  65. T. Berger, Z. Zhang, and H. Viswanathan, “The ceo problem [multiterminal source coding],” IEEE Transactions on Information Theory, vol. 42, no. 3, pp. 887–902, 1996.
  66. H. Viswanathan and T. Berger, “The quadratic gaussian ceo problem,” IEEE Transactions on Information Theory, vol. 43, no. 5, pp. 1549–1559, 1997.
  67. K. Eswaran and M. Gastpar, “Remote source coding under gaussian noise: Dueling roles of power and entropy power,” IEEE Transactions on Information Theory, vol. 65, no. 7, pp. 4486–4498, 2019.
  68. Y. Polyanskiy and Y. Wu, “Information theory: From coding to learning,” Book draft, 2022.
  69. T. M. Cover, “A note on the two-armed bandit problem with finite memory,” Information and Control, vol. 12, no. 5, pp. 371–377, 1968.
  70. T. M. Cover et al., “Hypothesis testing with finite statistics,” The Annals of Mathematical Statistics, vol. 40, no. 3, pp. 828–835, 1969.
  71. J. Koplowitz, “Necessary and sufficient memory size for m-hypothesis testing,” IEEE Transactions on Information Theory, vol. 21, no. 1, pp. 44–46, 1975.
  72. T. M. Cover, M. A. Freedman, and M. E. Hellman, “Optimal finite memory learning algorithms for the finite sample problem,” Information and Control, vol. 30, no. 1, pp. 49–85, 1976.
  73. D. P. Woodruff, Efficient and private distance approximation in the communication and streaming models. PhD thesis, Massachusetts Institute of Technology, 2007.
  74. D. M. Kane, J. Nelson, and D. P. Woodruff, “On the exact space complexity of sketching and streaming small norms,” in Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms, pp. 1161–1178, SIAM, 2010.
  75. T. S. Jayram and D. P. Woodruff, “Optimal bounds for johnson-lindenstrauss transforms and streaming problems with subconstant error,” ACM Transactions on Algorithms (TALG), vol. 9, no. 3, pp. 1–17, 2013.
  76. M. Crouch, A. McGregor, G. Valiant, and D. P. Woodruff, “Stochastic streams: Sample complexity vs. space complexity,” in 24th Annual European Symposium on Algorithms (ESA 2016), Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2016.
  77. Y. Dagan and O. Shamir, “Detecting correlations with little memory and communication,” in Conference On Learning Theory, pp. 1145–1198, PMLR, 2018.
  78. I. Dinur, “On the streaming indistinguishability of a random permutation and a random function,” in Advances in Cryptology–EUROCRYPT 2020: 39th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Zagreb, Croatia, May 10–14, 2020, Proceedings, Part II 30, pp. 433–460, Springer, 2020.
  79. I. Diakonikolas, T. Gouleakis, D. M. Kane, and S. Rao, “Communication and memory efficient testing of discrete distributions,” in Conference on Learning Theory, pp. 1070–1106, PMLR, 2019.
  80. I. Shahaf, O. Ordentlich, and G. Segev, “An information-theoretic proof of the streaming switching lemma for symmetric encryption,” in 2020 IEEE International Symposium on Information Theory (ISIT), pp. 858–863, IEEE, 2020.
  81. J. Jaeger and S. Tessaro, “Tight time-memory trade-offs for symmetric encryption,” in Advances in Cryptology–EUROCRYPT 2019: 38th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Darmstadt, Germany, May 19–23, 2019, Proceedings, Part I 38, pp. 467–497, Springer, 2019.
  82. O. Shamir, “Fundamental limits of online and distributed algorithms for statistical learning and estimation,” Advances in Neural Information Processing Systems, vol. 27, 2014.
  83. J. Steinhardt, G. Valiant, and S. Wager, “Memory, communication, and statistical queries,” in Conference on Learning Theory, pp. 1490–1516, PMLR, 2016.
  84. M. E. Hellman and T. M. Cover, “Learning with finite memory,” The Annals of Mathematical Statistics, pp. 765–782, 1970.
  85. T. Berg, O. Ordentlich, and O. Shayevitz, “Binary hypothesis testing with deterministic finite-memory decision rules,” in 2020 IEEE International Symposium on Information Theory (ISIT), pp. 1259–1264, IEEE, 2020.
  86. R. Raz, “Fast learning requires good memory: A time-space lower bound for parity learning,” 2016.
  87. T. Berg, O. Ordentlich, and O. Shayevitz, “Deterministic finite-memory bias estimation,” in Conference on Learning Theory, pp. 566–585, PMLR, 2021.
  88. T. Berg, O. Ordentlich, and O. Shayevitz, “On the memory complexity of uniformity testing,” in Conference on Learning Theory, pp. 3506–3523, PMLR, 2022.
  89. M. Horstein, “Sequential transmission using noiseless feedback,” IEEE Transactions on Information Theory, vol. 9, no. 3, pp. 136–143, 1963.
  90. R. Nowak, “Generalized binary search,” in 2008 46th Annual Allerton Conference on Communication, Control, and Computing, pp. 568–574, IEEE, 2008.
  91. O. Shayevitz and M. Feder, “Optimal feedback communication via posterior matching,” IEEE Transactions on Information Theory, vol. 57, no. 3, pp. 1186–1222, 2011.
  92. T. Wagner, “Estimation of the mean with time-varying finite memory (corresp.),” IEEE Transactions on Information Theory, vol. 18, no. 4, pp. 523–525, 1972.
  93. J. Isbell, “On a problem of robbins,” The Annals of Mathematical Statistics, vol. 30, no. 2, pp. 606–610, 1959.
  94. C. V. Smith and R. Pyke, “The robbins-isbell two-armed-bandit problem with finite memory,” The Annals of Mathematical Statistics, pp. 1375–1386, 1965.
  95. S. M. Samuels, “Randomized rules for the two-armed-bandit with finite memory,” The Annals of Mathematical Statistics, vol. 39, no. 6, pp. 2103–2107, 1968.
  96. M. E. Hellman and T. M. Cover, “On memory saved by randomization,” The Annals of Mathematical Statistics, vol. 42, no. 3, pp. 1075–1078, 1971.
  97. M. Hellman, “The effects of randomization on finite-memory decision schemes,” IEEE Transactions on Information Theory, vol. 18, no. 4, pp. 499–502, 1972.
  98. M. Hellman and T. Cover, “A review of recent results on learning with finite memory,” in 2nd International Symposium on Information Theory, pp. 289–294, 1973.
  99. B. Shubert and C. Anderson, “Testing a simple symmetric hypothesis by a finite-memory deterministic algorithm,” IEEE Transactions on Information Theory, vol. 19, no. 5, pp. 644–647, 1973.
  100. G. Kol, R. Raz, and A. Tal, “Time-space hardness of learning sparse parities,” in Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, pp. 1067–1080, 2017.
  101. R. Raz, “A time-space lower bound for a large class of learning problems,” in 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS), pp. 732–742, IEEE, 2017.
  102. S. Garg, P. K. Kothari, P. Liu, and R. Raz, “Memory-sample lower bounds for learning parity with noise,” arXiv preprint arXiv:2107.02320, 2021.
  103. T. Cover and M. Hellman, “The two-armed-bandit problem with time-invariant finite memory,” IEEE Transactions on Information Theory, vol. 16, no. 2, pp. 185–195, 1970.
  104. J. Horos and M. Hellman, “A confidence model for finite-memory learning systems (corresp.),” IEEE Transactions on Information Theory, vol. 18, no. 6, pp. 811–813, 1972.
  105. R. Flower and M. Hellman, “Hypothesis testing with finite memory in finite time (corresp.),” IEEE Transactions on Information Theory, vol. 18, no. 3, pp. 429–431, 1972.
  106. R. Roberts and J. Tooley, “Estimation with finite memory,” IEEE Transactions on Information Theory, vol. 16, no. 6, pp. 685–691, 1970.
  107. J. Koplowitz and R. Roberts, “Sequential estimation with a finite statistic,” IEEE Transactions on Information Theory, vol. 19, no. 5, pp. 631–635, 1973.
  108. M. E. Hellman, “Finite-memory algorithms for estimating the mean of a gaussian distribution (corresp.),” IEEE Transactions on Information Theory, vol. 20, no. 3, pp. 382–384, 1974.
  109. F. J. Samaniego, “Estimating a binomial parameter with finite memory,” IEEE Transactions on Information Theory, vol. 19, no. 5, pp. 636–643, 1973.
  110. F. Leighton and R. Rivest, “Estimating a probability using finite memory,” IEEE Transactions on Information Theory, vol. 32, no. 6, pp. 733–742, 1986.
  111. V. Anantharam and P. Tsoucas, “A proof of the markov chain tree theorem,” Statistics & Probability Letters, vol. 8, no. 2, pp. 189–192, 1989.
  112. J. Von Neumann, “13. various techniques used in connection with random digits,” Appl. Math Ser, vol. 12, no. 36-38, p. 5, 1951.
  113. J. Steinhardt and J. Duchi, “Minimax rates for memory-bounded sparse linear regression,” in Conference on Learning Theory, pp. 1564–1587, PMLR, 2015.
  114. B. Yu, “Assouad, fano, and le cam,” in Festschrift for Lucien Le Cam: research papers in probability and statistics, pp. 423–435, Springer, 1997.
  115. V. Sharan, A. Sidford, and G. Valiant, “Memory-sample tradeoffs for linear regression with small error,” in Proceedings of the 51st Annual ACM SIGACT Symposium on Theory of Computing, pp. 890–901, 2019.
  116. A. Jain and H. Tyagi, “Effective memory shrinkage in estimation,” in 2018 IEEE International Symposium on Information Theory (ISIT), pp. 1071–1075, IEEE, 2018.
  117. L. A. Kontorovich, “Statistical estimation with bounded memory,” Statistics and Computing, vol. 22, no. 5, pp. 1155–1164, 2012.
  118. N. Merhav and M. Feder, “Universal schemes for sequential decision from individual data sequences,” IEEE Transactions on Information Theory, vol. 39, no. 4, pp. 1280–1292, 1993.
  119. N. Merhav, M. Feder, and M. Gutman, “Some properties of sequential predictors for binary markov sources,” IEEE Transactions on Information Theory, vol. 39, no. 3, pp. 887–892, 1993.
  120. D. Rajwan and M. Feder, “Universal finite memory machines for coding binary sequences,” in Proceedings DCC 2000. Data Compression Conference, pp. 113–122, IEEE, 2000.
  121. E. Meron and M. Feder, “Optimal finite state universal coding of individual sequences,” in Data Compression Conference, 2004. Proceedings. DCC 2004, pp. 332–341, IEEE, 2004.
  122. O. Goldreich, “The uniform distribution is complete with respect to testing identity to a fixed distribution.,” in Electronic Colloquium on Computational Complexity (ECCC), vol. 23, p. 1, 2016.
  123. O. Goldreich and D. Ron, “On testing expansion in bounded-degree graphs,” in Electronic Colloquium on Computational Complexity (ECCC), vol. 20, 2000.
  124. L. Paninski, “A coincidence-based test for uniformity given very sparsely sampled discrete data,” IEEE Transactions on Information Theory, vol. 54, no. 10, pp. 4750–4755, 2008.
  125. I. Diakonikolas, D. M. Kane, and V. Nikishkin, “Testing identity of structured distributions,” in Proceedings of the twenty-sixth annual ACM-SIAM symposium on Discrete algorithms, pp. 1841–1854, SIAM, 2014.
  126. C. L. Canonne and J. Q. Yang, “Simpler distribution testing with little memory,” arXiv preprint arXiv:2311.01145, 2023.
  127. I. Diakonikolas, T. Gouleakis, J. Peebles, and E. Price, “Sample-optimal identity testing with high probability,” in 45th International Colloquium on Automata, Languages, and Programming (ICALP 2018), Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2018.
  128. S. Roy and Y. Vasudev, “Testing properties of distributions in the streaming model,” arXiv preprint arXiv:2309.03245, 2023.
  129. S. Chakraborty, E. Fischer, Y. Goldhirsh, and A. Matsliah, “On the power of conditional samples in distribution testing,” in Proceedings of the 4th conference on Innovations in Theoretical Computer Science, pp. 561–580, 2013.
  130. C. L. Canonne, D. Ron, and R. A. Servedio, “Testing probability distributions using conditional samples,” SIAM Journal on Computing, vol. 44, no. 3, pp. 540–616, 2015.
  131. G. P. Basharin, “On a statistical estimate for the entropy of a sequence of independent random variables,” Theory of Probability & Its Applications, vol. 4, no. 3, pp. 333–336, 1959.
  132. L. Paninski, “Estimating entropy on m bins given fewer than m samples,” IEEE Transactions on Information Theory, vol. 50, no. 9, pp. 2200–2203, 2004.
  133. G. Valiant and P. Valiant, “A clt and tight lower bounds for estimating entropy.,” in Electron. Colloquium Comput. Complex., vol. 17, p. 179, 2010.
  134. G. Valiant and P. Valiant, “Estimating the unseen: an n/log (n)-sample estimator for entropy and support size, shown optimal via new clts,” in Proceedings of the forty-third annual ACM symposium on Theory of computing, pp. 685–694, 2011.
  135. G. Valiant and P. Valiant, “The power of linear estimators,” in 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science, pp. 403–412, IEEE, 2011.
  136. S. Chien, K. Ligett, and A. McGregor, “Space-efficient estimation of robust statistics and distribution testing.,” in ICS, pp. 251–265, Citeseer, 2010.
  137. T. Berg, O. Ordentlich, and O. Shayevitz, “Entropy estimation with less memory,” in Draft - to be submitted, 2023.
  138. R. Morris, “Counting large numbers of events in small registers,” Communications of the ACM, vol. 21, no. 10, pp. 840–842, 1978.
  139. J. Nelson and H. Yu, “Optimal bounds for approximate counting,” arXiv preprint arXiv:2010.02116, 2020.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 19 likes about this paper.

Sign Up for Free