Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On the Optimal Bounds for Noisy Computing (2306.11951v1)

Published 21 Jun 2023 in cs.DS, cs.AI, cs.IT, cs.LG, math.IT, math.ST, and stat.TH

Abstract: We revisit the problem of computing with noisy information considered in Feige et al. 1994, which includes computing the OR function from noisy queries, and computing the MAX, SEARCH and SORT functions from noisy pairwise comparisons. For $K$ given elements, the goal is to correctly recover the desired function with probability at least $1-\delta$ when the outcome of each query is flipped with probability $p$. We consider both the adaptive sampling setting where each query can be adaptively designed based on past outcomes, and the non-adaptive sampling setting where the query cannot depend on past outcomes. The prior work provides tight bounds on the worst-case query complexity in terms of the dependence on $K$. However, the upper and lower bounds do not match in terms of the dependence on $\delta$ and $p$. We improve the lower bounds for all the four functions under both adaptive and non-adaptive query models. Most of our lower bounds match the upper bounds up to constant factors when either $p$ or $\delta$ is bounded away from $0$, while the ratio between the best prior upper and lower bounds goes to infinity when $p\rightarrow 0$ or $p\rightarrow 1/2$. On the other hand, we also provide matching upper and lower bounds for the number of queries in expectation, improving both the upper and lower bounds for the variable-length query model.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Learning with limited rounds of adaptivity: Coin tossing, multi-armed bandits, and ranking from pairwise comparisons. In Proceedings of the 2017 Conference on Learning Theory, volume 65 of Proceedings of Machine Learning Research, pages 39–75. PMLR, 07–10 Jul 2017.
  2. N. Ailon. Active learning ranking from pairwise preferences with almost optimal query complexity. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.
  3. Aggregating inconsistent information: Ranking and clustering. J. ACM, 55(5), nov 2008. ISSN 0004-5411. doi: 10.1145/1411509.1411513.
  4. A new active learning scheme with applications to learning to rank from pairwise preferences. 2011.
  5. Sorting and selection with imprecise comparisons. In ACM Trans. Algorithms, volume 12, pages 37–48, 07 2009. doi: 10.1007/978-3-642-02927-1˙5.
  6. Best arm identification in multi-armed bandits. In COLT, pages 41–53, 2010.
  7. Gambling in a rigged casino: The adversarial multi-armed bandit problem. In Proceedings of IEEE 36th annual foundations of computer science, pages 322–331. IEEE, 1995.
  8. E. R. Berlekamp. Block coding with noiseless feedback. Ph.D. thesis, MIT, Cambridge, MA, USA, 1964.
  9. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324–345, 1952. ISSN 00063444.
  10. M. Braverman and E. Mossel. Sorting from noisy information. 2009. URL https://arxiv.org/abs/0910.1191.
  11. J. Bretagnolle and C. Huber. Estimation des densités: risque minimax. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 47(2):119–137, 1979. doi: 10.1007/BF00535278.
  12. Pure exploration in multi-armed bandits problems. In International conference on Algorithmic learning theory, pages 23–37. Springer, 2009.
  13. M. V. Burnashev. Data transmission over a discrete channel with feedback. random transmission time. Problemy Peredachi Informatsii, 12(4):10–30, 1976.
  14. M. V. Burnashev and K. Zigangirov. An interval estimation problem for controlled observations. Problemy Peredachi Informatsii, 10(3):51–61, 1974.
  15. Asymptotically optimal sequential design for rank aggregation. Math. Oper. Res., 10 2017. doi: 10.1287/moor.2021.1209.
  16. Y. Chen and C. Suh. Spectral MLE: Top-K rank aggregation from pairwise comparisons. In F. Bach and D. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 371–380, Lille, France, 07–09 Jul 2015. PMLR.
  17. R. L. Dobrushin and S. Ortyukov. Lower bound for the redundancy of self-correcting arrangements of unreliable functional elements. Problemy Peredachi Informatsii, 13(1):82–89, 1977a.
  18. R. L. Dobrushin and S. Ortyukov. Upper bound on the redundancy of self-correcting arrangements of unreliable functional elements. Problemy Peredachi Informatsii, 13(3):56–76, 1977b.
  19. W. Evans and N. Pippenger. Average-case lower bounds for noisy boolean decision trees. SIAM Journal on Computing, 28(2):433–446, 1998.
  20. Maximum selection and ranking under noisy comparisons. In D. Precup and Y. W. Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1088–1096. PMLR, 06–11 Aug 2017.
  21. Computing with noisy information. SIAM Journal on Computing, 23(5):1001–1018, 1994.
  22. Best arm identification: A unified approach to fixed budget and fixed confidence. Advances in Neural Information Processing Systems, 25, 2012.
  23. P. Gács and A. Gál. Lower bounds for the complexity of reliable boolean circuits with noisy gates. IEEE Transactions on Information Theory, 40(2):579–583, 1994.
  24. A. Garivier and E. Kaufmann. Optimal best arm identification with fixed confidence. In Conference on Learning Theory, 2016.
  25. Y. Gu and Y. Xu. Optimal bounds for noisy sorting, 2023.
  26. Active ranking from pairwise comparisons and when parametric assumptions do not help. Ann. Stat., 47(6):3099 – 3126, 2019.
  27. M. Horstein. Sequential transmission using noiseless feedback. IEEE Trans. Inf. Theory, 9(3):136–143, 1963. doi: 10.1109/TIT.1963.1057832.
  28. Nonparametric goodness-of-fit testing under Gaussian models, volume 169. Springer Science & Business Media, 2003.
  29. K. Jamieson and R. Nowak. Best-arm identification algorithms for multi-armed bandits in the fixed confidence setting. In 48th Annual Conference on Information Sciences and Systems, pages 1–6, 2014.
  30. Active ranking using pairwise comparisons. In Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS’11, page 2240–2248, Red Hook, NY, USA, 2011. Curran Associates Inc. ISBN 9781618395993.
  31. R. M. Karp and R. Kleinberg. Noisy binary search and its applications. In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’07, page 881–890, USA, 2007. Society for Industrial and Applied Mathematics. ISBN 9780898716245.
  32. On the complexity of best arm identification in multi-armed bandit models. Journal of Machine Learning Research, 17:1–42, 2016.
  33. L. LeCam. Convergence of estimates under dimensionality restrictions. The Annals of Statistics, pages 38–53, 1973.
  34. Minimax rates and efficient algorithms for noisy sorting. In Proceedings of Algorithmic Learning Theory, volume 83 of Proceedings of Machine Learning Research, pages 821–847. PMLR, 07–09 Apr 2018.
  35. M. Mitzenmacher and E. Upfal. Probability and computing: Randomization and probabilistic techniques in algorithms and data analysis. Cambridge university press, 2017.
  36. Active learning for top-k𝑘kitalic_k rank aggregation from noisy comparisons. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 2488–2497. JMLR.org, 2017.
  37. Iterative ranking from pair-wise comparisons. In Advances in Neural Information Processing Systems, volume 25, 2012.
  38. A. Pelc. Searching with known error probability. Theoretical Computer Science, 63(2):185–202, 1989.
  39. On a lower bound for the redundancy of reliable networks with noisy gates. IEEE Transactions on Information Theory, 37(3):639–643, 1991.
  40. A. Rajkumar and S. Agarwal. A statistical convergence perspective of algorithms for rank aggregation from pairwise data. In Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32, page I–118–I–126, 2014.
  41. R. Reischuk and B. Schmeltz. Reliable computation with noisy circuits and decision trees-a general n log n lower bound. In [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science, pages 602–611. IEEE Computer Society, 1991.
  42. Pac ranking from pairwise and listwise queries: Lower bounds and upper bounds. 2018.
  43. A. Rényi. On a problem of information theory. MTA Mat. Kut. Int. Kozl. B, 6(MR143666):505–516, 1961.
  44. Simple, robust and optimal ranking from pairwise comparisons. J. Mach. Learn. Res., 18(199):1–38, 2018.
  45. Stochastically transitive models for pairwise comparisons: Statistical and computational issues. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, page 11–20. JMLR.org, 2016.
  46. A. B. Tsybakov. Introduction to nonparametric estimation. Springer, 9(10), 2004.
  47. S. Ulam. Adventures of a mathematician. Charles Scribner’s Sons, New York, NY, USA, 1976.
  48. J. Von Neumann. Probabilistic logics and the synthesis of reliable organisms from unreliable components. Automata studies, 34:43–98, 1956.
  49. Noisy sorting capacity. In 2022 IEEE International Symposium on Information Theory (ISIT), pages 2541–2546. IEEE, 2022.
  50. Noisy sorting capacity, 2023. URL https://arxiv.org/abs/2202.01446.
  51. Efficient ranking from pairwise comparisons. In Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 109–117, Atlanta, Georgia, USA, 17–19 Jun 2013.
  52. B. Yu. Assouad, fano, and le cam. In Festschrift for Lucien Le Cam, pages 423–435. Springer, 1997.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now