Papers
Topics
Authors
Recent
2000 character limit reached

Implicitly normalized forecaster with clipping for linear and non-linear heavy-tailed multi-armed bandits (2305.06743v3)

Published 11 May 2023 in cs.LG, math.OC, and stat.ML

Abstract: The Implicitly Normalized Forecaster (INF) algorithm is considered to be an optimal solution for adversarial multi-armed bandit (MAB) problems. However, most of the existing complexity results for INF rely on restrictive assumptions, such as bounded rewards. Recently, a related algorithm was proposed that works for both adversarial and stochastic heavy-tailed MAB settings. However, this algorithm fails to fully exploit the available data. In this paper, we propose a new version of INF called the Implicitly Normalized Forecaster with clipping (INF-clip) for MAB problems with heavy-tailed reward distributions. We establish convergence results under mild assumptions on the rewards distribution and demonstrate that INF-clip is optimal for linear heavy-tailed stochastic MAB problems and works well for non-linear ones. Furthermore, we show that INF-clip outperforms the best-of-both-worlds algorithm in cases where it is difficult to distinguish between different arms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Robbins, H.: Some aspects of the sequential design of experiments (1952) Berry and Fristedt [1985] Berry, D.A., Fristedt, B.: Bandit problems: sequential allocation of experiments (monographs on statistics and applied probability). London: Chapman and Hall 5(71-87), 7–7 (1985) Gittins et al. [2011] Gittins, J., Glazebrook, K., Weber, R.: Multi-armed Bandit Allocation Indices. John Wiley & Sons, Chichester (2011) Cesa-Bianchi and Lugosi [2006] Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge university press, New York (2006) Slivkins et al. [2019] Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Berry, D.A., Fristedt, B.: Bandit problems: sequential allocation of experiments (monographs on statistics and applied probability). London: Chapman and Hall 5(71-87), 7–7 (1985) Gittins et al. [2011] Gittins, J., Glazebrook, K., Weber, R.: Multi-armed Bandit Allocation Indices. John Wiley & Sons, Chichester (2011) Cesa-Bianchi and Lugosi [2006] Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge university press, New York (2006) Slivkins et al. [2019] Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gittins, J., Glazebrook, K., Weber, R.: Multi-armed Bandit Allocation Indices. John Wiley & Sons, Chichester (2011) Cesa-Bianchi and Lugosi [2006] Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge university press, New York (2006) Slivkins et al. [2019] Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge university press, New York (2006) Slivkins et al. [2019] Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  2. Berry, D.A., Fristedt, B.: Bandit problems: sequential allocation of experiments (monographs on statistics and applied probability). London: Chapman and Hall 5(71-87), 7–7 (1985) Gittins et al. [2011] Gittins, J., Glazebrook, K., Weber, R.: Multi-armed Bandit Allocation Indices. John Wiley & Sons, Chichester (2011) Cesa-Bianchi and Lugosi [2006] Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge university press, New York (2006) Slivkins et al. [2019] Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gittins, J., Glazebrook, K., Weber, R.: Multi-armed Bandit Allocation Indices. John Wiley & Sons, Chichester (2011) Cesa-Bianchi and Lugosi [2006] Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge university press, New York (2006) Slivkins et al. [2019] Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge university press, New York (2006) Slivkins et al. [2019] Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  3. Gittins, J., Glazebrook, K., Weber, R.: Multi-armed Bandit Allocation Indices. John Wiley & Sons, Chichester (2011) Cesa-Bianchi and Lugosi [2006] Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge university press, New York (2006) Slivkins et al. [2019] Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge university press, New York (2006) Slivkins et al. [2019] Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  4. Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge university press, New York (2006) Slivkins et al. [2019] Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  5. Slivkins, A., et al.: Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning 12(1-2), 1–286 (2019) Bubeck et al. [2012] Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  6. Bubeck, S., Cesa-Bianchi, N., et al.: Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends® in Machine Learning 5(1), 1–122 (2012) Sutton and Barto [2018] Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  7. Sutton, R.S., Barto, A.G.: Reinforcement Learning – an Introduction. Adaptive Computation and Machine Learning, 2nd edn. MIT press, Cambridge, MA (2018) Tsetlin [1969] Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  8. Tsetlin, M.: Issledovaniya po teorii avtomatov i modelirovaniyu biologicheskikh sistem (Studies in the Theory of Finite State Machines and Simulation of Biological Systems). Nauka Moscow (1969) Tsetlin [1973] Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  9. Tsetlin, M.: Automaton Theory and Modeling of Biological Systems. Academic Press, New York (1973) Varšavskij [1973] Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  10. Varšavskij, V.I.: Kollektivnoe Povedenie Avtomatov. Nauka, Moscow (1973) Nazin and Poznyak [1986] Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  11. Nazin, A., Poznyak, A.: Adaptive choice of variants. Nauka, Moscow (1986) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  12. Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM journal on computing 32(1), 48–77 (2002) Flaxman et al. [2004] Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  13. Flaxman, A.D., Kalai, A.T., McMahan, H.B.: Online convex optimization in the bandit setting: gradient descent without a gradient. arXiv preprint cs/0408007 (2004) Orabona [2019] Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  14. Orabona, F.: A modern introduction to online learning. arXiv preprint arXiv:1912.13213 (2019) Hazan et al. [2016] Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  15. Hazan, E., et al.: Introduction to online convex optimization. Foundations and Trends® in Optimization 2(3-4), 157–325 (2016) Auer et al. [2002] Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  16. Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 235–256 (2002) (ed.) [2003] (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  17. (ed.), R.S.T.: Handbook of Heavy Tailed Distributions in Finance: Handbooks in Finance, Book 1. Elsevier, North Holland (2003) Wang et al. [2019] Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  18. Wang, P., Xu, H., Jin, X., Wang, T.: Flash: efficient dynamic routing for offchain networks. In: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies, pp. 370–381 (2019) Choi et al. [2020] Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  19. Choi, D., Chun, S., Oh, H., Han, J., Kwon, T.: Rumor propagation is amplified by echo chambers in social media. Scientific reports 10(1), 310 (2020) Dhara et al. [2020] Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  20. Dhara, S., Hofstad, R., Leeuwaarden, J.S., Sen, S.: Heavy-tailed configuration models at criticality (2020) Barabási and Albert [1999] Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  21. Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. science 286(5439), 509–512 (1999) Bubeck et al. [2013] Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  22. Bubeck, S., Cesa-Bianchi, N., Lugosi, G.: Bandits with heavy tail. IEEE Transactions on Information Theory 59(11), 7711–7717 (2013) Medina and Yang [2016] Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  23. Medina, A.M., Yang, S.: No-regret algorithms for heavy-tailed linear bandits. In: International Conference on Machine Learning, pp. 1642–1650 (2016). PMLR Shao et al. [2018] Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  24. Shao, H., Yu, X., King, I., Lyu, M.R.: Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. Advances in Neural Information Processing Systems 31 (2018) Zhong et al. [2021] Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  25. Zhong, H., Huang, J., Yang, L., Wang, L.: Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. Advances in Neural Information Processing Systems 34, 15710–15720 (2021) Lu et al. [2019] Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  26. Lu, S., Wang, G., Hu, Y., Zhang, L.: Optimal algorithms for lipschitz bandits with heavy-tailed rewards. In: International Conference on Machine Learning, pp. 4154–4163 (2019). PMLR Lee et al. [2020] Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  27. Lee, K., Yang, H., Lim, S., Oh, S.: Optimal algorithms for stochastic multi-armed bandits with heavy tailed rewards. Advances in Neural Information Processing Systems 33, 8452–8462 (2020) Zimmert and Seldin [2019] Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  28. Zimmert, J., Seldin, Y.: An optimal algorithm for stochastic and adversarial bandits. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 467–475 (2019). PMLR [30] Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  29. Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization 19(4), 1574–1609 (2009) Nemirovskij and Yudin [1983] Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  30. Nemirovskij, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization (1983) Zhang et al. [2020] Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  31. Zhang, J., Karimireddy, S.P., Veit, A., Kim, S., Reddi, S.J., Kumar, S., Sra, S.: Why are adaptive methods good for attention models? Advances in Neural Information Processing Systems 33 (2020) Vural et al. [2022] Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  32. Vural, N.M., Yu, L., Balasubramanian, K., Volgushev, S., Erdogdu, M.A.: Mirror descent strikes again: Optimal stochastic convex optimization under infinite noise variance. In: Conference on Learning Theory, pp. 65–102 (2022). PMLR Cutkosky and Mehta [2021] Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  33. Cutkosky, A., Mehta, H.: High-probability bounds for non-convex stochastic optimization with heavy tails. Advances in Neural Information Processing Systems 34 (2021) Sadiev et al. [2023] Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  34. Sadiev, A., Danilova, M., Gorbunov, E., Horváth, S., Gidel, G., Dvurechensky, P., Gasnikov, A., Richtárik, P.: High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. arXiv preprint arXiv:2302.00999 (2023) Zhang and Cutkosky [2022] Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  35. Zhang, J., Cutkosky, A.: Parameter-free regret in high probability with heavy tails. arXiv preprint arXiv:2210.14355 (2022) Huang et al. [2022] Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  36. Huang, J., Dai, Y., Huang, L.: Adaptive best-of-both-worlds algorithm for heavy-tailed multi-armed bandits. In: International Conference on Machine Learning, pp. 9173–9200 (2022). PMLR Dann et al. [2023] Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  37. Dann, C., Wei, C.-Y., Zimmert, J.: A blackbox approach to best of both worlds in bandits and beyond. arXiv preprint arXiv:2302.09739 (2023) Littlestone and Warmuth [1994] Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  38. Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Information and computation 108(2), 212–261 (1994) Cesa-Bianchi et al. [1997] Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  39. Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E., Warmuth, M.K.: How to use expert advice. Journal of the ACM (JACM) 44(3), 427–485 (1997) Freund and Schapire [1997] Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  40. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of computer and system sciences 55(1), 119–139 (1997) Shamir [2017] Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  41. Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. The Journal of Machine Learning Research 18(1), 1703–1713 (2017) Kornilov et al. [2023] Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  42. Kornilov, N., Gasnikov, A., Dvurechensky, P., Dvinskikh, D.: Gradient free methods for non-smooth convex optimization with heavy tails on convex compact. arXiv preprint arXiv:2304.02442 (2023) Gasnikov et al. [2022] Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  43. Gasnikov, A., Novitskii, A., Novitskii, V., Abdukhakimov, F., Kamzolov, D., Beznosikov, A., Takáč, M., Dvurechensky, P., Gu, B.: The power of first-order smooth optimization for black-box non-smooth problems. arXiv preprint arXiv:2201.12289 (2022) Gorbunov et al. [2019] Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  44. Gorbunov, E., Vorontsova, E.A., Gasnikov, A.V.: On the upper bound for the expectation of the norm of a vector uniformly distributed on the sphere and the phenomenon of concentration of uniform measure on the sphere. Mathematical Notes 106 (2019) Ben-Tal and Nemirovski [2001] Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001) Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
  45. Ben-Tal, A., Nemirovski, A.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, Philadelphia (2001)
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Paper to Video (Beta)

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up