2000 character limit reached
Linear Submodular Maximization with Bandit Feedback (2407.02601v1)
Published 2 Jul 2024 in cs.LG and cs.DS
Abstract: Submodular optimization with bandit feedback has recently been studied in a variety of contexts. In a number of real-world applications such as diversified recommender systems and data summarization, the submodular function exhibits additional linear structure. We consider developing approximation algorithms for the maximization of a submodular objective function $f:2U\to\mathbb{R}_{\geq 0}$, where $f=\sum_{i=1}dw_iF_{i}$. It is assumed that we have value oracle access to the functions $F_i$, but the coefficients $w_i$ are unknown, and $f$ can only be accessed via noisy queries. We develop algorithms for this setting inspired by adaptive allocation algorithms in the best-arm identification for linear bandit, with approximation guarantees arbitrarily close to the setting where we have value oracle access to $f$. Finally, we empirically demonstrate that our algorithms make vast improvements in terms of sample efficiency compared to algorithms that do not exploit the linear structure of $f$ on instances of move recommendation.
- Improved algorithms for linear stochastic bandits. Advances in neural information processing systems, 24, 2011.
- Fast algorithms for maximizing submodular functions. In Proceedings of the twenty-fifth annual ACM-SIAM symposium on Discrete algorithms, pages 1497–1514. SIAM, 2014.
- Almost optimal streaming algorithms for coverage problems. In Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures, pages 13–23, 2017.
- Interactive submodular bandit. Advances in Neural Information Processing Systems, 30, 2017.
- Combinatorial pure exploration of multi-armed bandits. Advances in neural information processing systems, 27, 2014.
- Submodular cost submodular cover with an approximate oracle. In International Conference on Machine Learning, pages 1426–1435. PMLR, 2019.
- Gamification of pure exploration for linear bandits. In International Conference on Machine Learning, pages 2432–2442. PMLR, 2020.
- Beyond keyword search: discovering relevant scientific literature. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 439–447. ACM, 2011.
- Pac bounds for multi-armed bandit and markov decision processes. In Computational Learning Theory: 15th Annual Conference on Computational Learning Theory, COLT 2002 Sydney, Australia, July 8–10, 2002 Proceedings 15, pages 255–270. Springer, 2002.
- The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):1–19, 2015.
- Robust guarantees of stochastic greedy algorithms. In International Conference on Machine Learning, pages 1424–1432. PMLR, 2017a.
- Submodular optimization under noise. In Conference on Learning Theory, pages 1069–1122. PMLR, 2017b.
- Cascading linear submodular bandits: Accounting for position bias and diversity in online learning to rank. In Uncertainty in Artificial Intelligence, pages 722–732. PMLR, 2020.
- Maximization of approximately submodular functions. Advances in neural information processing systems, 29, 2016.
- Pac subset selection in stochastic multi-armed bandits. In ICML, volume 12, pages 655–662, 2012.
- Stochastic submodular maximization: The case of coverage functions. Advances in Neural Information Processing Systems, 30, 2017.
- Maximizing the spread of influence through a social network. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 137–146, 2003.
- Cost-effective outbreak detection in networks. In Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 420–429. ACM, 2007.
- A class of submodular functions for document summarization. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 510–520. Association for Computational Linguistics, 2011.
- Leveraging sparsity for efficient submodular data summarization. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems, pages 3414–3422, 2016.
- An analysis of approximations for maximizing submodular set functions—i. Mathematical programming, 14:265–294, 1978.
- Saga: A submodular greedy algorithm for group recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
- Binghui Peng. Dynamic influence maximization. Advances in Neural Information Processing Systems, 34:10718–10731, 2021.
- Noisy submodular maximization via adaptive sampling with applications to crowdsourced image collection summarization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016.
- Best-arm identification in linear bandits. Advances in Neural Information Processing Systems, 27, 2014.
- Submodular bandit problem under multiple constraints. In Conference on Uncertainty in Artificial Intelligence, pages 191–200. PMLR, 2020.
- Learning mixtures of submodular functions for image collection summarization. Advances in neural information processing systems, 27, 2014.
- A fully adaptive algorithm for pure exploration in linear bandits. In International Conference on Artificial Intelligence and Statistics, pages 843–851. PMLR, 2018.
- Linear submodular bandits with a knapsack constraint. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016.
- Linear submodular bandits and their application to diversified retrieval. Advances in Neural Information Processing Systems, 24, 2011.