Maximizing the Success Probability of Policy Allocations in Online Systems (2312.16267v1)
Abstract: The effectiveness of advertising in e-commerce largely depends on the ability of merchants to bid on and win impressions for their targeted users. The bidding procedure is highly complex due to various factors such as market competition, user behavior, and the diverse objectives of advertisers. In this paper we consider the problem at the level of user timelines instead of individual bid requests, manipulating full policies (i.e. pre-defined bidding strategies) and not bid values. In order to optimally allocate policies to users, typical multiple treatments allocation methods solve knapsack-like problems which aim at maximizing an expected value under constraints. In the industrial contexts such as online advertising, we argue that optimizing for the probability of success is a more suited objective than expected value maximization, and we introduce the SuccessProbaMax algorithm that aims at finding the policy allocation which is the most likely to outperform a fixed reference policy. Finally, we conduct comprehensive experiments both on synthetic and real-world data to evaluate its performance. The results demonstrate that our proposed algorithm outperforms conventional expected-value maximization algorithms in terms of success rate.
- LBCF: A Large-Scale Budget-Constrained Causal Forest Algorithm. In ACM Web Conference.
- E-Commerce Promotions Personalization via Online Multiple-Choice Knapsack with Uplift Modeling. In ACM International Conference on Information & Knowledge Management.
- Recursive partitioning for heterogeneous causal effects. National Academy of Sciences, 113(27): 7353–7360.
- Differentially Private Individual Treatment Effect Estimation from Aggregated Data. PPML Workshop.
- Robust label attribution for real-time bidding. arXiv preprint arXiv:2012.01767.
- Causal Models for Real Time Bidding with Repeated User Interactions. In ACM SIGKDD Conference on Knowledge Discovery & Data Mining.
- JAX: composable transformations of Python+NumPy programs.
- A Unifying Framework for Online Optimization with Long-Term Constraints. arXiv preprint arXiv:2209.07454.
- Multiplicative Pacing Equilibria in Auction Markets. Operations Research, 70(2): 963–989.
- Causally Motivated Attribution for Online Advertising. In International Workshop on Data Mining for Online Advertising and Internet Economy (AdKDD).
- An investigation into prediction+ optimisation for the knapsack problem. Integration of Constraint Programming, Artificial Intelligence, and Operations Research.
- CVXPY: A Python-embedded modeling language for convex optimization. Journal of Machine Learning Research, 17(83): 1–5.
- A large scale benchmark for individual treatment effect prediction and uplift modeling. arXiv preprint arXiv:2111.10106.
- Improve User Retention with Causal Learning. In ACM SIGKDD Workshop on Causal Discovery.
- Efficient projections onto the l1-ball for learning in high dimensions. In International Conference on Machine Learning.
- Efron, B. 1979. Bootstrap Methods: Another Look at the Jackknife. The Annals of Statistics, 7(1): 1 – 26.
- A comparison of methods for treatment assignment with an application to playlist generation. Information Systems Research.
- Additional Multi-Touch Attribution for Online Advertising. AAAI Conference on Artificial Intelligence.
- Kleber, M. 2019. Turtledove.
- Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643.
- A Real-World Implementation of Unbiased Lift-based Bidding System. In IEEE International Conference on Big Data.
- Rubin, D. B. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5): 688.
- Trust region policy optimization. In International conference on machine learning, 1889–1897. PMLR.
- Lectures on stochastic programming: modeling and theory. Society for Industrial and Applied Mathematics.
- The Multiple-Choice Knapsack Problem. Operations Research, 27(3): 503–515.
- Reinforcement learning: An introduction. MIT press.
- A simple approximation for bivariate normal integral based on error function and its application on probit model with binary endogenous regressor. Technical report, Institute of Economics, Academia Sinica, Taipei, Taiwan.
- Personalized treatment selection using causal heterogeneity. In ACM Web Conference.
- Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523): 1228–1242.
- Williams, R. J. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Reinforcement Learning, 5–32.
- An End-to-End Framework for Marketing Effectiveness Optimization under Budget Constraint. arXiv preprint arXiv:2302.04477.
- A unified framework for marketing budget allocation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 1820–1830.
- Direct Heterogeneous Causal Learning for Resource Allocation Problems in Marketing. AAAI Conference on Artificial Intelligence.