Disentangling Exploration from Exploitation (2404.19116v1)
Abstract: Starting from Robbins (1952), the literature on experimentation via multi-armed bandits has wed exploration and exploitation. Nonetheless, in many applications, agents' exploration and exploitation need not be intertwined: a policymaker may assess new policies different than the status quo; an investor may evaluate projects outside her portfolio. We characterize the optimal experimentation policy when exploration and exploitation are disentangled in the case of Poisson bandits, allowing for general news structures. The optimal policy features complete learning asymptotically, exhibits lots of persistence, but cannot be identified by an index a la Gittins. Disentanglement is particularly valuable for intermediate parameter values.
- Best arm identification in multi-armed bandits. In COLT, pp. 41–53.
- Early-career discrimination: Spiraling or self-correcting? mimeo.
- Venture capital financing, moral hazard, and learning. Journal of Banking & Finance 22(6-8), 703–735.
- Bandit problems.
- Strategic experimentation. Econometrica 67(2), 349–374.
- Pure exploration in finitely-armed and continuous-armed bandits. Theoretical Computer Science 412(19), 1832–1852.
- Recommender systems as mechanisms for social learning. The Quarterly Journal of Economics 133(2), 871–925.
- Optimal dynamic allocation of attention. American Economic Review 109(8), 2993–3029.
- Uncertainty and learning in pharmaceutical demand. Econometrica 73(4), 1137–1173.
- Currie, J. M. and W. B. MacLeod (2020). Understanding doctor decision making: The case of depression treatment. Econometrica 88(3), 847–878.
- Learning while experimenting. The Economic Journal 130(625), 65–92.
- Dickstein, M. J. et al. (2021). Efficient provision of experience goods: Evidence from antidepressant choice.
- On optimal scheduling. American Economic Journal: Microeconomics.
- Multi-armed bandit allocation indices. John Wiley & Sons.
- Gittins, J. C. (1979). Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society Series B: Statistical Methodology 41(2), 148–164.
- Gittins, J. C. and D. M. Jones (1979). A dynamic allocation index for the discounted multiarmed bandit problem. Biometrika 66(3), 561–565.
- Guo, Y. (2016). Dynamic delegation of experimentation. American Economic Review 106(8), 1969–2008.
- Incentives for experimenting agents. The RAND Journal of Economics 44(4), 632–663.
- Jovanovic, B. (1979). Job matching and the theory of turnover. Journal of Political Economy 87(5, Part 1), 972–990.
- Strategic experimentation with poisson bandits. Theoretical Economics 5(2), 275–311.
- Strategic experimentation with exponential bandits. Econometrica 73(1), 39–68.
- Complementary information and learning traps. The Quarterly Journal of Economics 135(1), 389–448.
- Dynamically aggregating diverse information. Econometrica 90(1), 47–80.
- Rational inattention: A review. Journal of Economic Literature 61(1), 226–273.
- Miller, R. A. (1984). Job matching and occupational choice. Journal of Political Economy 92(6), 1086–1120.
- Robbins, H. (1952). Some aspects of the sequential design of experiments.
- Rothschild, M. (1974). A two-armed bandit theory of market pricing. Journal of Economic Theory 9(2), 185–202.
- Sims, C. A. (2003). Implications of rational inattention. Journal of monetary Economics 50(3), 665–690.
- Strulovici, B. (2010). Learning while voting: Determinants of collective experimentation. Econometrica 78(3), 933–971.
- Thompson, W. R. (1933). On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3-4), 285–294.
- Wald, A. (1947). Foundations of a general theory of sequential decision functions. Econometrica, 279–313.
- Zhuo, R. (2023). Exploit or explore? an empirical study of resource allocation in research labs. mimeo.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.