Secretary Problems with Random Number of Candidates: How Prior Distributional Information Helps (2310.07884v1)
Abstract: We study variants of the secretary problem, where $N$, the number of candidates, is a random variable, and the decision maker wants to maximize the probability of success -- picking the largest number among the $N$ candidates -- using only the relative ranks of the candidates revealed so far. We consider three forms of prior information about $\mathbf p$, the probability distribution of $N$. In the full information setting, we assume $\mathbf p$ to be fully known. In that case, we show that single-threshold type of strategies can achieve $1/e$-approximation to the maximum probability of success among all possible strategies. In the upper bound setting, we assume that $N\leq \overline{n}$ (or $\mathbb E[N]\leq \bar{\mu}$), where $\bar{n}$ (or $\bar{\mu}$) is known. In that case, we show that randomization over single-threshold type of strategies can achieve the optimal worst case probability of success of $\frac{1}{\log(\bar{n})}$ (or $\frac{1}{\log(\bar{\mu})}$) asymptotically. Surprisingly, there is a single-threshold strategy (depending on $\overline{n}$) that can succeed with probability $2/e2$ for all but an exponentially small fraction of distributions supported on $[\bar{n}]$. In the sampling setting, we assume that we have access to $m$ samples $N{(1)},\ldots,N{(m)}\sim_{iid} \mathbf p$. In that case, we show that if $N\leq T$ with probability at least $1-O(\epsilon)$ for some $T\in \mathbb N$, $m\gtrsim \frac{1}{\epsilon2}\max(\log(\frac{1}{\epsilon}),\epsilon \log(\frac{\log(T)}{\epsilon}))$ is enough to learn a strategy that is at least $\epsilon$-suboptimal, and we provide a lower bound of $\Omega(\frac{1}{\epsilon2})$, showing that the sampling algorithm is optimal when $\epsilon=O(\frac{1}{\log\log(T)})$.
- A.R. Abdel-Hamid, J.A. Bather and G.B. Trustrum “The Secretary Problem with an Unknown Number of Candidates” In Journal of Applied Probability 19.3 Applied Probability Trust, 1982, pp. 619–630 URL: http://www.jstor.org/stable/3213519
- Shipra Agrawal, Zizhuo Wang and Yinyu Ye “A Dynamic Near-Optimal Algorithm for Online Linear Programming” In Oper. Res. 62.4 Linthicum, MD, USA: INFORMS, 2014, pp. 876–890
- “Nearly Optimal Pricing Algorithms for Production Constrained and Laminar Bayesian Selection” In Proceedings of the 2019 ACM Conference on Economics and Computation, EC ’19 Phoenix, AZ, USA: Association for Computing Machinery, 2019, pp. 91–92 DOI: 10.1145/3328526.3329652
- Pablo D. Azar, Robert Kleinberg and S.Matthew Weinberg “Prophet Inequalities with Limited Information” In Proceedings of the 2014 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 1358–1377 DOI: 10.1137/1.9781611973402.100
- Moshe Babaioff, Nicole Immorlica and Robert Kleinberg “Matroids, Secretary Problems, and Online Mechanisms” In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’07 New Orleans, Louisiana: Society for IndustrialApplied Mathematics, 2007, pp. 434–443
- “Online Auctions and Generalized Secretary Problems” In SIGecom Exch. 7.2 New York, NY, USA: Association for Computing Machinery, 2008 DOI: 10.1145/1399589.1399596
- Santiago Balseiro, Christian Kroer and Rachitesh Kumar “Online Resource Allocation under Horizon Uncertainty” In SIGMETRICS Perform. Eval. Rev. 51.1 New York, NY, USA: Association for Computing Machinery, 2023, pp. 63–64 DOI: 10.1145/3606376.3593559
- Mark Braverman, Mahsa Derakhshan and Antonio Molina Lovett “Max-Weight Online Stochastic Matching: Improved Approximations Against the Online Benchmark” In Proceedings of the 23rd ACM Conference on Economics and Computation, EC ’22 Boulder, CO, USA: Association for Computing Machinery, 2022, pp. 967–985 DOI: 10.1145/3490486.3538315
- Niv Buchbinder, Kamal Jain and Mohit Singh “Secretary Problems via Linear Programming” In Mathematics of Operations Research 39.1, 2014, pp. 190–206 DOI: 10.1287/moor.2013.0604
- Niv Buchbinder and Joseph (Seffi) Naor “The Design of Competitive Online Algorithms via a Primal–Dual Approach” In Foundations and Trends® in Theoretical Computer Science 3.2–3, 2009, pp. 93–263 DOI: 10.1561/0400000024
- “Improved Competitive Ratio for the Matroid Secretary Problem” In Proceedings of the Twenty-Third Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’12 Kyoto, Japan: Society for IndustrialApplied Mathematics, 2012, pp. 1702–1712
- T-H.Hubert Chan, Fei Chen and Shaofeng H.-C. Jiang “Revealing Optimal Thresholds for Generalized Secretary Problem via Continuous LP: Impacts on Online K-Item Auction and Bipartite K-Matching with Random Arrival Order” In Proceedings of the Twenty-Sixth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’15 San Diego, California: Society for IndustrialApplied Mathematics, 2015, pp. 1169–1188
- Yuan Shih Chow, Herbert Robbins and David Siegmund “The theory of optimal stopping” In The theory of optimal stopping New York: Dover, 1971 - 1991
- “Recent Developments in Prophet Inequalities” In SIGecom Exch. 17.1 New York, NY, USA: Association for Computing Machinery, 2019, pp. 61–70 DOI: 10.1145/3331033.3331039
- “Posted Price Mechanisms for a Random Stream of Customers” In Proceedings of the 2017 ACM Conference on Economics and Computation, EC ’17 Cambridge, Massachusetts, USA: Association for Computing Machinery, 2017, pp. 169–186 DOI: 10.1145/3033274.3085137
- “Prophet Inequalities for I.I.D. Random Variables from an Unknown Distribution” In Proceedings of the 2019 ACM Conference on Economics and Computation, EC ’19 Phoenix, AZ, USA: Association for Computing Machinery, 2019, pp. 3–17 DOI: 10.1145/3328526.3329627
- “Sample-Driven Optimal Stopping: From the Secretary Problem to the i.i.d. Prophet Inequality” In Mathematics of Operations Research 0.0, 0, pp. null DOI: 10.1287/moor.2023.1363
- “The Secretary Problem with Independent Sampling” In Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms, SODA 2021, Virtual Conference, January 10 - 13, 2021 Virtual, United States: Society for Industrial and Applied Mathematics, 2021, pp. 2047–2058 DOI: 10.1137/1.9781611976465.122
- “Unknown I.I.D. Prophets: Better Bounds, Streaming Algorithms, and a New Impossibility” In 12th Innovations in Theoretical Computer Science Conference (ITCS 2021) 185, Leibniz International Proceedings in Informatics (LIPIcs) Dagstuhl, Germany: Schloss Dagstuhl–Leibniz-Zentrum für Informatik, 2021, pp. 86:1–86:1 DOI: 10.4230/LIPIcs.ITCS.2021.86
- “Near Optimal Online Algorithms and Fast Approximation Algorithms for Resource Allocation Problems” In J. ACM 66.1 New York, NY, USA: Association for Computing Machinery, 2019 DOI: 10.1145/3284177
- “Dynamic Pricing in the Presence of Inventory Considerations: Research Overview, Current Practices, and Future Directions” In Management Science 49.10, 2003, pp. 1287–1309 DOI: 10.1287/mnsc.49.10.1287.17315
- Moran Feldman, Ola Svensson and Rico Zenklusen “A Simple O(log log(rank))-Competitive Algorithm for the Matroid Secretary Problem” In Mathematics of Operations Research 43.2, 2018, pp. 638–650 DOI: 10.1287/moor.2017.0876
- Thomas S. Ferguson “Who Solved the Secretary Problem?” In Statistical Science 4.3 Institute of Mathematical Statistics, 1989, pp. 282–289 DOI: 10.1214/ss/1177012493
- John P. Gilbert and Frederick Mosteller “Recognizing the Maximum of a Sequence” In Journal of the American Statistical Association 61.313 [American Statistical Association, Taylor & Francis, Ltd.], 1966, pp. 35–73 URL: http://www.jstor.org/stable/2283044
- Yanjun Han, Jiantao Jiao and Tsachy Weissman “Minimax estimation of discrete distributions” In 2015 IEEE International Symposium on Information Theory (ISIT), 2015, pp. 2291–2295 DOI: 10.1109/ISIT.2015.7282864
- “Minimax-Optimal Strategies for the Best-Choice Problem When a Bound is Known for the Expected Number of Objects” In SIAM Journal on Control and Optimization 32.4, 1994, pp. 937–951 DOI: 10.1137/S0363012992234724
- Theodore P. Hill and Robert P. Kertz “Stop Rule Inequalities for Uniformly Bounded Sequences of Random Variables” In Transactions of the American Mathematical Society 278.1 American Mathematical Society, 1983, pp. 197–207 URL: http://www.jstor.org/stable/1999311
- Theodore P. Hill and Ulrich Krengel “Minimax-Optimal Stop Rules and Distributions in Secretary Problems” In The Annals of Probability 19.1 Institute of Mathematical Statistics, 1991, pp. 342–353 DOI: 10.1214/aop/1176990548
- Robert P Kertz “Stop rule and supremum expectations of i.i.d. random variables: A complete comparison by conjugate duality” In Journal of Multivariate Analysis 19.1, 1986, pp. 88–112 DOI: https://doi.org/10.1016/0047-259X(86)90095-3
- “Primal Beats Dual on Online Packing LPs in the Random-Order Model” In Proceedings of the Forty-Sixth Annual ACM Symposium on Theory of Computing, STOC ’14 New York, New York: Association for Computing Machinery, 2014, pp. 303–312 DOI: 10.1145/2591796.2591810
- Robert Kleinberg “A Multiple-Choice Secretary Algorithm with Applications to Online Auctions” In Proceedings of the Sixteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’05 Vancouver, British Columbia: Society for IndustrialApplied Mathematics, 2005, pp. 630–631
- Oded Lachish “O(log log Rank) Competitive Ratio for the Matroid Secretary Problem” In 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, 2014, pp. 326–335 DOI: 10.1109/FOCS.2014.42
- Jeffrey I. McGill and Garrett J. Ryzin “Revenue Management: Research Overview and Prospects” In Transportation Science 33.2, 1999, pp. 233–256 DOI: 10.1287/trsc.33.2.233
- “AdWords and Generalized Online Matching” In J. ACM 54.5 New York, NY, USA: Association for Computing Machinery, 2007, pp. 22–es DOI: 10.1145/1284320.1284321
- “Geometry of Online Packing Linear Programs” In Automata, Languages, and Programming Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 701–713
- Shayan Oveis Gharan and Jan Vondrák “On Variants of the Matroid Secretary Problem” In Algorithms – ESA 2011 Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 335–346
- “Online Stochastic Max-Weight Bipartite Matching: Beyond Prophet Inequalities” In Proceedings of the 22nd ACM Conference on Economics and Computation, EC ’21 Budapest, Hungary: Association for Computing Machinery, 2021, pp. 763–764 DOI: 10.1145/3465456.3467613
- “The Best Choice Problem for a Random Number of Objects” In Theory of Probability & Its Applications 17.4, 1973, pp. 657–668 DOI: 10.1137/1117078
- Aviad Rubinstein, Jack Z. Wang and S.Matthew Weinberg “Optimal Single-Choice Prophet Inequalities from Samples” In 11th Innovations in Theoretical Computer Science Conference (ITCS 2020) 151, Leibniz International Proceedings in Informatics (LIPIcs) Dagstuhl, Germany: Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, 2020, pp. 60:1–60:10 DOI: 10.4230/LIPIcs.ITCS.2020.60
- Igal Sason “On Reverse Pinsker Inequalities”, 2015 arXiv:1503.07118 [cs.IT]
- Martin J. Wainwright “High-Dimensional Statistics: A Non-Asymptotic Viewpoint”, Cambridge Series in Statistical and Probabilistic Mathematics Cambridge University Press, 2019 DOI: 10.1017/9781108627771