Finding the Second-Best Candidate under the Mallows Model (2011.07666v3)

Abstract: The well-known secretary problem in sequential analysis and optimal stopping theory asks one to maximize the probability of finding the optimal candidate in a sequentially examined list under the constraint that accept/reject decisions are made in real-time. A version of the problem is the so-called postdoc problem, for which the question of interest is to devise a strategy that identifies the second-best candidate with highest possible probability of success. We study the postdoc problem in its combinatorial form. In this setting, a permutation $\pi$ of length $N$ is sampled according to some distribution on the symmetric group $S_N$ and the elements of $\pi$ are revealed one-by-one from left to right so that at each step, one can only observe the relative orders of the elements. At each step, one must decide to either accept or reject the currently presented element and cannot recall the decision in the future. The question of interest is to find the optimal strategy for selecting the position of the second-largest value. We solve the postdoc problem for the untraditional setting where the candidates are not presented uniformly at random but rather according to permutations drawn from the Mallows distribution. The Mallows distribution assigns to each permutation $\pi \in S_N$ a weight $\theta^{c(\pi)}$, where the function c counts the number of inversions in $\pi$. To identify the optimal stopping criteria for the significantly more challenging postdoc problem, we adopt a combinatorial methodology that includes new proof techniques and novel methodological extensions compared to the analysis first introduced in the setting of the secretary problem. The optimal strategies depend on the parameter $\theta$ of the Mallows distribution and can be determined exactly by solving well-defined recurrence relations.