New Techniques for Constructing Rare-Case Hard Functions

Abstract: We say that a function is rare-case hard against a given class of algorithms (the adversary) if all algorithms in the class can compute the function only on an $o(1)$-fraction of instances of size $n$ for large enough $n$. Starting from any NP-complete language, for each $\alpha > 0$, we construct a function that cannot be computed correctly even on a $1/n^\alpha$-fraction of instances for polynomial-sized circuit families if NP $\not \subset$ P/POLY and by polynomial-time algorithms if NP $\not \subset$ BPP - functions that are rare-case hard against polynomial-sized circuits and polynomial-time randomized algorithms. The constructed function is a number-theoretic polynomial evaluated over specific finite fields. For NP-complete languages that admit parsimonious reductions from all of NP (for example, SAT), the constructed functions are hard to compute even on a $1/n^\alpha$-fraction of instances by polynomial-time randomized algorithms and polynomial-sized circuit families simply if P# $\not \subset$ BPP and P# $\not \subset$ P/POLY, respectively. We also show that if the Randomized Exponential Time Hypothesis (RETH) is true, none of these constructed functions can be computed even on a $1/n^\alpha$-fraction of instances in subexponential time. These functions are very hard, almost always. While one may not be able to efficiently compute the values of these constructed functions themselves, in polynomial time, one can verify that the evaluation of a function, $s = f(x)$, is correct simply by asking a prover to compute $f(y)$ on targeted queries. We have extended our work to give an alternative proof of a variant of Lipton's theorem (Lipton, 1989). We also compare our techniques for constructing rare-case hard functions with two other existing methods in the literature (Sudan et al., 2001; Feige and Lund, 1996).