On Data-Driven Prescriptive Analytics with Side Information: A Regularized Nadaraya-Watson Approach

Abstract: We consider generic stochastic optimization problems in the presence of side information which enables a more insightful decision. The side information constitutes observable exogenous covariates that alter the conditional probability distribution of the random problem parameters. A decision maker who adapts her decisions according to the observed side information solves an optimization problem where the objective function is specified by the conditional expectation of the random cost. If the joint probability distribution is unknown, then the conditional expectation can be approximated in a data-driven manner using the Nadaraya-Watson (NW) kernel regression. While the emerging approximation scheme has found successful applications in diverse decision problems under uncertainty, it is largely unknown whether the scheme can provide any reasonable out-of-sample performance guarantees. In this paper, we establish guarantees for the generic problems by leveraging techniques from moderate deviations theory. Our analysis motivates the use of a variance-based regularization scheme which, in general, leads to a non-convex optimization problem. We adopt ideas from distributionally robust optimization to obtain tractable formulations. We present numerical experiments for newsvendor and wind energy commitment problems to highlight the effectiveness of our regularization scheme.