Dual dynamic programming for stochastic programs over an infinite horizon

Abstract: We consider solving stochastic programs over an infinite horizon. By leveraging the stationarity of problem, we develop a novel continually-exploring infinite-horizon explorative dual dynamic programming (CE-Inf-EDDP) algorithm that matches state-of-the-art complexity while providing encouraging numerical performance on the newsvendor and hydrothermal planning problem. CE-Inf-EDDP conceptually differs from previous dual dynamic programming approaches by exploring the feasible region longer and updating the cutting-plane model more frequently. In addition, our algorithm can handle both simple linear to more complex nonlinear costs. To demonstrate this, we extend our algorithm to handle the so-called hierarchical stationary stochastic program, where the cost function is a parametric multi-stage stochastic program. The hierarchical program can model problems with a hierarchy of decision-making, e.g., how long-term decisions influence day-to-day operations. As a concrete example, we introduce a newsvendor problem that includes a second-stage multi-product assembly serving as a secondary market.