Constant Approximation for Network Revenue Management with Markovian-Correlated Customer Arrivals (2305.05829v3)
Abstract: The Network Revenue Management (NRM) problem is a well-known challenge in dynamic decision-making under uncertainty. In this problem, fixed resources must be allocated to serve customers over a finite horizon, while customers arrive according to a stochastic process. The typical NRM model assumes that customer arrivals are independent over time. However, in this paper, we explore a more general setting where customer arrivals over different periods can be correlated. We propose a model that assumes the existence of a system state, which determines customer arrivals for the current period. This system state evolves over time according to a time-inhomogeneous Markov chain. We show our model can be used to represent correlation in various settings. To solve the NRM problem under our correlated model, we derive a new linear programming (LP) approximation of the optimal policy. Our approximation provides an upper bound on the total expected value collected by the optimal policy. We use our LP to develop a new bid price policy, which computes bid prices for each system state and time period in a backward induction manner. The decision is then made by comparing the reward of the customer against the associated bid prices. Our policy guarantees to collect at least $1/(1+L)$ fraction of the total reward collected by the optimal policy, where $L$ denotes the maximum number of resources required by a customer. In summary, our work presents a Markovian model for correlated customer arrivals in the NRM problem and provides a new LP approximation for solving the problem under this model. We derive a new bid price policy and provides a theoretical guarantee of the performance of the policy.
- D. Adelman. Dynamic bid prices in revenue management. Operations Research, 55(4):647–661, 2007.
- E. Altman. Constrained Markov decision processes, volume 7. CRC press, 1999.
- A. Aouad and W. Ma. A nonparametric framework for online stochastic matching with correlated arrivals. arXiv preprint arXiv:2208.02229, 2022.
- Y. Aviv and A. Pazgal. A partially observed markov decision process for dynamic pricing. Management science, 51(9):1400–1416, 2005.
- J. Baek and W. Ma. Bifurcating constraints to improve approximation ratios for network revenue management with reusable resources. Operations Research, 2022.
- Fluid approximations for revenue management under high-variance demand. Management science (forthcoming), 2022.
- Survey of dynamic resource constrained reward collection problems: Unified model and analysis. Operations Research (forthcoming), 2021.
- Neuro-dynamic programming: an overview. In Proceedings of 1995 34th IEEE conference on decision and control, volume 1, pages 560–564. IEEE, 1995.
- The multisecretary problem with many types. arXiv preprint arXiv:2205.09078, 2022.
- R. Bray. Logarithmic regret in multisecretary and online linear programming problems with continuous valuations. arXiv preprint arXiv:1912.08917, 2022.
- Information relaxations, duality, and convex stochastic dynamic programs. Operations Research, 62(6):1394–1415, 2014.
- P. Bumpensanti and H. Wang. A re-solving heuristic with uniformly bounded loss for network revenue management. Management Science, 66(7):2993–3009, 2020.
- F. Chen and J.-S. Song. Optimal policies for multiechelon inventory problems with markov-modulated demand. Operations Research, 49(2):226–234, 2001.
- Dynamic pricing in an evolving and unknown marketplace. Available at SSRN 3382957, 2019.
- D. P. De Farias and B. Van Roy. The linear programming approach to approximate dynamic programming. Operations research, 51(6):850–865, 2003.
- Dynamic pricing policies for an inventory model with random windows of opportunities. Naval Research Logistics (NRL), 65(8):660–675, 2018.
- Prophet inequalities made easy: Stochastic optimization by pricing nonstochastic inputs. SIAM Journal on Computing, 49(3):540–582, 2020.
- Exploration-exploitation in constrained mdps. arXiv preprint arXiv:2003.02189, 2020.
- V. F. Farias and B. Van Roy. An approximate dynamic programming approach to network revenue management. Preprint, 2007.
- Y. Feng and G. Gallego. Perishable asset revenue management with markovian time dependent demand intensities. Management science, 46(7):941–956, 2000.
- D. Freund and S. Banerjee. Good prophets know when the end is near. Available at SSRN 3479189, 2019.
- D. Freund and J. Zhao. Overbooking with bounded loss. In Mathematics of Operations Research, forthcoming, 2022.
- G. Gallego and G. Van Ryzin. Optimal dynamic pricing of inventories with stochastic demand over finite horizons. Management science, 40(8):999–1020, 1994.
- A best-of-both-worlds algorithm for constrained mdps with long-term constraints. arXiv preprint arXiv:2304.14326, 2023.
- Real-time optimization of personalized assortments. Management Science, 60(6):1532–1551, 2014.
- On the complexity of approximating k-set packing. computational complexity, 15(1):20–39, 2006.
- Revenue sharing in airline alliances. Management Science, 59(5):1177–1195, 2013.
- S. Jasin and S. Kumar. A re-solving heuristic with bounded revenue loss for network revenue management with customer choice. Mathematics of Operations Research, 37(2):313–345, 2012.
- S. Jasin and S. Kumar. Analysis of deterministic lp-based booking limit and bid price controls for revenue management. Operations Research, 61(6):1312–1320, 2013.
- Online resource allocation in markov chains. In Proceedings of the ACM Web Conference 2023, pages 3498–3507, 2023.
- Online stochastic optimization with wasserstein based non-stationarity. arXiv preprint arXiv:2012.06961, 2020.
- Degeneracy is ok: Logarithmic regret for network revenue management with indiscrete distributions. arXiv preprint arXiv:2210.07996, 2022.
- N. B. Keskin and M. Li. Selling quality-differentiated products in a markovian market with unknown transition probabilities. Available at SSRN 3526568, 2022.
- U. Krengel and L. Sucheston. On semiamarts, amarts, and processes with finite value. Probability on Banach spaces, 4:197–266, 1978.
- S. Kunnumkal and K. Talluri. On a piecewise-linear approximation for network revenue management. Mathematics of Operations Research, 41(1):72–91, 2016.
- Revenue management with calendar-aware and dependent demands: Asymptotically tight fluid approximations. Available at SSRN 4543277, 2023.
- An approximation algorithm for network revenue management under nonstationary arrivals. Operations Research, 68(3):834–855, 2020.
- J. Meissner and A. Strauss. Network revenue management with inventory-sensitive bid prices and customer choice. European Journal of Operational Research, 216(2):459–468, 2012.
- W. B. Powell. Approximate Dynamic Programming: Solving the curses of dimensionality, volume 703. John Wiley & Sons, 2007.
- Upper confidence primal-dual reinforcement learning for cmdp with adversarial loss. Advances in Neural Information Processing Systems, 33:15277–15287, 2020.
- M. I. Reiman and Q. Wang. An asymptotically optimal policy for a quantity-based network revenue management problem. Mathematics of Operations Research, 33(2):257–282, 2008.
- A. Rustichini and A. Wolinsky. Learning about variable demand in the long run. Journal of economic Dynamics and Control, 19(5-7):1283–1292, 1995.
- P. J. Schweitzer and A. Seidmann. Generalized polynomial approximations in markovian decision processes. Journal of mathematical analysis and applications, 110(2):568–582, 1985.
- S. P. Sethi and F. Cheng. Optimality of (s, s) policies in inventory models with markovian demand. Operations Research, 45(6):931–939, 1997.
- Handbook of quantitative supply chain analysis: modeling in the e-business era, volume 74. Springer Science & Business Media, 2004.
- Online matching with reusable network resources and decaying rewards: A unified framework. Available at SSRN 3981123, 2022.
- J.-S. Song and P. Zipkin. Inventory control in a fluctuating demand environment. Operations Research, 41(2):351–370, 1993.
- K. Talluri and G. Van Ryzin. An analysis of bid-price controls for network revenue management. Management science, 44(11-part-1):1577–1593, 1998.
- C. Tong and H. Topaloglu. On the approximate linear programming approach for network revenue management problems. INFORMS Journal on Computing, 26(1):121–134, 2014.
- H. Topaloglu. Using lagrangian relaxation to compute capacity-dependent bid prices in network revenue management. Operations Research, 57(3):637–649, 2009.
- V.-A. Truong and X. Wang. Prophet inequality with correlated arrival probabilities, with application to two sided matchings. arXiv preprint arXiv:1901.02552, 2019.
- A. Vera and S. Banerjee. The bayesian prophet: A low-regret framework for online decision making. Management Science, 67(3):1368–1391, 2021.
- Online allocation and pricing: Constant regret via bellman inequalities. Operations Research, 69(3):821–840, 2021.
- T. W. Vossen and D. Zhang. A dynamic disaggregation approach to approximate linear programs for network revenue management. Production and Operations Management, 24(3):469–487, 2015a.
- T. W. Vossen and D. Zhang. Reductions of approximate linear programs for network revenue management. Operations Research, 63(6):1352–1371, 2015b.
- Y. Wang and H. Wang. Constant regret resolving heuristics for price-based revenue management. Operations Research, 2022.
- Online learning in weakly coupled markov decision processes: A convergence time study. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 2(1):1–38, 2018.
- D. Zhang. An improved dynamic programming decomposition approach for network revenue management. Manufacturing & Service Operations Management, 13(1):35–52, 2011.
- D. Zhang and D. Adelman. An approximate dynamic programming approach to network revenue management with customer choice. Transportation Science, 43(3):381–394, 2009.
- Product-based approximate linear programs for network revenue management. Operations Research, 70(5):2837–2850, 2022.
- L. Zheng and L. Ratliff. Constrained upper confidence reinforcement learning. In Learning for Dynamics and Control, pages 620–629. PMLR, 2020.