Tacit algorithmic collusion in deep reinforcement learning guided price competition: A study using EV charge pricing game (2401.15108v2)
Abstract: Players in pricing games with complex structures are increasingly adopting AI aided learning algorithms to make pricing decisions for maximizing profits. This is raising concern for the antitrust agencies as the practice of using AI may promote tacit algorithmic collusion among otherwise independent players. Recent studies of games in canonical forms have shown contrasting claims ranging from none to a high level of tacit collusion among AI-guided players. In this paper, we examine the concern for tacit collusion by considering a practical game where EV charging hubs compete by dynamically varying their prices. Such a game is likely to be commonplace in the near future as EV adoption grows in all sectors of transportation. The hubs source power from the day-ahead (DA) and real-time (RT) electricity markets as well as from in-house battery storage systems. Their goal is to maximize profits via pricing and efficiently managing the cost of power usage. To aid our examination, we develop a two-step data-driven methodology. The first step obtains the DA commitment by solving a stochastic model. The second step generates the pricing strategies by solving a competitive Markov decision process model using a multi-agent deep reinforcement learning (MADRL) framework. We evaluate the resulting pricing strategies using an index for the level of tacit algorithmic collusion. An index value of zero indicates no collusion (perfect competition) and one indicates full collusion (monopolistic behavior). Results from our numerical case study yield collusion index values between 0.14 and 0.45, suggesting a low to moderate level of collusion.
- IEA. Global EV Outlook 2023. Technical report, IEA, 2023. License: CC BY 4.0.
- Infrastructure planning for ride-hailing services using shared autonomous electric vehicles. International Journal of Sustainable Transportation, pages 1–16, 2023a.
- Tesla. Supercharger. https://www.tesla.com/supercharger, 2023. Accessed: 2023-07-30.
- A distributionally robust approach for day-ahead power procurement by ev charging hubs. In 2023 IEEE Power & Energy Society General Meeting (PESGM), pages 1–5. IEEE, 2023.
- Optimized operational cost reduction for an ev charging station integrated with battery energy storage and pv generation. IEEE Transactions on Smart Grid, 10(2):2096–2106, 2018.
- A deep reinforcement learning approach for power management of battery-assisted fast-charging ev hubs participating in day-ahead and real-time electricity markets. Energy, 283:129097, 2023b.
- Deep reinforcement learning for dynamic pricing of perishable products. In Optimization and Learning: 4th International Conference, OLA 2021, Catania, Italy, June 21-23, 2021, Proceedings 4, pages 132–143. Springer, 2021.
- Dynamic pricing under competition using reinforcement learning. Journal of Revenue and Pricing Management, pages 1–14, 2021.
- Electric vehicle optimum charging-discharging scheduling with dynamic pricing employing multi agent deep neural network. Computers and Electrical Engineering, 105:108555, 2023.
- Dynamic pricing for electric vehicle extreme fast charging. IEEE Transactions on Intelligent Transportation Systems, 22(1):531–541, 2020.
- A dynamic pricing system for electric vehicle charging management using reinforcement learning. IEEE Intelligent Transportation Systems Magazine, 14(6):122–134, 2022.
- Dynamic pricing for differentiated pev charging services using deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems, 23(2):1415–1427, 2020.
- Dynamic pricing strategy of electric vehicle aggregators based on ddpg reinforcement learning algorithm. IEEE access, 9:21556–21566, 2021.
- Reinforcement mechanism design for electric vehicle demand response in microgrid charging stations. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2020.
- Artificial intelligence, algorithmic pricing, and collusion. American Economic Review, 110(10):3267–3297, 2020.
- Filip Mellgren. Tacit collusion with deep multi-agent reinforcement learning, 2020.
- Artificial collusion: Examining supracompetitive pricing by q-learning algorithms. Amsterdam Law School Research Paper, (2022-25), 2022.
- Weipeng Zhang. Pricing via artificial intelligence: The impact of neural network architecture on algorithmic collusion. Available at SSRN 4489688, 2023.
- Fast charging stations with stationary batteries: A techno-economic comparison of fast charging along highways and in cities. Transportation Research Procedia, 48:3832–3849, 2020.
- Optimal sizing of battery energy storage system in a fast ev charging station considering power outages. IEEE Transactions on Transportation Electrification, 6(2):453–463, 2020.
- FDOT. Traffic information. https://www.fdot.gov/statistics/trafficdata, 2023. (accessed : 03 March 2023).
- McKinsey and Company. Why the automotive future is electric. https://www.mckinsey.com/industries/automotive-and-assembly/our-insights/why-the-automotive-future-is-electric, 2021.
- Demand modeling of a dc fast charging station. In 2018 North American Power Symposium (NAPS), pages 1–6. IEEE, 2018.
- Idaho National Laboratory. Plug-in electric vehicle and infrastructure analysis. https://inldigitallibrary.inl.gov/sites/sti/sti/6799570.pdf, 2015.
- PJM. Energy market. https://www.pjm.com/markets-and-operations/energy.aspx, 2022. (accessed : 2 July 2022).
- Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.
- Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905, 2018.
- Attention is all you need. Advances in neural information processing systems, 30, 2017.
- Divide and conquer? k-means clustering of demand data allows rapid and accurate simulations of the british electricity system. IEEE Transactions on Engineering Management, 61(2):251–260, 2014.