Gradient Methods for Scalable Multi-value Electricity Network Expansion Planning (2404.01255v1)
Abstract: We consider multi-value expansion planning (MEP), a general bilevel optimization model in which a planner optimizes arbitrary functions of the dispatch outcome in the presence of a partially controllable, competitive electricity market. The MEP problem can be used to jointly plan various grid assets, such as transmission, generation, and battery storage capacities; examples include identifying grid investments that minimize emissions in the absence of a carbon tax, maximizing the profit of a portfolio of renewable investments and long-term energy contracts, or reducing price inequities between different grid stakeholders. The MEP problem, however, is in general nonconvex, making it difficult to solve exactly for large real-world systems. Therefore, we propose a fast stochastic implicit gradient-based heuristic method that scales well to large networks with many scenarios. We use a strong duality reformulation and the McCormick envelope to provide a lower bound on the performance of our algorithm via convex relaxation. We test the performance of our method on a large model of the U.S. Western Interconnect and demonstrate that it scales linearly with network size and number of scenarios and can be efficiently parallelized on large machines. We find that for medium-sized 16 hour cases, gradient descent on average finds a 5.3x lower objective value in 16.5x less time compared to a traditional reformulation-based approach solved with an interior point method. We conclude with a large example in which we jointly plan transmission, generation, and storage for a 768 hour case on 100 node system, showing that emissions penalization leads to additional 40.0% reduction in carbon intensity at an additional cost of $17.1/MWh.
- Net-zero america: Potential pathways, infrastructure, and impacts: Final report. Princeton University, 2021.
- Paul L Joskow. Transmission capacity expansion is needed to decarbonize the electricity sector efficiently. Joule, 4(1):1–3, January 2020.
- The value of inter-regional coordination and transmission in decarbonizing the US electricity system. Joule, 5(1):115–134, January 2021.
- Synergies of sector coupling and transmission reinforcement in a cost-optimised, highly renewable european energy system. Energy, 160:720–739, October 2018.
- Paul L Joskow. Facilitating transmission expansion to support efficient decarbonization of the electricity sector. Economics of Energy & Environmental Policy, 10(2), 2021.
- Using market simulations for economic assessment of transmission upgrades: Application of the california ISO approach. Restructured Electric Power Systems, pages 241–270, 2010.
- A bilevel approach to transmission expansion planning within a market environment. IEEE Transactions on Power Systems, 24(3):1513–1522, August 2009.
- A note on linearized reformulations for a class of bilevel linear integer problems. Annals of Operations Research, 272(1-2):99–117, January 2019.
- Deep learning. Nature, 521(7553):436–444, May 2015.
- State-of-the-art generation expansion planning: A review. Applied energy, 230:563–589, November 2018.
- Transmission expansion planning: Literature review and classification. IEEE systems journal, 13(3):3129–3140, September 2019.
- Energy storage system expansion planning in power systems: a review. IET renewable power generation, 12(11):1203–1221, August 2018.
- Review on generation and transmission expansion co‐planning models under a market environment. IET Generation, Transmission and Distribution, 14(6):931–944, March 2020.
- Proactive planning and valuation of transmission investments in restructured electricity markets. Journal of regulatory economics, 30(3):358–387, December 2006.
- If you build it, he will come: Anticipative power transmission planning. Energy economics, 36:135–146, March 2013.
- A three-level static MILP model for generation and transmission expansion planning. IEEE transactions on power systems : a publication of the Power Engineering Society, 28(1):202–210, February 2013.
- Basic theoretical foundations and insights on bilevel models and their applications to power systems. Annals of operations research, 254(1-2):303–334, July 2017.
- Efficient proactive transmission planning to accommodate renewables. In 2012 IEEE Power and Energy Society General Meeting, pages 1–7. IEEE, July 2012.
- Wind power investment within a market environment. Applied Energy, 88(9):3239–3247, September 2011.
- Transmission and wind investment in a deregulated electricity industry. IEEE Transactions on Power Systems, 30(3):1633–1643, May 2015.
- Merchant transmission capacity investment: A mathematical program with equilibrium constraint formulation. Electric Power Components & Systems, 44(1):82–89, January 2016.
- Bolun Xu. Batteries in Electricity Markets: Economic Planning and Operations. PhD thesis, University of Washington, 2018.
- Emissions-aware electricity network expansion planning via implicit differentiation. In NeurIPS 2021 Workshop on Tackling Climate Change with Machine Learning. Climate Change AI, 2021.
- Efficiently solving linear bilevel programming problems using off-the-shelf optimization software. Optimization and engineering, 19(1):187–211, March 2018.
- Solving certain complementarity problems in power markets via convex programming. Top, 30(3):465–491, October 2022.
- Mixed-integer bilevel optimization for capacity planning with rational markets. Computers & chemical engineering, 86:33–47, March 2016.
- Bilevel optimization based transmission expansion planning considering phase shifting transformer. In 2017 North American Power Symposium (NAPS), pages 1–6, September 2017.
- Fabian Pedregosa. Hyperparameter optimization with approximate gradient. arXiv [stat.ML], pages 737–746, February 2016.
- Optimizing millions of hyperparameters by implicit differentiation. International Conference on Artificial Intelligence and Statistics, 108:1540–1552, November 2019.
- Diffusion models: A comprehensive survey of methods and applications. arXiv [cs.LG], September 2022.
- Asen L Dontchev and R Tyrrell Rockafellar. Implicit functions and solution mappings: A view from variational analysis, volume 616 of Springer Series in Operations Research and Financial Engineering. Springer, New York, NY, 2 edition, June 2014.
- Sébastien Bubeck. Convex optimization: Algorithms and complexity. Foundations and Trends® in Machine Learning, 8(3-4):231–357, 2015.
- Convex Optimization. Cambridge University Press, Cambridge, England, March 2004.
- A decomposition approach to automated generation/transmission expansion planning. IEEE Transactions on Power Apparatus and Systems, PER-5(11):30–31, November 1985.
- Market-clearing electricity prices and energy uplift. Technical report, Harvard Electricity Policy Group, 2007.
- Len Garver. Transmission network estimation using linear programming. IEEE transactions on power apparatus and systems, PAS-89(7):1688–1697, September 1970.
- A hierarchical decomposition approach for transmission network expansion planning. IEEE transactions on power systems : a publication of the Power Engineering Society, 9(1):373–380, February 1994.
- Lloyd N Trefethen and David Bau, III. Numerical Linear Algebra. Society for Industrial and Applied Mathematics, Philadelphia, PA, January 1997.
- Adversarially robust learning for security-constrained optimal power flow. arXiv [math.OC], November 2021.
- Application of sensitivity analysis of load supplying capability to interactive transmission expansion planning. IEEE transactions on power apparatus and systems, PAS-104(2):381–389, February 1985.
- Analysis of marginal carbon intensities in constrained power networks. Hawaii International Conference on System Sciences, pages 1–9, January 2010.
- Dynamic locational marginal emissions via implicit differentiation. IEEE transactions on power systems : a publication of the Power Engineering Society, 39:1138–1147, February 2023.
- Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT’2010, pages 177–186, Heidelberg, 2010. Physica-Verlag HD.
- Randomized rounding: A technique for provably good algorithms and algorithmic proofs. Combinatorica. An International Journal on Combinatorics and the Theory of Computing, 7(4):365–374, December 1987.
- Why there is no need to use a big-M in linear bilevel optimization: a computational study of two ready-to-use approaches. Computational management science, 20(1):1–12, December 2023.
- Stephan Dempe. Foundations of bilevel programming. Nonconvex Optimization and Its Applications. Springer, New York, NY, 2002 edition, May 2002.
- On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming, 106(1):25–57, March 2006.
- Jointly constrained biconvex programming. Mathematics of operations research, 8(2):273–286, May 1983.
- Convex relaxations for gas expansion planning. INFORMS journal on computing, 28(4):645–656, November 2016.
- An operational state aggregation technique for transmission expansion planning based on line benefits. IEEE transactions on power systems : a publication of the Power Engineering Society, 32(4):2744–2755, July 2017.
- Efficient linear network model for TEP based on piecewise McCormick relaxation. IET Generation, Transmission and Distribution, 13(23):5404–5412, December 2019.
- Tight convex relaxations for the expansion planning problem. Journal of optimization theory and applications, 194(1):325–352, July 2022.
- Julia: A fresh approach to numerical computing. SIAM Review, 59(1):65–98, January 2017.
- Convex optimization in julia. In 2014 First Workshop for High Performance Technical Computing in Dynamic Languages, pages 18–28. ieeexplore.ieee.org, November 2014.
- MOSEK ApS. Mosek.jl, 2022.
- Michael Innes. Don’t unroll adjoint: Differentiating SSA-form programs. CoRR, abs/1810.07951, October 2018.
- JuMP: A modeling language for mathematical optimization. SIAM Review, 59(2):295–320, January 2017.
- PyPSA-USA: An open-source optimisation model of the US power system, 2023.
- Solving directed laplacian systems in nearly-linear time through sparse LU factorizations. In 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS), pages 898–909. IEEE, October 2018.
- Xingyu Zhou. On the fenchel duality between strong convexity and lipschitz continuous gradient. arXiv [math.OC], March 2018.
- Boris T Poljak. Introduction to optimization. Optimization Software, 1987.
- G W Stewart. On the continuity of the generalized inverse. SIAM journal on applied mathematics, 17(1):33–45, January 1969.
- Primer on monotone operator methods. Applied and Computational Mathematics, 15(1):3–43, 2016.
- Technology data - energy storage. Technical report, Danish Energy Agency, 2018.