Reactive Power Optimization Strategy for Power Grid with High Proportion of Renewable Energy Based on Reinforcement Learning Algorithm
DOI:
https://doi.org/10.18486/ijcsnt/14.3.015Keywords:
DDPG, Power Grid Reactive Power Optimization, Reinforcement Learning, Action Space Decomposition, Actor-Critic NetworkAbstract
As the high proportion of renewable energy grid connection leads to intensified grid voltage fluctuations, traditional reactive power optimization methods have obvious deficiencies in dynamic regulation capability and real-time performance, making it difficult to cope with the voltage over-limit and network loss problems caused by the randomness of wind and solar output. To this end, this paper proposes a reactive power optimization strategy based on the deep deterministic policy gradient (DDPG) algorithm, constructs a hierarchical reinforcement learning framework that includes 12-dimensional state spaces such as node voltage, renewable energy output, and load rate, and continuous/discrete action spaces such as SVG and capacitor banks, designs a composite reward function that integrates voltage deviation, network loss cost, and equipment action penalty, and implements minute-level online training through the interactive interface between the power grid simulation platform PSS/E and Python. Tests in the improved IEEE 33-node system show that this method reduces the voltage deviation rate to 0.41% and the average network loss to 123.2kW, verifying that the strategy can effectively coordinate the collaborative control of new energy stations and traditional reactive equipment, providing a new way to solve the problem of dynamic reactive power optimization in high-penetration power grids.
References
Hu D., Ye Z., Gao Y., et al. “Multi-Agent Deep Reinforcement Learning for Voltage Control with Coordinated Active and Reactive Power Optimization.” IEEE Transactions on Smart Grid, vol. 13, no. 6, pp. 4873–4886, 2022. DOI: https://doi.org/10.1109/TSG.2022.3185975
Kryonidis G. C., Malamaki K. N. D., Gkavanoudis S. I., et al. “Distributed Reactive Power Control Scheme for the Voltage Regulation of Unbalanced LV Grids.” IEEE Transactions on Sustainable Energy, vol. 12, no. 2, pp. 1301–1310, 2020. DOI: https://doi.org/10.1109/TSTE.2020.3042855
Candanedo S. “Reactive Power Optimization of Power System Based on Distributed Cooperative Particle Swarm Optimization Algorithm.” Distributed Processing Systems, vol. 1, no. 2, pp. 46–53, 2020. DOI: https://doi.org/10.38007/DPS.2020.010206
Chandrasekaran K., Selvaraj J., Amaladoss C. R., et al. “Hybrid Renewable Energy Based Smart Grid System for Reactive Power Management and Voltage Profile Enhancement Using Artificial Neural Network.” Energy Sources, Part A: Recovery, Utilization, and Environmental Effects, vol. 43, no. 19, pp. 2419–2442, 2021. DOI: https://doi.org/10.1080/15567036.2021.1902430
Yang T., Guo Y., Deng L., et al. “A Linear Branch Flow Model for Radial Distribution Networks and Its Application to Reactive Power Optimization and Network Reconfiguration.” IEEE Transactions on Smart Grid, vol. 12, no. 3, pp. 2027–2036, 2020. DOI: https://doi.org/10.1109/TSG.2020.3039984
Yang Lei, Li Shengnan, Huang Wei, Zhang Dan, Yang Bo, and Zhang Xiaoshun. “Reactive Power Optimization of Power Grid with High Proportion of Wind and Solar Energy Based on Balanced Optimizer.” Journal of Electric Power System and Automation, vol. 33, no. 4, pp. 32–39, 2021.
Wang Zhongfu. “Reactive Power Optimization Strategy of New Energy Power Grid Based on Interval Modeling.” Southern Energy Construction, vol. 8, no. 4, pp. 95–106, 2021.
Jiang Zhijun, Yuan Xuan, Qiu Wenhao, Huang Licai, and He Wei. “Reactive Power Optimization Model of Active Distribution Network with New Energy and Electric Vehicle Charging Station Connected to the Grid.” Journal of Electric Power System and Automation, vol. 36, no. 2, pp. 116–125, 2024.
Ma Xiping, Dong Xiaoyang, Li Yaxin, Liang Chen, and Xu Rui. “Research on Two-Layer Optimization Strategy of Active and Reactive Power Coordination in Power Grid Considering the Access of New Energy.” Power System and Clean Energy, vol. 40, no. 1, pp. 137–142, 149, 2024.
Zhang Jingzhong, Meng Fei, Sun Yang, and Yu Huixia. “Reactive Power Optimization Control of New Energy in Distribution Network Considering Active Power Uncertainty.” China Electric Power, vol. 57, no. 3, pp. 51–59, 2024.
Wang X., Wang S., Liang X., et al. “Deep Reinforcement Learning: A Survey.” IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 4, pp. 5064–5078, 2022. DOI: https://doi.org/10.1109/TNNLS.2022.3207346
Moerland T. M., Broekens J., Plaat A., et al. “Model-Based Reinforcement Learning: A Survey.” Foundations and Trends® in Machine Learning, vol. 16, no. 1, pp. 1–118, 2023. DOI: https://doi.org/10.1561/2200000086
Raffin A., Hill A., Gleave A., et al. “Stable-Baselines3: Reliable Reinforcement Learning Implementations.” Journal of Machine Learning Research, vol. 22, no. 268, pp. 1–8, 2021.
Wang H., Liu N., Zhang Y., et al. “Deep Reinforcement Learning: A Survey.” Frontiers of Information Technology & Electronic Engineering, vol. 21, no. 12, pp. 1726–1744, 2020. DOI: https://doi.org/10.1631/FITEE.1900533
Zhu Z., Lin K., Jain A. K., et al. “Transfer Learning in Deep Reinforcement Learning: A Survey.” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 11, pp. 13344–13362, 2023. DOI: https://doi.org/10.1109/TPAMI.2023.3292075
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Hao Chen, Nan Yang, Zeliang Ma
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
