Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach

Wang, Yi, Qiu, Dawei, Sun, Mingyang, Strbac, Goran and Gao, Zhiwei (2023) Secure energy management of multi-energy microgrid: A physical-informed safe reinforcement learning approach. Applied Energy, 335. p. 120759. ISSN 0306-2619

1-s2.0-S030626192300123X-main.pdf - Published Version
Available under License Creative Commons Attribution 4.0.

Download (2MB) | Preview
Official URL:


The large-scale integration of distributed energy resources into the energy industry enables the fast transition to a decarbonized future but raises some potential challenges of insecure and unreliable operations. Multi-energy Microgrids (MEMGs), as localized small multi-energy systems, can effectively integrate a variety of energy components with multiple energy sectors, which have been recently recognized as a valid solution to improve the operational security and reliability. As a result, a massive amount of research has been conducted to investigate MEMG energy management problems, including both model-based optimization and model-free learning approaches. Compared to optimization approaches, reinforcement learning is being widely deployed in MEMG energy management problems owing to its ability to handle highly dynamic and stochastic processes without knowing any system knowledge. However, it is still difficult for conventional model-free reinforcement learning methods to capture the physical constraints of the MEMG model, which may therefore destroy its secure operation. To address this research challenge, this paper proposes a novel safe reinforcement learning method by learning a dynamic security assessment rule to abstract a physical-informed safety layer on top of the conventional model-free reinforcement learning energy management policy, which can respect all the physical constraints through mathematically solving an action correction formulation. In this setting, the secure energy management of the MEMG can be guaranteed for both training and test procedures. Extensive case studies based on two integrated systems (i.e., a small 6-bus power and 7-node gas network, and a large 33-bus power and 20-node gas network) are carried out to verify the superior performance of the proposed physical-informed reinforcement learning method in achieving a cost-effective MEMG energy management performance while respecting all the physical constraints, compared to conventional reinforcement learning and optimization approaches.

Item Type: Article
Additional Information: Funding information: This work was supported by two UK EPSRC projects: ‘Integrated Development of Low-Carbon Energy Systems (IDLES): A Whole-System Paradigm for Creating a National Strategy’ (project code: EP/R045518/1) and UK-China project - ‘Technology Transformation to Support Flexible and Resilient Local Energy Systems’ (project code: EP/T021780/1), and one Horizon Europe project: ‘Reliability, Resilience and Defense technology for the griD’ (Grant agreement ID: 101075714) as well as the National Natural Science Foundation of China under Grants 62103371, 52161135201, U20A20159, 62061130220.
Uncontrolled Keywords: Multi-energy microgrid, Energy management, Dynamic security assessment, Physical-informed safety layer, Reinforcement learning
Subjects: H800 Chemical, Process and Energy Engineering
Department: Faculties > Engineering and Environment > Mathematics, Physics and Electrical Engineering
Depositing User: Elena Carlaw
Date Deposited: 08 Feb 2023 11:49
Last Modified: 08 Feb 2023 12:00

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics