Integrating residential-level photovoltaic energy generation and energy storage for the on-grid system is essential to reduce electricity use for residential consumption from the grid. However, reaching a reliable and optimal control policy is highly challenging due to the intrinsic uncertainties in the renewable energy sources and fluctuating demand profile. In this work, we proposed and designed an ensemble deep reinforcement learning (DRL) algorithm combined with risk evaluation to solve the energy optimization problem under uncertainties. Meanwhile, the attention and masking layer, the state-of-the-art natural language processing techniques, were incorporated into the algorithm to handle the issue of hard constraints, which are frequently encountered in the renewable energy optimization problem. To the best of our knowledge, this work is the first attempt to tackle the energy optimization problem under uncertainty using a scenario-based ensemble DRL approach with a risk evaluation. Through a well-designed single household microgrid energy management system, we found that the attention and masking layer played a crucial role in fulfilling the hard constraint. The ensemble DRL with the increased number of agents showed a significantly improved energy management policy leading to ∼75% of the cost reduction compared with those obtained by using conventional DRL with a single agent. The risk evaluation revealed that the current ensemble DRL approach possessed a high-risk/high-profit feature, which could be significantly improved by designing a risk-aware reward function in future investigations.
Skip Nav Destination
Article navigation
July 2022
Research Article|
July 08 2022
Attention and masking embedded ensemble reinforcement learning for smart energy optimization and risk evaluation under uncertainties
Tomah Sogabe
;
Tomah Sogabe
a)
(Conceptualization, Funding acquisition, Project administration, Writing – original draft, Writing – review & editing)
1
Info-Powered Energy System Research Center, The University of Electro-Communications
, Tokyo, Japan
2
Engineering Department, The University of Electro-Communications
, Tokyo, Japan
3
Grid, Inc.
Tokyo, Japan
a)Author to whom correspondence should be addressed: sogabe@uec.ac.jp
Search for other works by this author on:
Dinesh Bahadur Malla;
Dinesh Bahadur Malla
(Data curation, Investigation, Methodology, Software, Visualization)
2
Engineering Department, The University of Electro-Communications
, Tokyo, Japan
3
Grid, Inc.
Tokyo, Japan
Search for other works by this author on:
Chih-Chieh Chen
;
Chih-Chieh Chen
(Data curation, Resources, Writing – review & editing)
3
Grid, Inc.
Tokyo, Japan
Search for other works by this author on:
Katsuyoshi Sakamoto
Katsuyoshi Sakamoto
(Formal analysis, Investigation)
2
Engineering Department, The University of Electro-Communications
, Tokyo, Japan
Search for other works by this author on:
a)Author to whom correspondence should be addressed: sogabe@uec.ac.jp
J. Renewable Sustainable Energy 14, 045501 (2022)
Article history
Received:
April 27 2022
Accepted:
June 19 2022
Citation
Tomah Sogabe, Dinesh Bahadur Malla, Chih-Chieh Chen, Katsuyoshi Sakamoto; Attention and masking embedded ensemble reinforcement learning for smart energy optimization and risk evaluation under uncertainties. J. Renewable Sustainable Energy 1 July 2022; 14 (4): 045501. https://doi.org/10.1063/5.0097344
Download citation file:
Pay-Per-View Access
$40.00
Sign In
You could not be signed in. Please check your credentials and make sure you have an active account and try again.
Citing articles via
Modification of wind turbine wakes by large-scale, convective atmospheric boundary layer structures
Lawrence Cheung, Gopal Yalla, et al.
Simulation and modeling of wind farms in baroclinic atmospheric boundary layers
J. H. Kasper, A. Stieren, et al.
Machine learning for modern power distribution systems: Progress and perspectives
Marija Marković, Matthew Bossart, et al.
Related Content
Optimal placement and sizing of plug in electric vehicles charging stations within distribution networks with high penetration of photovoltaic panels
J. Renewable Sustainable Energy (September 2013)
Pump as turbine for throttling energy recovery in water distribution networks
AIP Conf. Proc. (December 2019)