Three-Stage Deep Reinforcement Learning for Privacy-and Safety-Aware Smart Electric Vehicle Charging Station Scheduling and Volt/VAR Control
Sangyoon Lee, Dae‐Hyun Choi
Abstract
This article proposes a three-stage privacy-and safety-aware deep reinforcement learning framework for coordinating smart electric vehicle charging stations (EVCSs) integrated with a photovoltaic system/ energy storage system (ESS) and volt/VAR control in a power distribution system. The proposed framework aims to maximize the EVCS profit and minimize the network real power loss while ensuring zero ESS state of charge (SOC) and voltage violation as well as preserving the privacy of the EVCS net load schedule data. In Stage 1 with 30-min resolution, each charging station operator (CSO) agent of the EVCS performs day-ahead profitable real power charging/discharging of the ESS without violating its SOC constraint via a safety layer during training. In Stage 2, using the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> -differential privacy method, the CSO agents encrypt the EVCS net load schedule data delivered from Stage 1. In Stage 3 with 5-min resolution, the distribution system operator agent conducts real-time reactive power charging/discharging of the ESSs to minimize the real power loss while removing voltage violations completely via iterative safe exploration of the agent with iteration penalties during training. The proposed framework was assessed on the IEEE 33-bus system for its privacy preserving and safety performances.