Deep Reinforcement learning for optimisation of LV distribution network operation with Integrated Flexibility in real-Time (DRIFT)

The modern power system is facing the challenges of large shares of renewable generation and increasing consumption of electric vehicles and heat pumps, affecting to a large extent power network operation, especially at distribution level. To avoid massive and costly network reinforcement, advanced distribution network control is needed, which must exploit the flexibility offered by active network users. The state-of-the-art distribution network control algorithms are usually based on off-line algorithms or, at best, on on-line optimisation algorithms based on the current state of the network. The major drawback of off-line algorithms is their low adaptability to variable network conditions, and the optimisation algorithms suffer from the ever-increasing complexity of the network, resulting in computationally intensive calculation process. The solution of these issues lies in the direction of a real-time approach to network control enabled by advanced concepts of artificial intelligence. To overcome the obstacles of novel algorithm acceptance, thorough field testing is needed, accompanied by explanation of their operation to distribution network operators. Further on, for a truly effective solution in terms of distribution network control, such solutions should be replicable and scalable. Therefore, scalability and replicability studies are crucial for adoption of new algorithms. The DRIFT project aims at addressing the above challenges with the development of a network control algorithm based on deep reinforcement learning (DRL) that can exploit the available flexibility of active network users in an optimal way. The optimality refers to a minimal use of users’ flexibility for the achievement of required network operating conditions. In the frame of DRIFT, an efficient deep neural network architecture will be designed and a framework for the reinforcement learning that allows to incorporate domain specifics will be defined and extended to include the possibility of guidance of the agent. The algorithm will be first developed and tested off-line. For the learning process of the model historical measurement data will be used and, to provide enough data for training, additional data will be generated by using the existing Distribution network simulation tool. The developed network control algorithm will be implemented in an actual low-voltage distribution network. The goal of the proposed system is to improve voltage profiles and avoid overloading of transformers and, in this way, increase the network hosting capacity (in terms of capability for renewables and new loads connection) without reinforcement of the primary network infrastructure. The controllable components in the LV network are primarily users’ heat pumps, solar power plants and battery storage. The final stage of the project is the study of the replicability of the proposed methodology to other distribution networks, and its scalability in terms of required data, measurement infrastructure, needed computational power and associated costs in case of wide-scale implementation. We aim at implementing the proposed solutions by combining the knowledge of network modelling and control (UL Faculty of Electrical Engineering), artificial intelligence (UL Faculty of Computer and Information Science) and network operation (DSO Elektro Gorenjska). Within UL FE as well as EG, the knowledge from the network operation and artificial intelligence domains is available, facilitating the interaction between the two. The co-financers of the project are the national Electricity distribution system operator SODO, the distribution system operator Elektro Gorenjska, and the transmission system operator ELES. Their cooperation shows the interest of key stakeholders in the implementation of the advanced network control concepts and provides appropriate support in the implementation of the project.