Residential demand response programs aim to activate demand flexibility at the household level.In recent years,reinforcement learning(RL)has gained significant attention for these type of applications.A major challeng...Residential demand response programs aim to activate demand flexibility at the household level.In recent years,reinforcement learning(RL)has gained significant attention for these type of applications.A major challenge of RL algorithms is data efficiency.New RL algorithms,such as proximal policy optimisation(PPO),have tried to increase data efficiency.Addi tionally,combining RL with transfer learning has been proposed in an effort to mitigate this challenge.In this work,we further improve upon state-of-the-art transfer learning performance by incorporating demand response domain knowledge into the learning pipeline.We evaluate our approach on a demand response use case where peak shaving and self-consumption is incentivised by means of a capacity tariff.We show our adapted version of PPO,combined with transfer learming,reduces cost by 14.51%compared to a regular hysteresis controller and by 6.68%compared to traditional PPO.展开更多
A number of decarbonization scenarios for the energy sector are built on simultaneous electrification of energy demand,and decarbonization of electricity generation through renewable energy sources.However,increased e...A number of decarbonization scenarios for the energy sector are built on simultaneous electrification of energy demand,and decarbonization of electricity generation through renewable energy sources.However,increased electricity demand due to heat and transport electrification and the variability associated with renewables have the potential to disrupt stable electric grid operation.To address these issues using demand response,researchers and practitioners have increasingly turned towards automated decision support tools which utilize machine learning and optimization algorithms.However,when applied naively,these algorithms suffer from high sample complexity,which means that it is often impractical to fit sufficiently complex models because of a lack of observed data.Recent advances have shown that techniques such as transfer learning can address this problem and improve their performance considerably—both in supervised and reinforcement learning contexts.Such formulations allow models to leverage existing domain knowledge and human expertise in addition to sparse observational data.More formally,transfer learning embodies all techniques where one aims to increase(learning)performance in a target domain or task,by using knowledge gained in a source domain or task.This paper provides a detailed overview of state-of-the-art techniques on applying transfer learning in demand response,showing improvements that can exceed 30%in a variety of tasks.We observe that most research to date has focused on transfer learning in the context of electricity demand prediction,although reinforcement learning based controllers have also seen increasing attention.However,a number of limitations remain in these studies,including a lack of benchmarks,systematic performance improvement tracking,and consensus on techniques that can help avoid negative transfer.展开更多
文摘Residential demand response programs aim to activate demand flexibility at the household level.In recent years,reinforcement learning(RL)has gained significant attention for these type of applications.A major challenge of RL algorithms is data efficiency.New RL algorithms,such as proximal policy optimisation(PPO),have tried to increase data efficiency.Addi tionally,combining RL with transfer learning has been proposed in an effort to mitigate this challenge.In this work,we further improve upon state-of-the-art transfer learning performance by incorporating demand response domain knowledge into the learning pipeline.We evaluate our approach on a demand response use case where peak shaving and self-consumption is incentivised by means of a capacity tariff.We show our adapted version of PPO,combined with transfer learming,reduces cost by 14.51%compared to a regular hysteresis controller and by 6.68%compared to traditional PPO.
基金Hussain Kazmi acknowledges support from Research Foundation-Flanders(FWO),Belgium(research fellowship 1262921N)in the preparation of this manuscript.Thijs Peirelinck,Brida V Mbuwir,Chris Hermans and Fred Spiessens acknowledge support from the Flemish Institute for Technological Research(VITO),Belgium in the preparation of this manuscript.Johan Suykens acknowledges support from ERC AdG E-DUALITY,Belgium(787960),KU Leuven C14/18/068,FWO GOA4917N,EU H2020 ICT-48 Network TAILOR,Ford KU Leuven Research Alliance KUL0076,Impulsfonds AI:VR 20192203 DOC.0318/1QUATER,KU Leuven AI institute.
文摘A number of decarbonization scenarios for the energy sector are built on simultaneous electrification of energy demand,and decarbonization of electricity generation through renewable energy sources.However,increased electricity demand due to heat and transport electrification and the variability associated with renewables have the potential to disrupt stable electric grid operation.To address these issues using demand response,researchers and practitioners have increasingly turned towards automated decision support tools which utilize machine learning and optimization algorithms.However,when applied naively,these algorithms suffer from high sample complexity,which means that it is often impractical to fit sufficiently complex models because of a lack of observed data.Recent advances have shown that techniques such as transfer learning can address this problem and improve their performance considerably—both in supervised and reinforcement learning contexts.Such formulations allow models to leverage existing domain knowledge and human expertise in addition to sparse observational data.More formally,transfer learning embodies all techniques where one aims to increase(learning)performance in a target domain or task,by using knowledge gained in a source domain or task.This paper provides a detailed overview of state-of-the-art techniques on applying transfer learning in demand response,showing improvements that can exceed 30%in a variety of tasks.We observe that most research to date has focused on transfer learning in the context of electricity demand prediction,although reinforcement learning based controllers have also seen increasing attention.However,a number of limitations remain in these studies,including a lack of benchmarks,systematic performance improvement tracking,and consensus on techniques that can help avoid negative transfer.
