摘要
针对分布式电源优化调度面临的隐私保护和实时决策问题,提出了基于联邦强化学习的多智能体分布式协同优化策略。首先,构建了基于联邦强化学习的配电网分布式协同优化框架,利用联邦学习避免在多智能体深度强化学习过程中泄露隐私数据。在此框架下,提出了多智能体约束策略优化方法,利用离线训练缩短在线决策时间,支持智能体实时分布式决策。同时,该方法为智能体构建了考虑潮流方程等约束条件的可行域,允许智能体在训练过程中自由探索,提高了收敛速度,并确保实时调度策略满足电力系统安全运行约束。最后,通过算例进行仿真验证,结果表明离线训练时各智能体仅利用局部信息即可实现全局优化,并保证了实时决策和调度策略的安全性。
Aiming at the privacy preservation and real-time decision-making problems of the optimal dispatch for distributed generators,a multi-agent distributed collaborative optimization strategy based on federated reinforcement learning is proposed.First,a distributed collaborative optimization framework for the distribution network based on federated reinforcement learning is constructed,which uses federated learning to avoid leaking private data in the process of multi-agent deep reinforcement learning.Under this framework,a multi-agent constrained policy optimization method is proposed,which uses off-line training to shorten the online decision-making time for supporting the real-time distributed decision-making of agents.At the same time,the proposed method constructs a feasible region for agents considering the constraints such as power flow equations,allowing them to explore freely during the training process,which improves the convergence speed and ensures that the real-time dispatch strategy can meet the power system security operation constraints.Finally,the simulation results show that each agent can achieve the global optimization only by using local information during off-line training,and the proposed method ensures the security of real-time decision-making and dispatch strategy.
作者
蒲天骄
杜帅
李烨
王新迎
PU Tianjiao;DU Shuai;LI Ye;WANG Xinying(China Electric Power Research Institute,Beijing 100192,China)
出处
《电力系统自动化》
EI
CSCD
北大核心
2023年第8期62-70,共9页
Automation of Electric Power Systems
基金
国家重点研发计划资助项目(2020YFB0905900)
国家自然科学基金资助项目(U2066213)。
关键词
分布式电源
分布式协同优化
深度强化学习
联邦强化学习
隐私保护
distributed generator
distributed collaborative optimization
deep reinforcement learning
federated reinforcement learning
privacy preservation
