摘要
针对低地球轨道卫星点波束频繁切换过程中存在缺乏可用带宽导致用户连接中断,影响用户QoS问题,提出了基于Dueling-DDQN的星上带宽资源预留算法研究。通过建立Dueling-DDQN神经网络作为决策评估器,来为呼叫分配带宽,有效避免了人工干扰。Dueling-DDQN神经网络结构采用对偶网络,可以增加学习性能,并在目标网络函数构建时应用DDQN方法,解决Q值过高估计问题,与动作空间探索时采用的ε-贪心策略不同,通Noisy方法来实现探索过程,增加模型的探索能力,实现低轨卫星网络通信系统信道带宽资源的动态预留。仿真结果表明:所提出的星上带宽资源动态预留方法可以降低用户的切换失败率和新呼叫阻塞率,增加带宽利用率,来提高用户QoS满意度,且相对于传统的启发式方法有更优的结果。
Aiming at the problem that the lack of available bandwidth leads to the interruption of users’connection and affects users’QoS in the process of frequent spot beam switching of LEO satellites,research on on-board bandwidth resource reservation algorithm based on Dueling-DDQN is proposed.By establishing Dueling-DDQN neural network as a decision evaluator,bandwidth is allocated for calls,which effectively avoids human interference.Dueling-DDQN neural network structure adopts dual network,which can improve the learning performance.In addition,DDQN method is applied to construct the objective network function to solve the problem of over-estimation of Q value.Different fromε-greedy strategy used in action space exploration,Noisy method is used to realize the exploration process,increase the exploration ability of the model,and realize the dynamic reservation of channel bandwidth resources in LEO satellite network communication system.The simulation results show that the proposed dynamic reservation method of on-board bandwidth resources can reduce the handover failure rate and new call blocking rate of users,increase bandwidth utilization rate,and improve users’QoS satisfaction,and it has better results than the traditional heuristic method.
作者
刘治国
张姣姣
潘成胜
LIU Zhiguo;ZHANG Jiaojiao;PAN Chengsheng(School of Information Engineering,Dalian University,Dalian 116600,China;Key Laboratory of Communication and Network,Dalian University,Dalian 116600,China;School of Electronics and Information Engineering,Nanjing University of Information Science and Technology,Nanjing 211800,China)
出处
《兵器装备工程学报》
CAS
CSCD
北大核心
2023年第12期272-277,284,共7页
Journal of Ordnance Equipment Engineering
基金
国家自然科学基金项目(61931004)。
关键词
卫星网络
用户服务质量
深度强化学习
带宽资源
动态预留
satellite network
user service quality
deep reinforcement learning
bandwidth resources
dynamic reservation
