Cybertwin-enabled 6th Generation(6G)network is envisioned to support artificial intelligence-native management to meet changing demands of 6G applications.Multi-Agent Deep Reinforcement Learning(MADRL)technologies dri...Cybertwin-enabled 6th Generation(6G)network is envisioned to support artificial intelligence-native management to meet changing demands of 6G applications.Multi-Agent Deep Reinforcement Learning(MADRL)technologies driven by Cybertwins have been proposed for adaptive task offloading strategies.However,the existence of random transmission delay between Cybertwin-driven agents and underlying networks is not considered in related works,which destroys the standard Markov property and increases the decision reaction time to reduce the task offloading strategy performance.In order to address this problem,we propose a pipelining task offloading method to lower the decision reaction time and model it as a delay-aware Markov Decision Process(MDP).Then,we design a delay-aware MADRL algorithm to minimize the weighted sum of task execution latency and energy consumption.Firstly,the state space is augmented using the lastly-received state and historical actions to rebuild the Markov property.Secondly,Gate Transformer-XL is introduced to capture historical actions'importance and maintain the consistent input dimension dynamically changed due to random transmission delays.Thirdly,a sampling method and a new loss function with the difference between the current and target state value and the difference between real state-action value and augmented state-action value are designed to obtain state transition trajectories close to the real ones.Numerical results demonstrate that the proposed methods are effective in reducing reaction time and improving the task offloading performance in the random-delay Cybertwin-enabled 6G networks.展开更多
The Internet of Things(IoTs)has become an essential component of the 5th Generation(5G)network and beyond,accelerating the transition to digital society.The increasing signaling traffic generated by billions of IoT de...The Internet of Things(IoTs)has become an essential component of the 5th Generation(5G)network and beyond,accelerating the transition to digital society.The increasing signaling traffic generated by billions of IoT devices has placed significant strain on the 5G Core network(5GC)control plane.To address this issue,the 3rd Gener-ation Partnership Project(3GPP)first proposed a Service-Based Architecture(SBA),intending to create a flexible,scalable,and agile cloud-native 5GC.However,considering the coupling of protocol states and functions,there are still many challenges to fully utilize the benefits of the cloud computing and orchestrate the 5GC in a cloud-native manner.We propose a Message-Level StateLess Design(ML-SLD)to provide a cloud-native 5GC from an architectural standpoint in this paper.Firstly,we propose an innovative mechanism for servitization of the N2 interface to maintain the connection between Radio Access Network(RAN)and the 5GC,avoiding interruptions and dropouts of large-scale user data.Furthermore,we propose an On-demand Message Forwarding(OMF)al-gorithm to reduce the impact of cloud fluctuations on the performance of cloud-native 5GC.Finally,we create a prototype that is based on the OpenAirInterface(OAI)5G core network projects,with all Network Functions(NFs)packaged in dockers and deployed in a kubernetes-based cloud environment.Several experiments have been built with UERANSIM and Chaosblade simulation tools.The findings demonstrate the viability and efficiency of our proposed methods.展开更多
基金funded by the National Key Research and Development Program of China under Grant 2019YFB1803301Beijing Natural Science Foundation (L202002)。
文摘Cybertwin-enabled 6th Generation(6G)network is envisioned to support artificial intelligence-native management to meet changing demands of 6G applications.Multi-Agent Deep Reinforcement Learning(MADRL)technologies driven by Cybertwins have been proposed for adaptive task offloading strategies.However,the existence of random transmission delay between Cybertwin-driven agents and underlying networks is not considered in related works,which destroys the standard Markov property and increases the decision reaction time to reduce the task offloading strategy performance.In order to address this problem,we propose a pipelining task offloading method to lower the decision reaction time and model it as a delay-aware Markov Decision Process(MDP).Then,we design a delay-aware MADRL algorithm to minimize the weighted sum of task execution latency and energy consumption.Firstly,the state space is augmented using the lastly-received state and historical actions to rebuild the Markov property.Secondly,Gate Transformer-XL is introduced to capture historical actions'importance and maintain the consistent input dimension dynamically changed due to random transmission delays.Thirdly,a sampling method and a new loss function with the difference between the current and target state value and the difference between real state-action value and augmented state-action value are designed to obtain state transition trajectories close to the real ones.Numerical results demonstrate that the proposed methods are effective in reducing reaction time and improving the task offloading performance in the random-delay Cybertwin-enabled 6G networks.
基金funded by the National Key Research and Development Program of China under Grant 2019YFB1803301Beijing Natural Science Foundation(L202002).
文摘The Internet of Things(IoTs)has become an essential component of the 5th Generation(5G)network and beyond,accelerating the transition to digital society.The increasing signaling traffic generated by billions of IoT devices has placed significant strain on the 5G Core network(5GC)control plane.To address this issue,the 3rd Gener-ation Partnership Project(3GPP)first proposed a Service-Based Architecture(SBA),intending to create a flexible,scalable,and agile cloud-native 5GC.However,considering the coupling of protocol states and functions,there are still many challenges to fully utilize the benefits of the cloud computing and orchestrate the 5GC in a cloud-native manner.We propose a Message-Level StateLess Design(ML-SLD)to provide a cloud-native 5GC from an architectural standpoint in this paper.Firstly,we propose an innovative mechanism for servitization of the N2 interface to maintain the connection between Radio Access Network(RAN)and the 5GC,avoiding interruptions and dropouts of large-scale user data.Furthermore,we propose an On-demand Message Forwarding(OMF)al-gorithm to reduce the impact of cloud fluctuations on the performance of cloud-native 5GC.Finally,we create a prototype that is based on the OpenAirInterface(OAI)5G core network projects,with all Network Functions(NFs)packaged in dockers and deployed in a kubernetes-based cloud environment.Several experiments have been built with UERANSIM and Chaosblade simulation tools.The findings demonstrate the viability and efficiency of our proposed methods.
