It is a significant research direction for highly complex musculoskeletal robots that how to develop the ability of motion learning and generalization.The cooperations of multiple brain regions are crucial to improvin...It is a significant research direction for highly complex musculoskeletal robots that how to develop the ability of motion learning and generalization.The cooperations of multiple brain regions are crucial to improving motion performance.Inspired by the neural mechanisms of structures,functions,and interconnections of basal ganglia and cerebellum,a biologically inspired integration model for motor learning of musculoskeletal robots is proposed.Based on the neural characteristics of the basal ganglia,the basal ganglia actor network,which mainly simulates the dorsal striatum,outputs motion commands,and the basal ganglia critic network,which simulates the ventral striatum,estimates actionstate values.Their network parameters are updated using the soft actor-critic method.Based on the sensorimotor prediction mechanism of the cerebellum,the cerebellum network evaluates the state feature extraction quality of the basal ganglia actor network and then updates the weights of its feature layer.This learning method is proven to converge to the optimal policy.Furthermore,drawing on the mechanism of dopaminergic dynamic regulation in the basal ganglia,the adaptive adjustment of target entropy and the dopaminergic experience replay are proposed to further improve the integration model,which contributes to the exploration-exploitation trade-off of motor learning.The bio-inspired integration model is validated on a musculoskeletal system.Experimental results indicate that this model can effectively control the musculoskeletal robot to accomplish the motion task from random starting locations to random target positions with high precision and robustness.展开更多
基金supported by Major Project of Science and Technology Innovation 2030 Brain Science and Brain-Inspired Intelligence under Grant No.2021ZD0200408the National Natural Science Foundation of China under Grant Nos.62203439 and 62203443Major program of the National Natural Science Foundation of China under Grant Nos.T2293720,T2293723,and T2293724.
文摘It is a significant research direction for highly complex musculoskeletal robots that how to develop the ability of motion learning and generalization.The cooperations of multiple brain regions are crucial to improving motion performance.Inspired by the neural mechanisms of structures,functions,and interconnections of basal ganglia and cerebellum,a biologically inspired integration model for motor learning of musculoskeletal robots is proposed.Based on the neural characteristics of the basal ganglia,the basal ganglia actor network,which mainly simulates the dorsal striatum,outputs motion commands,and the basal ganglia critic network,which simulates the ventral striatum,estimates actionstate values.Their network parameters are updated using the soft actor-critic method.Based on the sensorimotor prediction mechanism of the cerebellum,the cerebellum network evaluates the state feature extraction quality of the basal ganglia actor network and then updates the weights of its feature layer.This learning method is proven to converge to the optimal policy.Furthermore,drawing on the mechanism of dopaminergic dynamic regulation in the basal ganglia,the adaptive adjustment of target entropy and the dopaminergic experience replay are proposed to further improve the integration model,which contributes to the exploration-exploitation trade-off of motor learning.The bio-inspired integration model is validated on a musculoskeletal system.Experimental results indicate that this model can effectively control the musculoskeletal robot to accomplish the motion task from random starting locations to random target positions with high precision and robustness.
