Objective image quality assessment(IQA)plays an important role in various visual communication systems,which can automatically and efficiently predict the perceived quality of images.The human eye is the ultimate eval...Objective image quality assessment(IQA)plays an important role in various visual communication systems,which can automatically and efficiently predict the perceived quality of images.The human eye is the ultimate evaluator for visual experience,thus the modeling of human visual system(HVS)is a core issue for objective IQA and visual experience optimization.The traditional model based on black box fitting has low interpretability and it is difficult to guide the experience optimization effectively,while the model based on physiological simulation is hard to integrate into practical visual communication services due to its high computational complexity.For bridging the gap between signal distortion and visual experience,in this paper,we propose a novel perceptual no-reference(NR)IQA algorithm based on structural computational modeling of HVS.According to the mechanism of the human brain,we divide the visual signal processing into a low-level visual layer,a middle-level visual layer and a high-level visual layer,which conduct pixel information processing,primitive information processing and global image information processing,respectively.The natural scene statistics(NSS)based features,deep features and free-energy based features are extracted from these three layers.The support vector regression(SVR)is employed to aggregate features to the final quality prediction.Extensive experimental comparisons on three widely used benchmark IQA databases(LIVE,CSIQ and TID2013)demonstrate that our proposed metric is highly competitive with or outperforms the state-of-the-art NR IQA measures.展开更多
Visual odometry is critical in visual simultaneous localization and mapping for robot navigation.However,the pose estimation performance of most current visual odometry algorithms degrades in scenes with unevenly dist...Visual odometry is critical in visual simultaneous localization and mapping for robot navigation.However,the pose estimation performance of most current visual odometry algorithms degrades in scenes with unevenly distributed features because dense features occupy excessive weight.Herein,a new human visual attention mechanism for point-and-line stereo visual odometry,which is called point-line-weight-mechanism visual odometry(PLWM-VO),is proposed to describe scene features in a global and balanced manner.A weight-adaptive model based on region partition and region growth is generated for the human visual attention mechanism,where sufficient attention is assigned to position-distinctive objects(sparse features in the environment).Furthermore,the sum of absolute differences algorithm is used to improve the accuracy of initialization for line features.Compared with the state-of-the-art method(ORB-VO),PLWM-VO show a 36.79%reduction in the absolute trajectory error on the Kitti and Euroc datasets.Although the time consumption of PLWM-VO is higher than that of ORB-VO,online test results indicate that PLWM-VO satisfies the real-time demand.The proposed algorithm not only significantly promotes the environmental adaptability of visual odometry,but also quantitatively demonstrates the superiority of the human visual attention mechanism.展开更多
红外小目标的检测一直是红外追踪系统的关键技术,针对现有红外小目标检测方法在复杂背景下易造成虚警、检测速度慢的不足,从人类视觉系统的角度出发,参考了多尺度局部能量因子检测方法(multiscale local contrast measure using a local...红外小目标的检测一直是红外追踪系统的关键技术,针对现有红外小目标检测方法在复杂背景下易造成虚警、检测速度慢的不足,从人类视觉系统的角度出发,参考了多尺度局部能量因子检测方法(multiscale local contrast measure using a local energy factor,MLCM-LEF),提出了一种基于双层局部能量因子的红外小目标检测方法.从局部能量差异与局部亮度差异两个角度进行目标检测,使用双层局部能量因子从能量角度描述小目标与背景的相异程度,同时采取加权亮度差因子从亮度角度对图像进行目标检测,通过二维高斯融合上述二者的处理结果,最终利用图像均值和标准差进行自适应阈值分割,提取红外小目标.经过公开数据集实验测试,该方法在抑制背景噪声、减低虚警概率的表现上比主流的检测方法有所提升,与MLCM-LEF算法相比,基于双层局部能量因子的方法将单帧检测时间降低至三分之一.展开更多
The concept of receptive field(RF) is central to sensory neuroscience. Neuronal RF properties have been substantially studied in animals,while those in humans remain nearly unexplored. Here, we measured neuronal RFs w...The concept of receptive field(RF) is central to sensory neuroscience. Neuronal RF properties have been substantially studied in animals,while those in humans remain nearly unexplored. Here, we measured neuronal RFs with intracranial local field potentials(LFPs) and spiking activity in human visual cortex(V1/V2/V3). We recorded LFPs via macro-contacts and discovered that RF sizes estimated from lowfrequency activity(LFA, 0.5–30 Hz) were larger than those estimated from low-gamma activity(LGA, 30–60 Hz) and high-gamma activity(HGA, 60–150 Hz). We then took a rare opportunity to record LFPs and spiking activity via microwires in V1 simultaneously. We found that RF sizes and temporal profiles measured from LGA and HGA closely matched those from spiking activity. In sum, this study reveals that spiking activity of neurons in human visual cortex could be well approximated by LGA and HGA in RF estimation and temporal profile measurement, implying the pivotal functions of LGA and HGA in early visual information processing.展开更多
Recent studies on no-reference image quality assessment (NR-IQA) methods usually learn to evaluate the image quality by regressing from human subjective scores of the training samples. This study presented an NR-IQA m...Recent studies on no-reference image quality assessment (NR-IQA) methods usually learn to evaluate the image quality by regressing from human subjective scores of the training samples. This study presented an NR-IQA method based on the basic image visual parameters without using human scored image databases in learning. We demonstrated that these features comprised the most basic characteristics for constructing an image and influencing the visual quality of an image. In this paper, the definitions, computational method, and relationships among these visual metrics were described. We subsequently proposed a no-reference assessment function, which was referred to as a visual parameter measurement index (VPMI), based on the integration of these visual metrics to assess image quality. It is established that the maximum of VPMI corresponds to the best quality of the color image. We verified this method using the popular assessment database—image quality assessment database (LIVE), and the results indicated that the proposed method matched better with the subjective assessment of human vision. Compared with other image quality assessment models, it is highly competitive. VPMI has low computational complexity, which makes it promising to implement in real-time image assessment systems.展开更多
Bicycle sharing system has emerged as a new mode of transportation in many big cities over the past decade.Since the large number of bicycle stations distribute widely in the city,it is difficult to identify their uni...Bicycle sharing system has emerged as a new mode of transportation in many big cities over the past decade.Since the large number of bicycle stations distribute widely in the city,it is difficult to identify their unique attributes and characteristics directly.Oriented to the real bicycle hire dataset in Hangzhou,China,the clustering analysis for the bicycle stations based on the temporal flow data was carried out firstly.Then,based on the spatial distribution and temporal attributes of calculated clusters,visual diagram and map were used to vividly analyze the bicycle hire behavior related to station category and study the travel rules of citizens.The experimental results demonstrate the relation between human mobility,the time of day,day of week and the station location.展开更多
基金This work was supported by National Natural Science Foundation of China(Nos.61831015 and 61901260)Key Research and Development Program of China(No.2019YFB1405902).
文摘Objective image quality assessment(IQA)plays an important role in various visual communication systems,which can automatically and efficiently predict the perceived quality of images.The human eye is the ultimate evaluator for visual experience,thus the modeling of human visual system(HVS)is a core issue for objective IQA and visual experience optimization.The traditional model based on black box fitting has low interpretability and it is difficult to guide the experience optimization effectively,while the model based on physiological simulation is hard to integrate into practical visual communication services due to its high computational complexity.For bridging the gap between signal distortion and visual experience,in this paper,we propose a novel perceptual no-reference(NR)IQA algorithm based on structural computational modeling of HVS.According to the mechanism of the human brain,we divide the visual signal processing into a low-level visual layer,a middle-level visual layer and a high-level visual layer,which conduct pixel information processing,primitive information processing and global image information processing,respectively.The natural scene statistics(NSS)based features,deep features and free-energy based features are extracted from these three layers.The support vector regression(SVR)is employed to aggregate features to the final quality prediction.Extensive experimental comparisons on three widely used benchmark IQA databases(LIVE,CSIQ and TID2013)demonstrate that our proposed metric is highly competitive with or outperforms the state-of-the-art NR IQA measures.
基金Supported by Tianjin Municipal Natural Science Foundation of China(Grant No.19JCJQJC61600)Hebei Provincial Natural Science Foundation of China(Grant Nos.F2020202051,F2020202053).
文摘Visual odometry is critical in visual simultaneous localization and mapping for robot navigation.However,the pose estimation performance of most current visual odometry algorithms degrades in scenes with unevenly distributed features because dense features occupy excessive weight.Herein,a new human visual attention mechanism for point-and-line stereo visual odometry,which is called point-line-weight-mechanism visual odometry(PLWM-VO),is proposed to describe scene features in a global and balanced manner.A weight-adaptive model based on region partition and region growth is generated for the human visual attention mechanism,where sufficient attention is assigned to position-distinctive objects(sparse features in the environment).Furthermore,the sum of absolute differences algorithm is used to improve the accuracy of initialization for line features.Compared with the state-of-the-art method(ORB-VO),PLWM-VO show a 36.79%reduction in the absolute trajectory error on the Kitti and Euroc datasets.Although the time consumption of PLWM-VO is higher than that of ORB-VO,online test results indicate that PLWM-VO satisfies the real-time demand.The proposed algorithm not only significantly promotes the environmental adaptability of visual odometry,but also quantitatively demonstrates the superiority of the human visual attention mechanism.
文摘红外小目标的检测一直是红外追踪系统的关键技术,针对现有红外小目标检测方法在复杂背景下易造成虚警、检测速度慢的不足,从人类视觉系统的角度出发,参考了多尺度局部能量因子检测方法(multiscale local contrast measure using a local energy factor,MLCM-LEF),提出了一种基于双层局部能量因子的红外小目标检测方法.从局部能量差异与局部亮度差异两个角度进行目标检测,使用双层局部能量因子从能量角度描述小目标与背景的相异程度,同时采取加权亮度差因子从亮度角度对图像进行目标检测,通过二维高斯融合上述二者的处理结果,最终利用图像均值和标准差进行自适应阈值分割,提取红外小目标.经过公开数据集实验测试,该方法在抑制背景噪声、减低虚警概率的表现上比主流的检测方法有所提升,与MLCM-LEF算法相比,基于双层局部能量因子的方法将单帧检测时间降低至三分之一.
基金supported by the National Science and Technology Innovation 2030 Major Program(2022ZD0204802,2022ZD0204804)the National Natural Science Foundation of China(31930053,32171039)Beijing Academy of Artificial Intelligence(BAAI)。
文摘The concept of receptive field(RF) is central to sensory neuroscience. Neuronal RF properties have been substantially studied in animals,while those in humans remain nearly unexplored. Here, we measured neuronal RFs with intracranial local field potentials(LFPs) and spiking activity in human visual cortex(V1/V2/V3). We recorded LFPs via macro-contacts and discovered that RF sizes estimated from lowfrequency activity(LFA, 0.5–30 Hz) were larger than those estimated from low-gamma activity(LGA, 30–60 Hz) and high-gamma activity(HGA, 60–150 Hz). We then took a rare opportunity to record LFPs and spiking activity via microwires in V1 simultaneously. We found that RF sizes and temporal profiles measured from LGA and HGA closely matched those from spiking activity. In sum, this study reveals that spiking activity of neurons in human visual cortex could be well approximated by LGA and HGA in RF estimation and temporal profile measurement, implying the pivotal functions of LGA and HGA in early visual information processing.
基金supported by the National Natural Science Foundation of China under Grants No.61773094,No.61573080,No.91420105,and No.61375115National Program on Key Basic Research Project(973 Program)under Grant No.2013CB329401+1 种基金National High-Tech R&D Program of China(863 Program)under Grant No.2015AA020505Sichuan Province Science and Technology Project under Grants No.2015SZ0141 and No.2018ZA0138
文摘Recent studies on no-reference image quality assessment (NR-IQA) methods usually learn to evaluate the image quality by regressing from human subjective scores of the training samples. This study presented an NR-IQA method based on the basic image visual parameters without using human scored image databases in learning. We demonstrated that these features comprised the most basic characteristics for constructing an image and influencing the visual quality of an image. In this paper, the definitions, computational method, and relationships among these visual metrics were described. We subsequently proposed a no-reference assessment function, which was referred to as a visual parameter measurement index (VPMI), based on the integration of these visual metrics to assess image quality. It is established that the maximum of VPMI corresponds to the best quality of the color image. We verified this method using the popular assessment database—image quality assessment database (LIVE), and the results indicated that the proposed method matched better with the subjective assessment of human vision. Compared with other image quality assessment models, it is highly competitive. VPMI has low computational complexity, which makes it promising to implement in real-time image assessment systems.
基金the Public Projects of Zhejiang Province,China(Nos.2016C33110,2015C33067)National Natural Science Foundations of China(Nos.61602141,61473108,61402141)
文摘Bicycle sharing system has emerged as a new mode of transportation in many big cities over the past decade.Since the large number of bicycle stations distribute widely in the city,it is difficult to identify their unique attributes and characteristics directly.Oriented to the real bicycle hire dataset in Hangzhou,China,the clustering analysis for the bicycle stations based on the temporal flow data was carried out firstly.Then,based on the spatial distribution and temporal attributes of calculated clusters,visual diagram and map were used to vividly analyze the bicycle hire behavior related to station category and study the travel rules of citizens.The experimental results demonstrate the relation between human mobility,the time of day,day of week and the station location.
