Inpainting has been continuously studied in the field of computer vision.As artificial intelligence technology developed,deep learning technology was introduced in inpainting research,helping to improve performance.Cu...Inpainting has been continuously studied in the field of computer vision.As artificial intelligence technology developed,deep learning technology was introduced in inpainting research,helping to improve performance.Currently,the input target of an inpainting algorithm using deep learning has been studied from a single image to a video.However,deep learning-based inpainting technology for panoramic images has not been actively studied.We propose a 360-degree panoramic image inpainting method using generative adversarial networks(GANs).The proposed network inputs a 360-degree equirectangular format panoramic image converts it into a cube map format,which has relatively little distortion and uses it as a training network.Since the cube map format is used,the correlation of the six sides of the cube map should be considered.Therefore,all faces of the cube map are used as input for the whole discriminative network,and each face of the cube map is used as input for the slice discriminative network to determine the authenticity of the generated image.The proposed network performed qualitatively better than existing single-image inpainting algorithms and baseline algorithms.展开更多
An effective algorithm of electronic image stabilization (EIS) of catadioptric panoramic imaging system for track robots is presented. The key techniques of this algorithm are as follows:① A model of electronic image...An effective algorithm of electronic image stabilization (EIS) of catadioptric panoramic imaging system for track robots is presented. The key techniques of this algorithm are as follows:① A model of electronic image stabilization is built by analyzing the imaging theory and the principle of EIS, and the image shift function of unwrapped panoramic image is deduced;② The relationship equation between motion estimation parameters of annular panoramic image and motion estimation parameters of unwrapped panoramic image is developed according to the constrained aspect ratio of real objects, motion parameters of annular panoramic image are firstly estimated, and then motion parameters among the image shift function are carried out according to the relationship equation;③ An excessive stabilization threshold is presented to prevent the phenomena of excessive stabilization, and the Kalman filtering is adopted to smooth the image sequences. Numerical experimental results show that this algorithm can effectively smooth out the unwanted motion and follow the intentional camera movement under certain resolutions.展开更多
A novel image stabilization system is presented,which consists of a global feature point tracking based motion estimation,a Kalman filtering based motion smoothing and an image mosaic based panoramic compensation.The ...A novel image stabilization system is presented,which consists of a global feature point tracking based motion estimation,a Kalman filtering based motion smoothing and an image mosaic based panoramic compensation.The global motion is estimated using feature point matching and iteration with the least-square method.Then,the Kalman filter is applied to smooth the original motion vectors to effectively alleviate unwanted camera vibrations and follow the intentional camera scan.Lastly,the loss information of image boundary due to the motion compensation is reconstructed with image mosaic to improve the visual quality.The experimental results show that this system can smooth unwanted translation or rotation of the video sequences and realize a panoramic stabilization at real-time speed.展开更多
Inspired by the unique structure of insect compound eyes,a multi-channel image acquisition system is designed to photograph a cylindrical panorama of its surroundings with one shot. The hardware structure consists of ...Inspired by the unique structure of insect compound eyes,a multi-channel image acquisition system is designed to photograph a cylindrical panorama of its surroundings with one shot. The hardware structure consists of an embedded ARM system and one array of 16 micro-image sensors. The system achieves the synchronization of captured photos in 10 ms,as well as 10 f /s video capture. The software architecture includes the TCP /IP protocol,video capture procedures in"Poll/Read"or"video streaming"modes,thread pool monitoring in multi-threading mutex,synchronization control with the"event""mutex signal"and"critical region"functions,and a synthetic image algorithm characterized by its portability,modularity,and remote transmission. The panoramic imaging system is expected to be a vision sensor for mobile robotics.展开更多
Panoramic images are widely used in many scenes,especially in virtual reality and street view capture.However,they are new for street furniture identification which is usually based on mobile laser scanning point clou...Panoramic images are widely used in many scenes,especially in virtual reality and street view capture.However,they are new for street furniture identification which is usually based on mobile laser scanning point cloud data or conventional 2D images.This study proposes to perform semantic segmentation on panoramic images and transformed images to separate light poles and traffic signs from background implemented by pre-trained Fully Convolutional Networks(FCN).FCN is the most important model for deep learning applied on semantic segmentation for its end to end training process and pixel-wise prediction.In this study,we use FCN-8s model that pre-trained on cityscape dataset and finetune it by our own data.Then replace cross entropy loss function with focal loss function in the FCN model and train it again to produce the predictions.The results show that in all results from pre-trained model,fine-tuning,and FCN model with focal loss,the light poles and traffic signs are detected well and the transformed images have better performance than panoramic images in the prediction according to the Recall and IoU evaluation.展开更多
The recent advancements in the field of Virtual Reality(VR)and Augmented Reality(AR)have a substantial impact on modern day technology by digitizing each and everything related to human life and open the doors to the ...The recent advancements in the field of Virtual Reality(VR)and Augmented Reality(AR)have a substantial impact on modern day technology by digitizing each and everything related to human life and open the doors to the next generation Software Technology(Soft Tech).VR and AR technology provide astonishing immersive contents with the help of high quality stitched panoramic contents and 360°imagery that widely used in the education,gaming,entertainment,and production sector.The immersive quality of VR and AR contents are greatly dependent on the perceptual quality of panoramic or 360°images,in fact a minor visual distortion can significantly degrade the overall quality.Thus,to ensure the quality of constructed panoramic contents for VR and AR applications,numerous Stitched Image Quality Assessment(SIQA)methods have been proposed to assess the quality of panoramic contents before using in VR and AR.In this survey,we provide a detailed overview of the SIQA literature and exclusively focus on objective SIQA methods presented till date.For better understanding,the objective SIQA methods are classified into two classes namely Full-Reference SIQA and No-Reference SIQA approaches.Each class is further categorized into traditional and deep learning-based methods and examined their performance for SIQA task.Further,we shortlist the publicly available benchmark SIQA datasets and evaluation metrices used for quality assessment of panoramic contents.In last,we highlight the current challenges in this area based on the existing SIQA methods and suggest future research directions that need to be target for further improvement in SIQA domain.展开更多
In light of the limited efficacy of conventional methods for identifying pavement cracks and the absence of comprehensive depth and location data in two-dimensional photographs,this study presents an intelligent strat...In light of the limited efficacy of conventional methods for identifying pavement cracks and the absence of comprehensive depth and location data in two-dimensional photographs,this study presents an intelligent strategy for extracting road cracks.This methodology involves the integration of laser point cloud data obtained from a vehicle-mounted system and a panoramic sequence of images.The study employs a vehicle-mounted LiDAR measurement system to acquire laser point cloud and panoramic sequence image data simultaneously.A convolutional neural network is utilized to extract cracks from the panoramic sequence image.The extracted sequence image is then aligned with the laser point cloud,enabling the assignment of RGB information to the vehicle-mounted three dimensional(3D)point cloud and location information to the two dimensional(2D)panoramic image.Additionally,a threshold value is set based on the crack elevation change to extract the aligned roadway point cloud.The three-dimensional data pertaining to the cracks can be acquired.The experimental findings demonstrate that the use of convolutional neural networks has yielded noteworthy outcomes in the extraction of road cracks.The utilization of point cloud and image alignment techniques enables the extraction of precise location data pertaining to road cracks.This approach exhibits superior accuracy when compared to conventional methods.Moreover,it facilitates rapid and accurate identification and localization of road cracks,thereby playing a crucial role in ensuring road maintenance and traffic safety.Consequently,this technique finds extensive application in the domains of intelligent transportation and urbanization development.The technology exhibits significant promise for use in the domains of intelligent transportation and city development.展开更多
基金Korea Electric Power Corporation(Grant No.R18XA02).
文摘Inpainting has been continuously studied in the field of computer vision.As artificial intelligence technology developed,deep learning technology was introduced in inpainting research,helping to improve performance.Currently,the input target of an inpainting algorithm using deep learning has been studied from a single image to a video.However,deep learning-based inpainting technology for panoramic images has not been actively studied.We propose a 360-degree panoramic image inpainting method using generative adversarial networks(GANs).The proposed network inputs a 360-degree equirectangular format panoramic image converts it into a cube map format,which has relatively little distortion and uses it as a training network.Since the cube map format is used,the correlation of the six sides of the cube map should be considered.Therefore,all faces of the cube map are used as input for the whole discriminative network,and each face of the cube map is used as input for the slice discriminative network to determine the authenticity of the generated image.The proposed network performed qualitatively better than existing single-image inpainting algorithms and baseline algorithms.
基金Supported by State Key Laboratory of Explosion Science and Technology Foundation(ZDKT08-05)
文摘An effective algorithm of electronic image stabilization (EIS) of catadioptric panoramic imaging system for track robots is presented. The key techniques of this algorithm are as follows:① A model of electronic image stabilization is built by analyzing the imaging theory and the principle of EIS, and the image shift function of unwrapped panoramic image is deduced;② The relationship equation between motion estimation parameters of annular panoramic image and motion estimation parameters of unwrapped panoramic image is developed according to the constrained aspect ratio of real objects, motion parameters of annular panoramic image are firstly estimated, and then motion parameters among the image shift function are carried out according to the relationship equation;③ An excessive stabilization threshold is presented to prevent the phenomena of excessive stabilization, and the Kalman filtering is adopted to smooth the image sequences. Numerical experimental results show that this algorithm can effectively smooth out the unwanted motion and follow the intentional camera movement under certain resolutions.
基金supported by the National "863 Program" on High Technology Research and Development of China (Grant No.2006AA01Z127)
文摘A novel image stabilization system is presented,which consists of a global feature point tracking based motion estimation,a Kalman filtering based motion smoothing and an image mosaic based panoramic compensation.The global motion is estimated using feature point matching and iteration with the least-square method.Then,the Kalman filter is applied to smooth the original motion vectors to effectively alleviate unwanted camera vibrations and follow the intentional camera scan.Lastly,the loss information of image boundary due to the motion compensation is reconstructed with image mosaic to improve the visual quality.The experimental results show that this system can smooth unwanted translation or rotation of the video sequences and realize a panoramic stabilization at real-time speed.
基金Supported by the National Natural Science Foundation of China(61233014)the China Postdoctoral Science Foundation(2012M5210711,20123218110031)the National Natural Science Major International Cooperation Projects(61161120323)
文摘Inspired by the unique structure of insect compound eyes,a multi-channel image acquisition system is designed to photograph a cylindrical panorama of its surroundings with one shot. The hardware structure consists of an embedded ARM system and one array of 16 micro-image sensors. The system achieves the synchronization of captured photos in 10 ms,as well as 10 f /s video capture. The software architecture includes the TCP /IP protocol,video capture procedures in"Poll/Read"or"video streaming"modes,thread pool monitoring in multi-threading mutex,synchronization control with the"event""mutex signal"and"critical region"functions,and a synthetic image algorithm characterized by its portability,modularity,and remote transmission. The panoramic imaging system is expected to be a vision sensor for mobile robotics.
文摘Panoramic images are widely used in many scenes,especially in virtual reality and street view capture.However,they are new for street furniture identification which is usually based on mobile laser scanning point cloud data or conventional 2D images.This study proposes to perform semantic segmentation on panoramic images and transformed images to separate light poles and traffic signs from background implemented by pre-trained Fully Convolutional Networks(FCN).FCN is the most important model for deep learning applied on semantic segmentation for its end to end training process and pixel-wise prediction.In this study,we use FCN-8s model that pre-trained on cityscape dataset and finetune it by our own data.Then replace cross entropy loss function with focal loss function in the FCN model and train it again to produce the predictions.The results show that in all results from pre-trained model,fine-tuning,and FCN model with focal loss,the light poles and traffic signs are detected well and the transformed images have better performance than panoramic images in the prediction according to the Recall and IoU evaluation.
文摘The recent advancements in the field of Virtual Reality(VR)and Augmented Reality(AR)have a substantial impact on modern day technology by digitizing each and everything related to human life and open the doors to the next generation Software Technology(Soft Tech).VR and AR technology provide astonishing immersive contents with the help of high quality stitched panoramic contents and 360°imagery that widely used in the education,gaming,entertainment,and production sector.The immersive quality of VR and AR contents are greatly dependent on the perceptual quality of panoramic or 360°images,in fact a minor visual distortion can significantly degrade the overall quality.Thus,to ensure the quality of constructed panoramic contents for VR and AR applications,numerous Stitched Image Quality Assessment(SIQA)methods have been proposed to assess the quality of panoramic contents before using in VR and AR.In this survey,we provide a detailed overview of the SIQA literature and exclusively focus on objective SIQA methods presented till date.For better understanding,the objective SIQA methods are classified into two classes namely Full-Reference SIQA and No-Reference SIQA approaches.Each class is further categorized into traditional and deep learning-based methods and examined their performance for SIQA task.Further,we shortlist the publicly available benchmark SIQA datasets and evaluation metrices used for quality assessment of panoramic contents.In last,we highlight the current challenges in this area based on the existing SIQA methods and suggest future research directions that need to be target for further improvement in SIQA domain.
基金founded by National Key R&D Program of China (No.2021YFB2601200)National Natural Science Foundation of China (No.42171416)Teacher Support Program for Pyramid Talent Training Project of Beijing University of Civil Engineering and Architecture (No.JDJQ20200307).
文摘In light of the limited efficacy of conventional methods for identifying pavement cracks and the absence of comprehensive depth and location data in two-dimensional photographs,this study presents an intelligent strategy for extracting road cracks.This methodology involves the integration of laser point cloud data obtained from a vehicle-mounted system and a panoramic sequence of images.The study employs a vehicle-mounted LiDAR measurement system to acquire laser point cloud and panoramic sequence image data simultaneously.A convolutional neural network is utilized to extract cracks from the panoramic sequence image.The extracted sequence image is then aligned with the laser point cloud,enabling the assignment of RGB information to the vehicle-mounted three dimensional(3D)point cloud and location information to the two dimensional(2D)panoramic image.Additionally,a threshold value is set based on the crack elevation change to extract the aligned roadway point cloud.The three-dimensional data pertaining to the cracks can be acquired.The experimental findings demonstrate that the use of convolutional neural networks has yielded noteworthy outcomes in the extraction of road cracks.The utilization of point cloud and image alignment techniques enables the extraction of precise location data pertaining to road cracks.This approach exhibits superior accuracy when compared to conventional methods.Moreover,it facilitates rapid and accurate identification and localization of road cracks,thereby playing a crucial role in ensuring road maintenance and traffic safety.Consequently,this technique finds extensive application in the domains of intelligent transportation and urbanization development.The technology exhibits significant promise for use in the domains of intelligent transportation and city development.
