Feature matching plays a key role in computer vision. However, due to the limitations of the descriptors, the putative matches are inevitably contaminated by massive outliers.This paper attempts to tackle the outlier ...Feature matching plays a key role in computer vision. However, due to the limitations of the descriptors, the putative matches are inevitably contaminated by massive outliers.This paper attempts to tackle the outlier filtering problem from two aspects. First, a robust and efficient graph interaction model,is proposed, with the assumption that matches are correlated with each other rather than independently distributed. To this end, we construct a graph based on the local relationships of matches and formulate the outlier filtering task as a binary labeling energy minimization problem, where the pairwise term encodes the interaction between matches. We further show that this formulation can be solved globally by graph cut algorithm. Our new formulation always improves the performance of previous localitybased method without noticeable deterioration in processing time,adding a few milliseconds. Second, to construct a better graph structure, a robust and geometrically meaningful topology-aware relationship is developed to capture the topology relationship between matches. The two components in sum lead to topology interaction matching(TIM), an effective and efficient method for outlier filtering. Extensive experiments on several large and diverse datasets for multiple vision tasks including general feature matching, as well as relative pose estimation, homography and fundamental matrix estimation, loop-closure detection, and multi-modal image matching, demonstrate that our TIM is more competitive than current state-of-the-art methods, in terms of generality, efficiency, and effectiveness. The source code is publicly available at http://github.com/YifanLu2000/TIM.展开更多
Three-dimensional(3D)reconstruction based on aerial images has broad prospects,and feature matching is an important step of it.However,for high-resolution aerial images,there are usually problems such as long time,mis...Three-dimensional(3D)reconstruction based on aerial images has broad prospects,and feature matching is an important step of it.However,for high-resolution aerial images,there are usually problems such as long time,mismatching and sparse feature pairs using traditional algorithms.Therefore,an algorithm is proposed to realize fast,accurate and dense feature matching.The algorithm consists of four steps.Firstly,we achieve a balance between the feature matching time and the number of matching pairs by appropriately reducing the image resolution.Secondly,to realize further screening of the mismatches,a feature screening algorithm based on similarity judgment or local optimization is proposed.Thirdly,to make the algorithm more widely applicable,we combine the results of different algorithms to get dense results.Finally,all matching feature pairs in the low-resolution images are restored to the original images.Comparisons between the original algorithms and our algorithm show that the proposed algorithm can effectively reduce the matching time,screen out the mismatches,and improve the number of matches.展开更多
Feature matching is of significance in the field of computer vision.In this paper,a trifocal tensor based feature matching algorithm is proposed for three views,including a trinocular vision system.Initial matching po...Feature matching is of significance in the field of computer vision.In this paper,a trifocal tensor based feature matching algorithm is proposed for three views,including a trinocular vision system.Initial matching point-pairs can be determined according to generic matching algorithms,on which an initial trifocal tensor of three views can be confirmed.Then the initial matching point-pairs should be re-selected.Meanwhile,the trifocal tensor will be recomputed.Iteratively,the optimized trifocal tensor can be obtained.Compatible fundamental matrix of every two views can be determined.Furthermore,in the trinocular vision sensor,the trifocal tensor can be calculated based on the intrinsic parameter matrix of each camera.With the strict constraint provided by the trifocal tensor,feature matching results will be optimized.Experiments show that our proposed algorithm has the characteristics of feasibility and precision.展开更多
We present GeoGlue,a novel method using high-resolution UAV imagery for accurate feature matching,which is normally challenging due to the complicated scenes.Current feature detection methods are performed without gui...We present GeoGlue,a novel method using high-resolution UAV imagery for accurate feature matching,which is normally challenging due to the complicated scenes.Current feature detection methods are performed without guidance of geometric priors(e.g.,geometric lines),lacking enough attention given to salient geometric features which are indispensable for accurate matching due to their stable existence across views.In this work,geometric lines arefirstly detected by a CNN-based geometry detector(GD)which is pre-trained in a self-supervised manner through automatically generated images.Then,geometric lines are naturally vectorized based on GD and thus non-significant features can be disregarded as judged by their disordered geometric morphology.A graph attention network(GAT)is utilized forfinal feature matching,spanning across the image pair with geometric priors informed by GD.Comprehensive experiments show that GeoGlue outperforms other state-of-the-art methods in feature-matching accuracy and performance stability,achieving pose estimation with maximum rotation and translation errors under 1%in challenging scenes from benchmark datasets,Tanks&Temples and ETH3D.This study also proposes thefirst self-supervised deep-learning model for curved line detection,generating geometric priors for matching so that more attention is put on prominent features and improving the visual effect of 3D reconstruction.展开更多
A robust and eficient feature matching method is necessary for visual navigation in asteroid-landing missions.Based on the visual navigation framework and motion characteristics of asteroids,a robust and efficient tem...A robust and eficient feature matching method is necessary for visual navigation in asteroid-landing missions.Based on the visual navigation framework and motion characteristics of asteroids,a robust and efficient template feature matching method is proposed to adapt to feature distortion and scale change cases for visual navigation of asteroids.The proposed method is primarily based on a motion-constrained discriminative correlation filter(DCF).The prior information provided by the motion constraints between sequence images is used to provide a predicted search region for template feature matching.Additionally,some specific template feature samples are generated using the motion constraints for correlation filter learning,which is beneficial for training a scale and feature distortion adaptive correlation filter for accurate feature matching.Moreover,average peak-to-correlation energy(APCE)and jointly consistent measurements(JCMs)were used to eliminate false matching.Images captured by the Touch And Go Camera System(TAGCAMS)of the Bennu asteroid were used to evaluate the performance of the proposed method.In particular,both the robustness and accuracy of region matching and template center matching are evaluated.The qualitative and quantitative results illustrate the advancement of the proposed method in adapting to feature distortions and large-scale changes during spacecraft landing.展开更多
The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an...The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an inaccurate pose estimation or failed tracking.To address the challenge above,a new method of feature point matching is proposed in this paper,which combines the variable velocity model with the reverse optical flow method.First,the constant velocity model is extended to a new variable velocity model,and the expanded variable velocity model is used to provide the initial pixel shifting for the reverse optical flow method.Then the search range of feature points is accurately determined according to the results of the reverse optical flow method,thereby improving the accuracy and reliability of feature matching,with strengthened interframe tracking effects.Finally,we tested on TUM data set based on the RGB-D camera.Experimental results show that this method can reduce the probability of tracking failure and improve localization accuracy on SLAM(Simultaneous Localization and Mapping)systems.Compared with the traditional ORB-SLAM2,the test error of this method on each sequence in the TUM data set is significantly reduced,and the root mean square error is only 63.8%of the original system under the optimal condition.展开更多
Augmented solar images were used to research the adaptability of four representative image extraction and matching algorithms in space weather domain.These include the scale-invariant feature transform algorithm,speed...Augmented solar images were used to research the adaptability of four representative image extraction and matching algorithms in space weather domain.These include the scale-invariant feature transform algorithm,speeded-up robust features algorithm,binary robust invariant scalable keypoints algorithm,and oriented fast and rotated brief algorithm.The performance of these algorithms was estimated in terms of matching accuracy,feature point richness,and running time.The experiment result showed that no algorithm achieved high accuracy while keeping low running time,and all algorithms are not suitable for image feature extraction and matching of augmented solar images.To solve this problem,an improved method was proposed by using two-frame matching to utilize the accuracy advantage of the scale-invariant feature transform algorithm and the speed advantage of the oriented fast and rotated brief algorithm.Furthermore,our method and the four representative algorithms were applied to augmented solar images.Our application experiments proved that our method achieved a similar high recognition rate to the scale-invariant feature transform algorithm which is significantly higher than other algorithms.Our method also obtained a similar low running time to the oriented fast and rotated brief algorithm,which is significantly lower than other algorithms.展开更多
Domain adaptation and adversarial networks are two main approaches for transfer learning.Domain adaptation methods match the mean values of source and target domains,which requires a very large batch size during train...Domain adaptation and adversarial networks are two main approaches for transfer learning.Domain adaptation methods match the mean values of source and target domains,which requires a very large batch size during training.However,adversarial networks are usually unstable when training.In this paper,we propose a joint method of feature matching and adversarial networks to reduce domain discrepancy and mine domaininvariant features from the local and global aspects.At the same time,our method improves the stability of training.Moreover,the method is embedded into a unified convolutional neural network that can be easily optimized by gradient descent.Experimental results show that our joint method can yield the state-of-the-art results on three common public datasets.展开更多
A critical component of visual simultaneous localization and mapping is loop closure detection(LCD),an operation judging whether a robot has come to a pre-visited area.Concretely,given a query image(i.e.,the latest vi...A critical component of visual simultaneous localization and mapping is loop closure detection(LCD),an operation judging whether a robot has come to a pre-visited area.Concretely,given a query image(i.e.,the latest view observed by the robot),it proceeds by first exploring images with similar semantic information,followed by solving the relative relationship between candidate pairs in the 3D space.In this work,a novel appearance-based LCD system is proposed.Specifically,candidate frame selection is conducted via the combination of Superfeatures and aggregated selective match kernel(ASMK).We incorporate an incremental strategy into the vanilla ASMK to make it applied in the LCD task.It is demonstrated that this setting is memory-wise efficient and can achieve remarkable performance.To dig up consistent geometry between image pairs during loop closure verification,we propose a simple yet surprisingly effective feature matching algorithm,termed locality preserving matching with global consensus(LPM-GC).The major objective of LPM-GC is to retain the local neighborhood information of true feature correspondences between candidate pairs,where a global constraint is further designed to effectively remove false correspondences in challenging sceneries,e.g.,containing numerous repetitive structures.Meanwhile,we derive a closed-form solution that enables our approach to provide reliable correspondences within only a few milliseconds.The performance of the proposed approach has been experimentally evaluated on ten publicly available and challenging datasets.Results show that our method can achieve better performance over the state-of-the-art in both feature matching and LCD tasks.We have released our code of LPM-GC at https://github.com/jiayi-ma/LPM-GC.展开更多
Individual identification of dairy cows is the prerequisite for automatic analysis and intelligent perception of dairy cows'behavior.At present,individual identification of dairy cows based on deep convolutional n...Individual identification of dairy cows is the prerequisite for automatic analysis and intelligent perception of dairy cows'behavior.At present,individual identification of dairy cows based on deep convolutional neural network had the disadvantages in prolonged training at the additions of new cows samples.Therefore,a cow individual identification framework was proposed based on deep feature extraction and matching,and the individual identification of dairy cows based on this framework could avoid repeated training.Firstly,the trained convolutional neural network model was used as the feature extractor;secondly,the feature extraction was used to extract features and stored the features into the template feature library to complete the enrollment;finally,the identifies of dairy cows were identified.Based on this framework,when new cows joined the herd,enrollment could be completed quickly.In order to evaluate the application performance of this method in closed-set and open-set individual identification of dairy cows,back images of 524 cows were collected,among which the back images of 150 cows were selected as the training data to train feature extractor.The data of the remaining 374 cows were used to generate the template data set and the data to be identified.The experiment results showed that in the closed-set individual identification of dairy cows,the highest identification accuracy of top-1 was 99.73%,the highest identification accuracy from top-2 to top-5 was 100%,and the identification time of a single cow was 0.601 s,this method was verified to be effective.In the open-set individual identification of dairy cows,the recall was 90.38%,and the accuracy was 89.46%.When false accept rate(FAR)=0.05,true accept rate(TAR)=84.07%,this method was verified that the application had certain research value in open-set individual identification of dairy cows,which provided a certain idea for the application of individual identification in the field of intelligent animal husbandry.展开更多
Background Image matching is crucial in numerous computer vision tasks such as 3D reconstruction and simultaneous visual localization and mapping.The accuracy of the matching significantly impacted subsequent studies....Background Image matching is crucial in numerous computer vision tasks such as 3D reconstruction and simultaneous visual localization and mapping.The accuracy of the matching significantly impacted subsequent studies.Because of their local similarity,when image pairs contain comparable patterns but feature pairs are positioned differently,incorrect recognition can occur as global motion consistency is disregarded.Methods This study proposes an image-matching filtering algorithm based on global motion consistency.It can be used as a subsequent matching filter for the initial matching results generated by other matching algorithms based on the principle of motion smoothness.A particular matching algorithm can first be used to perform the initial matching;then,the rotation and movement information of the global feature vectors are combined to effectively identify outlier matches.The principle is that if the matching result is accurate,the feature vectors formed by any matched point should have similar rotation angles and moving distances.Thus,global motion direction and global motion distance consistencies were used to reject outliers caused by similar patterns in different locations.Results Four datasets were used to test the effectiveness of the proposed method.Three datasets with similar patterns in different locations were used to test the results for similar images that could easily be incorrectly matched by other algorithms,and one commonly used dataset was used to test the results for the general image-matching problem.The experimental results suggest that the proposed method is more accurate than other state-of-the-art algorithms in identifying mismatches in the initial matching set.Conclusions The proposed outlier rejection matching method can significantly improve the matching accuracy for similar images with locally similar feature pairs in different locations and can provide more accurate matching results for subsequent computer vision tasks.展开更多
A new method based on adaptive Hessian matrix threshold of finding key SRUF( speeded up robust features) features is proposed and is applied to an unmanned vehicle for its dynamic object recognition and guided navigat...A new method based on adaptive Hessian matrix threshold of finding key SRUF( speeded up robust features) features is proposed and is applied to an unmanned vehicle for its dynamic object recognition and guided navigation. First,the object recognition algorithm based on SURF feature matching for unmanned vehicle guided navigation is introduced. Then,the standard local invariant feature extraction algorithm SRUF is analyzed,the Hessian Metrix is especially discussed,and a method of adaptive Hessian threshold is proposed which is based on correct matching point pairs threshold feedback under a close loop frame. At last,different dynamic object recognition experiments under different weather light conditions are discussed. The experimental result shows that the key SURF feature abstract algorithm and the dynamic object recognition method can be used for unmanned vehicle systems.展开更多
In recent years,with the massive growth of image data,how to match the image required by users quickly and efficiently becomes a challenge.Compared with single-view feature,multi-view feature is more accurate to descr...In recent years,with the massive growth of image data,how to match the image required by users quickly and efficiently becomes a challenge.Compared with single-view feature,multi-view feature is more accurate to describe image information.The advantages of hash method in reducing data storage and improving efficiency also make us study how to effectively apply to large-scale image retrieval.In this paper,a hash algorithm of multi-index image retrieval based on multi-view feature coding is proposed.By learning the data correlation between different views,this algorithm uses multi-view data with deeper level image semantics to achieve better retrieval results.This algorithm uses a quantitative hash method to generate binary sequences,and uses the hash code generated by the association features to construct database inverted index files,so as to reduce the memory burden and promote the efficient matching.In order to reduce the matching error of hash code and ensure the retrieval accuracy,this algorithm uses inverted multi-index structure instead of single-index structure.Compared with other advanced image retrieval method,this method has better retrieval performance.展开更多
The task of indoor visual localization, utilizing camera visual information for user pose calculation, was a core component of Augmented Reality (AR) and Simultaneous Localization and Mapping (SLAM). Existing indoor l...The task of indoor visual localization, utilizing camera visual information for user pose calculation, was a core component of Augmented Reality (AR) and Simultaneous Localization and Mapping (SLAM). Existing indoor localization technologies generally used scene-specific 3D representations or were trained on specific datasets, making it challenging to balance accuracy and cost when applied to new scenes. Addressing this issue, this paper proposed a universal indoor visual localization method based on efficient image retrieval. Initially, a Multi-Layer Perceptron (MLP) was employed to aggregate features from intermediate layers of a convolutional neural network, obtaining a global representation of the image. This approach ensured accurate and rapid retrieval of reference images. Subsequently, a new mechanism using Random Sample Consensus (RANSAC) was designed to resolve relative pose ambiguity caused by the essential matrix decomposition based on the five-point method. Finally, the absolute pose of the queried user image was computed, thereby achieving indoor user pose estimation. The proposed indoor localization method was characterized by its simplicity, flexibility, and excellent cross-scene generalization. Experimental results demonstrated a positioning error of 0.09 m and 2.14° on the 7Scenes dataset, and 0.15 m and 6.37° on the 12Scenes dataset. These results convincingly illustrated the outstanding performance of the proposed indoor localization method.展开更多
The matching of local descriptors represents at this moment a key tool in computer vision, with a wide variety of methods designed for tasks such as image classification, object recognition and tracking, image stitchi...The matching of local descriptors represents at this moment a key tool in computer vision, with a wide variety of methods designed for tasks such as image classification, object recognition and tracking, image stitching, or data mining relying on it. Local feature description techniques are usually developed so as to provide invariance to photometric variations specific to the acquisition of natural images, but are nonetheless used in association with biomedical imaging as well. It has been previously shown that the matching of gradient based descriptors is affected by image modifications specific to Confocal Scanning Laser Microscopy (CSLM). In this paper we extend our previous work in this direction and show how specific acquisition or post-processing methods alleviate or accentuate this problem.展开更多
In nowadays society,the safety of the elderly population is becoming a pressing concern,especially for those who live alone.There might be daily risks such as accidental falling or treatment attack on them.Aiming at t...In nowadays society,the safety of the elderly population is becoming a pressing concern,especially for those who live alone.There might be daily risks such as accidental falling or treatment attack on them.Aiming at these problems,indoor positioning could be a critical way to monitor their states.With the rapidly development of the imaging techniques,wearable and portable cameras are very popular,which could be set on human individual.And in view of the advantages of the visual positioning,the authors propose a binocular visual positioning algorithm to real-timely locate the elderly indoor.In this paper,the imaging model has been established with the corrected image data from the binocular camera;then feature extraction has been completed to provide reference to adjacent image matching based on the binary robust independent elementary feature(BRIEF)descriptor,finally the camera movement and the states of the elderly have been estimated to distinguish their falling risk.In the experiments,the real-sense D435i sensors were adopted as the binocular cameras to obtain indoor images,and three experimental scenarios have been carried out to test the proposed method.The results show that the proposed algorithm can effectively locate the elderly indoor and improve the real-time monitoring capability.展开更多
A method is proposed for the prospecting prediction of subsurface mineral deposits based on soil geochemistry data and a deep convolutional neural network model.This method uses three techniques(window offset,scaling,...A method is proposed for the prospecting prediction of subsurface mineral deposits based on soil geochemistry data and a deep convolutional neural network model.This method uses three techniques(window offset,scaling,and rotation)to enhance the number of training data for the model.A window area is used to extract the spatial distribution characteristics of soil geochemistry and measure their correspondence with the occurrence of known subsurface deposits.Prospecting prediction is achieved by matching the characteristics of the window area of an unknown area with the relationships established in the known area.This method can efficiently predict mineral prospective areas where there are few ore deposits used for generating the training dataset,meaning that the deep-learning method can be effectively used for deposit prospecting prediction.Using soil active geochemical measurement data,this method was applied in the Daqiao area,Gansu Province,for which seven favorable gold prospecting target areas were predicted.The Daqiao orogenic gold deposit of latest Jurassic and Early Jurassic age in the southern domain has more than 105 t of gold resources at an average grade of 3-4 g/t.In 2020,the project team drilled and verified the K prediction area,and found 66 m gold mineralized bodies.The new method should be applicable to prospecting prediction using conventional geochemical data in other areas.展开更多
Current research of binocular vision systems mainly need to resolve the camera’s intrinsic parameters before the reconstruction of three-dimensional(3D)objects.The classical Zhang’calibration is hardly to calculate ...Current research of binocular vision systems mainly need to resolve the camera’s intrinsic parameters before the reconstruction of three-dimensional(3D)objects.The classical Zhang’calibration is hardly to calculate all errors caused by perspective distortion and lens distortion.Also,the image-matching algorithm of the binocular vision system still needs to be improved to accelerate the reconstruction speed of welding pool surfaces.In this paper,a preset coordinate system was utilized for camera calibration instead of Zhang’calibration.The binocular vision system was modified to capture images of welding pool surfaces by suppressing the strong arc interference during gas metal arc welding.Combining and improving the algorithms of speeded up robust features,binary robust invariant scalable keypoints,and KAZE,the feature information of points(i.e.,RGB values,pixel coordinates)was extracted as the feature vector of the welding pool surface.Based on the characteristics of the welding images,a mismatch-elimination algorithm was developed to increase the accuracy of image-matching algorithms.The world coordinates of matching feature points were calculated to reconstruct the 3D shape of the welding pool surface.The effectiveness and accuracy of the reconstruction of welding pool surfaces were verified by experimental results.This research proposes the development of binocular vision algorithms that can reconstruct the surface of welding pools accurately to realize intelligent welding control systems in the future.展开更多
We present a real-time monocular simultaneous localization and mapping(SLAM)system with a new distributed structure for multi-UAV collaboration tasks.The system is different from other general SLAM systems in two aspe...We present a real-time monocular simultaneous localization and mapping(SLAM)system with a new distributed structure for multi-UAV collaboration tasks.The system is different from other general SLAM systems in two aspects:First,it does not aim to build a global map,but to estimate the latest relative position between nearby vehicles;Second,there is no centralized structure in the proposed system,and each vehicle owns an individual metric map and an ego-motion estimator to obtain the relative position between its own map and the neighboring vehicles'.To realize the above characteristics in real time,we demonstrate an innovative feature description and matching algorithm to avoid catastrophic expansion of feature point matching workload due to the increased number of UAVs.Based on the hash and principal component analysis,the matching time complexity of this algorithm can be reduced from 0(logN)to 0(1).To evaluate the performance,the algorithm is verified on the acknowledged multi-view stereo benchmark dataset,and excellent results are obtained.Finally,through the simulation and real flight experiments,this improved SLAM system with the proposed algorithm is validated.展开更多
Simultaneous location and mapping(SLAM)plays the crucial role in VR/AR application,autonomous robotics navigation,UAV remote control,etc.The traditional SLAM is not good at handle the data acquired by camera with fast...Simultaneous location and mapping(SLAM)plays the crucial role in VR/AR application,autonomous robotics navigation,UAV remote control,etc.The traditional SLAM is not good at handle the data acquired by camera with fast movement or severe jittering,and the efficiency need to be improved.The paper proposes an improved SLAM algorithm,which mainly improves the real-time performance of classical SLAM algorithm,applies KDtree for efficient organizing feature points,and accelerates the feature points correspondence building.Moreover,the background map reconstruction thread is optimized,the SLAM parallel computation ability is increased.The color images experiments demonstrate that the improved SLAM algorithm holds better realtime performance than the classical SLAM.展开更多
基金supported by the National Natural Science Foundation of China (62276192)。
文摘Feature matching plays a key role in computer vision. However, due to the limitations of the descriptors, the putative matches are inevitably contaminated by massive outliers.This paper attempts to tackle the outlier filtering problem from two aspects. First, a robust and efficient graph interaction model,is proposed, with the assumption that matches are correlated with each other rather than independently distributed. To this end, we construct a graph based on the local relationships of matches and formulate the outlier filtering task as a binary labeling energy minimization problem, where the pairwise term encodes the interaction between matches. We further show that this formulation can be solved globally by graph cut algorithm. Our new formulation always improves the performance of previous localitybased method without noticeable deterioration in processing time,adding a few milliseconds. Second, to construct a better graph structure, a robust and geometrically meaningful topology-aware relationship is developed to capture the topology relationship between matches. The two components in sum lead to topology interaction matching(TIM), an effective and efficient method for outlier filtering. Extensive experiments on several large and diverse datasets for multiple vision tasks including general feature matching, as well as relative pose estimation, homography and fundamental matrix estimation, loop-closure detection, and multi-modal image matching, demonstrate that our TIM is more competitive than current state-of-the-art methods, in terms of generality, efficiency, and effectiveness. The source code is publicly available at http://github.com/YifanLu2000/TIM.
基金This work was supported by the Equipment Pre-Research Foundation of China(6140001020310).
文摘Three-dimensional(3D)reconstruction based on aerial images has broad prospects,and feature matching is an important step of it.However,for high-resolution aerial images,there are usually problems such as long time,mismatching and sparse feature pairs using traditional algorithms.Therefore,an algorithm is proposed to realize fast,accurate and dense feature matching.The algorithm consists of four steps.Firstly,we achieve a balance between the feature matching time and the number of matching pairs by appropriately reducing the image resolution.Secondly,to realize further screening of the mismatches,a feature screening algorithm based on similarity judgment or local optimization is proposed.Thirdly,to make the algorithm more widely applicable,we combine the results of different algorithms to get dense results.Finally,all matching feature pairs in the low-resolution images are restored to the original images.Comparisons between the original algorithms and our algorithm show that the proposed algorithm can effectively reduce the matching time,screen out the mismatches,and improve the number of matches.
基金Supported by the Key Research and Development Programs of Shandong Province(2018GGX101040)Applied Basic Research Programs of Qingdao(18-2-2-62-jch)。
文摘Feature matching is of significance in the field of computer vision.In this paper,a trifocal tensor based feature matching algorithm is proposed for three views,including a trinocular vision system.Initial matching point-pairs can be determined according to generic matching algorithms,on which an initial trifocal tensor of three views can be confirmed.Then the initial matching point-pairs should be re-selected.Meanwhile,the trifocal tensor will be recomputed.Iteratively,the optimized trifocal tensor can be obtained.Compatible fundamental matrix of every two views can be determined.Furthermore,in the trinocular vision sensor,the trifocal tensor can be calculated based on the intrinsic parameter matrix of each camera.With the strict constraint provided by the trifocal tensor,feature matching results will be optimized.Experiments show that our proposed algorithm has the characteristics of feasibility and precision.
基金supported by the Strategic Priority Research Program of the Chinese Academy of Sciences[Grant No.XDA19080101]the Director Fund of the International Research Center of Big Data for Sus-tainable Development Goals[Grant No.CBAS2022DF015]+1 种基金the National Natural Science Foundation of China[Grant No.41901328 and 41974108]the National Key Research and Development Program of China[Grant No.2022YFC3800700].
文摘We present GeoGlue,a novel method using high-resolution UAV imagery for accurate feature matching,which is normally challenging due to the complicated scenes.Current feature detection methods are performed without guidance of geometric priors(e.g.,geometric lines),lacking enough attention given to salient geometric features which are indispensable for accurate matching due to their stable existence across views.In this work,geometric lines arefirstly detected by a CNN-based geometry detector(GD)which is pre-trained in a self-supervised manner through automatically generated images.Then,geometric lines are naturally vectorized based on GD and thus non-significant features can be disregarded as judged by their disordered geometric morphology.A graph attention network(GAT)is utilized forfinal feature matching,spanning across the image pair with geometric priors informed by GD.Comprehensive experiments show that GeoGlue outperforms other state-of-the-art methods in feature-matching accuracy and performance stability,achieving pose estimation with maximum rotation and translation errors under 1%in challenging scenes from benchmark datasets,Tanks&Temples and ETH3D.This study also proposes thefirst self-supervised deep-learning model for curved line detection,generating geometric priors for matching so that more attention is put on prominent features and improving the visual effect of 3D reconstruction.
基金funded by the National Natural Science Foundation of China under Grant Nos.41822106 and 42101447the Dawn Scholar of Shanghai Program under Grant No.18SG22+2 种基金the Science and Technology on Aerospace Flight Dynamics Laboratory,China,under Grant No.KGJ6142210110305State Key Laboratory of Disaster Reduction in Civil Engineering under Grant No.SLDRCE19-B-35Fundamental Research Funds for the Central Universities of China.
文摘A robust and eficient feature matching method is necessary for visual navigation in asteroid-landing missions.Based on the visual navigation framework and motion characteristics of asteroids,a robust and efficient template feature matching method is proposed to adapt to feature distortion and scale change cases for visual navigation of asteroids.The proposed method is primarily based on a motion-constrained discriminative correlation filter(DCF).The prior information provided by the motion constraints between sequence images is used to provide a predicted search region for template feature matching.Additionally,some specific template feature samples are generated using the motion constraints for correlation filter learning,which is beneficial for training a scale and feature distortion adaptive correlation filter for accurate feature matching.Moreover,average peak-to-correlation energy(APCE)and jointly consistent measurements(JCMs)were used to eliminate false matching.Images captured by the Touch And Go Camera System(TAGCAMS)of the Bennu asteroid were used to evaluate the performance of the proposed method.In particular,both the robustness and accuracy of region matching and template center matching are evaluated.The qualitative and quantitative results illustrate the advancement of the proposed method in adapting to feature distortions and large-scale changes during spacecraft landing.
基金This work was supported by The National Natural Science Foundation of China under Grant No.61304205 and NO.61502240The Natural Science Foundation of Jiangsu Province under Grant No.BK20191401 and No.BK20201136Postgraduate Research&Practice Innovation Program of Jiangsu Province under Grant No.SJCX21_0364 and No.SJCX21_0363.
文摘The ORB-SLAM2 based on the constant velocity model is difficult to determine the search window of the reprojection of map points when the objects are in variable velocity motion,which leads to a false matching,with an inaccurate pose estimation or failed tracking.To address the challenge above,a new method of feature point matching is proposed in this paper,which combines the variable velocity model with the reverse optical flow method.First,the constant velocity model is extended to a new variable velocity model,and the expanded variable velocity model is used to provide the initial pixel shifting for the reverse optical flow method.Then the search range of feature points is accurately determined according to the results of the reverse optical flow method,thereby improving the accuracy and reliability of feature matching,with strengthened interframe tracking effects.Finally,we tested on TUM data set based on the RGB-D camera.Experimental results show that this method can reduce the probability of tracking failure and improve localization accuracy on SLAM(Simultaneous Localization and Mapping)systems.Compared with the traditional ORB-SLAM2,the test error of this method on each sequence in the TUM data set is significantly reduced,and the root mean square error is only 63.8%of the original system under the optimal condition.
基金Supported by the Key Research Program of the Chinese Academy of Sciences(ZDRE-KT-2021-3)。
文摘Augmented solar images were used to research the adaptability of four representative image extraction and matching algorithms in space weather domain.These include the scale-invariant feature transform algorithm,speeded-up robust features algorithm,binary robust invariant scalable keypoints algorithm,and oriented fast and rotated brief algorithm.The performance of these algorithms was estimated in terms of matching accuracy,feature point richness,and running time.The experiment result showed that no algorithm achieved high accuracy while keeping low running time,and all algorithms are not suitable for image feature extraction and matching of augmented solar images.To solve this problem,an improved method was proposed by using two-frame matching to utilize the accuracy advantage of the scale-invariant feature transform algorithm and the speed advantage of the oriented fast and rotated brief algorithm.Furthermore,our method and the four representative algorithms were applied to augmented solar images.Our application experiments proved that our method achieved a similar high recognition rate to the scale-invariant feature transform algorithm which is significantly higher than other algorithms.Our method also obtained a similar low running time to the oriented fast and rotated brief algorithm,which is significantly lower than other algorithms.
基金the Aerospace Science and Technology Foundation(No.20115557007)the National Natural Science Foundation of China(No.61673262)the Military Science and Technology Foundation of China(No.18-H863-03-ZT-001-006-06)
文摘Domain adaptation and adversarial networks are two main approaches for transfer learning.Domain adaptation methods match the mean values of source and target domains,which requires a very large batch size during training.However,adversarial networks are usually unstable when training.In this paper,we propose a joint method of feature matching and adversarial networks to reduce domain discrepancy and mine domaininvariant features from the local and global aspects.At the same time,our method improves the stability of training.Moreover,the method is embedded into a unified convolutional neural network that can be easily optimized by gradient descent.Experimental results show that our joint method can yield the state-of-the-art results on three common public datasets.
基金supported by the Key Research and Development Program of Hubei Province(2020BAB113)。
文摘A critical component of visual simultaneous localization and mapping is loop closure detection(LCD),an operation judging whether a robot has come to a pre-visited area.Concretely,given a query image(i.e.,the latest view observed by the robot),it proceeds by first exploring images with similar semantic information,followed by solving the relative relationship between candidate pairs in the 3D space.In this work,a novel appearance-based LCD system is proposed.Specifically,candidate frame selection is conducted via the combination of Superfeatures and aggregated selective match kernel(ASMK).We incorporate an incremental strategy into the vanilla ASMK to make it applied in the LCD task.It is demonstrated that this setting is memory-wise efficient and can achieve remarkable performance.To dig up consistent geometry between image pairs during loop closure verification,we propose a simple yet surprisingly effective feature matching algorithm,termed locality preserving matching with global consensus(LPM-GC).The major objective of LPM-GC is to retain the local neighborhood information of true feature correspondences between candidate pairs,where a global constraint is further designed to effectively remove false correspondences in challenging sceneries,e.g.,containing numerous repetitive structures.Meanwhile,we derive a closed-form solution that enables our approach to provide reliable correspondences within only a few milliseconds.The performance of the proposed approach has been experimentally evaluated on ten publicly available and challenging datasets.Results show that our method can achieve better performance over the state-of-the-art in both feature matching and LCD tasks.We have released our code of LPM-GC at https://github.com/jiayi-ma/LPM-GC.
基金Supported by the National Key Research and Development Program of China(2019YFE0125600)China Agriculture Research System(CARS-36)。
文摘Individual identification of dairy cows is the prerequisite for automatic analysis and intelligent perception of dairy cows'behavior.At present,individual identification of dairy cows based on deep convolutional neural network had the disadvantages in prolonged training at the additions of new cows samples.Therefore,a cow individual identification framework was proposed based on deep feature extraction and matching,and the individual identification of dairy cows based on this framework could avoid repeated training.Firstly,the trained convolutional neural network model was used as the feature extractor;secondly,the feature extraction was used to extract features and stored the features into the template feature library to complete the enrollment;finally,the identifies of dairy cows were identified.Based on this framework,when new cows joined the herd,enrollment could be completed quickly.In order to evaluate the application performance of this method in closed-set and open-set individual identification of dairy cows,back images of 524 cows were collected,among which the back images of 150 cows were selected as the training data to train feature extractor.The data of the remaining 374 cows were used to generate the template data set and the data to be identified.The experiment results showed that in the closed-set individual identification of dairy cows,the highest identification accuracy of top-1 was 99.73%,the highest identification accuracy from top-2 to top-5 was 100%,and the identification time of a single cow was 0.601 s,this method was verified to be effective.In the open-set individual identification of dairy cows,the recall was 90.38%,and the accuracy was 89.46%.When false accept rate(FAR)=0.05,true accept rate(TAR)=84.07%,this method was verified that the application had certain research value in open-set individual identification of dairy cows,which provided a certain idea for the application of individual identification in the field of intelligent animal husbandry.
基金Supported by the Natural Science Foundation of China(62072388,62276146)the Industry Guidance Project Foundation of Science technology Bureau of Fujian province(2020H0047)+2 种基金the Natural Science Foundation of Science Technology Bureau of Fujian province(2019J01601)the Creation Fund project of Science Technology Bureau of Fujian province(JAT190596)Putian University Research Project(2022034)。
文摘Background Image matching is crucial in numerous computer vision tasks such as 3D reconstruction and simultaneous visual localization and mapping.The accuracy of the matching significantly impacted subsequent studies.Because of their local similarity,when image pairs contain comparable patterns but feature pairs are positioned differently,incorrect recognition can occur as global motion consistency is disregarded.Methods This study proposes an image-matching filtering algorithm based on global motion consistency.It can be used as a subsequent matching filter for the initial matching results generated by other matching algorithms based on the principle of motion smoothness.A particular matching algorithm can first be used to perform the initial matching;then,the rotation and movement information of the global feature vectors are combined to effectively identify outlier matches.The principle is that if the matching result is accurate,the feature vectors formed by any matched point should have similar rotation angles and moving distances.Thus,global motion direction and global motion distance consistencies were used to reject outliers caused by similar patterns in different locations.Results Four datasets were used to test the effectiveness of the proposed method.Three datasets with similar patterns in different locations were used to test the results for similar images that could easily be incorrectly matched by other algorithms,and one commonly used dataset was used to test the results for the general image-matching problem.The experimental results suggest that the proposed method is more accurate than other state-of-the-art algorithms in identifying mismatches in the initial matching set.Conclusions The proposed outlier rejection matching method can significantly improve the matching accuracy for similar images with locally similar feature pairs in different locations and can provide more accurate matching results for subsequent computer vision tasks.
基金Supported by the National Natural Science Foundation of China(61103157)Beijing Municipal Education Commission Project(SQKM201311417010)
文摘A new method based on adaptive Hessian matrix threshold of finding key SRUF( speeded up robust features) features is proposed and is applied to an unmanned vehicle for its dynamic object recognition and guided navigation. First,the object recognition algorithm based on SURF feature matching for unmanned vehicle guided navigation is introduced. Then,the standard local invariant feature extraction algorithm SRUF is analyzed,the Hessian Metrix is especially discussed,and a method of adaptive Hessian threshold is proposed which is based on correct matching point pairs threshold feedback under a close loop frame. At last,different dynamic object recognition experiments under different weather light conditions are discussed. The experimental result shows that the key SURF feature abstract algorithm and the dynamic object recognition method can be used for unmanned vehicle systems.
基金supported in part by the National Natural Science Foundation of China under Grant 61772561,author J.Q,http://www.nsfc.gov.cn/in part by the Key Research and Development Plan of Hunan Province under Grant 2018NK2012,author J.Q,http://kjt.hunan.gov.cn/+7 种基金in part by the Key Research and Development Plan of Hunan Province under Grant 2019SK2022,author Y.T,http://kjt.hunan.gov.cn/in part by the Science Research Projects of Hunan Provincial Education Department under Grant 18A174,author X.X,http://kxjsc.gov.hnedu.cn/in part by the Science Research Projects of Hunan Provincial Education Department under Grant 19B584,author Y.T,http://kxjsc.gov.hnedu.cn/in part by the Degree&Postgraduate Education Reform Project of Hunan Province under Grant 2019JGYB154,author J.Q,http://xwb.gov.hnedu.cn/in part by the Postgraduate Excellent teaching team Project of Hunan Province under Grant[2019]370-133,author J.Q,http://xwb.gov.hnedu.cn/in part by the Postgraduate Education and Teaching Reform Project of Central South University of Forestry&Technology under Grant 2019JG013,author X.X,http://jwc.csuft.edu.cn/in part by the Natural Science Foundation of Hunan Province(No.2020JJ4140),author Y.T,http://kjt.hunan.gov.cn/in part by the Natural Science Foundation of Hunan Province(No.2020JJ4141),author X.X,http://kjt.hunan.gov.cn/.
文摘In recent years,with the massive growth of image data,how to match the image required by users quickly and efficiently becomes a challenge.Compared with single-view feature,multi-view feature is more accurate to describe image information.The advantages of hash method in reducing data storage and improving efficiency also make us study how to effectively apply to large-scale image retrieval.In this paper,a hash algorithm of multi-index image retrieval based on multi-view feature coding is proposed.By learning the data correlation between different views,this algorithm uses multi-view data with deeper level image semantics to achieve better retrieval results.This algorithm uses a quantitative hash method to generate binary sequences,and uses the hash code generated by the association features to construct database inverted index files,so as to reduce the memory burden and promote the efficient matching.In order to reduce the matching error of hash code and ensure the retrieval accuracy,this algorithm uses inverted multi-index structure instead of single-index structure.Compared with other advanced image retrieval method,this method has better retrieval performance.
文摘The task of indoor visual localization, utilizing camera visual information for user pose calculation, was a core component of Augmented Reality (AR) and Simultaneous Localization and Mapping (SLAM). Existing indoor localization technologies generally used scene-specific 3D representations or were trained on specific datasets, making it challenging to balance accuracy and cost when applied to new scenes. Addressing this issue, this paper proposed a universal indoor visual localization method based on efficient image retrieval. Initially, a Multi-Layer Perceptron (MLP) was employed to aggregate features from intermediate layers of a convolutional neural network, obtaining a global representation of the image. This approach ensured accurate and rapid retrieval of reference images. Subsequently, a new mechanism using Random Sample Consensus (RANSAC) was designed to resolve relative pose ambiguity caused by the essential matrix decomposition based on the five-point method. Finally, the absolute pose of the queried user image was computed, thereby achieving indoor user pose estimation. The proposed indoor localization method was characterized by its simplicity, flexibility, and excellent cross-scene generalization. Experimental results demonstrated a positioning error of 0.09 m and 2.14° on the 7Scenes dataset, and 0.15 m and 6.37° on the 12Scenes dataset. These results convincingly illustrated the outstanding performance of the proposed indoor localization method.
基金The UEFISCDIPN-II-PT-PCCA-2011-3.2-1162 Research Grant The CRUS SCIEX NMS-CH Fellowship nr. 12.135
文摘The matching of local descriptors represents at this moment a key tool in computer vision, with a wide variety of methods designed for tasks such as image classification, object recognition and tracking, image stitching, or data mining relying on it. Local feature description techniques are usually developed so as to provide invariance to photometric variations specific to the acquisition of natural images, but are nonetheless used in association with biomedical imaging as well. It has been previously shown that the matching of gradient based descriptors is affected by image modifications specific to Confocal Scanning Laser Microscopy (CSLM). In this paper we extend our previous work in this direction and show how specific acquisition or post-processing methods alleviate or accentuate this problem.
基金This work was supported by the National Natural Science Foundation of China(No.61803203).
文摘In nowadays society,the safety of the elderly population is becoming a pressing concern,especially for those who live alone.There might be daily risks such as accidental falling or treatment attack on them.Aiming at these problems,indoor positioning could be a critical way to monitor their states.With the rapidly development of the imaging techniques,wearable and portable cameras are very popular,which could be set on human individual.And in view of the advantages of the visual positioning,the authors propose a binocular visual positioning algorithm to real-timely locate the elderly indoor.In this paper,the imaging model has been established with the corrected image data from the binocular camera;then feature extraction has been completed to provide reference to adjacent image matching based on the binary robust independent elementary feature(BRIEF)descriptor,finally the camera movement and the states of the elderly have been estimated to distinguish their falling risk.In the experiments,the real-sense D435i sensors were adopted as the binocular cameras to obtain indoor images,and three experimental scenarios have been carried out to test the proposed method.The results show that the proposed algorithm can effectively locate the elderly indoor and improve the real-time monitoring capability.
基金funded by a pilot project entitled“Deep Geological Survey of Benxi-Linjiang Area”(1212011220247)of the 3D Geological Mapping and Deep Geological Survey of China Geological Survey。
文摘A method is proposed for the prospecting prediction of subsurface mineral deposits based on soil geochemistry data and a deep convolutional neural network model.This method uses three techniques(window offset,scaling,and rotation)to enhance the number of training data for the model.A window area is used to extract the spatial distribution characteristics of soil geochemistry and measure their correspondence with the occurrence of known subsurface deposits.Prospecting prediction is achieved by matching the characteristics of the window area of an unknown area with the relationships established in the known area.This method can efficiently predict mineral prospective areas where there are few ore deposits used for generating the training dataset,meaning that the deep-learning method can be effectively used for deposit prospecting prediction.Using soil active geochemical measurement data,this method was applied in the Daqiao area,Gansu Province,for which seven favorable gold prospecting target areas were predicted.The Daqiao orogenic gold deposit of latest Jurassic and Early Jurassic age in the southern domain has more than 105 t of gold resources at an average grade of 3-4 g/t.In 2020,the project team drilled and verified the K prediction area,and found 66 m gold mineralized bodies.The new method should be applicable to prospecting prediction using conventional geochemical data in other areas.
基金Supported by National Natural Science Foundation of China(Grant No.51775313)Major Program of Shandong Province Natural Science Foundation(Grant No.ZR2018ZC1760)Young Scholars Program of Shandong University(Grant No.2017WLJH24).
文摘Current research of binocular vision systems mainly need to resolve the camera’s intrinsic parameters before the reconstruction of three-dimensional(3D)objects.The classical Zhang’calibration is hardly to calculate all errors caused by perspective distortion and lens distortion.Also,the image-matching algorithm of the binocular vision system still needs to be improved to accelerate the reconstruction speed of welding pool surfaces.In this paper,a preset coordinate system was utilized for camera calibration instead of Zhang’calibration.The binocular vision system was modified to capture images of welding pool surfaces by suppressing the strong arc interference during gas metal arc welding.Combining and improving the algorithms of speeded up robust features,binary robust invariant scalable keypoints,and KAZE,the feature information of points(i.e.,RGB values,pixel coordinates)was extracted as the feature vector of the welding pool surface.Based on the characteristics of the welding images,a mismatch-elimination algorithm was developed to increase the accuracy of image-matching algorithms.The world coordinates of matching feature points were calculated to reconstruct the 3D shape of the welding pool surface.The effectiveness and accuracy of the reconstruction of welding pool surfaces were verified by experimental results.This research proposes the development of binocular vision algorithms that can reconstruct the surface of welding pools accurately to realize intelligent welding control systems in the future.
文摘We present a real-time monocular simultaneous localization and mapping(SLAM)system with a new distributed structure for multi-UAV collaboration tasks.The system is different from other general SLAM systems in two aspects:First,it does not aim to build a global map,but to estimate the latest relative position between nearby vehicles;Second,there is no centralized structure in the proposed system,and each vehicle owns an individual metric map and an ego-motion estimator to obtain the relative position between its own map and the neighboring vehicles'.To realize the above characteristics in real time,we demonstrate an innovative feature description and matching algorithm to avoid catastrophic expansion of feature point matching workload due to the increased number of UAVs.Based on the hash and principal component analysis,the matching time complexity of this algorithm can be reduced from 0(logN)to 0(1).To evaluate the performance,the algorithm is verified on the acknowledged multi-view stereo benchmark dataset,and excellent results are obtained.Finally,through the simulation and real flight experiments,this improved SLAM system with the proposed algorithm is validated.
基金This work is supported by the National Natural Science Foundation of China(Grant No.61672279)Project of“Six Talents Peak”in Jiangsu(2012-WLW-023)Open Foundation of State Key Laboratory of Hydrology-Water Resources and Hydraulic Engineering,Nanjing Hydraulic Research Institute,China(2016491411).
文摘Simultaneous location and mapping(SLAM)plays the crucial role in VR/AR application,autonomous robotics navigation,UAV remote control,etc.The traditional SLAM is not good at handle the data acquired by camera with fast movement or severe jittering,and the efficiency need to be improved.The paper proposes an improved SLAM algorithm,which mainly improves the real-time performance of classical SLAM algorithm,applies KDtree for efficient organizing feature points,and accelerates the feature points correspondence building.Moreover,the background map reconstruction thread is optimized,the SLAM parallel computation ability is increased.The color images experiments demonstrate that the improved SLAM algorithm holds better realtime performance than the classical SLAM.
