Target detection is the fundamental work for perceiving the behavior of cows using video analysis automatically.The videos captured in farming scenes often suffer from a complex background,which leads to difficulty in...Target detection is the fundamental work for perceiving the behavior of cows using video analysis automatically.The videos captured in farming scenes often suffer from a complex background,which leads to difficulty in detecting the target and inconvenience in the subsequent images analysis.In this study,a method was proposed to detect the moving target accurately for cows based on background subtraction.Firstly,the bounding rectangle of cows was calculated using the frames difference method to extract the local background in frames,which were averaged and spliced into one image as the entire background image.Secondly,the size and location of a cow’s body were determined by the bounding rectangle of cows,and the body area was tracked through the video by the binary images.Thirdly,the summation coefficients on RGB channels were adjusted to improve the contrast between the target and background images.Finally,taking the body area in every frame as reference area,the performance of target detection was evaluated by the reference area to determine the optimal summation coefficients on RGB channels,and then background subtraction was processed again to finish the detection.A total of 129 videos were used to test the detection algorithm,and the accuracy of the algorithm was 88.34%,which was 24.85%higher than the classical background subtraction method.The study shows that the algorithm proposed in this study is feasible to detect the target accurately and timely when cows are walking straight in the farming environment under natural light,and this method can improve the detection performance and is an extension to the classical background subtraction method.展开更多
On grounds of the advent of real-time applications,like autonomous driving,visual surveillance,and sports analysis,there is an augmenting focus of attention towards Multiple-Object Tracking(MOT).The tracking-by-detect...On grounds of the advent of real-time applications,like autonomous driving,visual surveillance,and sports analysis,there is an augmenting focus of attention towards Multiple-Object Tracking(MOT).The tracking-by-detection paradigm,a commonly utilized approach,connects the existing recognition hypotheses to the formerly assessed object trajectories by comparing the simila-rities of the appearance or the motion between them.For an efficient detection and tracking of the numerous objects in a complex environment,a Pearson Simi-larity-centred Kuhn-Munkres(PS-KM)algorithm was proposed in the present study.In this light,the input videos were,initially,gathered from the MOT dataset and converted into frames.The background subtraction occurred whichfiltered the inappropriate data concerning the frames after the frame conversion stage.Then,the extraction of features from the frames was executed.Afterwards,the higher dimensional features were transformed into lower-dimensional features,and feature reduction process was performed with the aid of Information Gain-centred Singular Value Decomposition(IG-SVD).Next,using the Modified Recurrent Neural Network(MRNN)method,classification was executed which identified the categories of the objects additionally.The PS-KM algorithm identi-fied that the recognized objects were tracked.Finally,the experimental outcomes exhibited that numerous targets were precisely tracked by the proposed system with 97%accuracy with a low false positive rate(FPR)of 2.3%.It was also proved that the present techniques viz.RNN,CNN,and KNN,were effective with regard to the existing models.展开更多
Behavioral analysis of macaques provides important experimental evidence in the field of neuroscience.In recent years,video-based automatic animal behavior analysis has received widespread attention.However,methods ca...Behavioral analysis of macaques provides important experimental evidence in the field of neuroscience.In recent years,video-based automatic animal behavior analysis has received widespread attention.However,methods capable of extracting and analyzing daily movement trajectories of macaques in their daily living cages remain underdeveloped,with previous approaches usually requiring specific environments to reduce interference from occlusion or environmental change.Here,we introduce a novel method,called MonkeyTrail,which satisfies the above requirements by frequently generating virtual empty backgrounds and using background subtraction to accurately obtain the foreground of moving animals.The empty background is generated by combining the frame difference method(FDM)and deep learning-based model(YOLOv5).The entire setup can be operated with low-cost hardware and can be applied to the daily living environments of individually caged macaques.To test MonkeyTrail performance,we labeled a dataset containing>8000 video frames with the bounding boxes of macaques under various conditions as ground-truth.Results showed that the tracking accuracy and stability of MonkeyTrail exceeded that of two deep learningbased methods(YOLOv5 and Single-Shot MultiBox Detector),traditional frame difference method,and na?ve background subtraction method.Using MonkeyTrail to analyze long-term surveillance video recordings,we successfully assessed changes in animal behavior in terms of movement amount and spatial preference.Thus,these findings demonstrate that MonkeyTrail enables low-cost,large-scale daily behavioral analysis of macaques.展开更多
The atomic force microscope(AFM)can measure nanoscale morphology and mechanical properties and has a wide range of applications.The traditional method for measuring the mechanical properties of a sample does so for th...The atomic force microscope(AFM)can measure nanoscale morphology and mechanical properties and has a wide range of applications.The traditional method for measuring the mechanical properties of a sample does so for the longitudinal and transverse properties separately,ignoring the coupling between them.In this paper,a data processing and multidimensional mechanical information extraction algorithm for the composite mode of peak force tapping and torsional resonance is proposed.On the basis of a tip–sample interaction model for the AFM,longitudinal peak force data are used to decouple amplitude and phase data of transverse torsional resonance,accurately identify the tip–sample longitudinal contact force in each peak force cycle,and synchronously obtain the corresponding characteristic images of the transverse amplitude and phase.Experimental results show that the measured longitudinal mechanical characteristics are consistent with the transverse amplitude and phase characteristics,which verifies the effectiveness of the method.Thus,a new method is provided for the measurement of multidimensional mechanical characteristics using the AFM.展开更多
An exponential growth in advanced technologies has resulted in the exploration of Ocean spaces.It has paved the way for new opportunities that can address questions relevant to diversity,uniqueness,and difficulty of m...An exponential growth in advanced technologies has resulted in the exploration of Ocean spaces.It has paved the way for new opportunities that can address questions relevant to diversity,uniqueness,and difficulty of marine life.Underwater Wireless Sensor Networks(UWSNs)are widely used to leverage such opportunities while these networks include a set of vehicles and sensors to monitor the environmental conditions.In this scenario,it is fascinating to design an automated fish detection technique with the help of underwater videos and computer vision techniques so as to estimate and monitor fish biomass in water bodies.Several models have been developed earlier for fish detection.However,they lack robustness to accommodate considerable differences in scenes owing to poor luminosity,fish orientation,structure of seabed,aquatic plantmovement in the background and distinctive shapes and texture of fishes from different genus.With this motivation,the current research article introduces an Intelligent Deep Learning based Automated Fish Detection model for UWSN,named IDLAFD-UWSN model.The presented IDLAFD-UWSN model aims at automatic detection of fishes from underwater videos,particularly in blurred and crowded environments.IDLAFD-UWSN model makes use of Mask Region Convolutional Neural Network(Mask RCNN)with Capsule Network as a baseline model for fish detection.Besides,in order to train Mask RCNN,background subtraction process using GaussianMixtureModel(GMM)model is applied.This model makes use of motion details of fishes in video which consequently integrates the outcome with actual image for the generation of fish-dependent candidate regions.Finally,Wavelet Kernel Extreme Learning Machine(WKELM)model is utilized as a classifier model.The performance of the proposed IDLAFD-UWSN model was tested against benchmark underwater video dataset and the experimental results achieved by IDLAFD-UWSN model were promising in comparison with other state-of-the-art methods under different aspects with the maximum accuracy of 98%and 97%on the applied blurred and crowded datasets respectively.展开更多
In order to realize automatic and accurate grading of cucumber, the first thing is to make sure the high accuracy and integrity in cucumber shape segmentation. As the core processor of this dissertation, DSP TMS320DM6...In order to realize automatic and accurate grading of cucumber, the first thing is to make sure the high accuracy and integrity in cucumber shape segmentation. As the core processor of this dissertation, DSP TMS320DM6437 acquired and processed digital image, it solved the common shadowing problem associated with the natural light. Ultimately, the background subtraction was proposed. Compared with the result of above-mentioned image data processing, the error rate of classic background subtraction method was often high. The result of optimization showed that the improved background subtraction method worked well, and it could meet an accurate segmentation of the fruit in comparison with the original methods.展开更多
In this paper,we propose an efficient fall detection system in enclosed environments based on single Gaussian model using the maximum likelihood method.Online video clips are used to extract the features from two came...In this paper,we propose an efficient fall detection system in enclosed environments based on single Gaussian model using the maximum likelihood method.Online video clips are used to extract the features from two cameras.After the model is constructed,a threshold is set,and the probability for an incoming sample under the single Gaussian model is compared with that threshold to make a decision.Experimental results show that if a proper threshold is set,a good recognition rate for fall activities can be achieved.展开更多
In this paper we present a novel automatic background substitution approach for live video. The objective of background substitution is to extract the foreground from the input video and then combine it with a new bac...In this paper we present a novel automatic background substitution approach for live video. The objective of background substitution is to extract the foreground from the input video and then combine it with a new background. In this paper, we use a color line model to improve the Gaussian mixture model in the background cut method to obtain a binary foreground segmentation result that is less sensitive to brightness differences. Based on the high quality binary segmentation results, we can automatically create a reliable trimap for alpha matting to refine the segmentation boundary. To make the composition result more realistic, an automatic foreground color adjustment step is added to make the foreground look consistent with the new background. Compared to previous approaches, our method can produce higher quality binary segmentation results, and to the best of our knowledge, this is the first time such an automatic and integrated background substitution system has been proposed which can run in real time, which makes it practical for everyday applications.展开更多
Moving object segmentation(MOS),aiming at segmenting moving objects from video frames,is an important and challenging task in computer vision and with various applications.With the development of deep learning(DL),MOS...Moving object segmentation(MOS),aiming at segmenting moving objects from video frames,is an important and challenging task in computer vision and with various applications.With the development of deep learning(DL),MOS has also entered the era of deep models toward spatiotemporal feature learning.This paper aims to provide the latest review of recent DL-based MOS methods proposed during the past three years.Specifically,we present a more up-to-date categorization based on model characteristics,then compare and discuss each category from feature learning(FL),and model training and evaluation perspectives.For FL,the methods reviewed are divided into three types:spatial FL,temporal FL,and spatiotemporal FL,then analyzed from input and model architectures aspects,three input types,and four typical preprocessing subnetworks are summarized.In terms of training,we discuss ideas for enhancing model transferability.In terms of evaluation,based on a previous categorization of scene dependent evaluation and scene independent evaluation,and combined with whether used videos are recorded with static or moving cameras,we further provide four subdivided evaluation setups and analyze that of reviewed methods.We also show performance comparisons of some reviewed MOS methods and analyze the advantages and disadvantages of reviewed MOS methods in terms of technology.Finally,based on the above comparisons and discussions,we present research prospects and future directions.展开更多
An adaptive background model based on max-imum statistical probability and a shadow suppression scheme for indoor and outdoor people detection by exploiting hue saturation value(HSV)color information is proposed.To ob...An adaptive background model based on max-imum statistical probability and a shadow suppression scheme for indoor and outdoor people detection by exploiting hue saturation value(HSV)color information is proposed.To obtain the initial background scene,the frequency of R,G,and B component values for each pixel at the same position in the learning sequence are respec-tively calculated;the R,G,and B component values with the biggest ratios are incorporated to model the initial background.The background maintenance,or the so-called background re-initiation,is also proposed to adapt to scene changes such as illumination changes and scene geometry changes.Moving cast shadows generally exhibit a challenge for accurate moving target detection.Based on the observation that a shadow cast on a background region lowers its brightness but does not change its chro-maticity significantly,we address this problem in the ar-ticle by exploiting HSV color information.In addition,quantitative metrics is introduced to evaluate the algo-rithm on a benchmark suite of indoor and outdoor video sequences.The experimental results are given to show the performance of the algorithm.展开更多
基金financially support ed by the general program from the National Natural Science Foundation of China under grant 61473235.
文摘Target detection is the fundamental work for perceiving the behavior of cows using video analysis automatically.The videos captured in farming scenes often suffer from a complex background,which leads to difficulty in detecting the target and inconvenience in the subsequent images analysis.In this study,a method was proposed to detect the moving target accurately for cows based on background subtraction.Firstly,the bounding rectangle of cows was calculated using the frames difference method to extract the local background in frames,which were averaged and spliced into one image as the entire background image.Secondly,the size and location of a cow’s body were determined by the bounding rectangle of cows,and the body area was tracked through the video by the binary images.Thirdly,the summation coefficients on RGB channels were adjusted to improve the contrast between the target and background images.Finally,taking the body area in every frame as reference area,the performance of target detection was evaluated by the reference area to determine the optimal summation coefficients on RGB channels,and then background subtraction was processed again to finish the detection.A total of 129 videos were used to test the detection algorithm,and the accuracy of the algorithm was 88.34%,which was 24.85%higher than the classical background subtraction method.The study shows that the algorithm proposed in this study is feasible to detect the target accurately and timely when cows are walking straight in the farming environment under natural light,and this method can improve the detection performance and is an extension to the classical background subtraction method.
文摘On grounds of the advent of real-time applications,like autonomous driving,visual surveillance,and sports analysis,there is an augmenting focus of attention towards Multiple-Object Tracking(MOT).The tracking-by-detection paradigm,a commonly utilized approach,connects the existing recognition hypotheses to the formerly assessed object trajectories by comparing the simila-rities of the appearance or the motion between them.For an efficient detection and tracking of the numerous objects in a complex environment,a Pearson Simi-larity-centred Kuhn-Munkres(PS-KM)algorithm was proposed in the present study.In this light,the input videos were,initially,gathered from the MOT dataset and converted into frames.The background subtraction occurred whichfiltered the inappropriate data concerning the frames after the frame conversion stage.Then,the extraction of features from the frames was executed.Afterwards,the higher dimensional features were transformed into lower-dimensional features,and feature reduction process was performed with the aid of Information Gain-centred Singular Value Decomposition(IG-SVD).Next,using the Modified Recurrent Neural Network(MRNN)method,classification was executed which identified the categories of the objects additionally.The PS-KM algorithm identi-fied that the recognized objects were tracked.Finally,the experimental outcomes exhibited that numerous targets were precisely tracked by the proposed system with 97%accuracy with a low false positive rate(FPR)of 2.3%.It was also proved that the present techniques viz.RNN,CNN,and KNN,were effective with regard to the existing models.
基金supported by the National Key Research and Development Program of China(2017YFA0105203,2017YFA0105201)National Science Foundation of China(31771076,81925011)+2 种基金Strategic Priority Research Program of the Chinese Academy of Sciences(CAS)(XDB32040201)Beijing Academy of Artificial IntelligenceKey-Area Research and Development Program of Guangdong Province(2019B030335001)。
文摘Behavioral analysis of macaques provides important experimental evidence in the field of neuroscience.In recent years,video-based automatic animal behavior analysis has received widespread attention.However,methods capable of extracting and analyzing daily movement trajectories of macaques in their daily living cages remain underdeveloped,with previous approaches usually requiring specific environments to reduce interference from occlusion or environmental change.Here,we introduce a novel method,called MonkeyTrail,which satisfies the above requirements by frequently generating virtual empty backgrounds and using background subtraction to accurately obtain the foreground of moving animals.The empty background is generated by combining the frame difference method(FDM)and deep learning-based model(YOLOv5).The entire setup can be operated with low-cost hardware and can be applied to the daily living environments of individually caged macaques.To test MonkeyTrail performance,we labeled a dataset containing>8000 video frames with the bounding boxes of macaques under various conditions as ground-truth.Results showed that the tracking accuracy and stability of MonkeyTrail exceeded that of two deep learningbased methods(YOLOv5 and Single-Shot MultiBox Detector),traditional frame difference method,and na?ve background subtraction method.Using MonkeyTrail to analyze long-term surveillance video recordings,we successfully assessed changes in animal behavior in terms of movement amount and spatial preference.Thus,these findings demonstrate that MonkeyTrail enables low-cost,large-scale daily behavioral analysis of macaques.
基金This project is supported by the General Program of the National Natural Science Foundation of China(62073227)the National Natural Science Foundation of China(61927805 and 61903359).
文摘The atomic force microscope(AFM)can measure nanoscale morphology and mechanical properties and has a wide range of applications.The traditional method for measuring the mechanical properties of a sample does so for the longitudinal and transverse properties separately,ignoring the coupling between them.In this paper,a data processing and multidimensional mechanical information extraction algorithm for the composite mode of peak force tapping and torsional resonance is proposed.On the basis of a tip–sample interaction model for the AFM,longitudinal peak force data are used to decouple amplitude and phase data of transverse torsional resonance,accurately identify the tip–sample longitudinal contact force in each peak force cycle,and synchronously obtain the corresponding characteristic images of the transverse amplitude and phase.Experimental results show that the measured longitudinal mechanical characteristics are consistent with the transverse amplitude and phase characteristics,which verifies the effectiveness of the method.Thus,a new method is provided for the measurement of multidimensional mechanical characteristics using the AFM.
基金The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work under grant number(RGP 1/53/42),www.kku.edu.sa.This research was funded by the Deanship of Scientific Research at Princess Nourah bint Abdulrahman University through the Fast-track Research Funding Program.
文摘An exponential growth in advanced technologies has resulted in the exploration of Ocean spaces.It has paved the way for new opportunities that can address questions relevant to diversity,uniqueness,and difficulty of marine life.Underwater Wireless Sensor Networks(UWSNs)are widely used to leverage such opportunities while these networks include a set of vehicles and sensors to monitor the environmental conditions.In this scenario,it is fascinating to design an automated fish detection technique with the help of underwater videos and computer vision techniques so as to estimate and monitor fish biomass in water bodies.Several models have been developed earlier for fish detection.However,they lack robustness to accommodate considerable differences in scenes owing to poor luminosity,fish orientation,structure of seabed,aquatic plantmovement in the background and distinctive shapes and texture of fishes from different genus.With this motivation,the current research article introduces an Intelligent Deep Learning based Automated Fish Detection model for UWSN,named IDLAFD-UWSN model.The presented IDLAFD-UWSN model aims at automatic detection of fishes from underwater videos,particularly in blurred and crowded environments.IDLAFD-UWSN model makes use of Mask Region Convolutional Neural Network(Mask RCNN)with Capsule Network as a baseline model for fish detection.Besides,in order to train Mask RCNN,background subtraction process using GaussianMixtureModel(GMM)model is applied.This model makes use of motion details of fishes in video which consequently integrates the outcome with actual image for the generation of fish-dependent candidate regions.Finally,Wavelet Kernel Extreme Learning Machine(WKELM)model is utilized as a classifier model.The performance of the proposed IDLAFD-UWSN model was tested against benchmark underwater video dataset and the experimental results achieved by IDLAFD-UWSN model were promising in comparison with other state-of-the-art methods under different aspects with the maximum accuracy of 98%and 97%on the applied blurred and crowded datasets respectively.
基金Supported by Heilongjiang Provincial Scientific Research Projects(12521038)China Postdoctoral Science Foundation(20080430886)
文摘In order to realize automatic and accurate grading of cucumber, the first thing is to make sure the high accuracy and integrity in cucumber shape segmentation. As the core processor of this dissertation, DSP TMS320DM6437 acquired and processed digital image, it solved the common shadowing problem associated with the natural light. Ultimately, the background subtraction was proposed. Compared with the result of above-mentioned image data processing, the error rate of classic background subtraction method was often high. The result of optimization showed that the improved background subtraction method worked well, and it could meet an accurate segmentation of the fruit in comparison with the original methods.
文摘In this paper,we propose an efficient fall detection system in enclosed environments based on single Gaussian model using the maximum likelihood method.Online video clips are used to extract the features from two cameras.After the model is constructed,a threshold is set,and the probability for an incoming sample under the single Gaussian model is compared with that threshold to make a decision.Experimental results show that if a proper threshold is set,a good recognition rate for fall activities can be achieved.
基金supported by the National HighTech R&D Program of China (Project No. 2012AA011903)the National Natural Science Foundation of China (Project No. 61373069)+1 种基金the Research Grant of Beijing Higher Institution Engineering Research CenterTsinghua–Tencent Joint Laboratory for Internet Innovation Technology
文摘In this paper we present a novel automatic background substitution approach for live video. The objective of background substitution is to extract the foreground from the input video and then combine it with a new background. In this paper, we use a color line model to improve the Gaussian mixture model in the background cut method to obtain a binary foreground segmentation result that is less sensitive to brightness differences. Based on the high quality binary segmentation results, we can automatically create a reliable trimap for alpha matting to refine the segmentation boundary. To make the composition result more realistic, an automatic foreground color adjustment step is added to make the foreground look consistent with the new background. Compared to previous approaches, our method can produce higher quality binary segmentation results, and to the best of our knowledge, this is the first time such an automatic and integrated background substitution system has been proposed which can run in real time, which makes it practical for everyday applications.
基金National Natural Science Foundation of China(Nos.61702323 and 62172268)the Shanghai Municipal Natural Science Foundation,China(No.20ZR1423100)+2 种基金the Open Fund of Science and Technology on Thermal Energy and Power Laboratory(No.TPL2020C02)Wuhan 2nd Ship Design and Research Institute,Wuhan,China,the National Key Research and Development Program of China(No.2018YFB1306303)the Major Basic Research Projects of Natural Science Foundation of Shandong Province,China(No.ZR2019ZD07).
文摘Moving object segmentation(MOS),aiming at segmenting moving objects from video frames,is an important and challenging task in computer vision and with various applications.With the development of deep learning(DL),MOS has also entered the era of deep models toward spatiotemporal feature learning.This paper aims to provide the latest review of recent DL-based MOS methods proposed during the past three years.Specifically,we present a more up-to-date categorization based on model characteristics,then compare and discuss each category from feature learning(FL),and model training and evaluation perspectives.For FL,the methods reviewed are divided into three types:spatial FL,temporal FL,and spatiotemporal FL,then analyzed from input and model architectures aspects,three input types,and four typical preprocessing subnetworks are summarized.In terms of training,we discuss ideas for enhancing model transferability.In terms of evaluation,based on a previous categorization of scene dependent evaluation and scene independent evaluation,and combined with whether used videos are recorded with static or moving cameras,we further provide four subdivided evaluation setups and analyze that of reviewed methods.We also show performance comparisons of some reviewed MOS methods and analyze the advantages and disadvantages of reviewed MOS methods in terms of technology.Finally,based on the above comparisons and discussions,we present research prospects and future directions.
文摘An adaptive background model based on max-imum statistical probability and a shadow suppression scheme for indoor and outdoor people detection by exploiting hue saturation value(HSV)color information is proposed.To obtain the initial background scene,the frequency of R,G,and B component values for each pixel at the same position in the learning sequence are respec-tively calculated;the R,G,and B component values with the biggest ratios are incorporated to model the initial background.The background maintenance,or the so-called background re-initiation,is also proposed to adapt to scene changes such as illumination changes and scene geometry changes.Moving cast shadows generally exhibit a challenge for accurate moving target detection.Based on the observation that a shadow cast on a background region lowers its brightness but does not change its chro-maticity significantly,we address this problem in the ar-ticle by exploiting HSV color information.In addition,quantitative metrics is introduced to evaluate the algo-rithm on a benchmark suite of indoor and outdoor video sequences.The experimental results are given to show the performance of the algorithm.
