In this paper, the global asymptotic stability is investigated for a class of Cohen-Grossberg neural networks with time-varying and distributed delays. By using the Lyapunov-Krasovskii functional and equivalent descri...In this paper, the global asymptotic stability is investigated for a class of Cohen-Grossberg neural networks with time-varying and distributed delays. By using the Lyapunov-Krasovskii functional and equivalent descriptor form of the considered system, several delay-dependent sufficient conditions are obtained to guarantee the asymptotic stability of the addressed systems. These conditions are dependent on both time-varying and distributed delays and presented in terms of LMIs and therefore, the stability criteria of such systems can be checked readily by resorting to the Matlab LMI toolbox. Finally, an example is given to show the effectiveness and less conservatism of the proposed methods.展开更多
The stability analysis of Cohen-Grossberg neural networks with multiple delays is given. An approach combining the Lyapunov functional with the linear matrix inequality （LMI） is taken to obtain the sufficient condit...The stability analysis of Cohen-Grossberg neural networks with multiple delays is given. An approach combining the Lyapunov functional with the linear matrix inequality （LMI） is taken to obtain the sufficient conditions for the globally asymptotic stability of equilibrium point. By using the properties of matrix norm, a practical corollary is derived. All results are established without assuming the differentiability and monotonicity of activation functions. The simulation samples have proved the effectiveness of the conclusions.展开更多
In this paper, the Cohen-Grossberg neural networks with time-varying delays and impulses are considered. New sufficient conditions for the existence and global exponential stability of a unique equilibrium point are e...In this paper, the Cohen-Grossberg neural networks with time-varying delays and impulses are considered. New sufficient conditions for the existence and global exponential stability of a unique equilibrium point are established by using the fixed point theorem and Lyapunov functional. An example is given to demonstrate the effectiveness of our results.展开更多
The global exponential stability of Cohen-Grossberg neural networks with time-varying delays is studied. By constructing several suitable Lyapunov functionals and utilizing differential in-equality techniques, some su...The global exponential stability of Cohen-Grossberg neural networks with time-varying delays is studied. By constructing several suitable Lyapunov functionals and utilizing differential in-equality techniques, some sufficient criteria for the global exponential stability and the exponential convergence rate of the equilibrium point of the system are obtained. The criteria do not require the activation functions to be differentiable or monotone nondecreasing. Some stability results from previous works are extended and improved. Comparisons are made to demonstrate the advantage of our results.展开更多
A class of generalized Cohen-Grossberg neural networks（CGNNs） with variable de- lays are investigated. By introducing a new type of Lyapunov functional and applying the homeomorphism theory and inequality technique,...A class of generalized Cohen-Grossberg neural networks（CGNNs） with variable de- lays are investigated. By introducing a new type of Lyapunov functional and applying the homeomorphism theory and inequality technique, some new conditions axe derived ensuring the existence and uniqueness of the equilibrium point and its global exponential stability for CGNNs. These results obtained are independent of delays, develop the existent outcome in the earlier literature and are very easily checked in practice.展开更多
This paper addresses the coexistence and local stability of multiple equilibrium points for fractional-order Cohen-Grossberg neural networks(FOCGNNs)with time delays.Based on Brouwer's fixed point theorem,sufficie...This paper addresses the coexistence and local stability of multiple equilibrium points for fractional-order Cohen-Grossberg neural networks(FOCGNNs)with time delays.Based on Brouwer's fixed point theorem,sufficient conditions are established to ensure the existence of Πi=1^n(2Ki+1)equilibrium points for FOCGNNs.Through the use of Hardy inequality,fractional Halanay inequality,and Lyapunov theory,some criteria are established to ensure the local Lagrange stability and the local Lyapunov asymptotical stability of Πi=1^n(Ki+1)equilibrium points for FOCGNNs.The obtained results encompass those of integer-order Hopfield neural networks with or without delay as special cases.The activation functions are nonlinear and nonmonotonic.There could be many corner points in this general class of activation functions.The structure of activation functions makes FOCGNNs could have a lot of stable equilibrium points.Coexistence of multiple stable equilibrium points is necessary when neural networks come to pattern recognition and associative memories.Finally,two numerical examples are provided to illustrate the effectiveness of the obtained results.展开更多
Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been c...Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition.However,the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously.To push the development of this issue,we propose the pluggable diffractive neural networks(P-DNN),a general paradigm resorting to the cascaded metasurfaces,which can be applied to recognize various tasks by switching internal plug-ins.As the proof-of-principle,the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes.Encouragingly,the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed,low-power and versatile artificial intelligence systems.展开更多
In this paper, the μ-stability of multiple equilibrium points(EPs) in the Cohen-Grossberg neural networks(CGNNs) is addressed by designing a kind of discontinuous activation function(AF). Under some criteria, CGNNs w...In this paper, the μ-stability of multiple equilibrium points(EPs) in the Cohen-Grossberg neural networks(CGNNs) is addressed by designing a kind of discontinuous activation function(AF). Under some criteria, CGNNs with this AF are shown to possess at least 5^(n)EPs, of which 3^(n)EPs are locally μ-stable. Compared with the saturated AF or the sigmoidal AF, CGNNs with the designed AF can produce many more total/stable EPs. Therefore, when CGNNs with the designed discontinuous AF are applied to associative memory, they can store more prototype patterns. Moreover, the AF is expanded to a more general version to further increase the number of total/stable equilibria. The CGNNs with the expanded AF are found to produce(2k+3)^(n)EPs, of which (k+2)^(n)EPs are locally μ-stable. By adjusting two parameters in the AF, the number of sufficient conditions ensuring the μ-stability of multiple equilibria can be decreased. This finding implies that the computational complexity can be greatly reduced.Two numerical examples and an application to associative memory are illustrated to verify the correctness of the obtained results.展开更多
The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedd...The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.展开更多
An artificial neural network(ANN)method is introduced to predict drop size in two kinds of pulsed columns with small-scale data sets.After training,the deviation between calculate and experimental results are 3.8%and ...An artificial neural network(ANN)method is introduced to predict drop size in two kinds of pulsed columns with small-scale data sets.After training,the deviation between calculate and experimental results are 3.8%and 9.3%,respectively.Through ANN model,the influence of interfacial tension and pulsation intensity on the droplet diameter has been developed.Droplet size gradually increases with the increase of interfacial tension,and decreases with the increase of pulse intensity.It can be seen that the accuracy of ANN model in predicting droplet size outside the training set range is reach the same level as the accuracy of correlation obtained based on experiments within this range.For two kinds of columns,the drop size prediction deviations of ANN model are 9.6%and 18.5%and the deviations in correlations are 11%and 15%.展开更多
Physics-informed neural networks are a useful machine learning method for solving differential equations,but encounter challenges in effectively learning thin boundary layers within singular perturbation problems.To r...Physics-informed neural networks are a useful machine learning method for solving differential equations,but encounter challenges in effectively learning thin boundary layers within singular perturbation problems.To resolve this issue,multi-scale-matching neural networks are proposed to solve the singular perturbation problems.Inspired by matched asymptotic expansions,the solution is decomposed into inner solutions for small scales and outer solutions for large scales,corresponding to boundary layers and outer regions,respectively.Moreover,to conform neural networks,we introduce exponential stretched variables in the boundary layers to avoid semiinfinite region problems.Numerical results for the thin plate problem validate the proposed method.展开更多
Multi-scale system remains a classical scientific problem in fluid dynamics,biology,etc.In the present study,a scheme of multi-scale Physics-informed neural networks is proposed to solve the boundary layer flow at hig...Multi-scale system remains a classical scientific problem in fluid dynamics,biology,etc.In the present study,a scheme of multi-scale Physics-informed neural networks is proposed to solve the boundary layer flow at high Reynolds numbers without any data.The flow is divided into several regions with different scales based on Prandtl's boundary theory.Different regions are solved with governing equations in different scales.The method of matched asymptotic expansions is used to make the flow field continuously.A flow on a semi infinite flat plate at a high Reynolds number is considered a multi-scale problem because the boundary layer scale is much smaller than the outer flow scale.The results are compared with the reference numerical solutions,which show that the msPINNs can solve the multi-scale problem of the boundary layer in high Reynolds number flows.This scheme can be developed for more multi-scale problems in the future.展开更多
Smart Industrial environments use the Industrial Internet of Things(IIoT)for their routine operations and transform their industrial operations with intelligent and driven approaches.However,IIoT devices are vulnerabl...Smart Industrial environments use the Industrial Internet of Things(IIoT)for their routine operations and transform their industrial operations with intelligent and driven approaches.However,IIoT devices are vulnerable to cyber threats and exploits due to their connectivity with the internet.Traditional signature-based IDS are effective in detecting known attacks,but they are unable to detect unknown emerging attacks.Therefore,there is the need for an IDS which can learn from data and detect new threats.Ensemble Machine Learning(ML)and individual Deep Learning(DL)based IDS have been developed,and these individual models achieved low accuracy;however,their performance can be improved with the ensemble stacking technique.In this paper,we have proposed a Deep Stacked Neural Network(DSNN)based IDS,which consists of two stacked Convolutional Neural Network(CNN)models as base learners and Extreme Gradient Boosting(XGB)as the meta learner.The proposed DSNN model was trained and evaluated with the next-generation dataset,TON_IoT.Several pre-processing techniques were applied to prepare a dataset for the model,including ensemble feature selection and the SMOTE technique.Accuracy,precision,recall,F1-score,and false positive rates were used to evaluate the performance of the proposed ensemble model.Our experimental results showed that the accuracy for binary classification is 99.61%,which is better than in the baseline individual DL and ML models.In addition,the model proposed for IDS has been compared with similar models.The proposed DSNN achieved better performance metrics than the other models.The proposed DSNN model will be used to develop enhanced IDS for threat mitigation in smart industrial environments.展开更多
Artificial neural networks(ANNs)have led to landmark changes in many fields,but they still differ significantly fromthemechanisms of real biological neural networks and face problems such as high computing costs,exces...Artificial neural networks(ANNs)have led to landmark changes in many fields,but they still differ significantly fromthemechanisms of real biological neural networks and face problems such as high computing costs,excessive computing power,and so on.Spiking neural networks(SNNs)provide a new approach combined with brain-like science to improve the computational energy efficiency,computational architecture,and biological credibility of current deep learning applications.In the early stage of development,its poor performance hindered the application of SNNs in real-world scenarios.In recent years,SNNs have made great progress in computational performance and practicability compared with the earlier research results,and are continuously producing significant results.Although there are already many pieces of literature on SNNs,there is still a lack of comprehensive review on SNNs from the perspective of improving performance and practicality as well as incorporating the latest research results.Starting from this issue,this paper elaborates on SNNs along the complete usage process of SNNs including network construction,data processing,model training,development,and deployment,aiming to provide more comprehensive and practical guidance to promote the development of SNNs.Therefore,the connotation and development status of SNNcomputing is reviewed systematically and comprehensively from four aspects:composition structure,data set,learning algorithm,software/hardware development platform.Then the development characteristics of SNNs in intelligent computing are summarized,the current challenges of SNNs are discussed and the future development directions are also prospected.Our research shows that in the fields of machine learning and intelligent computing,SNNs have comparable network scale and performance to ANNs and the ability to challenge large datasets and a variety of tasks.The advantages of SNNs over ANNs in terms of energy efficiency and spatial-temporal data processing have been more fully exploited.And the development of programming and deployment tools has lowered the threshold for the use of SNNs.SNNs show a broad development prospect for brain-like computing.展开更多
A significant demand rises for energy-efficient deep neural networks to support power-limited embedding devices with successful deep learning applications in IoT and edge computing fields.An accurate energy prediction...A significant demand rises for energy-efficient deep neural networks to support power-limited embedding devices with successful deep learning applications in IoT and edge computing fields.An accurate energy prediction approach is critical to provide measurement and lead optimization direction.However,the current energy prediction approaches lack accuracy and generalization ability due to the lack of research on the neural network structure and the excessive reliance on customized training dataset.This paper presents a novel energy prediction model,NeurstrucEnergy.NeurstrucEnergy treats neural networks as directed graphs and applies a bi-directional graph neural network training on a randomly generated dataset to extract structural features for energy prediction.NeurstrucEnergy has advantages over linear approaches because the bi-directional graph neural network collects structural features from each layer's parents and children.Experimental results show that NeurstrucEnergy establishes state-of-the-art results with mean absolute percentage error of 2.60%.We also evaluate NeurstrucEnergy in a randomly generated dataset,achieving the mean absolute percentage error of 4.83%over 10 typical convolutional neural networks in recent years and 7 efficient convolutional neural networks created by neural architecture search.Our code is available at https://github.com/NEUSoftGreenAI/NeurstrucEnergy.git.展开更多
Recent advances in deep neural networks have shed new light on physics,engineering,and scientific computing.Reconciling the data-centered viewpoint with physical simulation is one of the research hotspots.The physicsi...Recent advances in deep neural networks have shed new light on physics,engineering,and scientific computing.Reconciling the data-centered viewpoint with physical simulation is one of the research hotspots.The physicsinformedneural network(PINN)is currently the most general framework,which is more popular due to theconvenience of constructing NNs and excellent generalization ability.The automatic differentiation(AD)-basedPINN model is suitable for the homogeneous scientific problem;however,it is unclear how AD can enforce fluxcontinuity across boundaries between cells of different properties where spatial heterogeneity is represented bygrid cells with different physical properties.In this work,we propose a criss-cross physics-informed convolutionalneural network(CC-PINN)learning architecture,aiming to learn the solution of parametric PDEs with spatialheterogeneity of physical properties.To achieve the seamless enforcement of flux continuity and integration ofphysicalmeaning into CNN,a predefined 2D convolutional layer is proposed to accurately express transmissibilitybetween adjacent cells.The efficacy of the proposedmethodwas evaluated through predictions of several petroleumreservoir problems with spatial heterogeneity and compared against state-of-the-art(PINN)through numericalanalysis as a benchmark,which demonstrated the superiority of the proposed method over the PINN.展开更多
As neural radiance fields continue to advance in 3D content representation,the copyright issues surrounding 3D models oriented towards implicit representation become increasingly pressing.In response to this challenge...As neural radiance fields continue to advance in 3D content representation,the copyright issues surrounding 3D models oriented towards implicit representation become increasingly pressing.In response to this challenge,this paper treats the embedding and extraction of neural radiance field watermarks as inverse problems of image transformations and proposes a scheme for protecting neural radiance field copyrights using invertible neural network watermarking.Leveraging 2D image watermarking technology for 3D scene protection,the scheme embeds watermarks within the training images of neural radiance fields through the forward process in invertible neural networks and extracts them from images rendered by neural radiance fields through the reverse process,thereby ensuring copyright protection for both the neural radiance fields and associated 3D scenes.However,challenges such as information loss during rendering processes and deliberate tampering necessitate the design of an image quality enhancement module to increase the scheme’s robustness.This module restores distorted images through neural network processing before watermark extraction.Additionally,embedding watermarks in each training image enables watermark information extraction from multiple viewpoints.Our proposed watermarking method achieves a PSNR(Peak Signal-to-Noise Ratio)value exceeding 37 dB for images containing watermarks and 22 dB for recovered watermarked images,as evaluated on the Lego,Hotdog,and Chair datasets,respectively.These results demonstrate the efficacy of our scheme in enhancing copyright protection.展开更多
In a very recent study,Prof.Lingling Huang and co-workers proposed and demonstrated reconfigurable optical neural networks based on cascaded metasurfaces.By fixing one metasurface and switching the other pluggable met...In a very recent study,Prof.Lingling Huang and co-workers proposed and demonstrated reconfigurable optical neural networks based on cascaded metasurfaces.By fixing one metasurface and switching the other pluggable metasurfaces,the neural networks,which operate at near-infrared wavelengths,can perform distinct recognition tasks for handwritten digits and fashion products.This innovative device opens up an avenue for all-optical,high-speed,low-power,and multifunctional artificial intelligence systems.展开更多
Recent research advances in implicit neural representation have shown that a wide range of video data distributions are achieved by sharing model weights for Neural Representation for Videos(NeRV).While explicit metho...Recent research advances in implicit neural representation have shown that a wide range of video data distributions are achieved by sharing model weights for Neural Representation for Videos(NeRV).While explicit methods exist for accurately embedding ownership or copyright information in video data,the nascent NeRV framework has yet to address this issue comprehensively.In response,this paper introduces MarkINeRV,a scheme designed to embed watermarking information into video frames using an invertible neural network watermarking approach to protect the copyright of NeRV,which models the embedding and extraction of watermarks as a pair of inverse processes of a reversible network and employs the same network to achieve embedding and extraction of watermarks.It is just that the information flow is in the opposite direction.Additionally,a video frame quality enhancement module is incorporated to mitigate watermarking information losses in the rendering process and the possibility ofmalicious attacks during transmission,ensuring the accurate extraction of watermarking information through the invertible network’s inverse process.This paper evaluates the accuracy,robustness,and invisibility of MarkINeRV through multiple video datasets.The results demonstrate its efficacy in extracting watermarking information for copyright protection of NeRV.MarkINeRV represents a pioneering investigation into copyright issues surrounding NeRV.展开更多
The lethal brain tumor “Glioblastoma” has the propensity to grow over time. To improve patient outcomes, it is essential to classify GBM accurately and promptly in order to provide a focused and individualized treat...The lethal brain tumor “Glioblastoma” has the propensity to grow over time. To improve patient outcomes, it is essential to classify GBM accurately and promptly in order to provide a focused and individualized treatment plan. Despite this, deep learning methods, particularly Convolutional Neural Networks (CNNs), have demonstrated a high level of accuracy in a myriad of medical image analysis applications as a result of recent technical breakthroughs. The overall aim of the research is to investigate how CNNs can be used to classify GBMs using data from medical imaging, to improve prognosis precision and effectiveness. This research study will demonstrate a suggested methodology that makes use of the CNN architecture and is trained using a database of MRI pictures with this tumor. The constructed model will be assessed based on its overall performance. Extensive experiments and comparisons with conventional machine learning techniques and existing classification methods will also be made. It will be crucial to emphasize the possibility of early and accurate prediction in a clinical workflow because it can have a big impact on treatment planning and patient outcomes. The paramount objective is to not only address the classification challenge but also to outline a clear pathway towards enhancing prognosis precision and treatment effectiveness.展开更多
基金the National Natural Science Foundation of China (No.60574006).
文摘In this paper, the global asymptotic stability is investigated for a class of Cohen-Grossberg neural networks with time-varying and distributed delays. By using the Lyapunov-Krasovskii functional and equivalent descriptor form of the considered system, several delay-dependent sufficient conditions are obtained to guarantee the asymptotic stability of the addressed systems. These conditions are dependent on both time-varying and distributed delays and presented in terms of LMIs and therefore, the stability criteria of such systems can be checked readily by resorting to the Matlab LMI toolbox. Finally, an example is given to show the effectiveness and less conservatism of the proposed methods.
基金This work was supported by the National Natural Science Foundation of China(No. 60534010, 60572070), Liaoning Natural Science Foundation ofChina(No.20052027), and the Program for Changjiang Scholars and Innovative Research Team in University.
文摘The stability analysis of Cohen-Grossberg neural networks with multiple delays is given. An approach combining the Lyapunov functional with the linear matrix inequality （LMI） is taken to obtain the sufficient conditions for the globally asymptotic stability of equilibrium point. By using the properties of matrix norm, a practical corollary is derived. All results are established without assuming the differentiability and monotonicity of activation functions. The simulation samples have proved the effectiveness of the conclusions.
文摘In this paper, the Cohen-Grossberg neural networks with time-varying delays and impulses are considered. New sufficient conditions for the existence and global exponential stability of a unique equilibrium point are established by using the fixed point theorem and Lyapunov functional. An example is given to demonstrate the effectiveness of our results.
基金National Natural Science Foundation of China (No70471049)
文摘The global exponential stability of Cohen-Grossberg neural networks with time-varying delays is studied. By constructing several suitable Lyapunov functionals and utilizing differential in-equality techniques, some sufficient criteria for the global exponential stability and the exponential convergence rate of the equilibrium point of the system are obtained. The criteria do not require the activation functions to be differentiable or monotone nondecreasing. Some stability results from previous works are extended and improved. Comparisons are made to demonstrate the advantage of our results.
基金Supported by the Distinguished Expert Science Foundation of Naval Aeronautical Engineering Institutethe Younger Foundation of Yantai University (SX06Z9)
文摘A class of generalized Cohen-Grossberg neural networks（CGNNs） with variable de- lays are investigated. By introducing a new type of Lyapunov functional and applying the homeomorphism theory and inequality technique, some new conditions axe derived ensuring the existence and uniqueness of the equilibrium point and its global exponential stability for CGNNs. These results obtained are independent of delays, develop the existent outcome in the earlier literature and are very easily checked in practice.
基金Project supported by the Natural Science Foundation of Zhejiang Province of China(Grant Nos.LY18F030023,LY17F030016,LQ18F030015,and LY18F020028)the National Natural Science Foundation of China(Grant Nos.61503338,61773348,and 61972354).
文摘This paper addresses the coexistence and local stability of multiple equilibrium points for fractional-order Cohen-Grossberg neural networks(FOCGNNs)with time delays.Based on Brouwer's fixed point theorem,sufficient conditions are established to ensure the existence of Πi=1^n(2Ki+1)equilibrium points for FOCGNNs.Through the use of Hardy inequality,fractional Halanay inequality,and Lyapunov theory,some criteria are established to ensure the local Lagrange stability and the local Lyapunov asymptotical stability of Πi=1^n(Ki+1)equilibrium points for FOCGNNs.The obtained results encompass those of integer-order Hopfield neural networks with or without delay as special cases.The activation functions are nonlinear and nonmonotonic.There could be many corner points in this general class of activation functions.The structure of activation functions makes FOCGNNs could have a lot of stable equilibrium points.Coexistence of multiple stable equilibrium points is necessary when neural networks come to pattern recognition and associative memories.Finally,two numerical examples are provided to illustrate the effectiveness of the obtained results.
基金The authors acknowledge the funding provided by the National Key R&D Program of China(2021YFA1401200)Beijing Outstanding Young Scientist Program(BJJWZYJH01201910007022)+2 种基金National Natural Science Foundation of China(No.U21A20140,No.92050117,No.62005017)programBeijing Municipal Science&Technology Commission,Administrative Commission of Zhongguancun Science Park(No.Z211100004821009)This work was supported by the Synergetic Extreme Condition User Facility(SECUF).
文摘Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition.However,the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously.To push the development of this issue,we propose the pluggable diffractive neural networks(P-DNN),a general paradigm resorting to the cascaded metasurfaces,which can be applied to recognize various tasks by switching internal plug-ins.As the proof-of-principle,the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes.Encouragingly,the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed,low-power and versatile artificial intelligence systems.
基金supported by the National Natural Science Foundation of China(Grant Nos.62173214 and 61973199)the Shandong Provincial Natural Science Foundation(Grant Nos.ZR2021MF003 and ZR2022MF324)the Major Technologies Research and Development Special Program of Anhui Province(Grant No.202003a05020001)。
文摘In this paper, the μ-stability of multiple equilibrium points(EPs) in the Cohen-Grossberg neural networks(CGNNs) is addressed by designing a kind of discontinuous activation function(AF). Under some criteria, CGNNs with this AF are shown to possess at least 5^(n)EPs, of which 3^(n)EPs are locally μ-stable. Compared with the saturated AF or the sigmoidal AF, CGNNs with the designed AF can produce many more total/stable EPs. Therefore, when CGNNs with the designed discontinuous AF are applied to associative memory, they can store more prototype patterns. Moreover, the AF is expanded to a more general version to further increase the number of total/stable equilibria. The CGNNs with the expanded AF are found to produce(2k+3)^(n)EPs, of which (k+2)^(n)EPs are locally μ-stable. By adjusting two parameters in the AF, the number of sufficient conditions ensuring the μ-stability of multiple equilibria can be decreased. This finding implies that the computational complexity can be greatly reduced.Two numerical examples and an application to associative memory are illustrated to verify the correctness of the obtained results.
基金The Qian Xuesen Youth Innovation Foundation from China Aerospace Science and Technology Corporation(Grant Number 2022JY51).
文摘The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.
基金the support of the National Natural Science Foundation of China(22278234,21776151)。
文摘An artificial neural network(ANN)method is introduced to predict drop size in two kinds of pulsed columns with small-scale data sets.After training,the deviation between calculate and experimental results are 3.8%and 9.3%,respectively.Through ANN model,the influence of interfacial tension and pulsation intensity on the droplet diameter has been developed.Droplet size gradually increases with the increase of interfacial tension,and decreases with the increase of pulse intensity.It can be seen that the accuracy of ANN model in predicting droplet size outside the training set range is reach the same level as the accuracy of correlation obtained based on experiments within this range.For two kinds of columns,the drop size prediction deviations of ANN model are 9.6%and 18.5%and the deviations in correlations are 11%and 15%.
基金supported by the National Natural Science Foun-dation of China (NSFC) Basic Science Center Program for"Multiscale Problems in Nonlinear Mechanics"(Grant No. 11988102)supported by the National Natural Science Foundation of China (NSFC)(Grant No. 12202451)
文摘Physics-informed neural networks are a useful machine learning method for solving differential equations,but encounter challenges in effectively learning thin boundary layers within singular perturbation problems.To resolve this issue,multi-scale-matching neural networks are proposed to solve the singular perturbation problems.Inspired by matched asymptotic expansions,the solution is decomposed into inner solutions for small scales and outer solutions for large scales,corresponding to boundary layers and outer regions,respectively.Moreover,to conform neural networks,we introduce exponential stretched variables in the boundary layers to avoid semiinfinite region problems.Numerical results for the thin plate problem validate the proposed method.
文摘Multi-scale system remains a classical scientific problem in fluid dynamics,biology,etc.In the present study,a scheme of multi-scale Physics-informed neural networks is proposed to solve the boundary layer flow at high Reynolds numbers without any data.The flow is divided into several regions with different scales based on Prandtl's boundary theory.Different regions are solved with governing equations in different scales.The method of matched asymptotic expansions is used to make the flow field continuously.A flow on a semi infinite flat plate at a high Reynolds number is considered a multi-scale problem because the boundary layer scale is much smaller than the outer flow scale.The results are compared with the reference numerical solutions,which show that the msPINNs can solve the multi-scale problem of the boundary layer in high Reynolds number flows.This scheme can be developed for more multi-scale problems in the future.
文摘Smart Industrial environments use the Industrial Internet of Things(IIoT)for their routine operations and transform their industrial operations with intelligent and driven approaches.However,IIoT devices are vulnerable to cyber threats and exploits due to their connectivity with the internet.Traditional signature-based IDS are effective in detecting known attacks,but they are unable to detect unknown emerging attacks.Therefore,there is the need for an IDS which can learn from data and detect new threats.Ensemble Machine Learning(ML)and individual Deep Learning(DL)based IDS have been developed,and these individual models achieved low accuracy;however,their performance can be improved with the ensemble stacking technique.In this paper,we have proposed a Deep Stacked Neural Network(DSNN)based IDS,which consists of two stacked Convolutional Neural Network(CNN)models as base learners and Extreme Gradient Boosting(XGB)as the meta learner.The proposed DSNN model was trained and evaluated with the next-generation dataset,TON_IoT.Several pre-processing techniques were applied to prepare a dataset for the model,including ensemble feature selection and the SMOTE technique.Accuracy,precision,recall,F1-score,and false positive rates were used to evaluate the performance of the proposed ensemble model.Our experimental results showed that the accuracy for binary classification is 99.61%,which is better than in the baseline individual DL and ML models.In addition,the model proposed for IDS has been compared with similar models.The proposed DSNN achieved better performance metrics than the other models.The proposed DSNN model will be used to develop enhanced IDS for threat mitigation in smart industrial environments.
基金supported by the National Natural Science Foundation of China(Nos.61974164,62074166,62004219,62004220,and 62104256).
文摘Artificial neural networks(ANNs)have led to landmark changes in many fields,but they still differ significantly fromthemechanisms of real biological neural networks and face problems such as high computing costs,excessive computing power,and so on.Spiking neural networks(SNNs)provide a new approach combined with brain-like science to improve the computational energy efficiency,computational architecture,and biological credibility of current deep learning applications.In the early stage of development,its poor performance hindered the application of SNNs in real-world scenarios.In recent years,SNNs have made great progress in computational performance and practicability compared with the earlier research results,and are continuously producing significant results.Although there are already many pieces of literature on SNNs,there is still a lack of comprehensive review on SNNs from the perspective of improving performance and practicality as well as incorporating the latest research results.Starting from this issue,this paper elaborates on SNNs along the complete usage process of SNNs including network construction,data processing,model training,development,and deployment,aiming to provide more comprehensive and practical guidance to promote the development of SNNs.Therefore,the connotation and development status of SNNcomputing is reviewed systematically and comprehensively from four aspects:composition structure,data set,learning algorithm,software/hardware development platform.Then the development characteristics of SNNs in intelligent computing are summarized,the current challenges of SNNs are discussed and the future development directions are also prospected.Our research shows that in the fields of machine learning and intelligent computing,SNNs have comparable network scale and performance to ANNs and the ability to challenge large datasets and a variety of tasks.The advantages of SNNs over ANNs in terms of energy efficiency and spatial-temporal data processing have been more fully exploited.And the development of programming and deployment tools has lowered the threshold for the use of SNNs.SNNs show a broad development prospect for brain-like computing.
基金supported by the Natural Science Foundation of Liaoning Province(2020-BS-054)the Fundamental Research Funds for the Central Universities(N2017005)the National Natural Science Foundation of China(62162050).
文摘A significant demand rises for energy-efficient deep neural networks to support power-limited embedding devices with successful deep learning applications in IoT and edge computing fields.An accurate energy prediction approach is critical to provide measurement and lead optimization direction.However,the current energy prediction approaches lack accuracy and generalization ability due to the lack of research on the neural network structure and the excessive reliance on customized training dataset.This paper presents a novel energy prediction model,NeurstrucEnergy.NeurstrucEnergy treats neural networks as directed graphs and applies a bi-directional graph neural network training on a randomly generated dataset to extract structural features for energy prediction.NeurstrucEnergy has advantages over linear approaches because the bi-directional graph neural network collects structural features from each layer's parents and children.Experimental results show that NeurstrucEnergy establishes state-of-the-art results with mean absolute percentage error of 2.60%.We also evaluate NeurstrucEnergy in a randomly generated dataset,achieving the mean absolute percentage error of 4.83%over 10 typical convolutional neural networks in recent years and 7 efficient convolutional neural networks created by neural architecture search.Our code is available at https://github.com/NEUSoftGreenAI/NeurstrucEnergy.git.
基金the National Natural Science Foundation of China(No.52274048)Beijing Natural Science Foundation(No.3222037)+1 种基金the CNPC 14th Five-Year Perspective Fundamental Research Project(No.2021DJ2104)the Science Foundation of China University of Petroleum,Beijing(No.2462021YXZZ010).
文摘Recent advances in deep neural networks have shed new light on physics,engineering,and scientific computing.Reconciling the data-centered viewpoint with physical simulation is one of the research hotspots.The physicsinformedneural network(PINN)is currently the most general framework,which is more popular due to theconvenience of constructing NNs and excellent generalization ability.The automatic differentiation(AD)-basedPINN model is suitable for the homogeneous scientific problem;however,it is unclear how AD can enforce fluxcontinuity across boundaries between cells of different properties where spatial heterogeneity is represented bygrid cells with different physical properties.In this work,we propose a criss-cross physics-informed convolutionalneural network(CC-PINN)learning architecture,aiming to learn the solution of parametric PDEs with spatialheterogeneity of physical properties.To achieve the seamless enforcement of flux continuity and integration ofphysicalmeaning into CNN,a predefined 2D convolutional layer is proposed to accurately express transmissibilitybetween adjacent cells.The efficacy of the proposedmethodwas evaluated through predictions of several petroleumreservoir problems with spatial heterogeneity and compared against state-of-the-art(PINN)through numericalanalysis as a benchmark,which demonstrated the superiority of the proposed method over the PINN.
基金supported by the National Natural Science Foundation of China,with Fund Numbers 62272478,62102451the National Defense Science and Technology Independent Research Project(Intelligent Information Hiding Technology and Its Applications in a Certain Field)and Science and Technology Innovation Team Innovative Research Project Research on Key Technologies for Intelligent Information Hiding”with Fund Number ZZKY20222102.
文摘As neural radiance fields continue to advance in 3D content representation,the copyright issues surrounding 3D models oriented towards implicit representation become increasingly pressing.In response to this challenge,this paper treats the embedding and extraction of neural radiance field watermarks as inverse problems of image transformations and proposes a scheme for protecting neural radiance field copyrights using invertible neural network watermarking.Leveraging 2D image watermarking technology for 3D scene protection,the scheme embeds watermarks within the training images of neural radiance fields through the forward process in invertible neural networks and extracts them from images rendered by neural radiance fields through the reverse process,thereby ensuring copyright protection for both the neural radiance fields and associated 3D scenes.However,challenges such as information loss during rendering processes and deliberate tampering necessitate the design of an image quality enhancement module to increase the scheme’s robustness.This module restores distorted images through neural network processing before watermark extraction.Additionally,embedding watermarks in each training image enables watermark information extraction from multiple viewpoints.Our proposed watermarking method achieves a PSNR(Peak Signal-to-Noise Ratio)value exceeding 37 dB for images containing watermarks and 22 dB for recovered watermarked images,as evaluated on the Lego,Hotdog,and Chair datasets,respectively.These results demonstrate the efficacy of our scheme in enhancing copyright protection.
基金support of the National Science Foundation(ECCS-1916839 and DMR-2202268).
文摘In a very recent study,Prof.Lingling Huang and co-workers proposed and demonstrated reconfigurable optical neural networks based on cascaded metasurfaces.By fixing one metasurface and switching the other pluggable metasurfaces,the neural networks,which operate at near-infrared wavelengths,can perform distinct recognition tasks for handwritten digits and fashion products.This innovative device opens up an avenue for all-optical,high-speed,low-power,and multifunctional artificial intelligence systems.
基金supported by the National Natural Science Foundation of China,with Fund Numbers 62272478,62102451the National Defense Science and Technology Independent Research Project(Intelligent Information Hiding Technology and Its Applications in a Certain Field)and Science and Technology Innovation Team Innovative Research Project“Research on Key Technologies for Intelligent Information Hiding”with Fund Number ZZKY20222102.
文摘Recent research advances in implicit neural representation have shown that a wide range of video data distributions are achieved by sharing model weights for Neural Representation for Videos(NeRV).While explicit methods exist for accurately embedding ownership or copyright information in video data,the nascent NeRV framework has yet to address this issue comprehensively.In response,this paper introduces MarkINeRV,a scheme designed to embed watermarking information into video frames using an invertible neural network watermarking approach to protect the copyright of NeRV,which models the embedding and extraction of watermarks as a pair of inverse processes of a reversible network and employs the same network to achieve embedding and extraction of watermarks.It is just that the information flow is in the opposite direction.Additionally,a video frame quality enhancement module is incorporated to mitigate watermarking information losses in the rendering process and the possibility ofmalicious attacks during transmission,ensuring the accurate extraction of watermarking information through the invertible network’s inverse process.This paper evaluates the accuracy,robustness,and invisibility of MarkINeRV through multiple video datasets.The results demonstrate its efficacy in extracting watermarking information for copyright protection of NeRV.MarkINeRV represents a pioneering investigation into copyright issues surrounding NeRV.
文摘The lethal brain tumor “Glioblastoma” has the propensity to grow over time. To improve patient outcomes, it is essential to classify GBM accurately and promptly in order to provide a focused and individualized treatment plan. Despite this, deep learning methods, particularly Convolutional Neural Networks (CNNs), have demonstrated a high level of accuracy in a myriad of medical image analysis applications as a result of recent technical breakthroughs. The overall aim of the research is to investigate how CNNs can be used to classify GBMs using data from medical imaging, to improve prognosis precision and effectiveness. This research study will demonstrate a suggested methodology that makes use of the CNN architecture and is trained using a database of MRI pictures with this tumor. The constructed model will be assessed based on its overall performance. Extensive experiments and comparisons with conventional machine learning techniques and existing classification methods will also be made. It will be crucial to emphasize the possibility of early and accurate prediction in a clinical workflow because it can have a big impact on treatment planning and patient outcomes. The paramount objective is to not only address the classification challenge but also to outline a clear pathway towards enhancing prognosis precision and treatment effectiveness.
