Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro...Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro-posed to improve the efficiency for edge inference of Deep Neural Networks(DNNs),existing PoT schemes require a huge amount of bit-wise manipulation and have large memory overhead,and their efficiency is bounded by the bottleneck of computation latency and memory footprint.To tackle this challenge,we present an efficient inference approach on the basis of PoT quantization and model compression.An integer-only scalar PoT quantization(IOS-PoT)is designed jointly with a distribution loss regularizer,wherein the regularizer minimizes quantization errors and training disturbances.Additionally,two-stage model compression is developed to effectively reduce memory requirement,and alleviate bandwidth usage in communications of networked heterogenous learning systems.The product look-up table(P-LUT)inference scheme is leveraged to replace bit-shifting with only indexing and addition operations for achieving low-latency computation and implementing efficient edge accelerators.Finally,comprehensive experiments on Residual Networks(ResNets)and efficient architectures with Canadian Institute for Advanced Research(CIFAR),ImageNet,and Real-world Affective Faces Database(RAF-DB)datasets,indicate that our approach achieves 2×∼10×improvement in the reduction of both weight size and computation cost in comparison to state-of-the-art methods.A P-LUT accelerator prototype is implemented on the Xilinx KV260 Field Programmable Gate Array(FPGA)platform for accelerating convolution operations,with performance results showing that P-LUT reduces memory footprint by 1.45×,achieves more than 3×power efficiency and 2×resource efficiency,compared to the conventional bit-shifting scheme.展开更多
Automated machine learning(AutoML)pruning methods aim at searching for a pruning strategy automatically to reduce the computational complexity of deep convolutional neural networks(deep CNNs).However,some previous wor...Automated machine learning(AutoML)pruning methods aim at searching for a pruning strategy automatically to reduce the computational complexity of deep convolutional neural networks(deep CNNs).However,some previous work found that the results of many Auto-ML pruning methods cannot even surpass the results of the uniformly pruning method.In this paper,the ineffectiveness of Auto-ML pruning,which is caused by unfull and unfair training of the supernet,is shown.A deep supernet suffers from unfull training because it contains too many candidates.To overcome the unfull training,a stage-wise pruning(SWP)method is proposed,which splits a deep supernet into several stage-wise supernets to reduce the candidate number and utilize inplace distillation to supervise the stage training.Besides,a wide supernet is hit by unfair training since the sampling probability of each channel is unequal.Therefore,the fullnet and the tinynet are sampled in each training iteration to ensure that each channel can be overtrained.Remarkably,the proxy performance of the subnets trained with SWP is closer to the actual performance than that of most of the previous AutoML pruning work.Furthermore,experiments show that SWP achieves the state-of-the-art in both CIFAR-10 and ImageNet under the mobile setting.展开更多
Facial emotion recognition achieves great success with the help of large neural models but also fails to be applied in practical situations due to the large model size of neural methods.To bridge this gap,in this pape...Facial emotion recognition achieves great success with the help of large neural models but also fails to be applied in practical situations due to the large model size of neural methods.To bridge this gap,in this paper,we combine two mainstream model compression methods(pruning and quantization)together,and propose a pruningthen-quantization framework to compress the neural models for facial emotion recognition tasks.Experiments on three datasets show that our model could achieve a high model compression ratio and maintain the model’s high performance well.Besides,We analyze the layer-wise compression performance of our proposed framework to explore its effect and adaptability in fine-grained modules.展开更多
Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the ...Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.展开更多
To better understand the failure behaviours and strength of bolt-reinforced blocky rocks,large scale extensive laboratory experiments are carried out on blocky rock-like specimens with and without rockbolt reinforceme...To better understand the failure behaviours and strength of bolt-reinforced blocky rocks,large scale extensive laboratory experiments are carried out on blocky rock-like specimens with and without rockbolt reinforcement.The results show that both shear failure and tensile failure along joint surfaces are observed but the shear failure is a main controlling factor for the peak strength of the rock mass with and without rockbolts.The rockbolts are necked and shear deformation simultaneously happens in bolt reinforced rock specimens.As the joint dip angle increases,the joint shear failure becomes more dominant.The number of rockbolts has a significant impact on the peak strain and uniaxial compressive strength(UCS),but little influence on the deformation modulus of the rock mass.Using the Winkler beam model to represent the rockbolt behaviours,an analytical model for the prediction of the strength of boltreinforced blocky rocks is proposed.Good agreement between the UCS values predicted by proposed model and obtained from experiments suggest an encouraging performance of the proposed model.In addition,the performance of the proposed model is further assessed using published results in the literature,indicating the proposed model can be used effectively in the prediction of UCS of bolt-reinforced blocky rocks.展开更多
BACKGROUND： Varying degrees of inflammatory responses occur during lumbar nerve root compression. Studies have shown that nitric oxide synthase （NOS） and calcitonin gene-related peptide （CGRP） are involved in sec...BACKGROUND： Varying degrees of inflammatory responses occur during lumbar nerve root compression. Studies have shown that nitric oxide synthase （NOS） and calcitonin gene-related peptide （CGRP） are involved in secondary disc inflammation. OBJECTIVE： To observe the effects of warm acupuncture on the ultrastructure of inflammatory mediators in a rat model of lumbar nerve root compression, including NOS and CGRP contents. DESIGN, TIME AND SETTING： Randomized, controlled study, with molecular biological analysis, was performed at the Experimental Center, Sixth People＇s Hospital Affiliated to Shanghai Jiao Tong University, between September 2006 and April 2007. MATERIALS： Acupuncture needles and refined Moxa grains were purchased from Shanghai Taicheng Technology Development Co., Ltd., China; Mobic tablets were purchased from Shanghai Boehringer Ingelheim Pharmaceuticals Co., Ltd., China; enzyme linked immunosorbent assay （ELISA） kits for NOS and CGRP were purchased from ADL Biotechnology, Inc., USA. METHODS： A total of 50, healthy, adult Sprague-Dawley rats, were randomly divided into five groups normal, model, warm acupuncture, acupuncture, and drug, with 10 rats in each group. Rats in the four groups, excluding the normal group, were used to establish models of lumbar nerve root compression. After 3 days, Jiaji points were set using reinforcing-reducing manipulation in the warm acupuncture group. Moxa grains were burned on each needle, with 2 grains each daily. The acupuncture group was the same as the warm acupuncture group, with the exception of non-moxibustion. Mobic suspension （3.75 mg/kg） was used in the oral drug group, once a day. Treatment of each group lasted for 14 consecutive days. Modeling and medication were not performed in the normal group. MAIN OUTCOME MEASURES： The ultrastructure of damaged nerve roots was observed with transmission electron microscopy; NOS and CGRP contents were measured using ELISA. RESULTS： The changes of the radicular ultramicrostructure were characterized by Wallerian degeneration; nerve fibers were clearly demyelinated; axons collapsed or degenerated; outer Schwann cell cytoplasm was swollen and its nucleus was compacted. Compared with the normal group, NOS and CGRP contents in the nerve root compression zone in the model group were significantly increased （P 〈 0.01）. Nerve root edema was improved in the drug, acupuncture and the warm acupuncture groups over the model group. NOS and CGRP expressions were also decreased with the warm acupuncture group having the lowest concentration （P 〈 0.01）. CONCLUSION： In comparison to the known effects of Mobic drug and acupuncture treatments, the warm acupuncture significantly decreased NOS and CGRP expression which helped improve the ultrastructure of the compressed nerve root.展开更多
A novel deep neural network compression model for airport object detection has been presented.This novel model aims at disadvantages of deep neural network,i.e.the complexity of the model and the great cost of calcula...A novel deep neural network compression model for airport object detection has been presented.This novel model aims at disadvantages of deep neural network,i.e.the complexity of the model and the great cost of calculation.According to the requirement of airport object detection,the model obtains temporal and spatial semantic rules from the uncompressed model.These spatial semantic rules are added to the model after parameter compression to assist the detection.The rules can improve the accuracy of the detection model in order to make up for the loss caused by parameter compression.The experiments show that the effect of the novel compression detection model is no worse than that of the uncompressed original model.Even some of the original model false detection can be eliminated through the prior knowledge.展开更多
[Objectives]To observe the effect of Xianlinggubao Capsule on osteoporotic vertebral compression fracture(OVCF)in rabbits and the influence mechanism of the repair of fractures.[Methods]Female June age 30 rabbits were...[Objectives]To observe the effect of Xianlinggubao Capsule on osteoporotic vertebral compression fracture(OVCF)in rabbits and the influence mechanism of the repair of fractures.[Methods]Female June age 30 rabbits were randomly divided into control group,model control group and Xianlinggubao group.After bilateral ovariectomy,the model control group and Xianlinggubao group were injected with dexamethasone continuously for 4 weeks,and then the OVCF compound model was established by surgery.The Xianlinggubao group was treated with Xianlinggubao at a dose of 300 mg/(kg·d)for 60 d,while the blank control group and the model control group were treated with the same amount of normal saline for 60 d.The number of blood vessels and the expression of bone morphogenetic protein-2(BMP-2)were detected by immunohistochemical staining and the bone mineral density(BMD)in the callus of the third lumbar fracture area of rabbits was measured.The content of serum phosphorus(P),alkaline phosphatase(ALP)and total calcium(TCa)in rabbit venous blood were measured by automatic biochemical analyzer.The content of vascular endothelial growth factor(VEGF)and platelet-derived growth factor(PDGF)in rabbit venous blood were measured by ELISA kit.[Results]The number of blood vessels and the expression of BMP-2 in the callus of the third lumbar fracture area of rabbits was high in Xianlinggubao group,the content of serum P,ALP,TCa,VEGF and PDGF was obviously increased,BMD was obviously increased,the bone microstructure of the third lumbar vertebrae fracture area of rabbits was basically restored.Compared with the model control group(P<0.05),the difference was statistically significant.[Conclusions]Xianlinggubao Capsule can increase calcium and phosphorus deposition,promote the formation of blood vessels in the fracture area of OVCF in rabbits,and have a strong repair effect on OVCF in rabbits.展开更多
Integrating Tiny Machine Learning(TinyML)with edge computing in remotely sensed images enhances the capabilities of road anomaly detection on a broader level.Constrained devices efficiently implement a Binary Neural N...Integrating Tiny Machine Learning(TinyML)with edge computing in remotely sensed images enhances the capabilities of road anomaly detection on a broader level.Constrained devices efficiently implement a Binary Neural Network(BNN)for road feature extraction,utilizing quantization and compression through a pruning strategy.The modifications resulted in a 28-fold decrease in memory usage and a 25%enhancement in inference speed while only experiencing a 2.5%decrease in accuracy.It showcases its superiority over conventional detection algorithms in different road image scenarios.Although constrained by computer resources and training datasets,our results indicate opportunities for future research,demonstrating that quantization and focused optimization can significantly improve machine learning models’accuracy and operational efficiency.ARM Cortex-M0 gives practical feasibility and substantial benefits while deploying our optimized BNN model on this low-power device:Advanced machine learning in edge computing.The analysis work delves into the educational significance of TinyML and its essential function in analyzing road networks using remote sensing,suggesting ways to improve smart city frameworks in road network assessment,traffic management,and autonomous vehicle navigation systems by emphasizing the importance of new technologies for maintaining and safeguarding road networks.展开更多
Deep neural networks excel at image identification and computer vision applications such as visual product search, facial recognition, medical image analysis, object detection, semantic segmentation,instance segmentat...Deep neural networks excel at image identification and computer vision applications such as visual product search, facial recognition, medical image analysis, object detection, semantic segmentation,instance segmentation, and many others. In image and video recognition applications, convolutional neural networks(CNNs) are widely employed. These networks provide better performance but at a higher cost of computation. With the advent of big data, the growing scale of datasets has made processing and model training a time-consuming operation, resulting in longer training times. Moreover, these large scale datasets contain redundant data points that have minimum impact on the final outcome of the model. To address these issues, an accelerated CNN system is proposed for speeding up training by eliminating the noncritical data points during training alongwith a model compression method. Furthermore, the identification of the critical input data is performed by aggregating the data points at two levels of granularity which are used for evaluating the impact on the model output.Extensive experiments are conducted using the proposed method on CIFAR-10 dataset on ResNet models giving a 40% reduction in number of FLOPs with a degradation of just 0.11% accuracy.展开更多
Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices...Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices,and it is not environmental-friendly with much power cost.In this paper,we focus on low-rank optimization for efficient deep learning techniques.In the space domain,DNNs are compressed by low rank approximation of the network parameters,which directly reduces the storage requirement with a smaller number of network parameters.In the time domain,the network parameters can be trained in a few subspaces,which enables efficient training for fast convergence.The model compression in the spatial domain is summarized into three categories as pre-train,pre-set,and compression-aware methods,respectively.With a series of integrable techniques discussed,such as sparse pruning,quantization,and entropy coding,we can ensemble them in an integration framework with lower computational complexity and storage.In addition to summary of recent technical advances,we have two findings for motivating future works.One is that the effective rank,derived from the Shannon entropy of the normalized singular values,outperforms other conventional sparse measures such as the?_1 norm for network compression.The other is a spatial and temporal balance for tensorized neural networks.For accelerating the training of tensorized neural networks,it is crucial to leverage redundancy for both model compression and subspace training.展开更多
Knowledge distillation,as a pivotal technique in the field of model compression,has been widely applied across various domains.However,the problem of student model performance being limited due to inherent biases in t...Knowledge distillation,as a pivotal technique in the field of model compression,has been widely applied across various domains.However,the problem of student model performance being limited due to inherent biases in the teacher model during the distillation process still persists.To address the inherent biases in knowledge distillation,we propose a de-biased knowledge distillation framework tailored for binary classification tasks.For the pre-trained teacher model,biases in the soft labels are mitigated through knowledge infusion and label de-biasing techniques.Based on this,a de-biased distillation loss is introduced,allowing the de-biased labels to replace the soft labels as the fitting target for the student model.This approach enables the student model to learn from the corrected model information,achieving high-performance deployment on lightweight student models.Experiments conducted on multiple real-world datasets demonstrate that deep learning models compressed under the de-biased knowledge distillation framework significantly outperform traditional response-based and feature-based knowledge distillation models across various evaluation metrics,highlighting the effectiveness and superiority of the de-biased knowledge distillation framework in model compression.展开更多
The high temperature split Hopkinson pressure bar （SHPB） compression experiment is conducted to obtain the data relationship among strain, strain rate and flow stress from room temperature to 550 C for aeronautical ...The high temperature split Hopkinson pressure bar （SHPB） compression experiment is conducted to obtain the data relationship among strain, strain rate and flow stress from room temperature to 550 C for aeronautical aluminum alloy 7050-T7451. Combined high-speed orthogonal cutting experiments with the cutting process simulations, the data relationship of high temperature, high strain rate and large strain in high-speed cutting is modified. The Johnson-Cook empirical model considering the effects of strain hardening, strain rate hardening and thermal softening is selected to describe the data relationship in high-speed cutting, and the material constants of flow stress constitutive model for aluminum alloy 7050-T7451 are determined. Finally, the constitutive model of aluminum alloy 7050-T7451 is established through experiment and simulation verification in high-speed cutting. The model is proved to be reasonable by matching the measured values of the cutting force with the estimated results from FEM simulations.展开更多
Deep learning technology has been widely used in computer vision,speech recognition,natural language processing,and other related fields.The deep learning algorithm has high precision and high reliability.However,the ...Deep learning technology has been widely used in computer vision,speech recognition,natural language processing,and other related fields.The deep learning algorithm has high precision and high reliability.However,the lack of resources in the edge terminal equipment makes it difficult to run deep learning algorithms that require more memory and computing power.In this paper,we propose MoTransFrame,a general model processing framework for deep learning models.Instead of designing a model compression algorithm with a high compression ratio,MoTransFrame can transplant popular convolutional neural networks models to resources-starved edge devices promptly and accurately.By the integration method,Deep learning models can be converted into portable projects for Arduino,a typical edge device with limited resources.Our experiments show that MoTransFrame has good adaptability in edge devices with limited memories.It is more flexible than other model transplantation methods.It can keep a small loss of model accuracy when the number of parameters is compressed by tens of times.At the same time,the computational resources needed in the reasoning process are less than what the edge node could handle.展开更多
In this article, we consider the blowup criterion for the local strong solution to the compressible fluid-particle interaction model in dimension three with vacuum. We establish a BKM type criterion for possible break...In this article, we consider the blowup criterion for the local strong solution to the compressible fluid-particle interaction model in dimension three with vacuum. We establish a BKM type criterion for possible breakdown of such solutions at critical time in terms of both the L^∞ （0, T; L^6）-norm of the density of particles and the ^L1（0, T; L^∞）-norm of the deformation tensor of velocity gradient.展开更多
The two-phase flow models are commonly used in industrial applications, such as nuclear, power, chemical-process, oil-and-gas, cryogenics, bio-medical, micro-technology and so on. This is a survey paper on the study o...The two-phase flow models are commonly used in industrial applications, such as nuclear, power, chemical-process, oil-and-gas, cryogenics, bio-medical, micro-technology and so on. This is a survey paper on the study of compressible nonconservative two-fluid model, drift-flux model and viscous liquid-gas two-phase flow model. We give the research developments of these three two-phase flow models, respectively. In the last part, we give some open problems about the above models.展开更多
Recently,many regression models have been presented for prediction of mechanical parameters of rocks regarding to rock index properties.Although statistical analysis is a common method for developing regression models...Recently,many regression models have been presented for prediction of mechanical parameters of rocks regarding to rock index properties.Although statistical analysis is a common method for developing regression models,but still selection of suitable transformation of the independent variables in a regression model is diffcult.In this paper,a genetic algorithm(GA)has been employed as a heuristic search method for selection of best transformation of the independent variables(some index properties of rocks)in regression models for prediction of uniaxial compressive strength(UCS)and modulus of elasticity(E).Firstly,multiple linear regression(MLR)analysis was performed on a data set to establish predictive models.Then,two GA models were developed in which root mean squared error(RMSE)was defned as ftness function.Results have shown that GA models are more precise than MLR models and are able to explain the relation between the intrinsic strength/elasticity properties and index properties of rocks by simple formulation and accepted accuracy.展开更多
In this article,we focus on the short time strong solution to a compressible quantum hydrodynamic model.We establish a blow-up criterion about the solutions of the compressible quantum hydrodynamic model in terms of t...In this article,we focus on the short time strong solution to a compressible quantum hydrodynamic model.We establish a blow-up criterion about the solutions of the compressible quantum hydrodynamic model in terms of the gradient of the velocity,the second spacial derivative of the square root of the density,and the first order time derivative and first order spacial derivative of the square root of the density.展开更多
With the high-quality development of urban buildings,higher requirements are come up with for lateral bearing capacity of laterally loaded piles.Consequently,a more accurate analysis to predict the lateral response of...With the high-quality development of urban buildings,higher requirements are come up with for lateral bearing capacity of laterally loaded piles.Consequently,a more accurate analysis to predict the lateral response of the pile within an allowable displacement is an important issue.However,the current p-y curve methods cannot fully take into account the pile-soil interaction,which will lead to a large calculation difference.In this paper,a new analytical p-y curve is established and a finite difference method for determining the lateral response of pile is proposed,which can consider the separation effect of pile-soil interface and the coefficient of circumferential friction resistance.In particular,an analytical expression is developed to determine the compressive soil pressure by dividing the compressive soil pressure into two parts:initial compressive soil pressure and increment of compressive soil pressure.In addition,the relationship between compressive soil pressure and horizontal displacement of the pile is established based on the reasonable assumption.The correctness of the proposed method is verified through four examples.Based on the verified method,a parametric analysis is also conducted to investigate the influences of factors on lateral response of the pile,including internal friction angle,pile length and elastic modulus of pile.展开更多
The isothermal compression tests were carried out on Gleeble-3500 thermal-mechanical simulation machine in a temperature range of 298-473 K and strain rate range of 0.001-10 s^-1. The experimental results show that th...The isothermal compression tests were carried out on Gleeble-3500 thermal-mechanical simulation machine in a temperature range of 298-473 K and strain rate range of 0.001-10 s^-1. The experimental results show that the flow stress data are negatively correlated with temperature for temperature softening, and the strain rates sensitivity of this composite increases with elevating temperature. Based on the experimental data, Johnson-Cook, modified Johnson-Cook and Arrhenius constitutive models were established. The accuracy of these three constitutive models was analyzed and compared. The results show that the values predicted by Johnson-Cook model could not agree well with the experimental values. The prediction accuracy of Arrhenius model is higher than that of Johnson-Cook model but lower than that of the Modified Johnson-Cook model.展开更多
基金This work was supported by Open Fund Project of State Key Laboratory of Intelligent Vehicle Safety Technology by Grant with No.IVSTSKL-202311Key Projects of Science and Technology Research Programme of Chongqing Municipal Education Commission by Grant with No.KJZD-K202301505+1 种基金Cooperation Project between Chongqing Municipal Undergraduate Universities and Institutes Affiliated to the Chinese Academy of Sciences in 2021 by Grant with No.HZ2021015Chongqing Graduate Student Research Innovation Program by Grant with No.CYS240801.
文摘Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro-posed to improve the efficiency for edge inference of Deep Neural Networks(DNNs),existing PoT schemes require a huge amount of bit-wise manipulation and have large memory overhead,and their efficiency is bounded by the bottleneck of computation latency and memory footprint.To tackle this challenge,we present an efficient inference approach on the basis of PoT quantization and model compression.An integer-only scalar PoT quantization(IOS-PoT)is designed jointly with a distribution loss regularizer,wherein the regularizer minimizes quantization errors and training disturbances.Additionally,two-stage model compression is developed to effectively reduce memory requirement,and alleviate bandwidth usage in communications of networked heterogenous learning systems.The product look-up table(P-LUT)inference scheme is leveraged to replace bit-shifting with only indexing and addition operations for achieving low-latency computation and implementing efficient edge accelerators.Finally,comprehensive experiments on Residual Networks(ResNets)and efficient architectures with Canadian Institute for Advanced Research(CIFAR),ImageNet,and Real-world Affective Faces Database(RAF-DB)datasets,indicate that our approach achieves 2×∼10×improvement in the reduction of both weight size and computation cost in comparison to state-of-the-art methods.A P-LUT accelerator prototype is implemented on the Xilinx KV260 Field Programmable Gate Array(FPGA)platform for accelerating convolution operations,with performance results showing that P-LUT reduces memory footprint by 1.45×,achieves more than 3×power efficiency and 2×resource efficiency,compared to the conventional bit-shifting scheme.
基金This work was supported by Natural Science Foundation of Zhejiang Province,China(No.LY21F030018)National Key R&D Program of China(No.2018YFB 1308400).
文摘Automated machine learning(AutoML)pruning methods aim at searching for a pruning strategy automatically to reduce the computational complexity of deep convolutional neural networks(deep CNNs).However,some previous work found that the results of many Auto-ML pruning methods cannot even surpass the results of the uniformly pruning method.In this paper,the ineffectiveness of Auto-ML pruning,which is caused by unfull and unfair training of the supernet,is shown.A deep supernet suffers from unfull training because it contains too many candidates.To overcome the unfull training,a stage-wise pruning(SWP)method is proposed,which splits a deep supernet into several stage-wise supernets to reduce the candidate number and utilize inplace distillation to supervise the stage training.Besides,a wide supernet is hit by unfair training since the sampling probability of each channel is unequal.Therefore,the fullnet and the tinynet are sampled in each training iteration to ensure that each channel can be overtrained.Remarkably,the proxy performance of the subnets trained with SWP is closer to the actual performance than that of most of the previous AutoML pruning work.Furthermore,experiments show that SWP achieves the state-of-the-art in both CIFAR-10 and ImageNet under the mobile setting.
基金supported in part by the Technological Breakthrough Project of Science,Technology and Innovation Commission of Shenzhen Municipality(No.JSGG20201102162000001)InnoHK Initiative of Hong Kong SAR Government,and the Laboratory for AI-Powered Financial Technologies Ltd.
文摘Facial emotion recognition achieves great success with the help of large neural models but also fails to be applied in practical situations due to the large model size of neural methods.To bridge this gap,in this paper,we combine two mainstream model compression methods(pruning and quantization)together,and propose a pruningthen-quantization framework to compress the neural models for facial emotion recognition tasks.Experiments on three datasets show that our model could achieve a high model compression ratio and maintain the model’s high performance well.Besides,We analyze the layer-wise compression performance of our proposed framework to explore its effect and adaptability in fine-grained modules.
基金supported by Sichuan Science and Technology Program(2023YFSY0026,2023YFH0004).
文摘Recent advancements in natural language processing have given rise to numerous pre-training language models in question-answering systems.However,with the constant evolution of algorithms,data,and computing power,the increasing size and complexity of these models have led to increased training costs and reduced efficiency.This study aims to minimize the inference time of such models while maintaining computational performance.It also proposes a novel Distillation model for PAL-BERT(DPAL-BERT),specifically,employs knowledge distillation,using the PAL-BERT model as the teacher model to train two student models:DPAL-BERT-Bi and DPAL-BERTC.This research enhances the dataset through techniques such as masking,replacement,and n-gram sampling to optimize knowledge transfer.The experimental results showed that the distilled models greatly outperform models trained from scratch.In addition,although the distilled models exhibit a slight decrease in performance compared to PAL-BERT,they significantly reduce inference time to just 0.25%of the original.This demonstrates the effectiveness of the proposed approach in balancing model performance and efficiency.
基金supported by the National Key Research and Development Projects of China(No.2021YFB2600402)National Natural Science Foundation of China(Nos.52209148 and 52374119)+1 种基金the opening fund of State Key Laboratory of Geomechanics and Geotechnical Engineering,Institute of Rock and Soil Mechanics,Chinese Academy of Sciences(No.SKLGME023023)the opening fund of Key Laboratory of Water Management and Water Security for Yellow River Basin,Ministry of Water Resources(No.2023-SYSJJ-02)。
文摘To better understand the failure behaviours and strength of bolt-reinforced blocky rocks,large scale extensive laboratory experiments are carried out on blocky rock-like specimens with and without rockbolt reinforcement.The results show that both shear failure and tensile failure along joint surfaces are observed but the shear failure is a main controlling factor for the peak strength of the rock mass with and without rockbolts.The rockbolts are necked and shear deformation simultaneously happens in bolt reinforced rock specimens.As the joint dip angle increases,the joint shear failure becomes more dominant.The number of rockbolts has a significant impact on the peak strain and uniaxial compressive strength(UCS),but little influence on the deformation modulus of the rock mass.Using the Winkler beam model to represent the rockbolt behaviours,an analytical model for the prediction of the strength of boltreinforced blocky rocks is proposed.Good agreement between the UCS values predicted by proposed model and obtained from experiments suggest an encouraging performance of the proposed model.In addition,the performance of the proposed model is further assessed using published results in the literature,indicating the proposed model can be used effectively in the prediction of UCS of bolt-reinforced blocky rocks.
基金Modern Projects of Traditional Chinese Medicine of Shanghai Science and Technology Commission, No.08DZ1973200Research Projects of Shanghai Bureau of Public Health,No.2006Q004L
文摘BACKGROUND： Varying degrees of inflammatory responses occur during lumbar nerve root compression. Studies have shown that nitric oxide synthase （NOS） and calcitonin gene-related peptide （CGRP） are involved in secondary disc inflammation. OBJECTIVE： To observe the effects of warm acupuncture on the ultrastructure of inflammatory mediators in a rat model of lumbar nerve root compression, including NOS and CGRP contents. DESIGN, TIME AND SETTING： Randomized, controlled study, with molecular biological analysis, was performed at the Experimental Center, Sixth People＇s Hospital Affiliated to Shanghai Jiao Tong University, between September 2006 and April 2007. MATERIALS： Acupuncture needles and refined Moxa grains were purchased from Shanghai Taicheng Technology Development Co., Ltd., China; Mobic tablets were purchased from Shanghai Boehringer Ingelheim Pharmaceuticals Co., Ltd., China; enzyme linked immunosorbent assay （ELISA） kits for NOS and CGRP were purchased from ADL Biotechnology, Inc., USA. METHODS： A total of 50, healthy, adult Sprague-Dawley rats, were randomly divided into five groups normal, model, warm acupuncture, acupuncture, and drug, with 10 rats in each group. Rats in the four groups, excluding the normal group, were used to establish models of lumbar nerve root compression. After 3 days, Jiaji points were set using reinforcing-reducing manipulation in the warm acupuncture group. Moxa grains were burned on each needle, with 2 grains each daily. The acupuncture group was the same as the warm acupuncture group, with the exception of non-moxibustion. Mobic suspension （3.75 mg/kg） was used in the oral drug group, once a day. Treatment of each group lasted for 14 consecutive days. Modeling and medication were not performed in the normal group. MAIN OUTCOME MEASURES： The ultrastructure of damaged nerve roots was observed with transmission electron microscopy; NOS and CGRP contents were measured using ELISA. RESULTS： The changes of the radicular ultramicrostructure were characterized by Wallerian degeneration; nerve fibers were clearly demyelinated; axons collapsed or degenerated; outer Schwann cell cytoplasm was swollen and its nucleus was compacted. Compared with the normal group, NOS and CGRP contents in the nerve root compression zone in the model group were significantly increased （P 〈 0.01）. Nerve root edema was improved in the drug, acupuncture and the warm acupuncture groups over the model group. NOS and CGRP expressions were also decreased with the warm acupuncture group having the lowest concentration （P 〈 0.01）. CONCLUSION： In comparison to the known effects of Mobic drug and acupuncture treatments, the warm acupuncture significantly decreased NOS and CGRP expression which helped improve the ultrastructure of the compressed nerve root.
文摘A novel deep neural network compression model for airport object detection has been presented.This novel model aims at disadvantages of deep neural network,i.e.the complexity of the model and the great cost of calculation.According to the requirement of airport object detection,the model obtains temporal and spatial semantic rules from the uncompressed model.These spatial semantic rules are added to the model after parameter compression to assist the detection.The rules can improve the accuracy of the detection model in order to make up for the loss caused by parameter compression.The experiments show that the effect of the novel compression detection model is no worse than that of the uncompressed original model.Even some of the original model false detection can be eliminated through the prior knowledge.
基金Supported by Shiyan Taihe Hospital Project(2021JJXM084)General Project of Hubei Provincial Health and Health Commission(ZY2021M006).
文摘[Objectives]To observe the effect of Xianlinggubao Capsule on osteoporotic vertebral compression fracture(OVCF)in rabbits and the influence mechanism of the repair of fractures.[Methods]Female June age 30 rabbits were randomly divided into control group,model control group and Xianlinggubao group.After bilateral ovariectomy,the model control group and Xianlinggubao group were injected with dexamethasone continuously for 4 weeks,and then the OVCF compound model was established by surgery.The Xianlinggubao group was treated with Xianlinggubao at a dose of 300 mg/(kg·d)for 60 d,while the blank control group and the model control group were treated with the same amount of normal saline for 60 d.The number of blood vessels and the expression of bone morphogenetic protein-2(BMP-2)were detected by immunohistochemical staining and the bone mineral density(BMD)in the callus of the third lumbar fracture area of rabbits was measured.The content of serum phosphorus(P),alkaline phosphatase(ALP)and total calcium(TCa)in rabbit venous blood were measured by automatic biochemical analyzer.The content of vascular endothelial growth factor(VEGF)and platelet-derived growth factor(PDGF)in rabbit venous blood were measured by ELISA kit.[Results]The number of blood vessels and the expression of BMP-2 in the callus of the third lumbar fracture area of rabbits was high in Xianlinggubao group,the content of serum P,ALP,TCa,VEGF and PDGF was obviously increased,BMD was obviously increased,the bone microstructure of the third lumbar vertebrae fracture area of rabbits was basically restored.Compared with the model control group(P<0.05),the difference was statistically significant.[Conclusions]Xianlinggubao Capsule can increase calcium and phosphorus deposition,promote the formation of blood vessels in the fracture area of OVCF in rabbits,and have a strong repair effect on OVCF in rabbits.
基金supported by the National Natural Science Foundation of China(61170147)Scientific Research Project of Zhejiang Provincial Department of Education in China(Y202146796)+2 种基金Natural Science Foundation of Zhejiang Province in China(LTY22F020003)Wenzhou Major Scientific and Technological Innovation Project of China(ZG2021029)Scientific and Technological Projects of Henan Province in China(202102210172).
文摘Integrating Tiny Machine Learning(TinyML)with edge computing in remotely sensed images enhances the capabilities of road anomaly detection on a broader level.Constrained devices efficiently implement a Binary Neural Network(BNN)for road feature extraction,utilizing quantization and compression through a pruning strategy.The modifications resulted in a 28-fold decrease in memory usage and a 25%enhancement in inference speed while only experiencing a 2.5%decrease in accuracy.It showcases its superiority over conventional detection algorithms in different road image scenarios.Although constrained by computer resources and training datasets,our results indicate opportunities for future research,demonstrating that quantization and focused optimization can significantly improve machine learning models’accuracy and operational efficiency.ARM Cortex-M0 gives practical feasibility and substantial benefits while deploying our optimized BNN model on this low-power device:Advanced machine learning in edge computing.The analysis work delves into the educational significance of TinyML and its essential function in analyzing road networks using remote sensing,suggesting ways to improve smart city frameworks in road network assessment,traffic management,and autonomous vehicle navigation systems by emphasizing the importance of new technologies for maintaining and safeguarding road networks.
文摘Deep neural networks excel at image identification and computer vision applications such as visual product search, facial recognition, medical image analysis, object detection, semantic segmentation,instance segmentation, and many others. In image and video recognition applications, convolutional neural networks(CNNs) are widely employed. These networks provide better performance but at a higher cost of computation. With the advent of big data, the growing scale of datasets has made processing and model training a time-consuming operation, resulting in longer training times. Moreover, these large scale datasets contain redundant data points that have minimum impact on the final outcome of the model. To address these issues, an accelerated CNN system is proposed for speeding up training by eliminating the noncritical data points during training alongwith a model compression method. Furthermore, the identification of the critical input data is performed by aggregating the data points at two levels of granularity which are used for evaluating the impact on the model output.Extensive experiments are conducted using the proposed method on CIFAR-10 dataset on ResNet models giving a 40% reduction in number of FLOPs with a degradation of just 0.11% accuracy.
基金supported by the National Natural Science Foundation of China(62171088,U19A2052,62020106011)the Medico-Engineering Cooperation Funds from University of Electronic Science and Technology of China(ZYGX2021YGLH215,ZYGX2022YGRH005)。
文摘Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices,and it is not environmental-friendly with much power cost.In this paper,we focus on low-rank optimization for efficient deep learning techniques.In the space domain,DNNs are compressed by low rank approximation of the network parameters,which directly reduces the storage requirement with a smaller number of network parameters.In the time domain,the network parameters can be trained in a few subspaces,which enables efficient training for fast convergence.The model compression in the spatial domain is summarized into three categories as pre-train,pre-set,and compression-aware methods,respectively.With a series of integrable techniques discussed,such as sparse pruning,quantization,and entropy coding,we can ensemble them in an integration framework with lower computational complexity and storage.In addition to summary of recent technical advances,we have two findings for motivating future works.One is that the effective rank,derived from the Shannon entropy of the normalized singular values,outperforms other conventional sparse measures such as the?_1 norm for network compression.The other is a spatial and temporal balance for tensorized neural networks.For accelerating the training of tensorized neural networks,it is crucial to leverage redundancy for both model compression and subspace training.
基金supported by the National Natural Science Foundation of China under Grant No.62172056Young Elite Scientists Sponsorship Program by CAST under Grant No.2022QNRC001.
文摘Knowledge distillation,as a pivotal technique in the field of model compression,has been widely applied across various domains.However,the problem of student model performance being limited due to inherent biases in the teacher model during the distillation process still persists.To address the inherent biases in knowledge distillation,we propose a de-biased knowledge distillation framework tailored for binary classification tasks.For the pre-trained teacher model,biases in the soft labels are mitigated through knowledge infusion and label de-biasing techniques.Based on this,a de-biased distillation loss is introduced,allowing the de-biased labels to replace the soft labels as the fitting target for the student model.This approach enables the student model to learn from the corrected model information,achieving high-performance deployment on lightweight student models.Experiments conducted on multiple real-world datasets demonstrate that deep learning models compressed under the de-biased knowledge distillation framework significantly outperform traditional response-based and feature-based knowledge distillation models across various evaluation metrics,highlighting the effectiveness and superiority of the de-biased knowledge distillation framework in model compression.
文摘The high temperature split Hopkinson pressure bar （SHPB） compression experiment is conducted to obtain the data relationship among strain, strain rate and flow stress from room temperature to 550 C for aeronautical aluminum alloy 7050-T7451. Combined high-speed orthogonal cutting experiments with the cutting process simulations, the data relationship of high temperature, high strain rate and large strain in high-speed cutting is modified. The Johnson-Cook empirical model considering the effects of strain hardening, strain rate hardening and thermal softening is selected to describe the data relationship in high-speed cutting, and the material constants of flow stress constitutive model for aluminum alloy 7050-T7451 are determined. Finally, the constitutive model of aluminum alloy 7050-T7451 is established through experiment and simulation verification in high-speed cutting. The model is proved to be reasonable by matching the measured values of the cutting force with the estimated results from FEM simulations.
基金supported by The National Key Research and Development Program of China(2018YFB1800202,2016YFB1000302,SQ2019ZD090149,2018YFB0204301)the CETC Joint Advanced Research Foundation(6141B08080101)+1 种基金The Major Special Science and Technology Project of Hainan Province(ZDKJ2019008)The New Generation of Artificial Intelligence Special Action Project(AI20191125008).
文摘Deep learning technology has been widely used in computer vision,speech recognition,natural language processing,and other related fields.The deep learning algorithm has high precision and high reliability.However,the lack of resources in the edge terminal equipment makes it difficult to run deep learning algorithms that require more memory and computing power.In this paper,we propose MoTransFrame,a general model processing framework for deep learning models.Instead of designing a model compression algorithm with a high compression ratio,MoTransFrame can transplant popular convolutional neural networks models to resources-starved edge devices promptly and accurately.By the integration method,Deep learning models can be converted into portable projects for Arduino,a typical edge device with limited resources.Our experiments show that MoTransFrame has good adaptability in edge devices with limited memories.It is more flexible than other model transplantation methods.It can keep a small loss of model accuracy when the number of parameters is compressed by tens of times.At the same time,the computational resources needed in the reasoning process are less than what the edge node could handle.
基金supported by the National Basic Research Program of China(973 Program)(2011CB808002)the National Natural Science Foundation of China(11371152,11128102,11071086,and 11571117)+3 种基金the Natural Science Foundation of Guangdong Province(S2012010010408)the Foundation for Distinguished Young Talents in Higher Education of Guangdong(2015KQNCX095)the Major Foundation of Hanshan Normal University(LZ201403)the Scientific Research Foundation of Graduate School of South China Normal University(2014ssxm04)
文摘In this article, we consider the blowup criterion for the local strong solution to the compressible fluid-particle interaction model in dimension three with vacuum. We establish a BKM type criterion for possible breakdown of such solutions at critical time in terms of both the L^∞ （0, T; L^6）-norm of the density of particles and the ^L1（0, T; L^∞）-norm of the deformation tensor of velocity gradient.
基金supported by the National Natural Science Foundation of China(11722104,11671150)supported by the National Natural Science Foundation of China(11571280,11331005)+3 种基金supported by the National Natural Science Foundation of China(11331005,11771150)by GDUPS(2016)the Fundamental Research Funds for the Central Universities of China(D2172260)FANEDD No.201315
文摘The two-phase flow models are commonly used in industrial applications, such as nuclear, power, chemical-process, oil-and-gas, cryogenics, bio-medical, micro-technology and so on. This is a survey paper on the study of compressible nonconservative two-fluid model, drift-flux model and viscous liquid-gas two-phase flow model. We give the research developments of these three two-phase flow models, respectively. In the last part, we give some open problems about the above models.
文摘Recently,many regression models have been presented for prediction of mechanical parameters of rocks regarding to rock index properties.Although statistical analysis is a common method for developing regression models,but still selection of suitable transformation of the independent variables in a regression model is diffcult.In this paper,a genetic algorithm(GA)has been employed as a heuristic search method for selection of best transformation of the independent variables(some index properties of rocks)in regression models for prediction of uniaxial compressive strength(UCS)and modulus of elasticity(E).Firstly,multiple linear regression(MLR)analysis was performed on a data set to establish predictive models.Then,two GA models were developed in which root mean squared error(RMSE)was defned as ftness function.Results have shown that GA models are more precise than MLR models and are able to explain the relation between the intrinsic strength/elasticity properties and index properties of rocks by simple formulation and accepted accuracy.
基金The first author is supported by the National Natural Science Foundation of China(11801107)the second author is supported by the National Natural Science Foundation of China(11731014).
文摘In this article,we focus on the short time strong solution to a compressible quantum hydrodynamic model.We establish a blow-up criterion about the solutions of the compressible quantum hydrodynamic model in terms of the gradient of the velocity,the second spacial derivative of the square root of the density,and the first order time derivative and first order spacial derivative of the square root of the density.
基金Project(52068004)supported by the National Natural Science Foundation of ChinaProject(2018JJA160134)supported by the Natural Science Foundation of Guangxi Province,ChinaProject(AB19245018)supported by Key Research Projects of Guangxi Province,China。
文摘With the high-quality development of urban buildings,higher requirements are come up with for lateral bearing capacity of laterally loaded piles.Consequently,a more accurate analysis to predict the lateral response of the pile within an allowable displacement is an important issue.However,the current p-y curve methods cannot fully take into account the pile-soil interaction,which will lead to a large calculation difference.In this paper,a new analytical p-y curve is established and a finite difference method for determining the lateral response of pile is proposed,which can consider the separation effect of pile-soil interface and the coefficient of circumferential friction resistance.In particular,an analytical expression is developed to determine the compressive soil pressure by dividing the compressive soil pressure into two parts:initial compressive soil pressure and increment of compressive soil pressure.In addition,the relationship between compressive soil pressure and horizontal displacement of the pile is established based on the reasonable assumption.The correctness of the proposed method is verified through four examples.Based on the verified method,a parametric analysis is also conducted to investigate the influences of factors on lateral response of the pile,including internal friction angle,pile length and elastic modulus of pile.
基金Funded by the Program of International S&T Cooperation(No.2013DFA51230)the Opening Subject Fund of Ningbo University(No.zj1226)
文摘The isothermal compression tests were carried out on Gleeble-3500 thermal-mechanical simulation machine in a temperature range of 298-473 K and strain rate range of 0.001-10 s^-1. The experimental results show that the flow stress data are negatively correlated with temperature for temperature softening, and the strain rates sensitivity of this composite increases with elevating temperature. Based on the experimental data, Johnson-Cook, modified Johnson-Cook and Arrhenius constitutive models were established. The accuracy of these three constitutive models was analyzed and compared. The results show that the values predicted by Johnson-Cook model could not agree well with the experimental values. The prediction accuracy of Arrhenius model is higher than that of Johnson-Cook model but lower than that of the Modified Johnson-Cook model.
