Convective storms and lightning are among the most important weather phenomena that are challenging to forecast.In this study,a novel multi-task learning(MTL)encoder-decoder U-net neural network was developed to forec...Convective storms and lightning are among the most important weather phenomena that are challenging to forecast.In this study,a novel multi-task learning(MTL)encoder-decoder U-net neural network was developed to forecast convective storms and lightning with lead times for up to 90 min,using GOES-16 geostationary satellite infrared brightness temperatures(IRBTs),lightning flashes from Geostationary Lightning Mapper(GLM),and vertically integrated liquid(VIL)from Next Generation Weather Radar(NEXRAD).To cope with the heavily skewed distribution of lightning data,a spatiotemporal exponent-weighted loss function and log-transformed lightning normalization approach were developed.The effects of MTL,single-task learning(STL),and IRBTs as auxiliary input features on convection and lightning nowcasting were investigated.The results showed that normalizing the heavily skew-distributed lightning data along with a log-transformation dramatically outperforms the min-max normalization method for nowcasting an intense lightning event.The MTL model significantly outperformed the STL model for both lightning nowcasting and VIL nowcasting,particularly for intense lightning events.The MTL also helped delay the lightning forecast performance decay with the lead times.Furthermore,incorporating satellite IRBTs as auxiliary input features substantially improved lightning nowcasting,but produced little difference in VIL forecasting.Finally,the MTL model performed better for forecasting both lightning and the VIL of organized convective storms than for isolated cells.展开更多
Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new...Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new tasks and datasets,which is inevitably time-consuming.This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components,three-type seismic damage,and four-type deterioration states.The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head,task-specific recognition subnetwork.The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures.The multi-head,task-specific recognition subnetwork consists of three individual self-attention pipelines,each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task.A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one.Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity.The results show that the proposed method can simultaneously recognize different structural components,seismic damage,and deterioration states,and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.展开更多
Aspect-based sentiment analysis(ABSA)is a fine-grained process.Its fundamental subtasks are aspect termextraction(ATE)and aspect polarity classification(APC),and these subtasks are dependent and closely related.Howeve...Aspect-based sentiment analysis(ABSA)is a fine-grained process.Its fundamental subtasks are aspect termextraction(ATE)and aspect polarity classification(APC),and these subtasks are dependent and closely related.However,most existing works on Arabic ABSA content separately address them,assume that aspect terms are preidentified,or use a pipeline model.Pipeline solutions design different models for each task,and the output from the ATE model is used as the input to the APC model,which may result in error propagation among different steps because APC is affected by ATE error.These methods are impractical for real-world scenarios where the ATE task is the base task for APC,and its result impacts the accuracy of APC.Thus,in this study,we focused on a multi-task learning model for Arabic ATE and APC in which the model is jointly trained on two subtasks simultaneously in a singlemodel.This paper integrates themulti-task model,namely Local Cotext Foucse-Aspect Term Extraction and Polarity classification(LCF-ATEPC)and Arabic Bidirectional Encoder Representation from Transformers(AraBERT)as a shred layer for Arabic contextual text representation.The LCF-ATEPC model is based on a multi-head selfattention and local context focus mechanism(LCF)to capture the interactive information between an aspect and its context.Moreover,data augmentation techniques are proposed based on state-of-the-art augmentation techniques(word embedding substitution with constraints and contextual embedding(AraBERT))to increase the diversity of the training dataset.This paper examined the effect of data augmentation on the multi-task model for Arabic ABSA.Extensive experiments were conducted on the original and combined datasets(merging the original and augmented datasets).Experimental results demonstrate that the proposed Multi-task model outperformed existing APC techniques.Superior results were obtained by AraBERT and LCF-ATEPC with fusion layer(AR-LCF-ATEPC-Fusion)and the proposed data augmentation word embedding-based method(FastText)on the combined dataset.展开更多
There is a growing amount of data uploaded to the internet every day and it is important to understand the volume of those data to find a better scheme to process them.However,the volume of internet data is beyond the...There is a growing amount of data uploaded to the internet every day and it is important to understand the volume of those data to find a better scheme to process them.However,the volume of internet data is beyond the processing capabilities of the current internet infrastructure.Therefore,engineering works using technology to organize and analyze information and extract useful information are interesting in both industry and academia.The goal of this paper is to explore the entity relationship based on deep learning,introduce semantic knowledge by using the prepared language model,develop an advanced entity relationship information extraction method by combining Robustly Optimized BERT Approach(RoBERTa)and multi-task learning,and combine the intelligent characters in the field of linguistic,called Robustly Optimized BERT Approach+Multi-Task Learning(RoBERTa+MTL).To improve the effectiveness of model interaction,multi-task teaching is used to implement the observation information of auxiliary tasks.Experimental results show that our method has achieved an accuracy of 88.95 entity relationship extraction,and a further it has achieved 86.35%of accuracy after being combined with multi-task learning.展开更多
Prevailing linguistic steganalysis approaches focus on learning sensitive features to distinguish a particular category of steganographic texts from non-steganographic texts,by performing binary classification.While i...Prevailing linguistic steganalysis approaches focus on learning sensitive features to distinguish a particular category of steganographic texts from non-steganographic texts,by performing binary classification.While it remains an unsolved problem and poses a significant threat to the security of cyberspace when various categories of non-steganographic or steganographic texts coexist.In this paper,we propose a general linguistic steganalysis framework named LS-MTL,which introduces the idea of multi-task learning to deal with the classification of various categories of steganographic and non-steganographic texts.LS-MTL captures sensitive linguistic features from multiple related linguistic steganalysis tasks and can concurrently handle diverse tasks with a constructed model.In the proposed framework,convolutional neural networks(CNNs)are utilized as private base models to extract sensitive features for each steganalysis task.Besides,a shared CNN is built to capture potential interaction information and share linguistic features among all tasks.Finally,LS-MTL incorporates the private and shared sensitive features to identify the detected text as steganographic or non-steganographic.Experimental results demonstrate that the proposed framework LS-MTL outperforms the baseline in the multi-category linguistic steganalysis task,while average Acc,Pre,and Rec are increased by 0.5%,1.4%,and 0.4%,respectively.More ablation experimental results show that LS-MTL with the shared module has robust generalization capability and achieves good detection performance even in the case of spare data.展开更多
Deep learning based methods have been successfully applied to semantic segmentation of optical remote sensing images.However,as more and more remote sensing data is available,it is a new challenge to comprehensively u...Deep learning based methods have been successfully applied to semantic segmentation of optical remote sensing images.However,as more and more remote sensing data is available,it is a new challenge to comprehensively utilize multi-modal remote sensing data to break through the performance bottleneck of single-modal interpretation.In addition,semantic segmentation and height estimation in remote sensing data are two tasks with strong correlation,but existing methods usually study individual tasks separately,which leads to high computational resource overhead.To this end,we propose a Multi-Task learning framework for Multi-Modal remote sensing images(MM_MT).Specifically,we design a Cross-Modal Feature Fusion(CMFF)method,which aggregates complementary information of different modalities to improve the accuracy of semantic segmentation and height estimation.Besides,a dual-stream multi-task learning method is introduced for Joint Semantic Segmentation and Height Estimation(JSSHE),extracting common features in a shared network to save time and resources,and then learning task-specific features in two task branches.Experimental results on the public multi-modal remote sensing image dataset Potsdam show that compared to training two tasks independently,multi-task learning saves 20%of training time and achieves competitive performance with mIoU of 83.02%for semantic segmentation and accuracy of 95.26%for height estimation.展开更多
The global nuclear mass based on the macroscopic-microscopic model was studied by applying a newly designed multi-task learning artificial neural network(MTL-ANN). First, the reported nuclear binding energies of 2095 ...The global nuclear mass based on the macroscopic-microscopic model was studied by applying a newly designed multi-task learning artificial neural network(MTL-ANN). First, the reported nuclear binding energies of 2095 nuclei(Z ≥ 8, N ≥ 8) released in the latest Atomic Mass Evaluation AME2020 and the deviations between the fitting result of the liquid drop model(LDM)and data from AME2020 for each nucleus were obtained.To compensate for the deviations and investigate the possible ignored physics in the LDM, the MTL-ANN method was introduced in the model. Compared to the single-task learning(STL) method, this new network has a powerful ability to simultaneously learn multi-nuclear properties,such as the binding energies and single neutron and proton separation energies. Moreover, it is highly effective in reducing the risk of overfitting and achieving better predictions. Consequently, good predictions can be obtained using this nuclear mass model for both the training and validation datasets and for the testing dataset. In detail, the global root mean square(RMS) of the binding energy is effectively reduced from approximately 2.4 MeV of LDM to the current 0.2 MeV, and the RMS of Sn, Spcan also reach approximately 0.2 MeV. Moreover, compared to STL, for the training and validation sets, 3-9% improvement can be achieved with the binding energy, and 20-30% improvement for S_(n), S_(p);for the testing sets, the reduction in deviations can even reach 30-40%, which significantly illustrates the advantage of the current MTL.展开更多
Sea state bias(SSB)is an important component of errors for the radar altimeter measurements of sea surface height(SSH).However,existing SSB estimation methods are almost all based on single-task learning(STL),where on...Sea state bias(SSB)is an important component of errors for the radar altimeter measurements of sea surface height(SSH).However,existing SSB estimation methods are almost all based on single-task learning(STL),where one model is built on the data from only one radar altimeter.In this paper,taking account of the data from multiple radar altimeters available,we introduced a multi-task learning method,called trace-norm regularized multi-task learning(TNR-MTL),for SSB estimation.Corresponding to each individual task,TNR-MLT involves only three parameters.Hence,it is easy to implement.More importantly,the convergence of TNR-MLT is theoretically guaranteed.Compared with the commonly used STL models,TNR-MTL can effectively utilize the shared information between data from multiple altimeters.During the training of TNR-MTL,we used the JASON-2 and JASON-3 cycle data to solve two correlated SSB estimation tasks.Then the optimal model was selected to estimate SSB on the JASON-2 and the HY-270-71 cycle intersection data.For the JSAON-2 cycle intersection data,the corrected variance(M)has been reduced by 0.60 cm^2 compared to the geophysical data records(GDR);while for the HY-2 cycle intersection data,M has been reduced by 1.30 cm^2 compared to GDR.Therefore,TNR-MTL is proved to be effective for the SSB estimation tasks.展开更多
With the rapid development of the mobile Internet,users generate massive data in different forms in social network every day,and different characteristics of users are reflected by these social media data.How to integ...With the rapid development of the mobile Internet,users generate massive data in different forms in social network every day,and different characteristics of users are reflected by these social media data.How to integrate multiple heterogeneous information and establish user profiles from multiple perspectives plays an important role in providing personalized services,marketing,and recommendation systems.In this paper,we propose Multi-source&Multi-task Learning for User Profiles in Social Network which integrates multiple social data sources and contains a multi-task learning framework to simultaneously predict various attributes of a user.Firstly,we design their own feature extraction models for multiple heterogeneous data sources.Secondly,we design a shared layer to fuse multiple heterogeneous data sources as general shared representation for multi-task learning.Thirdly,we design each task’s own unique presentation layer for discriminant output of specific-task.Finally,we design a weighted loss function to improve the learning efficiency and prediction accuracy of each task.Our experimental results on more than 5000 Sina Weibo users demonstrate that our approach outperforms state-of-the-art baselines for inferring gender,age and region of social media users.展开更多
Recently, several deep learning models have been successfully proposed and have been applied to solve different Natural Language Processing (NLP) tasks. However, these models solve the problem based on single-task sup...Recently, several deep learning models have been successfully proposed and have been applied to solve different Natural Language Processing (NLP) tasks. However, these models solve the problem based on single-task supervised learning and do not consider the correlation between the tasks. Based on this observation, in this paper, we implemented a multi-task learning model to joint learn two related NLP tasks simultaneously and conducted experiments to evaluate if learning these tasks jointly can improve the system performance compared with learning them individually. In addition, a comparison of our model with the state-of-the-art learning models, including multi-task learning, transfer learning, unsupervised learning and feature based traditional machine learning models is presented. This paper aims to 1) show the advantage of multi-task learning over single-task learning in training related NLP tasks, 2) illustrate the influence of various encoding structures to the proposed single- and multi-task learning models, and 3) compare the performance between multi-task learning and other learning models in literature on textual entailment task and semantic relatedness task.展开更多
Open-set recognition(OSR)is a realistic problem in wireless signal recogni-tion,which means that during the inference phase there may appear unknown classes not seen in the training phase.The method of intra-class spl...Open-set recognition(OSR)is a realistic problem in wireless signal recogni-tion,which means that during the inference phase there may appear unknown classes not seen in the training phase.The method of intra-class splitting(ICS)that splits samples of known classes to imitate unknown classes has achieved great performance.However,this approach relies too much on the predefined splitting ratio and may face huge performance degradation in new environment.In this paper,we train a multi-task learning(MTL)net-work based on the characteristics of wireless signals to improve the performance in new scenes.Besides,we provide a dynamic method to decide the splitting ratio per class to get more precise outer samples.To be specific,we make perturbations to the sample from the center of one class toward its adversarial direction and the change point of confidence scores during this process is used as the splitting threshold.We conduct several experi-ments on one wireless signal dataset collected at 2.4 GHz ISM band by LimeSDR and one open modulation recognition dataset,and the analytical results demonstrate the effective-ness of the proposed method.展开更多
Personality prediction on social network has become a hot topic.At present,most studies are using single-task classification/regression machine learning.However,this method ignores the potential association between mu...Personality prediction on social network has become a hot topic.At present,most studies are using single-task classification/regression machine learning.However,this method ignores the potential association between multiple tasks.Also an ideal prediction result is difficult to achieve based on the small scale training data,since it is not easy to get a lot of social network data with personality label samples.In this paper,a robust multi-task learning method(RMTL)is proposed to predict Big-Five personality on Micro-blog.We aim to learn five tasks simultaneously by extracting and utilizing appropriate shared information among multiple tasks as well as identifying irrelevant tasks.For a set of Sina Micro-blog users’information and personality labeled data retrieved by questionnaire,we validate the RMTL method by comparing it with 4 single-task learning methods and the mere multi-task learning.Our experiment demonstrates that the proposed RMTL can improve the precision rate,recall rate of the prediction and F value.展开更多
In order to accurately segment architectural features in highresolution remote sensing images,a semantic segmentation method based on U-net network multi-task learning is proposed.First,a boundary distance map was gen...In order to accurately segment architectural features in highresolution remote sensing images,a semantic segmentation method based on U-net network multi-task learning is proposed.First,a boundary distance map was generated based on the remote sensing image of the ground truth map of the building.The remote sensing image and its truth map were used as the input in the U-net network,followed by the addition of the building ground prediction layer at the end of the U-net network.Based on the ResNet network,a multi-task network with the boundary distance prediction layer was built.Experiments involving the ISPRS aerial remote sensing image building and feature annotation data set show that compared with the full convolutional network combined with the multi-layer perceptron method,the intersection ratio of VGG16 network,VGG16+boundary prediction,ResNet50 and the method in this paper were increased by 5.15%,6.946%,6.41%and 7.86%.The accuracy of the networks was increased to 94.71%,95.39%,95.30%and 96.10%respectively,which resulted in high-precision extraction of building features.展开更多
Increase in permeability of renewable energy sources(RESs)leads to the prominent problem of voltage stability in power system,so it is urgent to have a system strength evaluation method with both accuracy and practica...Increase in permeability of renewable energy sources(RESs)leads to the prominent problem of voltage stability in power system,so it is urgent to have a system strength evaluation method with both accuracy and practicability to control its access scale within a reasonable range.Therefore,a hybrid intelligence enhancement method is proposed by combining the advantages of mechanism method and data driven method.First,calculation of critical short circuit ratio(CSCR)is set as the direction of intelligent enhancement by taking the multiple renewable energy station short circuit ratio as the quantitative indicator.Then,the construction process of CSCR dataset is proposed,and a batch simulation program of samples is developed accordingly,which provides a data basis for subsequent research.Finally,a multi-task learning model based on progressive layered extraction is used to simultaneously predict CSCR of each RESs connection point,which significantly reduces evaluation error caused by weak links.Predictive performance and anti-noise performance of the proposed method are verified on the CEPRI-FS-102 bus system,which provides strong technical support for real-time monitoring of system strength.展开更多
Thoracic diseases pose significant risks to an individual's chest health and are among the most perilous medical diseases. They can impact either one or both lungs, which leads to a severe impairment of a person’...Thoracic diseases pose significant risks to an individual's chest health and are among the most perilous medical diseases. They can impact either one or both lungs, which leads to a severe impairment of a person’s ability to breathe normally. Some notable examples of such diseases encompass pneumonia, lung cancer, coronavirus disease 2019 (COVID-19), tuberculosis, and chronic obstructive pulmonary disease (COPD). Consequently, early and precise detection of these diseases is paramount during the diagnostic process. Traditionally, the primary methods employed for the detection involve the use of X-ray imaging or computed tomography (CT) scans. Nevertheless, due to the scarcity of proficient radiologists and the inherent similarities between these diseases, the accuracy of detection can be compromised, leading to imprecise or erroneous results. To address this challenge, scientists have turned to computer-based solutions, aiming for swift and accurate diagnoses. The primary objective of this study is to develop two machine learning models, utilizing single-task and multi-task learning frameworks, to enhance classification accuracy. Within the multi-task learning architecture, two principal approaches exist soft parameter sharing and hard parameter sharing. Consequently, this research adopts a multi-task deep learning approach that leverages CNNs to achieve improved classification performance for the specified tasks. These tasks, focusing on pneumonia and COVID-19, are processed and learned simultaneously within a multi-task model. To assess the effectiveness of the trained model, it is rigorously validated using three different real-world datasets for training and testing.展开更多
As a complex driving behaviour,lane-changing(LC)behaviour has a great influence on traffic flow.Improper lane-changing behaviour often leads to traffic accidents.Numerous studies are currently being conducted to predi...As a complex driving behaviour,lane-changing(LC)behaviour has a great influence on traffic flow.Improper lane-changing behaviour often leads to traffic accidents.Numerous studies are currently being conducted to predict lane-change trajectories to minimize dangers.However,most of their models focus on how to optimize input variables without considering the interaction between output variables.This study proposes an LC trajectory prediction model based on a multi-task deep learning framework to improve driving safety.Concretely,in this work,the coupling effect of lateral and longitudinal movement is considered in the L.C process.Trajectory changes in two directions will be modelled separately,and the information interaction is completed under the multi-task learing framework.In addition,the trajectory fragents are clustered by the driving features,and trajectory type recognition is added to the trajectory prediction framework as an auxiliary task.Finally,the prediction process of lateral and longitudinal trajectory and LC style is completed by long short-term memory(LSTM).The model training and testing are conducted with the data collected by the driving simulator,and the proposed method expresses better performance in LC trjectory prediction compared with several traditional models.The results of this study can enhance the trajectory prediction accuracy of advanced driving assistance systems(ADASs)and reduce the traffic accidents caused by lane changes.展开更多
Online advertising has gained much attention on various platforms as a hugely lucrative market.In promoting content and advertisements in real life,the acquisition of user target actions is usually a multi-step proces...Online advertising has gained much attention on various platforms as a hugely lucrative market.In promoting content and advertisements in real life,the acquisition of user target actions is usually a multi-step process,such as impres-sion→click→conversion,which means the process from the delivery of the recommended item to the user’s click to the final conversion.Due to data sparsity or sample selection bias,it is difficult for the trained model to achieve the business goal of the target campaign.Multi-task learning,a classical solution to this pro-blem,aims to generalize better on the original task given several related tasks by exploiting the knowledge between tasks to share the same feature and label space.Adaptively learned task relations bring better performance to make full use of the correlation between tasks.We train a general model capable of captur-ing the relationships between various tasks on all existing active tasks from a meta-learning perspective.In addition,this paper proposes a Multi-task Attention Network(MAN)to identify commonalities and differences between tasks in the feature space.The model performance is improved by explicitly learning the stacking of task relationships in the label space.To illustrate the effectiveness of our method,experiments are conducted on Alibaba Click and Conversion Pre-diction(Ali-CCP)dataset.Experimental results show that the method outperforms the state-of-the-art multi-task learning methods.展开更多
The high throughput prediction of the thermodynamic phase behavior of active pharmaceutical ingredients(APIs)with pharmaceutically relevant excipients remains a major scientific challenge in the screening of pharmaceu...The high throughput prediction of the thermodynamic phase behavior of active pharmaceutical ingredients(APIs)with pharmaceutically relevant excipients remains a major scientific challenge in the screening of pharmaceutical formulations.In this work,a developed machine-learning model efficiently predicts the solubility of APIs in polymers by learning the phase equilibrium principle and using a few molecular descriptors.Under the few-shot learning framework,thermodynamic theory(perturbed-chain statistical associating fluid theory)was used for data augmentation,and computational chemistry was applied for molecular descriptors'screening.The results showed that the developed machine-learning model can predict the API-polymer phase diagram accurately,broaden the solubility data of APIs in polymers,and reproduce the relationship between API solubility and the interaction mechanisms between API and polymer successfully,which provided efficient guidance for the development of pharmaceutical formulations.展开更多
BACKGROUND Liver transplantation(LT)is a life-saving intervention for patients with end-stage liver disease.However,the equitable allocation of scarce donor organs remains a formidable challenge.Prognostic tools are p...BACKGROUND Liver transplantation(LT)is a life-saving intervention for patients with end-stage liver disease.However,the equitable allocation of scarce donor organs remains a formidable challenge.Prognostic tools are pivotal in identifying the most suitable transplant candidates.Traditionally,scoring systems like the model for end-stage liver disease have been instrumental in this process.Nevertheless,the landscape of prognostication is undergoing a transformation with the integration of machine learning(ML)and artificial intelligence models.AIM To assess the utility of ML models in prognostication for LT,comparing their performance and reliability to established traditional scoring systems.METHODS Following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines,we conducted a thorough and standardized literature search using the PubMed/MEDLINE database.Our search imposed no restrictions on publication year,age,or gender.Exclusion criteria encompassed non-English studies,review articles,case reports,conference papers,studies with missing data,or those exhibiting evident methodological flaws.RESULTS Our search yielded a total of 64 articles,with 23 meeting the inclusion criteria.Among the selected studies,60.8%originated from the United States and China combined.Only one pediatric study met the criteria.Notably,91%of the studies were published within the past five years.ML models consistently demonstrated satisfactory to excellent area under the receiver operating characteristic curve values(ranging from 0.6 to 1)across all studies,surpassing the performance of traditional scoring systems.Random forest exhibited superior predictive capabilities for 90-d mortality following LT,sepsis,and acute kidney injury(AKI).In contrast,gradient boosting excelled in predicting the risk of graft-versus-host disease,pneumonia,and AKI.CONCLUSION This study underscores the potential of ML models in guiding decisions related to allograft allocation and LT,marking a significant evolution in the field of prognostication.展开更多
基金supported by the Science and Technology Grant No.520120210003,Jibei Electric Power Company of the State Grid Corporation of China。
文摘Convective storms and lightning are among the most important weather phenomena that are challenging to forecast.In this study,a novel multi-task learning(MTL)encoder-decoder U-net neural network was developed to forecast convective storms and lightning with lead times for up to 90 min,using GOES-16 geostationary satellite infrared brightness temperatures(IRBTs),lightning flashes from Geostationary Lightning Mapper(GLM),and vertically integrated liquid(VIL)from Next Generation Weather Radar(NEXRAD).To cope with the heavily skewed distribution of lightning data,a spatiotemporal exponent-weighted loss function and log-transformed lightning normalization approach were developed.The effects of MTL,single-task learning(STL),and IRBTs as auxiliary input features on convection and lightning nowcasting were investigated.The results showed that normalizing the heavily skew-distributed lightning data along with a log-transformation dramatically outperforms the min-max normalization method for nowcasting an intense lightning event.The MTL model significantly outperformed the STL model for both lightning nowcasting and VIL nowcasting,particularly for intense lightning events.The MTL also helped delay the lightning forecast performance decay with the lead times.Furthermore,incorporating satellite IRBTs as auxiliary input features substantially improved lightning nowcasting,but produced little difference in VIL forecasting.Finally,the MTL model performed better for forecasting both lightning and the VIL of organized convective storms than for isolated cells.
基金National Key R&D Program of China under Grant No.2019YFC1511005the National Natural Science Foundation of China under Grant Nos.51921006,52192661 and 52008138+2 种基金the China Postdoctoral Science Foundation under Grant Nos.BX20190102 and 2019M661286the Heilongjiang Natural Science Foundation under Grant No.LH2022E070the Heilongjiang Province Postdoctoral Science Foundation under Grant Nos.LBH-TZ2016 and LBH-Z19064。
文摘Recent studies for computer vision and deep learning-based,post-earthquake inspections on RC structures mainly perform well for specific tasks,while the trained models must be fine-tuned and re-trained when facing new tasks and datasets,which is inevitably time-consuming.This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components,three-type seismic damage,and four-type deterioration states.The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head,task-specific recognition subnetwork.The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures.The multi-head,task-specific recognition subnetwork consists of three individual self-attention pipelines,each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task.A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one.Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity.The results show that the proposed method can simultaneously recognize different structural components,seismic damage,and deterioration states,and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.
文摘Aspect-based sentiment analysis(ABSA)is a fine-grained process.Its fundamental subtasks are aspect termextraction(ATE)and aspect polarity classification(APC),and these subtasks are dependent and closely related.However,most existing works on Arabic ABSA content separately address them,assume that aspect terms are preidentified,or use a pipeline model.Pipeline solutions design different models for each task,and the output from the ATE model is used as the input to the APC model,which may result in error propagation among different steps because APC is affected by ATE error.These methods are impractical for real-world scenarios where the ATE task is the base task for APC,and its result impacts the accuracy of APC.Thus,in this study,we focused on a multi-task learning model for Arabic ATE and APC in which the model is jointly trained on two subtasks simultaneously in a singlemodel.This paper integrates themulti-task model,namely Local Cotext Foucse-Aspect Term Extraction and Polarity classification(LCF-ATEPC)and Arabic Bidirectional Encoder Representation from Transformers(AraBERT)as a shred layer for Arabic contextual text representation.The LCF-ATEPC model is based on a multi-head selfattention and local context focus mechanism(LCF)to capture the interactive information between an aspect and its context.Moreover,data augmentation techniques are proposed based on state-of-the-art augmentation techniques(word embedding substitution with constraints and contextual embedding(AraBERT))to increase the diversity of the training dataset.This paper examined the effect of data augmentation on the multi-task model for Arabic ABSA.Extensive experiments were conducted on the original and combined datasets(merging the original and augmented datasets).Experimental results demonstrate that the proposed Multi-task model outperformed existing APC techniques.Superior results were obtained by AraBERT and LCF-ATEPC with fusion layer(AR-LCF-ATEPC-Fusion)and the proposed data augmentation word embedding-based method(FastText)on the combined dataset.
文摘There is a growing amount of data uploaded to the internet every day and it is important to understand the volume of those data to find a better scheme to process them.However,the volume of internet data is beyond the processing capabilities of the current internet infrastructure.Therefore,engineering works using technology to organize and analyze information and extract useful information are interesting in both industry and academia.The goal of this paper is to explore the entity relationship based on deep learning,introduce semantic knowledge by using the prepared language model,develop an advanced entity relationship information extraction method by combining Robustly Optimized BERT Approach(RoBERTa)and multi-task learning,and combine the intelligent characters in the field of linguistic,called Robustly Optimized BERT Approach+Multi-Task Learning(RoBERTa+MTL).To improve the effectiveness of model interaction,multi-task teaching is used to implement the observation information of auxiliary tasks.Experimental results show that our method has achieved an accuracy of 88.95 entity relationship extraction,and a further it has achieved 86.35%of accuracy after being combined with multi-task learning.
基金This paper is partly supported by the National Natural Science Foundation of China unde rGrants 61972057 and 62172059Hunan ProvincialNatural Science Foundation of China underGrant 2022JJ30623 and 2019JJ50287Scientific Research Fund of Hunan Provincial Education Department of China under Grant 21A0211 and 19A265。
文摘Prevailing linguistic steganalysis approaches focus on learning sensitive features to distinguish a particular category of steganographic texts from non-steganographic texts,by performing binary classification.While it remains an unsolved problem and poses a significant threat to the security of cyberspace when various categories of non-steganographic or steganographic texts coexist.In this paper,we propose a general linguistic steganalysis framework named LS-MTL,which introduces the idea of multi-task learning to deal with the classification of various categories of steganographic and non-steganographic texts.LS-MTL captures sensitive linguistic features from multiple related linguistic steganalysis tasks and can concurrently handle diverse tasks with a constructed model.In the proposed framework,convolutional neural networks(CNNs)are utilized as private base models to extract sensitive features for each steganalysis task.Besides,a shared CNN is built to capture potential interaction information and share linguistic features among all tasks.Finally,LS-MTL incorporates the private and shared sensitive features to identify the detected text as steganographic or non-steganographic.Experimental results demonstrate that the proposed framework LS-MTL outperforms the baseline in the multi-category linguistic steganalysis task,while average Acc,Pre,and Rec are increased by 0.5%,1.4%,and 0.4%,respectively.More ablation experimental results show that LS-MTL with the shared module has robust generalization capability and achieves good detection performance even in the case of spare data.
基金National Key R&D Program of China(No.2022ZD0118401).
文摘Deep learning based methods have been successfully applied to semantic segmentation of optical remote sensing images.However,as more and more remote sensing data is available,it is a new challenge to comprehensively utilize multi-modal remote sensing data to break through the performance bottleneck of single-modal interpretation.In addition,semantic segmentation and height estimation in remote sensing data are two tasks with strong correlation,but existing methods usually study individual tasks separately,which leads to high computational resource overhead.To this end,we propose a Multi-Task learning framework for Multi-Modal remote sensing images(MM_MT).Specifically,we design a Cross-Modal Feature Fusion(CMFF)method,which aggregates complementary information of different modalities to improve the accuracy of semantic segmentation and height estimation.Besides,a dual-stream multi-task learning method is introduced for Joint Semantic Segmentation and Height Estimation(JSSHE),extracting common features in a shared network to save time and resources,and then learning task-specific features in two task branches.Experimental results on the public multi-modal remote sensing image dataset Potsdam show that compared to training two tasks independently,multi-task learning saves 20%of training time and achieves competitive performance with mIoU of 83.02%for semantic segmentation and accuracy of 95.26%for height estimation.
基金supported by the National Natural Science Foundation of China(Nos.1187050492,12005303,and 12175170).
文摘The global nuclear mass based on the macroscopic-microscopic model was studied by applying a newly designed multi-task learning artificial neural network(MTL-ANN). First, the reported nuclear binding energies of 2095 nuclei(Z ≥ 8, N ≥ 8) released in the latest Atomic Mass Evaluation AME2020 and the deviations between the fitting result of the liquid drop model(LDM)and data from AME2020 for each nucleus were obtained.To compensate for the deviations and investigate the possible ignored physics in the LDM, the MTL-ANN method was introduced in the model. Compared to the single-task learning(STL) method, this new network has a powerful ability to simultaneously learn multi-nuclear properties,such as the binding energies and single neutron and proton separation energies. Moreover, it is highly effective in reducing the risk of overfitting and achieving better predictions. Consequently, good predictions can be obtained using this nuclear mass model for both the training and validation datasets and for the testing dataset. In detail, the global root mean square(RMS) of the binding energy is effectively reduced from approximately 2.4 MeV of LDM to the current 0.2 MeV, and the RMS of Sn, Spcan also reach approximately 0.2 MeV. Moreover, compared to STL, for the training and validation sets, 3-9% improvement can be achieved with the binding energy, and 20-30% improvement for S_(n), S_(p);for the testing sets, the reduction in deviations can even reach 30-40%, which significantly illustrates the advantage of the current MTL.
基金This work was supported by the Major Project for New Generation of AI(No.2018AAA0100400)the National Natural Science Foundation of China(No.41706010)+1 种基金the Joint Fund of the Equipments Pre-Research and Ministry of Education of China(No.6141A020337)and the Fundamental Research Funds for the Central Universities of China.
文摘Sea state bias(SSB)is an important component of errors for the radar altimeter measurements of sea surface height(SSH).However,existing SSB estimation methods are almost all based on single-task learning(STL),where one model is built on the data from only one radar altimeter.In this paper,taking account of the data from multiple radar altimeters available,we introduced a multi-task learning method,called trace-norm regularized multi-task learning(TNR-MTL),for SSB estimation.Corresponding to each individual task,TNR-MLT involves only three parameters.Hence,it is easy to implement.More importantly,the convergence of TNR-MLT is theoretically guaranteed.Compared with the commonly used STL models,TNR-MTL can effectively utilize the shared information between data from multiple altimeters.During the training of TNR-MTL,we used the JASON-2 and JASON-3 cycle data to solve two correlated SSB estimation tasks.Then the optimal model was selected to estimate SSB on the JASON-2 and the HY-270-71 cycle intersection data.For the JSAON-2 cycle intersection data,the corrected variance(M)has been reduced by 0.60 cm^2 compared to the geophysical data records(GDR);while for the HY-2 cycle intersection data,M has been reduced by 1.30 cm^2 compared to GDR.Therefore,TNR-MTL is proved to be effective for the SSB estimation tasks.
基金This work is supported by State Grid Science and Technology Project under Grant No.520613180002,62061318C002the Fundamental Research Funds for the Central Universities(Grant No.HIT.NSRIF.201714)+4 种基金Weihai Science and Technology Development Program(2016DXGJMS15)Key Research and Development Program in Shandong Provincial(2017GGX90103)Sanming Science and Technology Project,Grant No.2015-G-6,Shandong province vocational education educational reform research project.Grant No.2017209Study and Development of Smart Agriculture Control System Based on Spark Big Data Decision(2017N0029)Jiangsu Province industrial Communication Technology Application Technology Innovation Team Project.
文摘With the rapid development of the mobile Internet,users generate massive data in different forms in social network every day,and different characteristics of users are reflected by these social media data.How to integrate multiple heterogeneous information and establish user profiles from multiple perspectives plays an important role in providing personalized services,marketing,and recommendation systems.In this paper,we propose Multi-source&Multi-task Learning for User Profiles in Social Network which integrates multiple social data sources and contains a multi-task learning framework to simultaneously predict various attributes of a user.Firstly,we design their own feature extraction models for multiple heterogeneous data sources.Secondly,we design a shared layer to fuse multiple heterogeneous data sources as general shared representation for multi-task learning.Thirdly,we design each task’s own unique presentation layer for discriminant output of specific-task.Finally,we design a weighted loss function to improve the learning efficiency and prediction accuracy of each task.Our experimental results on more than 5000 Sina Weibo users demonstrate that our approach outperforms state-of-the-art baselines for inferring gender,age and region of social media users.
文摘Recently, several deep learning models have been successfully proposed and have been applied to solve different Natural Language Processing (NLP) tasks. However, these models solve the problem based on single-task supervised learning and do not consider the correlation between the tasks. Based on this observation, in this paper, we implemented a multi-task learning model to joint learn two related NLP tasks simultaneously and conducted experiments to evaluate if learning these tasks jointly can improve the system performance compared with learning them individually. In addition, a comparison of our model with the state-of-the-art learning models, including multi-task learning, transfer learning, unsupervised learning and feature based traditional machine learning models is presented. This paper aims to 1) show the advantage of multi-task learning over single-task learning in training related NLP tasks, 2) illustrate the influence of various encoding structures to the proposed single- and multi-task learning models, and 3) compare the performance between multi-task learning and other learning models in literature on textual entailment task and semantic relatedness task.
文摘Open-set recognition(OSR)is a realistic problem in wireless signal recogni-tion,which means that during the inference phase there may appear unknown classes not seen in the training phase.The method of intra-class splitting(ICS)that splits samples of known classes to imitate unknown classes has achieved great performance.However,this approach relies too much on the predefined splitting ratio and may face huge performance degradation in new environment.In this paper,we train a multi-task learning(MTL)net-work based on the characteristics of wireless signals to improve the performance in new scenes.Besides,we provide a dynamic method to decide the splitting ratio per class to get more precise outer samples.To be specific,we make perturbations to the sample from the center of one class toward its adversarial direction and the change point of confidence scores during this process is used as the splitting threshold.We conduct several experi-ments on one wireless signal dataset collected at 2.4 GHz ISM band by LimeSDR and one open modulation recognition dataset,and the analytical results demonstrate the effective-ness of the proposed method.
文摘Personality prediction on social network has become a hot topic.At present,most studies are using single-task classification/regression machine learning.However,this method ignores the potential association between multiple tasks.Also an ideal prediction result is difficult to achieve based on the small scale training data,since it is not easy to get a lot of social network data with personality label samples.In this paper,a robust multi-task learning method(RMTL)is proposed to predict Big-Five personality on Micro-blog.We aim to learn five tasks simultaneously by extracting and utilizing appropriate shared information among multiple tasks as well as identifying irrelevant tasks.For a set of Sina Micro-blog users’information and personality labeled data retrieved by questionnaire,we validate the RMTL method by comparing it with 4 single-task learning methods and the mere multi-task learning.Our experiment demonstrates that the proposed RMTL can improve the precision rate,recall rate of the prediction and F value.
基金This research was supported by National Key Research and Development program[2018YFF0213606-03(Mu,Y.,Hu,T.L.,Gong,H.,Li,S.J.and Sun,Y.H.)http://www.most.gov.cn]the Jilin Province Science and Technology Development Plan focusing on research and development projects[20200402006NC(Mu,Y.,Hu,T.L.,Gong,H.and Li,S.J.)http://kjt.jl.gov.cn]+1 种基金the science and technology support project for key industries in southern Xinjiang[2018DB001(Gong,H.,and Li,S.J.)http://kjj.xjbt.gov.cn]the key technology R&D project of Changchun Science and Technology Bureau of Jilin Province[21ZGN29(Mu,Y.,Bao,H.P.,Wang X.B.)http://kjj.changchun.gov.cn].
文摘In order to accurately segment architectural features in highresolution remote sensing images,a semantic segmentation method based on U-net network multi-task learning is proposed.First,a boundary distance map was generated based on the remote sensing image of the ground truth map of the building.The remote sensing image and its truth map were used as the input in the U-net network,followed by the addition of the building ground prediction layer at the end of the U-net network.Based on the ResNet network,a multi-task network with the boundary distance prediction layer was built.Experiments involving the ISPRS aerial remote sensing image building and feature annotation data set show that compared with the full convolutional network combined with the multi-layer perceptron method,the intersection ratio of VGG16 network,VGG16+boundary prediction,ResNet50 and the method in this paper were increased by 5.15%,6.946%,6.41%and 7.86%.The accuracy of the networks was increased to 94.71%,95.39%,95.30%and 96.10%respectively,which resulted in high-precision extraction of building features.
文摘Increase in permeability of renewable energy sources(RESs)leads to the prominent problem of voltage stability in power system,so it is urgent to have a system strength evaluation method with both accuracy and practicability to control its access scale within a reasonable range.Therefore,a hybrid intelligence enhancement method is proposed by combining the advantages of mechanism method and data driven method.First,calculation of critical short circuit ratio(CSCR)is set as the direction of intelligent enhancement by taking the multiple renewable energy station short circuit ratio as the quantitative indicator.Then,the construction process of CSCR dataset is proposed,and a batch simulation program of samples is developed accordingly,which provides a data basis for subsequent research.Finally,a multi-task learning model based on progressive layered extraction is used to simultaneously predict CSCR of each RESs connection point,which significantly reduces evaluation error caused by weak links.Predictive performance and anti-noise performance of the proposed method are verified on the CEPRI-FS-102 bus system,which provides strong technical support for real-time monitoring of system strength.
文摘Thoracic diseases pose significant risks to an individual's chest health and are among the most perilous medical diseases. They can impact either one or both lungs, which leads to a severe impairment of a person’s ability to breathe normally. Some notable examples of such diseases encompass pneumonia, lung cancer, coronavirus disease 2019 (COVID-19), tuberculosis, and chronic obstructive pulmonary disease (COPD). Consequently, early and precise detection of these diseases is paramount during the diagnostic process. Traditionally, the primary methods employed for the detection involve the use of X-ray imaging or computed tomography (CT) scans. Nevertheless, due to the scarcity of proficient radiologists and the inherent similarities between these diseases, the accuracy of detection can be compromised, leading to imprecise or erroneous results. To address this challenge, scientists have turned to computer-based solutions, aiming for swift and accurate diagnoses. The primary objective of this study is to develop two machine learning models, utilizing single-task and multi-task learning frameworks, to enhance classification accuracy. Within the multi-task learning architecture, two principal approaches exist soft parameter sharing and hard parameter sharing. Consequently, this research adopts a multi-task deep learning approach that leverages CNNs to achieve improved classification performance for the specified tasks. These tasks, focusing on pneumonia and COVID-19, are processed and learned simultaneously within a multi-task model. To assess the effectiveness of the trained model, it is rigorously validated using three different real-world datasets for training and testing.
基金funded in part by the National Natural Science Foundation of China(Grant No.52172310)the Humanities and Social Sciences Foundation of the Ministry of Education(Grant No.21YJCZH147)the Innovation-Driven Project of Central South Univ ersity(Grant No.2020CX041).
文摘As a complex driving behaviour,lane-changing(LC)behaviour has a great influence on traffic flow.Improper lane-changing behaviour often leads to traffic accidents.Numerous studies are currently being conducted to predict lane-change trajectories to minimize dangers.However,most of their models focus on how to optimize input variables without considering the interaction between output variables.This study proposes an LC trajectory prediction model based on a multi-task deep learning framework to improve driving safety.Concretely,in this work,the coupling effect of lateral and longitudinal movement is considered in the L.C process.Trajectory changes in two directions will be modelled separately,and the information interaction is completed under the multi-task learing framework.In addition,the trajectory fragents are clustered by the driving features,and trajectory type recognition is added to the trajectory prediction framework as an auxiliary task.Finally,the prediction process of lateral and longitudinal trajectory and LC style is completed by long short-term memory(LSTM).The model training and testing are conducted with the data collected by the driving simulator,and the proposed method expresses better performance in LC trjectory prediction compared with several traditional models.The results of this study can enhance the trajectory prediction accuracy of advanced driving assistance systems(ADASs)and reduce the traffic accidents caused by lane changes.
基金Our work was supported by the research project of Yunnan University(Grant No.2021Y274)Natural Science Foundation of China(Grant No.61862064).
文摘Online advertising has gained much attention on various platforms as a hugely lucrative market.In promoting content and advertisements in real life,the acquisition of user target actions is usually a multi-step process,such as impres-sion→click→conversion,which means the process from the delivery of the recommended item to the user’s click to the final conversion.Due to data sparsity or sample selection bias,it is difficult for the trained model to achieve the business goal of the target campaign.Multi-task learning,a classical solution to this pro-blem,aims to generalize better on the original task given several related tasks by exploiting the knowledge between tasks to share the same feature and label space.Adaptively learned task relations bring better performance to make full use of the correlation between tasks.We train a general model capable of captur-ing the relationships between various tasks on all existing active tasks from a meta-learning perspective.In addition,this paper proposes a Multi-task Attention Network(MAN)to identify commonalities and differences between tasks in the feature space.The model performance is improved by explicitly learning the stacking of task relationships in the label space.To illustrate the effectiveness of our method,experiments are conducted on Alibaba Click and Conversion Pre-diction(Ali-CCP)dataset.Experimental results show that the method outperforms the state-of-the-art multi-task learning methods.
基金the financial support from the National Natural Science Foundation of China(22278070,21978047,21776046)。
文摘The high throughput prediction of the thermodynamic phase behavior of active pharmaceutical ingredients(APIs)with pharmaceutically relevant excipients remains a major scientific challenge in the screening of pharmaceutical formulations.In this work,a developed machine-learning model efficiently predicts the solubility of APIs in polymers by learning the phase equilibrium principle and using a few molecular descriptors.Under the few-shot learning framework,thermodynamic theory(perturbed-chain statistical associating fluid theory)was used for data augmentation,and computational chemistry was applied for molecular descriptors'screening.The results showed that the developed machine-learning model can predict the API-polymer phase diagram accurately,broaden the solubility data of APIs in polymers,and reproduce the relationship between API solubility and the interaction mechanisms between API and polymer successfully,which provided efficient guidance for the development of pharmaceutical formulations.
文摘BACKGROUND Liver transplantation(LT)is a life-saving intervention for patients with end-stage liver disease.However,the equitable allocation of scarce donor organs remains a formidable challenge.Prognostic tools are pivotal in identifying the most suitable transplant candidates.Traditionally,scoring systems like the model for end-stage liver disease have been instrumental in this process.Nevertheless,the landscape of prognostication is undergoing a transformation with the integration of machine learning(ML)and artificial intelligence models.AIM To assess the utility of ML models in prognostication for LT,comparing their performance and reliability to established traditional scoring systems.METHODS Following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines,we conducted a thorough and standardized literature search using the PubMed/MEDLINE database.Our search imposed no restrictions on publication year,age,or gender.Exclusion criteria encompassed non-English studies,review articles,case reports,conference papers,studies with missing data,or those exhibiting evident methodological flaws.RESULTS Our search yielded a total of 64 articles,with 23 meeting the inclusion criteria.Among the selected studies,60.8%originated from the United States and China combined.Only one pediatric study met the criteria.Notably,91%of the studies were published within the past five years.ML models consistently demonstrated satisfactory to excellent area under the receiver operating characteristic curve values(ranging from 0.6 to 1)across all studies,surpassing the performance of traditional scoring systems.Random forest exhibited superior predictive capabilities for 90-d mortality following LT,sepsis,and acute kidney injury(AKI).In contrast,gradient boosting excelled in predicting the risk of graft-versus-host disease,pneumonia,and AKI.CONCLUSION This study underscores the potential of ML models in guiding decisions related to allograft allocation and LT,marking a significant evolution in the field of prognostication.
