Motivated by the study of regularization for sparse problems,we propose a new regularization method for sparse vector recovery.We derive sufficient conditions on the well-posedness of the new regularization,and design...Motivated by the study of regularization for sparse problems,we propose a new regularization method for sparse vector recovery.We derive sufficient conditions on the well-posedness of the new regularization,and design an iterative algorithm,namely the iteratively reweighted algorithm(IR-algorithm),for efficiently computing the sparse solutions to the proposed regularization model.The convergence of the IR-algorithm and the setting of the regularization parameters are analyzed at length.Finally,we present numerical examples to illustrate the features of the new regularization and algorithm.展开更多
In point cloud registration applications,noise and poor initial conditions lead to many false matches.False matches significantly degrade registration accuracy and speed.A penalty function is adopted in many robust po...In point cloud registration applications,noise and poor initial conditions lead to many false matches.False matches significantly degrade registration accuracy and speed.A penalty function is adopted in many robust point-to-point registration methods to suppress the influence of false matches.However,after applying a penalty function,problems cannot be solved in their analytical forms based on the introduction of nonlinearity.Therefore,most existing methods adopt the descending method.In this paper,a novel iterative-reweighting-based method is proposed to overcome the limitations of existing methods.The proposed method iteratively solves the eigenvectors of a four-dimensional matrix,whereas the calculation of the descending method relies on solving an eight-dimensional matrix.Therefore,the proposed method can achieve increased computational efficiency.The proposed method was validated on simulated noise corruption data,and the results reveal that it obtains higher efficiency and precision than existing methods,particularly under very noisy conditions.Experimental results for the KITTI dataset demonstrate that the proposed method can be used in real-time localization processes with high accuracy and good efficiency.展开更多
The prediction of the heating and cooling loads of a building is an essential aspect in studies involving the analysis of energy consumption in buildings. An accurate estimation of heating and cooling load leads...The prediction of the heating and cooling loads of a building is an essential aspect in studies involving the analysis of energy consumption in buildings. An accurate estimation of heating and cooling load leads to better management of energy related tasks and progressing towards an energy efficient building. With increasing global energy demands and buildings being major energy consuming entities, there is renewed interest in studying the energy performance of buildings. Alternative technologies like Artificial Intelligence (AI) techniques are being widely used in energy studies involving buildings. This paper presents a review of research in the area of forecasting the heating and cooling load of buildings using AI techniques. The results discussed in this paper demonstrate the use of AI techniques in the estimation of the thermal loads of buildings. An accurate prediction of the heating and cooling loads of buildings is necessary for forecasting the energy expenditure in buildings. It can also help in the design and construction of energy efficient buildings.展开更多
Dropout and other feature noising schemes have shown promise in controlling over-fitting by artificially corrupting the training data. Though extensive studies have been performed for generalized linear models, little...Dropout and other feature noising schemes have shown promise in controlling over-fitting by artificially corrupting the training data. Though extensive studies have been performed for generalized linear models, little has been done for support vector machines （SVMs）, one of the most successful approaches for supervised learning. This paper presents dropout training for both linear SVMs and the nonlinear extension with latent representation learning. For linear SVMs, to deal with the intractable expectation of the non-smooth hinge loss under corrupting distributions, we develop an iteratively re-weighted least square （IRLS） algorithm by exploring data augmentation techniques. Our algorithm iteratively minimizes the expectation of a re- weighted least square problem, where the re-weights are analytically updated. For nonlinear latent SVMs, we con- sider learning one layer of latent representations in SVMs and extend the data augmentation technique in conjunction with first-order Taylor-expansion to deal with the intractable expected hinge loss and the nonlinearity of latent representa- tions. Finally, we apply the similar data augmentation ideas to develop a new IRLS algorithm for the expected logistic loss under corrupting distributions, and we further develop a non-linear extension of logistic regression by incorporating one layer of latent representations. Our algorithms offer insights on the connection and difference between the hinge loss and logistic loss in dropout training. Empirical results on several real datasets demonstrate the effectiveness of dropout training on significantly boosting the classification accuracy of both linear and nonlinear SVMs.展开更多
The l1 norm is the tight convex relaxation for the l0 norm and has been successfully applied for recovering sparse signals.However,for problems with fewer samples than required for accurate l1 recovery,one needs to ap...The l1 norm is the tight convex relaxation for the l0 norm and has been successfully applied for recovering sparse signals.However,for problems with fewer samples than required for accurate l1 recovery,one needs to apply nonconvex penalties such as lp norm.As one method for solving lp minimization problems,iteratively reweighted l1 minimization updates the weight for each component based on the value of the same component at the previous iteration.It assigns large weights on small components in magnitude and small weights on large components in magnitude.The set of the weights is not fixed,and it makes the analysis of this method difficult.In this paper,we consider a weighted l1 penalty with the set of the weights fixed,and the weights are assigned based on the sort of all the components in magnitude.The smallest weight is assigned to the largest component in magnitude.This new penalty is called nonconvex sorted l1.Then we propose two methods for solving nonconvex sorted l1 minimization problems:iteratively reweighted l1 minimization and iterative sorted thresholding,and prove that both methods will converge to a local minimizer of the nonconvex sorted l1 minimization problems.We also show that both methods are generalizations of iterative support detection and iterative hard thresholding,respectively.The numerical experiments demonstrate the better performance of assigning weights by sort compared to assigning by value.展开更多
Sparse representation has been widely used in signal processing,pattern recognition and computer vision etc.Excellent achievements have been made in both theoretical researches and practical applications.However,there...Sparse representation has been widely used in signal processing,pattern recognition and computer vision etc.Excellent achievements have been made in both theoretical researches and practical applications.However,there are two limitations on the application of classification.One is that sufficient training samples are required for each class,and the other is that samples should be uncorrupted.In order to alleviate above problems,a sparse and dense hybrid representation(SDR)framework has been proposed,where the training dictionary is decomposed into a class-specific dictionary and a non-class-specific dictionary.SDR putsℓ1 constraint on the coefficients of class-specific dictionary.Nevertheless,it over-emphasizes the sparsity and overlooks the correlation information in class-specific dictionary,which may lead to poor classification results.To overcome this disadvantage,an adaptive sparse and dense hybrid representation with non-convex optimization(ASDR-NO)is proposed in this paper.The trace norm is adopted in class-specific dictionary,which is different from general approaches.By doing so,the dictionary structure becomes adaptive and the representation ability of the dictionary will be improved.Meanwhile,a non-convex surrogate is used to approximate the rank function in dictionary decomposition in order to avoid a suboptimal solution of the original rank minimization,which can be solved by iteratively reweighted nuclear norm(IRNN)algorithm.Extensive experiments conducted on benchmark data sets have verified the effectiveness and advancement of the proposed algorithm compared with the state-of-the-art sparse representation methods.展开更多
基金Project supported by the National Natural Science Foundation of China(No.61603322)the Research Foundation of Education Bureau of Hunan Province of China(No.16C1542)
文摘Motivated by the study of regularization for sparse problems,we propose a new regularization method for sparse vector recovery.We derive sufficient conditions on the well-posedness of the new regularization,and design an iterative algorithm,namely the iteratively reweighted algorithm(IR-algorithm),for efficiently computing the sparse solutions to the proposed regularization model.The convergence of the IR-algorithm and the setting of the regularization parameters are analyzed at length.Finally,we present numerical examples to illustrate the features of the new regularization and algorithm.
基金the National Natural Science Foundation of China(No.U1764264)。
文摘In point cloud registration applications,noise and poor initial conditions lead to many false matches.False matches significantly degrade registration accuracy and speed.A penalty function is adopted in many robust point-to-point registration methods to suppress the influence of false matches.However,after applying a penalty function,problems cannot be solved in their analytical forms based on the introduction of nonlinearity.Therefore,most existing methods adopt the descending method.In this paper,a novel iterative-reweighting-based method is proposed to overcome the limitations of existing methods.The proposed method iteratively solves the eigenvectors of a four-dimensional matrix,whereas the calculation of the descending method relies on solving an eight-dimensional matrix.Therefore,the proposed method can achieve increased computational efficiency.The proposed method was validated on simulated noise corruption data,and the results reveal that it obtains higher efficiency and precision than existing methods,particularly under very noisy conditions.Experimental results for the KITTI dataset demonstrate that the proposed method can be used in real-time localization processes with high accuracy and good efficiency.
文摘The prediction of the heating and cooling loads of a building is an essential aspect in studies involving the analysis of energy consumption in buildings. An accurate estimation of heating and cooling load leads to better management of energy related tasks and progressing towards an energy efficient building. With increasing global energy demands and buildings being major energy consuming entities, there is renewed interest in studying the energy performance of buildings. Alternative technologies like Artificial Intelligence (AI) techniques are being widely used in energy studies involving buildings. This paper presents a review of research in the area of forecasting the heating and cooling load of buildings using AI techniques. The results discussed in this paper demonstrate the use of AI techniques in the estimation of the thermal loads of buildings. An accurate prediction of the heating and cooling loads of buildings is necessary for forecasting the energy expenditure in buildings. It can also help in the design and construction of energy efficient buildings.
文摘Dropout and other feature noising schemes have shown promise in controlling over-fitting by artificially corrupting the training data. Though extensive studies have been performed for generalized linear models, little has been done for support vector machines （SVMs）, one of the most successful approaches for supervised learning. This paper presents dropout training for both linear SVMs and the nonlinear extension with latent representation learning. For linear SVMs, to deal with the intractable expectation of the non-smooth hinge loss under corrupting distributions, we develop an iteratively re-weighted least square （IRLS） algorithm by exploring data augmentation techniques. Our algorithm iteratively minimizes the expectation of a re- weighted least square problem, where the re-weights are analytically updated. For nonlinear latent SVMs, we con- sider learning one layer of latent representations in SVMs and extend the data augmentation technique in conjunction with first-order Taylor-expansion to deal with the intractable expected hinge loss and the nonlinearity of latent representa- tions. Finally, we apply the similar data augmentation ideas to develop a new IRLS algorithm for the expected logistic loss under corrupting distributions, and we further develop a non-linear extension of logistic regression by incorporating one layer of latent representations. Our algorithms offer insights on the connection and difference between the hinge loss and logistic loss in dropout training. Empirical results on several real datasets demonstrate the effectiveness of dropout training on significantly boosting the classification accuracy of both linear and nonlinear SVMs.
基金This work is partially supported by European Research Council,the National Natural Science Foundation of China(No.11201079)the Fundamental Research Funds for the Central Universities of China(Nos.20520133238 and 20520131169)the National Natural Science Foundation of United States(Nos.DMS-0748839 and DMS-1317602).
文摘The l1 norm is the tight convex relaxation for the l0 norm and has been successfully applied for recovering sparse signals.However,for problems with fewer samples than required for accurate l1 recovery,one needs to apply nonconvex penalties such as lp norm.As one method for solving lp minimization problems,iteratively reweighted l1 minimization updates the weight for each component based on the value of the same component at the previous iteration.It assigns large weights on small components in magnitude and small weights on large components in magnitude.The set of the weights is not fixed,and it makes the analysis of this method difficult.In this paper,we consider a weighted l1 penalty with the set of the weights fixed,and the weights are assigned based on the sort of all the components in magnitude.The smallest weight is assigned to the largest component in magnitude.This new penalty is called nonconvex sorted l1.Then we propose two methods for solving nonconvex sorted l1 minimization problems:iteratively reweighted l1 minimization and iterative sorted thresholding,and prove that both methods will converge to a local minimizer of the nonconvex sorted l1 minimization problems.We also show that both methods are generalizations of iterative support detection and iterative hard thresholding,respectively.The numerical experiments demonstrate the better performance of assigning weights by sort compared to assigning by value.
基金The work described in this paper was partially supported by the National Natural Science Foundation of China(Grant Nos.61673249,61703252)the Union Fund of National Natural Science Foundation of China(U1805263)the Research Project Supported by Shanxi Scholarship Council of China(2016-004).
文摘Sparse representation has been widely used in signal processing,pattern recognition and computer vision etc.Excellent achievements have been made in both theoretical researches and practical applications.However,there are two limitations on the application of classification.One is that sufficient training samples are required for each class,and the other is that samples should be uncorrupted.In order to alleviate above problems,a sparse and dense hybrid representation(SDR)framework has been proposed,where the training dictionary is decomposed into a class-specific dictionary and a non-class-specific dictionary.SDR putsℓ1 constraint on the coefficients of class-specific dictionary.Nevertheless,it over-emphasizes the sparsity and overlooks the correlation information in class-specific dictionary,which may lead to poor classification results.To overcome this disadvantage,an adaptive sparse and dense hybrid representation with non-convex optimization(ASDR-NO)is proposed in this paper.The trace norm is adopted in class-specific dictionary,which is different from general approaches.By doing so,the dictionary structure becomes adaptive and the representation ability of the dictionary will be improved.Meanwhile,a non-convex surrogate is used to approximate the rank function in dictionary decomposition in order to avoid a suboptimal solution of the original rank minimization,which can be solved by iteratively reweighted nuclear norm(IRNN)algorithm.Extensive experiments conducted on benchmark data sets have verified the effectiveness and advancement of the proposed algorithm compared with the state-of-the-art sparse representation methods.
