This paper presents a new trust region algorithm for solving a class of composite nonsmooth optimizations. It is distinguished by the fact that this method does not enforce strict monotonicity of the objective functio...This paper presents a new trust region algorithm for solving a class of composite nonsmooth optimizations. It is distinguished by the fact that this method does not enforce strict monotonicity of the objective function values at successive iterates and that this method extends the existing results for this type of nonlinear optimization with smooth, or piecewise smooth, or convex objective functions or their composition. It is proved that this algorithm is globally convergent under certain conditions. Finally, some numerical results for several optimization problems are reported which show that the nonmonotonic trust region method is competitive with the usual trust region method.展开更多
The Nesterov accelerated dynamical approach serves as an essential tool for addressing convex optimization problems with accelerated convergence rates.Most previous studies in this field have primarily concentrated on...The Nesterov accelerated dynamical approach serves as an essential tool for addressing convex optimization problems with accelerated convergence rates.Most previous studies in this field have primarily concentrated on unconstrained smooth con-vex optimization problems.In this paper,on the basis of primal-dual dynamical approach,Nesterov accelerated dynamical approach,projection operator and directional gradient,we present two accelerated primal-dual projection neurodynamic approaches with time scaling to address convex optimization problems with smooth and nonsmooth objective functions subject to linear and set constraints,which consist of a second-order ODE(ordinary differential equation)or differential conclusion system for the primal variables and a first-order ODE for the dual vari-ables.By satisfying specific conditions for time scaling,we demonstrate that the proposed approaches have a faster conver-gence rate.This only requires assuming convexity of the objective function.We validate the effectiveness of our proposed two accel-erated primal-dual projection neurodynamic approaches through numerical experiments.展开更多
Some properties of a class of quasi-differentiable functions（the difference of two finite convex functions） are considered in this paper. And the convergence of the steepest descent algorithm for unconstrained and c...Some properties of a class of quasi-differentiable functions（the difference of two finite convex functions） are considered in this paper. And the convergence of the steepest descent algorithm for unconstrained and constrained quasi-differentiable programming is proved.展开更多
In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorith...In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorithm under reasonable assumptions.展开更多
By means of Lagrange duality of Hill＇s maximum plastic work principle theory of the convex program, a dual problem under Mises＇ yield condition has been derived and whereby a non-differentiable convex optimization m...By means of Lagrange duality of Hill＇s maximum plastic work principle theory of the convex program, a dual problem under Mises＇ yield condition has been derived and whereby a non-differentiable convex optimization model for the limit analysis is developed. With this model, it is not necessary to linearize the yield condition and its discrete form becomes a minimization problem of the sum of Euclidean norms subject to linear constraints. Aimed at resolving the non-differentiability of Euclidean norms, a smoothing algorithm for the limit analysis of perfect-plastic continuum media is proposed. Its efficiency is demonstrated by computing the limit load factor and the collapse state for some plane stress and plain strain problems.展开更多
Numerical methods for the solution of nonsmooth equations are studied. A new subdifferential for a locally Lipschitzian function is proposed. Based on this subdifferential, Newton methods for solving nonsmooth equatio...Numerical methods for the solution of nonsmooth equations are studied. A new subdifferential for a locally Lipschitzian function is proposed. Based on this subdifferential, Newton methods for solving nonsmooth equations are developed and their convergence is shown. Since this subdifferential is easy to be computed, the present Newton methods can be executed easily in some applications.展开更多
A new nonsmooth equations model of constrained minimax problem was de-rived. The generalized Newton method was applied for solving this system of nonsmooth equations system. A new algorithm for solving constrained min...A new nonsmooth equations model of constrained minimax problem was de-rived. The generalized Newton method was applied for solving this system of nonsmooth equations system. A new algorithm for solving constrained minimax problem was established. The local superlinear and quadratic convergences of the algorithm were discussed.展开更多
The paper is concerned with the filled functions for global optimization of a continuous function of several variables.More general forms of filled functions are presented for smooth and nonsmooth optimizations.These ...The paper is concerned with the filled functions for global optimization of a continuous function of several variables.More general forms of filled functions are presented for smooth and nonsmooth optimizations.These functions have either two adjustable parameters or one adjustable parameter.Conditions on functions and on the values of parameters are given so that the constructed functions are desired filled functions.展开更多
This paper studies a class of nonconvex composite optimization, whose objective is a summation of an average of nonconvex(weakly) smooth functions and a convex nonsmooth function, where the gradient of the former func...This paper studies a class of nonconvex composite optimization, whose objective is a summation of an average of nonconvex(weakly) smooth functions and a convex nonsmooth function, where the gradient of the former function has the H o¨lder continuity. By exploring the structure of such kind of problems, we first propose a proximal(quasi-)Newton algorithm wPQN(Proximal quasi-Newton algorithm for weakly smooth optimization) and investigate its theoretical complexities to find an approximate solution. Then we propose a stochastic variant algorithm wPSQN(Proximal stochastic quasi-Newton algorithm for weakly smooth optimization), which allows a random subset of component functions to be used at each iteration. Moreover, motivated by recent success of variance reduction techniques, we propose two variance reduced algorithms,wPSQN-SVRG and wPSQN-SARAH, and investigate their computational complexity separately.展开更多
In this paper, a bundle modification strategy is proposed for nonsmooth convex constrained min- imization problems. As a result, a new feasible point bundle method is presented by applying this strategy. Whenever the ...In this paper, a bundle modification strategy is proposed for nonsmooth convex constrained min- imization problems. As a result, a new feasible point bundle method is presented by applying this strategy. Whenever the stability center is updated, some points in the bundle will be substituted by new ones which have lower objective values and/or constraint values, aiming at getting a better bundle. The method generates feasible serious iterates on which the objective function is monotonically decreasing. Global convergence of the algorithm is established, and some preliminary numerical results show that our method performs better than the standard feasible point bundle method.展开更多
This paper presents a modified definition of the filled function for finding a global minimizer of a nonsmooth function on a closed bounded set, and then give a one-parameter filled function. Theoretical and numerical...This paper presents a modified definition of the filled function for finding a global minimizer of a nonsmooth function on a closed bounded set, and then give a one-parameter filled function. Theoretical and numerical properties of the proposed filled function are investigated and a corresponding solution algorithm is proposed. The proposed filled function＇s parameter is easier to be appropriately chosen than previous functions in literatures. Numerical results obtained indicate the efficiency of the proposed filled function method. An improved fingerprint recognition method using global filled function is also reported.展开更多
uv-decomposition method for solving a mathematical program with equilibrium constraints （MPEC） problem with linear complementarity constraints is presented. The problem is first converted into a nonlinear programmin...uv-decomposition method for solving a mathematical program with equilibrium constraints （MPEC） problem with linear complementarity constraints is presented. The problem is first converted into a nonlinear programming one. The structure of subdifferential a corresponding penalty function and results of its uv-decomposition are given. A conceptual algorithm for solving this problem with a superUnear convergence rate is then constructed in terms of the obtained results.展开更多
First-order proximal methods that solve linear and bilinear elliptic optimal control problems with a sparsity cost functional are discussed. In particular, fast convergence of these methods is proved. For benchmarking...First-order proximal methods that solve linear and bilinear elliptic optimal control problems with a sparsity cost functional are discussed. In particular, fast convergence of these methods is proved. For benchmarking purposes, inexact proximal schemes are compared to an inexact semismooth Newton method. Results of numerical experiments are presented to demonstrate the computational effectiveness of proximal schemes applied to infinite-dimensional elliptic optimal control problems and to validate the theoretical estimates.展开更多
optimal topology design of truss structures concerning stress and frictionless unilateral contact displacement constraints is investigated. The existence of ununique optimal solution under contact gaps is found. This ...optimal topology design of truss structures concerning stress and frictionless unilateral contact displacement constraints is investigated. The existence of ununique optimal solution under contact gaps is found. This shows that the contact conditions have an effect on structural topology, and different initial contact gaps may lead to different structural topologies. To avoid the singular optima in structural topology optimization in multiple loading cases, an epsilon-relaxed method is adopted to establish the relaxing topology optimization formulations. The problem is solved by means of a two-level optimization method. In the first sublevel, the solution of the frictionless unilateral contact problem is obtained by solving an equivalent quadratic programming. In the second sublevel, topology optimization of truss is carried out by an epsilon-relaxed method. The validity of the method proposed is verified by computational results.展开更多
A vu-decomposition method for solving a second-order cone problem is presented in this paper. It is first transformed into a nonlinear programming problem. Then, the structure of the Clarke subdifferential correspondi...A vu-decomposition method for solving a second-order cone problem is presented in this paper. It is first transformed into a nonlinear programming problem. Then, the structure of the Clarke subdifferential corresponding to the penalty function and some results of itsvu-decomposition are given. Under a certain condition, a twice continuously differentiable trajectory is computed to produce a second-order expansion of the objective function. A conceptual algorithm for solving this problem with a superlinear convergence rate is given.展开更多
A framework for the optimal sparse-control of the probability density function of a jump-diffusion process is presented. This framework is based on the partial integro-differential Fokker-Planck (FP) equation that gov...A framework for the optimal sparse-control of the probability density function of a jump-diffusion process is presented. This framework is based on the partial integro-differential Fokker-Planck (FP) equation that governs the time evolution of the probability density function of this process. In the stochastic process and, correspondingly, in the FP model the control function enters as a time-dependent coefficient. The objectives of the control are to minimize a discrete-in-time, resp. continuous-in-time, tracking functionals and its L2- and L1-costs, where the latter is considered to promote control sparsity. An efficient proximal scheme for solving these optimal control problems is considered. Results of numerical experiments are presented to validate the theoretical results and the computational effectiveness of the proposed control framework.展开更多
We introduce a new algorithm,extended regularized dual averaging(XRDA),for solving regularized stochastic optimization problems,which generalizes the regularized dual averaging(RDA)method.The main novelty of the metho...We introduce a new algorithm,extended regularized dual averaging(XRDA),for solving regularized stochastic optimization problems,which generalizes the regularized dual averaging(RDA)method.The main novelty of the method is that it allows a flexible control of the backward step size.For instance,the backward step size used in RDA grows without bound,while for XRDA the backward step size can be kept bounded.We demonstrate experimentally that additional control over the backward step size can speed up the convergence of the algorithm while preserving desired properties of the iterates,such as sparsity.Theoretically,we show that the XRDA method achieves the same convergence rate as RDA for general convex objectives.展开更多
In this paper,the relation between the shadow price and the Lagrange multiplier for nonsmooth optimization problem is explored.It is obtained that the Lagrange multipliers are upper bounds of the shadow price for a co...In this paper,the relation between the shadow price and the Lagrange multiplier for nonsmooth optimization problem is explored.It is obtained that the Lagrange multipliers are upper bounds of the shadow price for a convex optimization problem and a class of Lipschtzian optimization problems.This result can be used in pricing mechanisms for nonsmooth situation.Several nonsmooth functions involved in this class of Lipschtzian optimizations are listed.Finally,an application to electricity pricing is discussed.展开更多
In this work,we present probabilistic local convergence results for a stochastic semismooth Newton method for a class of stochastic composite optimization problems involving the sum of smooth nonconvex and nonsmooth c...In this work,we present probabilistic local convergence results for a stochastic semismooth Newton method for a class of stochastic composite optimization problems involving the sum of smooth nonconvex and nonsmooth convex terms in the objective function.We assume that the gradient and Hessian information of the smooth part of the objective function can only be approximated and accessed via calling stochastic firstand second-order oracles.The approach combines stochastic semismooth Newton steps,stochastic proximal gradient steps and a globalization strategy based on growth conditions.We present tail bounds and matrix concentration inequalities for the stochastic oracles that can be utilized to control the approximation errors via appropriately adjusting or increasing the sampling rates.Under standard local assumptions,we prove that the proposed algorithm locally turns into a pure stochastic semismooth Newton method and converges r-linearly or r-superlinearly with high probability.展开更多
Accelerated proximal gradient methods have recently been developed for solving quasi-static incremental problems of elastoplastic analysis with some different yield criteria.It has been demonstrated through numerical ...Accelerated proximal gradient methods have recently been developed for solving quasi-static incremental problems of elastoplastic analysis with some different yield criteria.It has been demonstrated through numerical experiments that these methods can outperform conventional optimization-based approaches in computational plasticity.However,in literature these algorithms are described individually for specific yield criteria,and hence there exists no guide for application of the algorithms to other yield criteria.This short paper presents a general form of algorithm design,independent of specific forms of yield criteria,that unifies the existing proximal gradient methods.Clear interpretation is also given to each step of the presented general algorithm so that each update rule is linked to the underlying physical laws in terms of mechanical quantities.展开更多
文摘This paper presents a new trust region algorithm for solving a class of composite nonsmooth optimizations. It is distinguished by the fact that this method does not enforce strict monotonicity of the objective function values at successive iterates and that this method extends the existing results for this type of nonlinear optimization with smooth, or piecewise smooth, or convex objective functions or their composition. It is proved that this algorithm is globally convergent under certain conditions. Finally, some numerical results for several optimization problems are reported which show that the nonmonotonic trust region method is competitive with the usual trust region method.
基金supported by the National Natural Science Foundation of China(62176218,62176027)the Fundamental Research Funds for the Central Universities(XDJK2020TY003)the Funds for Chongqing Talent Plan(cstc2024ycjh-bgzxm0082)。
文摘The Nesterov accelerated dynamical approach serves as an essential tool for addressing convex optimization problems with accelerated convergence rates.Most previous studies in this field have primarily concentrated on unconstrained smooth con-vex optimization problems.In this paper,on the basis of primal-dual dynamical approach,Nesterov accelerated dynamical approach,projection operator and directional gradient,we present two accelerated primal-dual projection neurodynamic approaches with time scaling to address convex optimization problems with smooth and nonsmooth objective functions subject to linear and set constraints,which consist of a second-order ODE(ordinary differential equation)or differential conclusion system for the primal variables and a first-order ODE for the dual vari-ables.By satisfying specific conditions for time scaling,we demonstrate that the proposed approaches have a faster conver-gence rate.This only requires assuming convexity of the objective function.We validate the effectiveness of our proposed two accel-erated primal-dual projection neurodynamic approaches through numerical experiments.
基金Supported by the State Foundations of Ph.D.Units(20020141013)Supported by the NSF of China(10001007)
文摘Some properties of a class of quasi-differentiable functions（the difference of two finite convex functions） are considered in this paper. And the convergence of the steepest descent algorithm for unconstrained and constrained quasi-differentiable programming is proved.
基金Supported by CERG: CityU 101005 of the Government of Hong Kong SAR, Chinathe National Natural ScienceFoundation of China, the Specialized Research Fund of Doctoral Program of Higher Education of China (Grant No.20040319003)the Natural Science Fund of Jiangsu Province of China (Grant No. BK2006214)
文摘In this paper we present a filter-trust-region algorithm for solving LC1 unconstrained optimization problems which uses the second Dini upper directional derivative. We establish the global convergence of the algorithm under reasonable assumptions.
基金Project supported by the National Natural Science Foundation of China (Nos.10572031, 10332010)
文摘By means of Lagrange duality of Hill＇s maximum plastic work principle theory of the convex program, a dual problem under Mises＇ yield condition has been derived and whereby a non-differentiable convex optimization model for the limit analysis is developed. With this model, it is not necessary to linearize the yield condition and its discrete form becomes a minimization problem of the sum of Euclidean norms subject to linear constraints. Aimed at resolving the non-differentiability of Euclidean norms, a smoothing algorithm for the limit analysis of perfect-plastic continuum media is proposed. Its efficiency is demonstrated by computing the limit load factor and the collapse state for some plane stress and plain strain problems.
文摘Numerical methods for the solution of nonsmooth equations are studied. A new subdifferential for a locally Lipschitzian function is proposed. Based on this subdifferential, Newton methods for solving nonsmooth equations are developed and their convergence is shown. Since this subdifferential is easy to be computed, the present Newton methods can be executed easily in some applications.
文摘A new nonsmooth equations model of constrained minimax problem was de-rived. The generalized Newton method was applied for solving this system of nonsmooth equations system. A new algorithm for solving constrained minimax problem was established. The local superlinear and quadratic convergences of the algorithm were discussed.
文摘The paper is concerned with the filled functions for global optimization of a continuous function of several variables.More general forms of filled functions are presented for smooth and nonsmooth optimizations.These functions have either two adjustable parameters or one adjustable parameter.Conditions on functions and on the values of parameters are given so that the constructed functions are desired filled functions.
基金Supported by National Natural Science Foundation of China(Grant No.11871453)The Major Key Project of PCL(Grant No.PCL2022A05).
文摘This paper studies a class of nonconvex composite optimization, whose objective is a summation of an average of nonconvex(weakly) smooth functions and a convex nonsmooth function, where the gradient of the former function has the H o¨lder continuity. By exploring the structure of such kind of problems, we first propose a proximal(quasi-)Newton algorithm wPQN(Proximal quasi-Newton algorithm for weakly smooth optimization) and investigate its theoretical complexities to find an approximate solution. Then we propose a stochastic variant algorithm wPSQN(Proximal stochastic quasi-Newton algorithm for weakly smooth optimization), which allows a random subset of component functions to be used at each iteration. Moreover, motivated by recent success of variance reduction techniques, we propose two variance reduced algorithms,wPSQN-SVRG and wPSQN-SARAH, and investigate their computational complexity separately.
基金Project supported by the National Natural Science Foundation of China(11761013,11771383)Guangxi Natural Science Foundation(2013GXNSFAA019013,2014GXNSFFA118001,2016GXNSFDA380019)the Open Project of Guangxi Colleges and Universities Key Laboratory of Complex System Optimization and Big Data Processing(2016CSOBDP0203)
文摘In this paper, a bundle modification strategy is proposed for nonsmooth convex constrained min- imization problems. As a result, a new feasible point bundle method is presented by applying this strategy. Whenever the stability center is updated, some points in the bundle will be substituted by new ones which have lower objective values and/or constraint values, aiming at getting a better bundle. The method generates feasible serious iterates on which the objective function is monotonically decreasing. Global convergence of the algorithm is established, and some preliminary numerical results show that our method performs better than the standard feasible point bundle method.
基金This research is supported by the National Natural Science Foundation of China under Grant No. 11001248.
文摘This paper presents a modified definition of the filled function for finding a global minimizer of a nonsmooth function on a closed bounded set, and then give a one-parameter filled function. Theoretical and numerical properties of the proposed filled function are investigated and a corresponding solution algorithm is proposed. The proposed filled function＇s parameter is easier to be appropriately chosen than previous functions in literatures. Numerical results obtained indicate the efficiency of the proposed filled function method. An improved fingerprint recognition method using global filled function is also reported.
基金Project supported by the National Natural Science Foundation of China(Nos.10372063,10771026 and 10471015)
文摘uv-decomposition method for solving a mathematical program with equilibrium constraints （MPEC） problem with linear complementarity constraints is presented. The problem is first converted into a nonlinear programming one. The structure of subdifferential a corresponding penalty function and results of its uv-decomposition are given. A conceptual algorithm for solving this problem with a superUnear convergence rate is then constructed in terms of the obtained results.
文摘First-order proximal methods that solve linear and bilinear elliptic optimal control problems with a sparsity cost functional are discussed. In particular, fast convergence of these methods is proved. For benchmarking purposes, inexact proximal schemes are compared to an inexact semismooth Newton method. Results of numerical experiments are presented to demonstrate the computational effectiveness of proximal schemes applied to infinite-dimensional elliptic optimal control problems and to validate the theoretical estimates.
基金the Postdoctoral Science and Research Foundation of Shanghai,China
文摘optimal topology design of truss structures concerning stress and frictionless unilateral contact displacement constraints is investigated. The existence of ununique optimal solution under contact gaps is found. This shows that the contact conditions have an effect on structural topology, and different initial contact gaps may lead to different structural topologies. To avoid the singular optima in structural topology optimization in multiple loading cases, an epsilon-relaxed method is adopted to establish the relaxing topology optimization formulations. The problem is solved by means of a two-level optimization method. In the first sublevel, the solution of the frictionless unilateral contact problem is obtained by solving an equivalent quadratic programming. In the second sublevel, topology optimization of truss is carried out by an epsilon-relaxed method. The validity of the method proposed is verified by computational results.
基金Project supported by the National Natural Science Foundation of China (No. 10771026)the Foundation of Dalian University of Technology (Nos. MXDUT73008 and MXDUT98009)
文摘A vu-decomposition method for solving a second-order cone problem is presented in this paper. It is first transformed into a nonlinear programming problem. Then, the structure of the Clarke subdifferential corresponding to the penalty function and some results of itsvu-decomposition are given. Under a certain condition, a twice continuously differentiable trajectory is computed to produce a second-order expansion of the objective function. A conceptual algorithm for solving this problem with a superlinear convergence rate is given.
文摘A framework for the optimal sparse-control of the probability density function of a jump-diffusion process is presented. This framework is based on the partial integro-differential Fokker-Planck (FP) equation that governs the time evolution of the probability density function of this process. In the stochastic process and, correspondingly, in the FP model the control function enters as a time-dependent coefficient. The objectives of the control are to minimize a discrete-in-time, resp. continuous-in-time, tracking functionals and its L2- and L1-costs, where the latter is considered to promote control sparsity. An efficient proximal scheme for solving these optimal control problems is considered. Results of numerical experiments are presented to validate the theoretical results and the computational effectiveness of the proposed control framework.
基金Verne M.Willaman Fund,the Center for Computational Mathematics and Applications(CCMA)at the Pennsylvania State University,and the National Science Foundation(Grant No.DMS-1819157).
文摘We introduce a new algorithm,extended regularized dual averaging(XRDA),for solving regularized stochastic optimization problems,which generalizes the regularized dual averaging(RDA)method.The main novelty of the method is that it allows a flexible control of the backward step size.For instance,the backward step size used in RDA grows without bound,while for XRDA the backward step size can be kept bounded.We demonstrate experimentally that additional control over the backward step size can speed up the convergence of the algorithm while preserving desired properties of the iterates,such as sparsity.Theoretically,we show that the XRDA method achieves the same convergence rate as RDA for general convex objectives.
基金supported by the National Natural Science Foundation of China(No.72071130).
文摘In this paper,the relation between the shadow price and the Lagrange multiplier for nonsmooth optimization problem is explored.It is obtained that the Lagrange multipliers are upper bounds of the shadow price for a convex optimization problem and a class of Lipschtzian optimization problems.This result can be used in pricing mechanisms for nonsmooth situation.Several nonsmooth functions involved in this class of Lipschtzian optimizations are listed.Finally,an application to electricity pricing is discussed.
基金supported by the Fundamental Research Fund—Shenzhen Research Institute for Big Data Startup Fund(Grant No.JCYJ-AM20190601)the Shenzhen Institute of Artificial Intelligence and Robotics for Society+2 种基金National Natural Science Foundation of China(Grant Nos.11831002 and 11871135)the Key-Area Research and Development Program of Guangdong Province(Grant No.2019B121204008)Beijing Academy of Artificial Intelligence。
文摘In this work,we present probabilistic local convergence results for a stochastic semismooth Newton method for a class of stochastic composite optimization problems involving the sum of smooth nonconvex and nonsmooth convex terms in the objective function.We assume that the gradient and Hessian information of the smooth part of the objective function can only be approximated and accessed via calling stochastic firstand second-order oracles.The approach combines stochastic semismooth Newton steps,stochastic proximal gradient steps and a globalization strategy based on growth conditions.We present tail bounds and matrix concentration inequalities for the stochastic oracles that can be utilized to control the approximation errors via appropriately adjusting or increasing the sampling rates.Under standard local assumptions,we prove that the proposed algorithm locally turns into a pure stochastic semismooth Newton method and converges r-linearly or r-superlinearly with high probability.
文摘Accelerated proximal gradient methods have recently been developed for solving quasi-static incremental problems of elastoplastic analysis with some different yield criteria.It has been demonstrated through numerical experiments that these methods can outperform conventional optimization-based approaches in computational plasticity.However,in literature these algorithms are described individually for specific yield criteria,and hence there exists no guide for application of the algorithms to other yield criteria.This short paper presents a general form of algorithm design,independent of specific forms of yield criteria,that unifies the existing proximal gradient methods.Clear interpretation is also given to each step of the presented general algorithm so that each update rule is linked to the underlying physical laws in terms of mechanical quantities.
