Sharp convergence rates of stochastic approximation algorithms are given for the case where the derivative of the unknown regression function at the sought-for root is zero. The convergence rates obtained are sharp fo...Sharp convergence rates of stochastic approximation algorithms are given for the case where the derivative of the unknown regression function at the sought-for root is zero. The convergence rates obtained are sharp for the general step size used in the algorithms in contrast to the previous work where they are not sharp for slowly decreasing step sizes; all possible limit points are found for the case where the first matrix coefficient in the expansion of the regression function is normal; and the estimation upper bound is shown to be achieved for the multi-dimensional case in contrast to the previous work where only the one-dimensional result is proved.展开更多
We study iterative processes of stochastic approximation for finding fixed points of weakly contractive and nonexpansive operators in Hilbert spaces under the condition that operators are given with random errors. We ...We study iterative processes of stochastic approximation for finding fixed points of weakly contractive and nonexpansive operators in Hilbert spaces under the condition that operators are given with random errors. We prove mean square convergence and convergence almost sure (a.s.) of iterative approximations and establish both asymptotic and nonasymptotic estimates of the convergence rate in degenerate and non-degenerate cases. Previously the stochastic approximation algorithms were studied mainly for optimization problems.展开更多
在MIMO-OFDM水声通信系统中,由于信道间的相互干扰和水声信道严重时延扩展产生的频率选择性衰落,系统的通信误码率较高。针对这一问题,研究了空频编码的MIMO-OFDM通信,提出空频迭代信道估计与均衡(Spatial Frequency Iterative Channel ...在MIMO-OFDM水声通信系统中,由于信道间的相互干扰和水声信道严重时延扩展产生的频率选择性衰落,系统的通信误码率较高。针对这一问题,研究了空频编码的MIMO-OFDM通信,提出空频迭代信道估计与均衡(Spatial Frequency Iterative Channel Estimation and Equalization,SFICEE)方法。该方法通过载波间的空频正交性进行各收发阵元对的信道估计,并通过空频均衡获得符号初始估计,迭代更新信道估计,而后通过符号后验软信息反馈进行迭代空频软均衡。仿真结果表明,当误码率为10^(-3)时,文中所提出的SFICEE方法经过二次迭代与STBC方法相比具有4.8 d B的性能增益,相对于SFBC方法有2.8 d B的性能提升。当输入信噪比相同时,文中所提出方法的星座图更加收敛,可以更好地降低水下通信系统的误码率。展开更多
In this paper,we study a stochastic Newton method for nonlinear equations,whose exact function information is difficult to obtain while only stochastic approximations are available.At each iteration of the proposed al...In this paper,we study a stochastic Newton method for nonlinear equations,whose exact function information is difficult to obtain while only stochastic approximations are available.At each iteration of the proposed algorithm,an inexact Newton step is first computed based on stochastic zeroth-and first-order oracles.To encourage the possible reduction of the optimality error,we then take the unit step size if it is acceptable by an inexact Armijo line search condition.Otherwise,a small step size will be taken to help induce desired good properties.Then we investigate convergence properties of the proposed algorithm and obtain the almost sure global convergence under certain conditions.We also explore the computational complexities to find an approximate solution in terms of calls to stochastic zeroth-and first-order oracles,when the proposed algorithm returns a randomly chosen output.Furthermore,we analyze the local convergence properties of the algorithm and establish the local convergence rate in high probability.At last we present preliminary numerical tests and the results demonstrate the promising performances of the proposed algorithm.展开更多
Alternating direction method of multipliers(ADMM)receives much attention in the recent years due to various demands from machine learning and big data related optimization.In 2013,Ouyang et al.extend the ADMM to the s...Alternating direction method of multipliers(ADMM)receives much attention in the recent years due to various demands from machine learning and big data related optimization.In 2013,Ouyang et al.extend the ADMM to the stochastic setting for solving some stochastic optimization problems,inspired by the structural risk minimization principle.In this paper,we consider a stochastic variant of symmetric ADMM,named symmetric stochastic linearized ADMM(SSL-ADMM).In particular,using the framework of variational inequality,we analyze the convergence properties of SSL-ADMM.Moreover,we show that,with high probability,SSL-ADMM has O((ln N)·N^(-1/2))constraint violation bound and objective error bound for convex problems,and has O((ln N)^(2)·N^(-1))constraint violation bound and objective error bound for strongly convex problems,where N is the iteration number.Symmetric ADMM can improve the algorithmic performance compared to classical ADMM,numerical experiments for statistical machine learning show that such an improvement is also present in the stochastic setting.展开更多
This paper considers the estimation for a partly linear model with case 1 interval censored data.We assume that the error distribution belongs to a known family of scale distributions with an unknown scaleparameter. T...This paper considers the estimation for a partly linear model with case 1 interval censored data.We assume that the error distribution belongs to a known family of scale distributions with an unknown scaleparameter. The sieve maximum likelihood estimator (MLE) for the model's parameter is shown to be stronglyconsistent, and the convergence rate of the estimator is obtained and discussed.展开更多
Customary stochastic programming with recourse assumes that the probability distribution of random parameters is independent of decision variables.Recent studies demonstrated that stochastic programming models with en...Customary stochastic programming with recourse assumes that the probability distribution of random parameters is independent of decision variables.Recent studies demonstrated that stochastic programming models with endogenous uncertainty can better reflect many real-world activities and applications accompanying with decision-dependent uncertainty.In this paper,we concentrate on a class of decision-dependent two-stage stochastic programs(DTSPs)and investigate their discrete approximation.To develop the discrete approximation methods for DTSPs,we first derive the quantitative stability results for DTSPs.Based on the stability conclusion,we examine two discretization schemes when the support set of random variables is bounded,and give the rates of convergence for the optimal value and optimal solution set of the discrete approximation problem to those of the original problem.Then we extend the proposed approaches to the general situation with an unbounded support set by using the truncating technique.As an illustration of our discretization schemes,we reformulate the discretization problems under specific structures of the decision-dependent distribution.Finally,an application and numerical results are presented to demonstrate our theoretical results.展开更多
文摘Sharp convergence rates of stochastic approximation algorithms are given for the case where the derivative of the unknown regression function at the sought-for root is zero. The convergence rates obtained are sharp for the general step size used in the algorithms in contrast to the previous work where they are not sharp for slowly decreasing step sizes; all possible limit points are found for the case where the first matrix coefficient in the expansion of the regression function is normal; and the estimation upper bound is shown to be achieved for the multi-dimensional case in contrast to the previous work where only the one-dimensional result is proved.
文摘We study iterative processes of stochastic approximation for finding fixed points of weakly contractive and nonexpansive operators in Hilbert spaces under the condition that operators are given with random errors. We prove mean square convergence and convergence almost sure (a.s.) of iterative approximations and establish both asymptotic and nonasymptotic estimates of the convergence rate in degenerate and non-degenerate cases. Previously the stochastic approximation algorithms were studied mainly for optimization problems.
文摘在MIMO-OFDM水声通信系统中,由于信道间的相互干扰和水声信道严重时延扩展产生的频率选择性衰落,系统的通信误码率较高。针对这一问题,研究了空频编码的MIMO-OFDM通信,提出空频迭代信道估计与均衡(Spatial Frequency Iterative Channel Estimation and Equalization,SFICEE)方法。该方法通过载波间的空频正交性进行各收发阵元对的信道估计,并通过空频均衡获得符号初始估计,迭代更新信道估计,而后通过符号后验软信息反馈进行迭代空频软均衡。仿真结果表明,当误码率为10^(-3)时,文中所提出的SFICEE方法经过二次迭代与STBC方法相比具有4.8 d B的性能增益,相对于SFBC方法有2.8 d B的性能提升。当输入信噪比相同时,文中所提出方法的星座图更加收敛,可以更好地降低水下通信系统的误码率。
基金supported by the National Natural Science Foundation of China (Nos.11731013,11871453 and 11971089)Young Elite Scientists Sponsorship Program by CAST (No.2018QNRC001)+1 种基金Youth Innovation Promotion Association,CASFundamental Research Funds for the Central Universities,UCAS.
文摘In this paper,we study a stochastic Newton method for nonlinear equations,whose exact function information is difficult to obtain while only stochastic approximations are available.At each iteration of the proposed algorithm,an inexact Newton step is first computed based on stochastic zeroth-and first-order oracles.To encourage the possible reduction of the optimality error,we then take the unit step size if it is acceptable by an inexact Armijo line search condition.Otherwise,a small step size will be taken to help induce desired good properties.Then we investigate convergence properties of the proposed algorithm and obtain the almost sure global convergence under certain conditions.We also explore the computational complexities to find an approximate solution in terms of calls to stochastic zeroth-and first-order oracles,when the proposed algorithm returns a randomly chosen output.Furthermore,we analyze the local convergence properties of the algorithm and establish the local convergence rate in high probability.At last we present preliminary numerical tests and the results demonstrate the promising performances of the proposed algorithm.
基金Supported by National Natural Science Foundation of China (61662036)。
文摘Alternating direction method of multipliers(ADMM)receives much attention in the recent years due to various demands from machine learning and big data related optimization.In 2013,Ouyang et al.extend the ADMM to the stochastic setting for solving some stochastic optimization problems,inspired by the structural risk minimization principle.In this paper,we consider a stochastic variant of symmetric ADMM,named symmetric stochastic linearized ADMM(SSL-ADMM).In particular,using the framework of variational inequality,we analyze the convergence properties of SSL-ADMM.Moreover,we show that,with high probability,SSL-ADMM has O((ln N)·N^(-1/2))constraint violation bound and objective error bound for convex problems,and has O((ln N)^(2)·N^(-1))constraint violation bound and objective error bound for strongly convex problems,where N is the iteration number.Symmetric ADMM can improve the algorithmic performance compared to classical ADMM,numerical experiments for statistical machine learning show that such an improvement is also present in the stochastic setting.
基金the National Natural Science Foundation of China(Grant No. 19631040).
文摘This paper considers the estimation for a partly linear model with case 1 interval censored data.We assume that the error distribution belongs to a known family of scale distributions with an unknown scaleparameter. The sieve maximum likelihood estimator (MLE) for the model's parameter is shown to be stronglyconsistent, and the convergence rate of the estimator is obtained and discussed.
文摘Customary stochastic programming with recourse assumes that the probability distribution of random parameters is independent of decision variables.Recent studies demonstrated that stochastic programming models with endogenous uncertainty can better reflect many real-world activities and applications accompanying with decision-dependent uncertainty.In this paper,we concentrate on a class of decision-dependent two-stage stochastic programs(DTSPs)and investigate their discrete approximation.To develop the discrete approximation methods for DTSPs,we first derive the quantitative stability results for DTSPs.Based on the stability conclusion,we examine two discretization schemes when the support set of random variables is bounded,and give the rates of convergence for the optimal value and optimal solution set of the discrete approximation problem to those of the original problem.Then we extend the proposed approaches to the general situation with an unbounded support set by using the truncating technique.As an illustration of our discretization schemes,we reformulate the discretization problems under specific structures of the decision-dependent distribution.Finally,an application and numerical results are presented to demonstrate our theoretical results.
