We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space,but with a non-convex constraint set introduced by m...We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space,but with a non-convex constraint set introduced by model parameterization.This observation allows us to repose such problems via a suitable relaxation as convex optimization problems in the space of distributions over the training parameters.We derive some simple relationships between the distribution-space problem and the original problem,e.g.,a distribution-space solution is at least as good as a solution in the original space.Moreover,we develop a numerical algorithm based on mixture distributions to perform approximate optimization directly in the distribution space.Consistency of this approximation is established and the numerical efficacy of the proposed algorithm is illustrated in simple examples.In both theory and practice,this formulation provides an alternative approach to large-scale optimization in machine learning.展开更多
In this paper, two framelet based deconvolution algorithms are proposed. The basic idea of framelet based approach is to convert the deconvolution problem to the problem of inpainting in a frame domain by constructing...In this paper, two framelet based deconvolution algorithms are proposed. The basic idea of framelet based approach is to convert the deconvolution problem to the problem of inpainting in a frame domain by constructing a framelet system with one of the masks being the given （discrete） convolution kernel via the unitary extension principle of [26], as introduced in [6-9] . The first algorithm unifies our previous works in high resolution image reconstruction and infra-red chopped and nodded image restoration, and the second one is a combination of our previous frame-based deconvolution algorithm and the iterative thresholding algorithm given by [14, 16]. The strong convergence of the algorithms in infinite dimensional settings is given by employing proximal forward-backward splitting （PFBS） method. Consequently, it unifies iterative algorithms of infinite and finite dimensional setting and simplifies the proof of the convergence of the aluorithms of [6].展开更多
As X-ray computed tomography （CT） is widely used in diagnosis and radiotherapy, it is important to reduce the radiation dose as low as reasonably achievable. For this pur- pose, one may use the TV based methods or w...As X-ray computed tomography （CT） is widely used in diagnosis and radiotherapy, it is important to reduce the radiation dose as low as reasonably achievable. For this pur- pose, one may use the TV based methods or wavelet frame based methods to reconstruct high quality images from reduced number of projections. Furthermore, by using the in- terior tomography scheme which only illuminates a region-of-interest （ROI）, one can save more radiation dose. In this paper, a robust wavelet frame regularization based model is proposed for both global reconstruction and interior tomography. The model can help to reduce the errors caused by mismatch of the huge sparse projection matrix. A three-system decomposition scheme is applied to decompose the reconstructed images into three differ- ent parts： cartoon, artifacts and noise. Therefore, by discarding the estimated artifacts and noise parts, the reconstructed images can be obtained with less noise and artifacts. Similar to other frame based image restoration models, the model can be efficiently solved by the split Bregman algorithm. Numerical simulations show that the proposed model outperforms the FBP and SART＋TV methods in terms of preservation of sharp edges, mean structural similarity （SSIM）, contrast-to-noise ratio, relative error and correlation- s. For example, for real sheep lung reconstruction, the proposed method can reach the mean structural similarity as high as 0.75 using only 100 projections while the FBP and the SART^TV methods need more than 200 projections. Additionally, the proposed ro- bust method is applicable for interior and exterior tomography with better performance compared to the FBP and the SART＋TV methods.展开更多
基金supported by the National Natural Science Foundation of China(Grant No.12201053)supported by the National Research Foundation,Singapore,under the NRF fellowship(Project No.NRF-NRFF13-2021-0005).
文摘We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space,but with a non-convex constraint set introduced by model parameterization.This observation allows us to repose such problems via a suitable relaxation as convex optimization problems in the space of distributions over the training parameters.We derive some simple relationships between the distribution-space problem and the original problem,e.g.,a distribution-space solution is at least as good as a solution in the original space.Moreover,we develop a numerical algorithm based on mixture distributions to perform approximate optimization directly in the distribution space.Consistency of this approximation is established and the numerical efficacy of the proposed algorithm is illustrated in simple examples.In both theory and practice,this formulation provides an alternative approach to large-scale optimization in machine learning.
文摘In this paper, two framelet based deconvolution algorithms are proposed. The basic idea of framelet based approach is to convert the deconvolution problem to the problem of inpainting in a frame domain by constructing a framelet system with one of the masks being the given （discrete） convolution kernel via the unitary extension principle of [26], as introduced in [6-9] . The first algorithm unifies our previous works in high resolution image reconstruction and infra-red chopped and nodded image restoration, and the second one is a combination of our previous frame-based deconvolution algorithm and the iterative thresholding algorithm given by [14, 16]. The strong convergence of the algorithms in infinite dimensional settings is given by employing proximal forward-backward splitting （PFBS） method. Consequently, it unifies iterative algorithms of infinite and finite dimensional setting and simplifies the proof of the convergence of the aluorithms of [6].
文摘As X-ray computed tomography （CT） is widely used in diagnosis and radiotherapy, it is important to reduce the radiation dose as low as reasonably achievable. For this pur- pose, one may use the TV based methods or wavelet frame based methods to reconstruct high quality images from reduced number of projections. Furthermore, by using the in- terior tomography scheme which only illuminates a region-of-interest （ROI）, one can save more radiation dose. In this paper, a robust wavelet frame regularization based model is proposed for both global reconstruction and interior tomography. The model can help to reduce the errors caused by mismatch of the huge sparse projection matrix. A three-system decomposition scheme is applied to decompose the reconstructed images into three differ- ent parts： cartoon, artifacts and noise. Therefore, by discarding the estimated artifacts and noise parts, the reconstructed images can be obtained with less noise and artifacts. Similar to other frame based image restoration models, the model can be efficiently solved by the split Bregman algorithm. Numerical simulations show that the proposed model outperforms the FBP and SART＋TV methods in terms of preservation of sharp edges, mean structural similarity （SSIM）, contrast-to-noise ratio, relative error and correlation- s. For example, for real sheep lung reconstruction, the proposed method can reach the mean structural similarity as high as 0.75 using only 100 projections while the FBP and the SART^TV methods need more than 200 projections. Additionally, the proposed ro- bust method is applicable for interior and exterior tomography with better performance compared to the FBP and the SART＋TV methods.
