Total variation （TV） is widely applied in image process-ing. The assumption of TV is that an image consists of piecewise constants, however, it suffers from the so-cal ed staircase effect. In order to reduce the sta...Total variation （TV） is widely applied in image process-ing. The assumption of TV is that an image consists of piecewise constants, however, it suffers from the so-cal ed staircase effect. In order to reduce the staircase effect and preserve the edges when textures of image are extracted, a new image decomposition model is proposed in this paper. The proposed model is based on the to-tal generalized variation method which involves and balances the higher order of the structure. We also derive a numerical algorithm based on a primal-dual formulation that can be effectively imple-mented. Numerical experiments show that the proposed method can achieve a better trade-off between noise removal and texture extraction, while avoiding the staircase effect efficiently.展开更多
In order to avoid staircasing effect and preserve small scale texture information for the classical total variation regularization, a new minimization energy functional model for image decomposition is proposed. First...In order to avoid staircasing effect and preserve small scale texture information for the classical total variation regularization, a new minimization energy functional model for image decomposition is proposed. Firstly, an adaptive regularization based on the local feature of images is introduced to substitute total variational regularization. The oscillatory component containing texture and/or noise is modeled in generalized function space div （BMO）. And then, the existence and uniqueness of the minimizer for proposed model are proved. Finally, the gradient descent flow of the Euler-Lagrange equations for the new model is numerically implemented by using a finite difference method. Experiments show that the proposed model is very robust to noise, and the staircasing effect is avoided efficiently, while edges and textures are well remained.展开更多
New models for image decomposition are proposed which separate an image into a cartoon, consisting only of geometric objects, and an oscillatory component, consisting of textures or noise. The proposed models are give...New models for image decomposition are proposed which separate an image into a cartoon, consisting only of geometric objects, and an oscillatory component, consisting of textures or noise. The proposed models are given in a variational formulation with adaptive regularization norms for both the cartoon and texture parts. The adaptive behavior preserves key features such as object boundaries and textures while avoiding staircasing in what should be smooth regions. This decomposition is computed by minimizing a convex functional which depends on the two variables u and v, alternatively in each variable. Experimental results and comparisons to validate the proposed models are presented.展开更多
We propose a layered image inpainting scheme based on image decomposition. The damaged image is first decomposed into three layers： cartoon, edge, and texture. The cartoon and edge layers are repaired using an adapti...We propose a layered image inpainting scheme based on image decomposition. The damaged image is first decomposed into three layers： cartoon, edge, and texture. The cartoon and edge layers are repaired using an adaptive offset operator that can fill-in damaged image blocks while preserving sharpness of edges. The missing information in the texture layer is generated with a texture synthesis method. By using discrete cosine transform （DCT） in image decomposition and trading between resolution and computation complexity in texture synthesis, the processing time is kept at a reasonable level.展开更多
In this paper we present a novel image decomposition method via credible data fitting with local total variation filter. The oscillation rate is used to measure the image complexity and characteristics. The filter par...In this paper we present a novel image decomposition method via credible data fitting with local total variation filter. The oscillation rate is used to measure the image complexity and characteristics. The filter parameter can be determined by a fitting curve which is reconstructed by oscillation rate. In addition, the approximate Gaussian algorithm and integral image are used to reduce the algorithm computation and the sensitivity of the filter window selection. Experiments show the new method is better than the exist- ing methods.展开更多
Intrinsic image decomposition is an important and long-standing computer vision problem.Given an input image,recovering the physical scene properties is ill-posed.Several physically motivated priors have been used to ...Intrinsic image decomposition is an important and long-standing computer vision problem.Given an input image,recovering the physical scene properties is ill-posed.Several physically motivated priors have been used to restrict the solution space of the optimization problem for intrinsic image decomposition.This work takes advantage of deep learning,and shows that it can solve this challenging computer vision problem with high efficiency.The focus lies in the feature encoding phase to extract discriminative features for different intrinsic layers from an input image.To achieve this goal,we explore the distinctive characteristics of different intrinsic components in the high-dimensional feature embedding space.We define feature distribution divergence to efficiently separate the feature vectors of different intrinsic components.The feature distributions are also constrained to fit the real ones through a feature distribution consistency.In addition,a data refinement approach is provided to remove data inconsistency from the Sintel dataset,making it more suitable for intrinsic image decomposition.Our method is also extended to intrinsic video decomposition based on pixel-wise correspondences between adjacent frames.Experimental results indicate that our proposed network structure can outperform the existing state-of-the-art.展开更多
Image smoothing is a crucial image processing topic and has wide applications. For images with rich texture, most of the existing image smoothing methods are difficult to obtain significant texture removal performance...Image smoothing is a crucial image processing topic and has wide applications. For images with rich texture, most of the existing image smoothing methods are difficult to obtain significant texture removal performance because texture containing obvious edges and large gradient changes is easy to be preserved as the main edges. In this paper, we propose a novel framework （DSHFG） for image smoothing combined with the constraint of sparse high frequency gradient for texture images. First, we decompose the image into two components： a smooth component （constant component） and a non-smooth （high frequency） component. Second, we remove the non-smooth component containing high frequency gradient and smooth the other component combining with the constraint of sparse high frequency gradient. Experimental results demonstrate the proposed method is more competitive on efficiently texture removing than the state-of-the-art methods. What is more, our approach has a variety of applications including edge detection, detail magnification, image abstraction, and image composition.展开更多
A satellite image adaptive restoration method was developed that avoids ringing artifacts at the image boundary and retains oriented features. The method combines periodic plus smooth image decom- position with comple...A satellite image adaptive restoration method was developed that avoids ringing artifacts at the image boundary and retains oriented features. The method combines periodic plus smooth image decom- position with complex wavelet packet transforms. The framework first decomposes a degraded satellite im- age into the sum of a ＂periodic component＂ and a ＂smooth component＂. The Bayesian method is then used to estimate the modulation transfer function degradation parameters and the noise. The periodic component is deconvoluted using complex wavelet packet transforms with the deconvolution result of the periodic component then combined with the smooth component to get the final recovered result. Tests show that this strategy effectively avoids ringing artifacts while preserving local image details （especially directional tex- tures） without amplifying the noise. Quantitative comparisons illustrate that the results are comparable with previous methods. Another benefit is that this approach can process large satellite images with parallel processing, which is important for practical use.展开更多
In this paper,we study to use nonlocal bounded variation(NLBV)techniques to decompose an image intensity into the illumination and reflectance components.By considering spatial smoothness of the illumination component...In this paper,we study to use nonlocal bounded variation(NLBV)techniques to decompose an image intensity into the illumination and reflectance components.By considering spatial smoothness of the illumination component and nonlocal total variation(NLTV)of the reflectance component in the decomposition framework,an energy functional is constructed.We establish the theoretical results of the space of NLBV functions such as lower semicontinuity,approximation and compactness.These essential properties of NLBV functions are important tools to show the existence of solution of the proposed energy functional.Experimental results on both grey-level and color images are shown to illustrate the usefulness of the nonlocal total variation image decomposition model,and demonstrate the performance of the proposed method is better than the other testing methods.展开更多
Underwater imaging posts a challenge due to the degradation by the absorption and scattering occurred during light propagation as well as poor lighting conditions in water medium Although image filtering techniques ar...Underwater imaging posts a challenge due to the degradation by the absorption and scattering occurred during light propagation as well as poor lighting conditions in water medium Although image filtering techniques are utilized to improve image quality effectively, problems of the distortion of image details and the bias of color correction still exist in output images due to the complexity of image texture distribution. This paper proposes a new underwater image enhancement method based on image struc- tural decomposition. By introducing a curvature factor into the Mumford_Shah_G decomposition algorithm, image details and struc- ture components are better preserved without the gradient effect. Thus, histogram equalization and Retinex algorithms are applied in the decomposed structure component for global image enhancement and non-uniform brightness correction for gray level and the color images, then the optical absorption spectrum in water medium is incorporate to improve the color correction. Finally, the en- hauced structure and preserved detail component are re.composed to generate the output. Experiments with real underwater images verify the image improvement by the proposed method in image contrast, brightness and color fidelity.展开更多
Oil spills pose a major threat to ocean ecosystems and their health. Synthetic aperture radar（SAR） sensors can detect oil spills on the sea surface. These oil spills appear as dark spots in SAR images. However, dark...Oil spills pose a major threat to ocean ecosystems and their health. Synthetic aperture radar（SAR） sensors can detect oil spills on the sea surface. These oil spills appear as dark spots in SAR images. However, dark formations can be caused by a number of phenomena. It is aimed to distinguishing oil spills or look-alike objects. A novel method based on a bidimensional empirical mode decomposition is proposed. The selected dark formations are first decomposed into several bidimensional intrinsic mode functions and the residue. Subsequently, 64 dimension feature sets are calculated using the Hilbert spectral analysis and five new features are extracted with a relief algorithm. Mahalanobis distances are then used for classification. Three data sets containing oil spills or look-alikes are used to test the accuracy rate of the method. The accuracy rate is more than 90%. The experimental results demonstrate that the novel method can detect oil spills validly and accurately.展开更多
With the bandwidth restriction in airborne Synthetic Aperture Radar (SAR)-based hu- man-in-the-loop applications, the acquired SAR images should be compressed with loss to overcome the conflict of image quantity and t...With the bandwidth restriction in airborne Synthetic Aperture Radar (SAR)-based hu- man-in-the-loop applications, the acquired SAR images should be compressed with loss to overcome the conflict of image quantity and the response time. In this letter a framework of SAR image compression is described. The SAR image is decomposed into two components, namely structural and textural components. The target region mask is used to retain the important target information by allocating more bits during compression, while less bits is allocated for the background region. The obtained results show that the compressed image using the proposed algorithm has better visual effect under the same bit rate compared with JPEG2000 algorithm.展开更多
This paper proposes a novel region based image fusion scheme based on multiresolution analysis. The low frequency band of the image multiresolution representation is segmented into important regions, sub-important reg...This paper proposes a novel region based image fusion scheme based on multiresolution analysis. The low frequency band of the image multiresolution representation is segmented into important regions, sub-important regions and background regions. Each feature of the regions is used to determine the region’s degree of membership in the multiresolution representation, and then to achieve multiresolution representation of the fusion result. The final image fusion result can be obtained by using the inverse multiresolution transform. Experiments showed that the proposed image fusion method can have better performance than existing image fusion methods.展开更多
Poor illumination greatly affects the quality of obtained images.In this paper,a novel convolutional neural network named DEANet is proposed on the basis of Retinex for low-light image enhancement.DEANet combines the ...Poor illumination greatly affects the quality of obtained images.In this paper,a novel convolutional neural network named DEANet is proposed on the basis of Retinex for low-light image enhancement.DEANet combines the frequency and content information of images and is divided into three subnetworks:decomposition,enhancement,and adjustment networks,which perform image decomposition;denoising,contrast enhancement,and detail preservation;and image adjustment and generation,respectively.The model is trained on the public LOL dataset,and the experimental results show that it outperforms the existing state-of-the-art methods regarding visual effects and image quality.展开更多
Background Monocular depth estimation aims to predict a dense depth map from a single RGB image,and has important applications in 3D reconstruction,automatic driving,and augmented reality.However,existing methods dire...Background Monocular depth estimation aims to predict a dense depth map from a single RGB image,and has important applications in 3D reconstruction,automatic driving,and augmented reality.However,existing methods directly feed the original RGB image into the model to extract depth features without avoiding the interference of depth-irrelevant information on depth-estimation accuracy,which leads to inferior performance.Methods To remove the influence of depth-irrelevant information and improve the depth-prediction accuracy,we propose RADepthNet,a novel reflectance-guided network that fuses boundary features.Specifically,our method predicts depth maps using the following three steps:(1)Intrinsic Image Decomposition.We propose a reflectance extraction module consisting of an encoder-decoder structure to extract the depth-related reflectance.Through an ablation study,we demonstrate that the module can reduce the influence of illumination on depth estimation.(2)Boundary Detection.A boundary extraction module,consisting of an encoder,refinement block,and upsample block,was proposed to better predict the depth at object boundaries utilizing gradient constraints.(3)Depth Prediction Module.We use an encoder different from(2)to obtain depth features from the reflectance map and fuse boundary features to predict depth.In addition,we proposed FIFADataset,a depth-estimation dataset applied in soccer scenarios.Results Extensive experiments on a public dataset and our proposed FIFADataset show that our method achieves state-of-the-art performance.展开更多
We propose a novel interactive lighting editing system for lighting a single indoor RGB image based on spherical harmonic lighting.It allows users to intuitively edit illumination and relight the complicated low-light...We propose a novel interactive lighting editing system for lighting a single indoor RGB image based on spherical harmonic lighting.It allows users to intuitively edit illumination and relight the complicated low-light indoor scene.Our method not only achieves plausible global relighting but also enhances the local details of the complicated scene according to the spatially-varying spherical harmonic lighting,which only requires a single RGB image along with a corresponding depth map.To this end,we first present a joint optimization algorithm,which is based on the geometric optimization of the depth map and intrinsic image decomposition avoiding texture-copy,for refining the depth map and obtaining the shading map.Then we propose a lighting estimation method based on spherical harmonic lighting,which not only achieves the global illumination estimation of the scene,but also further enhances local details of the complicated scene.Finally,we use a simple and intuitive interactive method to edit the environment lighting map to adjust lighting and relight the scene.Through extensive experimental results,we demonstrate that our proposed approach is simple and intuitive for relighting the low-light indoor scene,and achieve state-of-the-art results.展开更多
Under the perspective projection assumption,non-Lambertian photometric stereo is a highly non-linear problem.In this study,we present an optimized framework for reconstructing the surface normal and depth with non-Lam...Under the perspective projection assumption,non-Lambertian photometric stereo is a highly non-linear problem.In this study,we present an optimized framework for reconstructing the surface normal and depth with non-Lambertian reflection models under perspective projection.By decomposing the images into diffuse and specular components,we compute the surface normal and reflectance simultaneously.We also propose a variational formulation that is robust and useful for surface reconstruction.The experiments show that our method accurately reconstructs both the surface shape and reflectance of colorful objects with non-Lambertian surfaces.展开更多
基金supported by the National Natural Science Foundation of China(6127129461301229)+1 种基金the Doctoral Research Fund of Henan University of Science and Technology(0900170809001751)
文摘Total variation （TV） is widely applied in image process-ing. The assumption of TV is that an image consists of piecewise constants, however, it suffers from the so-cal ed staircase effect. In order to reduce the staircase effect and preserve the edges when textures of image are extracted, a new image decomposition model is proposed in this paper. The proposed model is based on the to-tal generalized variation method which involves and balances the higher order of the structure. We also derive a numerical algorithm based on a primal-dual formulation that can be effectively imple-mented. Numerical experiments show that the proposed method can achieve a better trade-off between noise removal and texture extraction, while avoiding the staircase effect efficiently.
基金supported by the Science and Technology Foundation Program of Chongqing Municipal Education Committee (KJ091208)
文摘In order to avoid staircasing effect and preserve small scale texture information for the classical total variation regularization, a new minimization energy functional model for image decomposition is proposed. Firstly, an adaptive regularization based on the local feature of images is introduced to substitute total variational regularization. The oscillatory component containing texture and/or noise is modeled in generalized function space div （BMO）. And then, the existence and uniqueness of the minimizer for proposed model are proved. Finally, the gradient descent flow of the Euler-Lagrange equations for the new model is numerically implemented by using a finite difference method. Experiments show that the proposed model is very robust to noise, and the staircasing effect is avoided efficiently, while edges and textures are well remained.
文摘New models for image decomposition are proposed which separate an image into a cartoon, consisting only of geometric objects, and an oscillatory component, consisting of textures or noise. The proposed models are given in a variational formulation with adaptive regularization norms for both the cartoon and texture parts. The adaptive behavior preserves key features such as object boundaries and textures while avoiding staircasing in what should be smooth regions. This decomposition is computed by minimizing a convex functional which depends on the two variables u and v, alternatively in each variable. Experimental results and comparisons to validate the proposed models are presented.
基金Project supported by the Shanghai Leading Academic Discipline Project(Grant No.T0102)
文摘We propose a layered image inpainting scheme based on image decomposition. The damaged image is first decomposed into three layers： cartoon, edge, and texture. The cartoon and edge layers are repaired using an adaptive offset operator that can fill-in damaged image blocks while preserving sharpness of edges. The missing information in the texture layer is generated with a texture synthesis method. By using discrete cosine transform （DCT） in image decomposition and trading between resolution and computation complexity in texture synthesis, the processing time is kept at a reasonable level.
基金Supported by National Nature Science Foundation of China(61103150)National Research Foundation for the Doctoral Program of Higher Education of China(20110131130004)Shandong University Outstanding Graduate Research Innovation Fund(No.yyx10122)
文摘In this paper we present a novel image decomposition method via credible data fitting with local total variation filter. The oscillation rate is used to measure the image complexity and characteristics. The filter parameter can be determined by a fitting curve which is reconstructed by oscillation rate. In addition, the approximate Gaussian algorithm and integral image are used to reduce the algorithm computation and the sensitivity of the filter window selection. Experiments show the new method is better than the exist- ing methods.
基金supported by the Special Funds for Creative Research(Grant No.2022C61540)the National Natural Science Foundation of China(NSFC,Grant Nos.61972012 and 61732016).
文摘Intrinsic image decomposition is an important and long-standing computer vision problem.Given an input image,recovering the physical scene properties is ill-posed.Several physically motivated priors have been used to restrict the solution space of the optimization problem for intrinsic image decomposition.This work takes advantage of deep learning,and shows that it can solve this challenging computer vision problem with high efficiency.The focus lies in the feature encoding phase to extract discriminative features for different intrinsic layers from an input image.To achieve this goal,we explore the distinctive characteristics of different intrinsic components in the high-dimensional feature embedding space.We define feature distribution divergence to efficiently separate the feature vectors of different intrinsic components.The feature distributions are also constrained to fit the real ones through a feature distribution consistency.In addition,a data refinement approach is provided to remove data inconsistency from the Sintel dataset,making it more suitable for intrinsic image decomposition.Our method is also extended to intrinsic video decomposition based on pixel-wise correspondences between adjacent frames.Experimental results indicate that our proposed network structure can outperform the existing state-of-the-art.
基金This work was supported by the National Natural Science Foundation of China under Grant Nos. 61373078, 61572292, 61602277, and 61332015, the Key Project of National Natural Science Foundation of China Joint Fund with Zhejiang Integration of Informatization and Industrialization under Grant No. U1609218, and the Natural Science Foundation of Shandong Province of China under Grant No. ZR2016FQ12.
文摘Image smoothing is a crucial image processing topic and has wide applications. For images with rich texture, most of the existing image smoothing methods are difficult to obtain significant texture removal performance because texture containing obvious edges and large gradient changes is easy to be preserved as the main edges. In this paper, we propose a novel framework （DSHFG） for image smoothing combined with the constraint of sparse high frequency gradient for texture images. First, we decompose the image into two components： a smooth component （constant component） and a non-smooth （high frequency） component. Second, we remove the non-smooth component containing high frequency gradient and smooth the other component combining with the constraint of sparse high frequency gradient. Experimental results demonstrate the proposed method is more competitive on efficiently texture removing than the state-of-the-art methods. What is more, our approach has a variety of applications including edge detection, detail magnification, image abstraction, and image composition.
基金Supported by the National High-Tech Research and Development (863) Program of China (No. 2007AA120408)
文摘A satellite image adaptive restoration method was developed that avoids ringing artifacts at the image boundary and retains oriented features. The method combines periodic plus smooth image decom- position with complex wavelet packet transforms. The framework first decomposes a degraded satellite im- age into the sum of a ＂periodic component＂ and a ＂smooth component＂. The Bayesian method is then used to estimate the modulation transfer function degradation parameters and the noise. The periodic component is deconvoluted using complex wavelet packet transforms with the deconvolution result of the periodic component then combined with the smooth component to get the final recovered result. Tests show that this strategy effectively avoids ringing artifacts while preserving local image details （especially directional tex- tures） without amplifying the noise. Quantitative comparisons illustrate that the results are comparable with previous methods. Another benefit is that this approach can process large satellite images with parallel processing, which is important for practical use.
基金supported by the National Natural Science Foundation of China(Grant No.11201341)China Postdoctoral Science Foundation funded project(No.2012M511126 and No.2013T60459)+1 种基金supported by RGC GRF Grant Number 202013HKBU FRG Grant Number FRG/12-13/065.
文摘In this paper,we study to use nonlocal bounded variation(NLBV)techniques to decompose an image intensity into the illumination and reflectance components.By considering spatial smoothness of the illumination component and nonlocal total variation(NLTV)of the reflectance component in the decomposition framework,an energy functional is constructed.We establish the theoretical results of the space of NLBV functions such as lower semicontinuity,approximation and compactness.These essential properties of NLBV functions are important tools to show the existence of solution of the proposed energy functional.Experimental results on both grey-level and color images are shown to illustrate the usefulness of the nonlocal total variation image decomposition model,and demonstrate the performance of the proposed method is better than the other testing methods.
基金supported by the National Natural Science Foundation of China (Grant Nos.60772058 and 61271406)
文摘Underwater imaging posts a challenge due to the degradation by the absorption and scattering occurred during light propagation as well as poor lighting conditions in water medium Although image filtering techniques are utilized to improve image quality effectively, problems of the distortion of image details and the bias of color correction still exist in output images due to the complexity of image texture distribution. This paper proposes a new underwater image enhancement method based on image struc- tural decomposition. By introducing a curvature factor into the Mumford_Shah_G decomposition algorithm, image details and struc- ture components are better preserved without the gradient effect. Thus, histogram equalization and Retinex algorithms are applied in the decomposed structure component for global image enhancement and non-uniform brightness correction for gray level and the color images, then the optical absorption spectrum in water medium is incorporate to improve the color correction. Finally, the en- hauced structure and preserved detail component are re.composed to generate the output. Experiments with real underwater images verify the image improvement by the proposed method in image contrast, brightness and color fidelity.
基金The National Science and Technology Support Project under contract No.2014BAB12B02the Natural Science Foundation of Liaoning Province under contract No.201602042
文摘Oil spills pose a major threat to ocean ecosystems and their health. Synthetic aperture radar（SAR） sensors can detect oil spills on the sea surface. These oil spills appear as dark spots in SAR images. However, dark formations can be caused by a number of phenomena. It is aimed to distinguishing oil spills or look-alike objects. A novel method based on a bidimensional empirical mode decomposition is proposed. The selected dark formations are first decomposed into several bidimensional intrinsic mode functions and the residue. Subsequently, 64 dimension feature sets are calculated using the Hilbert spectral analysis and five new features are extracted with a relief algorithm. Mahalanobis distances are then used for classification. Three data sets containing oil spills or look-alikes are used to test the accuracy rate of the method. The accuracy rate is more than 90%. The experimental results demonstrate that the novel method can detect oil spills validly and accurately.
文摘With the bandwidth restriction in airborne Synthetic Aperture Radar (SAR)-based hu- man-in-the-loop applications, the acquired SAR images should be compressed with loss to overcome the conflict of image quantity and the response time. In this letter a framework of SAR image compression is described. The SAR image is decomposed into two components, namely structural and textural components. The target region mask is used to retain the important target information by allocating more bits during compression, while less bits is allocated for the background region. The obtained results show that the compressed image using the proposed algorithm has better visual effect under the same bit rate compared with JPEG2000 algorithm.
基金Project supported by the National Natural Science Foundation of China (No. 60375008), China Aviation Science Foundation (No.02D57003), China Ph.D Discipline Special Foundation (No.20020248029), and Shanghai Key Scientific Project (No.02DZ15001), China
文摘This paper proposes a novel region based image fusion scheme based on multiresolution analysis. The low frequency band of the image multiresolution representation is segmented into important regions, sub-important regions and background regions. Each feature of the regions is used to determine the region’s degree of membership in the multiresolution representation, and then to achieve multiresolution representation of the fusion result. The final image fusion result can be obtained by using the inverse multiresolution transform. Experiments showed that the proposed image fusion method can have better performance than existing image fusion methods.
基金This work was supported by the Shanghai Aerospace Science and Technology Innovation Fund(No.SAST2019-048)the Cross-Media Intelligent Technology Project of Beijing National Research Center for Information Science and Technology(BNRist)(No.BNR2019TD01022).
文摘Poor illumination greatly affects the quality of obtained images.In this paper,a novel convolutional neural network named DEANet is proposed on the basis of Retinex for low-light image enhancement.DEANet combines the frequency and content information of images and is divided into three subnetworks:decomposition,enhancement,and adjustment networks,which perform image decomposition;denoising,contrast enhancement,and detail preservation;and image adjustment and generation,respectively.The model is trained on the public LOL dataset,and the experimental results show that it outperforms the existing state-of-the-art methods regarding visual effects and image quality.
基金Supported by the National Natural Science Foundation of China under Grants 61872241, 62077037 and 62077037Shanghai Municipal Science and Technology Major Project under Grant 2021SHZDZX0102。
文摘Background Monocular depth estimation aims to predict a dense depth map from a single RGB image,and has important applications in 3D reconstruction,automatic driving,and augmented reality.However,existing methods directly feed the original RGB image into the model to extract depth features without avoiding the interference of depth-irrelevant information on depth-estimation accuracy,which leads to inferior performance.Methods To remove the influence of depth-irrelevant information and improve the depth-prediction accuracy,we propose RADepthNet,a novel reflectance-guided network that fuses boundary features.Specifically,our method predicts depth maps using the following three steps:(1)Intrinsic Image Decomposition.We propose a reflectance extraction module consisting of an encoder-decoder structure to extract the depth-related reflectance.Through an ablation study,we demonstrate that the module can reduce the influence of illumination on depth estimation.(2)Boundary Detection.A boundary extraction module,consisting of an encoder,refinement block,and upsample block,was proposed to better predict the depth at object boundaries utilizing gradient constraints.(3)Depth Prediction Module.We use an encoder different from(2)to obtain depth features from the reflectance map and fuse boundary features to predict depth.In addition,we proposed FIFADataset,a depth-estimation dataset applied in soccer scenarios.Results Extensive experiments on a public dataset and our proposed FIFADataset show that our method achieves state-of-the-art performance.
基金supported by NSFC(No.61972298)Bingtuan Science and Technology Program(No.2019BC008).
文摘We propose a novel interactive lighting editing system for lighting a single indoor RGB image based on spherical harmonic lighting.It allows users to intuitively edit illumination and relight the complicated low-light indoor scene.Our method not only achieves plausible global relighting but also enhances the local details of the complicated scene according to the spatially-varying spherical harmonic lighting,which only requires a single RGB image along with a corresponding depth map.To this end,we first present a joint optimization algorithm,which is based on the geometric optimization of the depth map and intrinsic image decomposition avoiding texture-copy,for refining the depth map and obtaining the shading map.Then we propose a lighting estimation method based on spherical harmonic lighting,which not only achieves the global illumination estimation of the scene,but also further enhances local details of the complicated scene.Finally,we use a simple and intuitive interactive method to edit the environment lighting map to adjust lighting and relight the scene.Through extensive experimental results,we demonstrate that our proposed approach is simple and intuitive for relighting the low-light indoor scene,and achieve state-of-the-art results.
基金the Technological Program of Cultural Relics Preservation of Zhejiang Province,Chinathe Key Research and Development Program of Zhejiang Province,China(No.2018C03051)the National Standard Development Program of Cultural Relics Protection of China(No.581250-T0170B)。
文摘Under the perspective projection assumption,non-Lambertian photometric stereo is a highly non-linear problem.In this study,we present an optimized framework for reconstructing the surface normal and depth with non-Lambertian reflection models under perspective projection.By decomposing the images into diffuse and specular components,we compute the surface normal and reflectance simultaneously.We also propose a variational formulation that is robust and useful for surface reconstruction.The experiments show that our method accurately reconstructs both the surface shape and reflectance of colorful objects with non-Lambertian surfaces.
