Previous deep learning-based super-resolution(SR)methods rely on the assumption that the degradation process is predefined(e.g.,bicubic downsampling).Thus,their performance would suffer from deterioration if the real ...Previous deep learning-based super-resolution(SR)methods rely on the assumption that the degradation process is predefined(e.g.,bicubic downsampling).Thus,their performance would suffer from deterioration if the real degradation is not consistent with the assumption.To deal with real-world scenarios,existing blind SR methods are committed to estimating both the degradation and the super-resolved image with an extra loss or iterative scheme.However,degradation estimation that requires more computation would result in limited SR performance due to the accumulated estimation errors.In this paper,we propose a contrastive regularization built upon contrastive learning to exploit both the information of blurry images and clear images as negative and positive samples,respectively.Contrastive regularization ensures that the restored image is pulled closer to the clear image and pushed far away from the blurry image in the representation space.Furthermore,instead of estimating the degradation,we extract global statistical prior information to capture the character of the distortion.Considering the coupling between the degradation and the low-resolution image,we embed the global prior into the distortion-specific SR network to make our method adaptive to the changes of distortions.We term our distortion-specific network with contrastive regularization as CRDNet.The extensive experiments on synthetic and realworld scenes demonstrate that our lightweight CRDNet surpasses state-of-the-art blind super-resolution approaches.展开更多
The diagnosis of COVID-19 requires chest computed tomography(CT).High-resolution CT images can provide more diagnostic information to help doctors better diagnose the disease,so it is of clinical importance to study s...The diagnosis of COVID-19 requires chest computed tomography(CT).High-resolution CT images can provide more diagnostic information to help doctors better diagnose the disease,so it is of clinical importance to study super-resolution(SR)algorithms applied to CT images to improve the reso-lution of CT images.However,most of the existing SR algorithms are studied based on natural images,which are not suitable for medical images;and most of these algorithms improve the reconstruction quality by increasing the network depth,which is not suitable for machines with limited resources.To alleviate these issues,we propose a residual feature attentional fusion network for lightweight chest CT image super-resolution(RFAFN).Specifically,we design a contextual feature extraction block(CFEB)that can extract CT image features more efficiently and accurately than ordinary residual blocks.In addition,we propose a feature-weighted cascading strategy(FWCS)based on attentional feature fusion blocks(AFFB)to utilize the high-frequency detail information extracted by CFEB as much as possible via selectively fusing adjacent level feature information.Finally,we suggest a global hierarchical feature fusion strategy(GHFFS),which can utilize the hierarchical features more effectively than dense concatenation by progressively aggregating the feature information at various levels.Numerous experiments show that our method performs better than most of the state-of-the-art(SOTA)methods on the COVID-19 chest CT dataset.In detail,the peak signal-to-noise ratio(PSNR)is 0.11 dB and 0.47 dB higher on CTtest1 and CTtest2 at×3 SR compared to the suboptimal method,but the number of parameters and multi-adds are reduced by 22K and 0.43G,respectively.Our method can better recover chest CT image quality with fewer computational resources and effectively assist in COVID-19.展开更多
Super-resolution(SR)microscopy has dramatically enhanced our understanding of biological processes.However,scattering media in thick specimens severely limits the spatial resolution,often rendering the images unclear ...Super-resolution(SR)microscopy has dramatically enhanced our understanding of biological processes.However,scattering media in thick specimens severely limits the spatial resolution,often rendering the images unclear or indistinguishable.Additionally,live-cell imaging faces challenges in achieving high temporal resolution for fast-moving subcellular structures.Here,we present the principles of a synthetic wave microscopy(SWM)to extract three-dimensional information from thick unlabeled specimens,where photobleaching and phototoxicity are avoided.SWM exploits multiple-wave interferometry to reveal the specimen’s phase information in the area of interest,which is not affected by the scattering media in the optical path.SWM achieves~0.42λ/NA resolution at an imaging speed of up to 106 pixels/s.SWM proves better temporal resolution and sensitivity than the most conventional microscopes currently available while maintaining exceptional SR and anti-scattering capabilities.Penetrating through the scattering media is challenging for conventional imaging techniques.Remarkably,SWM retains its efficacy even in conditions of low signal-to-noise ratios.It facilitates the visualization of dynamic subcellular structures in live cells,encompassing tubular endoplasmic reticulum(ER),lipid droplets,mitochondria,and lysosomes.展开更多
Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,w...Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,which is beneficial for subsequent applications.The development of deep learning has promoted significant progress in hyperspectral image super-resolution,and the powerful expression capabilities of deep neural networks make the predicted results more reliable.Recently,several latest deep learning technologies have made the hyperspectral image super-resolution method explode.However,a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent.To this end,in this survey,we first introduce the concept of hyperspectral image super-resolution and classify the methods from the perspectives with or without auxiliary information.Then,we review the learning-based methods in three categories,including single hyperspectral image super-resolution,panchromatic-based hyperspectral image super-resolution,and multispectral-based hyperspectral image super-resolution.Subsequently,we summarize the commonly used hyperspectral dataset,and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively.Moreover,we briefly introduce several typical applications of hyperspectral image super-resolution,including ground object classification,urban change detection,and ecosystem monitoring.Finally,we provide the conclusion and challenges in existing learning-based methods,looking forward to potential future research directions.展开更多
Medical image super-resolution is a fundamental challenge due to absorption and scattering in tissues.These challenges are increasing the interest in the quality of medical images.Recent research has proven that the r...Medical image super-resolution is a fundamental challenge due to absorption and scattering in tissues.These challenges are increasing the interest in the quality of medical images.Recent research has proven that the rapid progress in convolutional neural networks(CNNs)has achieved superior performance in the area of medical image super-resolution.However,the traditional CNN approaches use interpolation techniques as a preprocessing stage to enlarge low-resolution magnetic resonance(MR)images,adding extra noise in the models and more memory consumption.Furthermore,conventional deep CNN approaches used layers in series-wise connection to create the deeper mode,because this later end layer cannot receive complete information and work as a dead layer.In this paper,we propose Inception-ResNet-based Network for MRI Image Super-Resolution known as IRMRIS.In our proposed approach,a bicubic interpolation is replaced with a deconvolution layer to learn the upsampling filters.Furthermore,a residual skip connection with the Inception block is used to reconstruct a high-resolution output image from a low-quality input image.Quantitative and qualitative evaluations of the proposed method are supported through extensive experiments in reconstructing sharper and clean texture details as compared to the state-of-the-art methods.展开更多
Image Super-Resolution(SR)research has achieved great success with powerful neural networks.The deeper networks with more parameters improve the restoration quality but add the computation complexity,which means more ...Image Super-Resolution(SR)research has achieved great success with powerful neural networks.The deeper networks with more parameters improve the restoration quality but add the computation complexity,which means more inference time would be cost,hindering image SR from practical usage.Noting the spatial distribution of the objects or things in images,a twostage local objects SR system is proposed,which consists of two modules,the object detection module and the SR module.Firstly,You Only Look Once(YOLO),which is efficient in generic object detection tasks,is selected to detect the input images for obtaining objects of interest,then put them into the SR module and output corresponding High-Resolution(HR)subimages.The computational power consumption of image SR is optimized by reducing the resolution of input images.In addition,we establish a dataset,TrafficSign500,for our experiment.Finally,the performance of the proposed system is evaluated under several State-Of-The-Art(SOTA)YOLOv5 and SISR models.Results show that our system can achieve a tremendous computation improvement in image SR.展开更多
The angular resolution of radar is of crucial signifi-cance to its tracking performance.In this paper,a super-resolu-tion parameter estimation algorithm based on wide-narrowband joint processing is proposed to improve...The angular resolution of radar is of crucial signifi-cance to its tracking performance.In this paper,a super-resolu-tion parameter estimation algorithm based on wide-narrowband joint processing is proposed to improve the angular resolution of wideband monopulse radar.The range cells containing resolv-able scattering points are detected in the wideband mode,and these range cells are adopted to estimate part of the target parameters by algorithms of low computational requirement.Then,the likelihood function of the echo is constructed in the narrow-band mode to estimate the rest of the parameters,and the parameters estimated in the wideband mode are employed to reduce computation and enhance estimation accuracy.Simu-lation results demonstrate that the proposed algorithm has higher estimation accuracy and lower computational complexity than the current algorithm and can avoid the risk of model mis-match.展开更多
Hyperspectral images can easily discriminate different materials due to their fine spectral resolution.However,obtaining a hyperspectral image(HSI)with a high spatial resolution is still a challenge as we are limited ...Hyperspectral images can easily discriminate different materials due to their fine spectral resolution.However,obtaining a hyperspectral image(HSI)with a high spatial resolution is still a challenge as we are limited by the high computing requirements.The spatial resolution of HSI can be enhanced by utilizing Deep Learning(DL)based Super-resolution(SR).A 3D-CNNHSR model is developed in the present investigation for 3D spatial super-resolution for HSI,without losing the spectral content.The 3DCNNHSR model was tested for the Hyperion HSI.The pre-processing of the HSI was done before applying the SR model so that the full advantage of hyperspectral data can be utilized with minimizing the errors.The key innovation of the present investigation is that it used 3D convolution as it simultaneously applies convolution in both the spatial and spectral dimensions and captures spatial-spectral features.By clustering contiguous spectral content together,a cube is formed and by convolving the cube with the 3D kernel a 3D convolution is realized.The 3D-CNNHSR model was compared with a 2D-CNN model,additionally,the assessment was based on higherresolution data from the Sentinel-2 satellite.Based on the evaluation metrics it was observed that the 3D-CNNHSR model yields better results for the SR of HSI with efficient computational speed,which is significantly less than previous studies.展开更多
Super-resolution techniques are used to reconstruct an image with a high resolution from one or more low-resolution image(s).In this paper,we proposed a single image super-resolution algorithm.It uses the nonlocal mea...Super-resolution techniques are used to reconstruct an image with a high resolution from one or more low-resolution image(s).In this paper,we proposed a single image super-resolution algorithm.It uses the nonlocal mean filter as a prior step to produce a denoised image.The proposed algorithm is based on curvelet transform.It converts the denoised image into low and high frequencies(sub-bands).Then we applied a multi-dimensional interpolation called Lancozos interpolation over both sub-bands.In parallel,we applied sparse representation with over complete dictionary for the denoised image.The proposed algorithm then combines the dictionary learning in the sparse representation and the interpolated sub-bands using inverse curvelet transform to have an image with a higher resolution.The experimental results of the proposed super-resolution algorithm show superior performance and obviously better-recovering images with enhanced edges.The comparison study shows that the proposed super-resolution algorithm outperforms the state-of-the-art.The mean absolute error is 0.021±0.008 and the structural similarity index measure is 0.89±0.08.展开更多
A Generative Adversarial Neural(GAN)network is designed based on deep learning for the Super-Resolution(SR)reconstruction task of temperaturefields(comparable to downscaling in the meteorologicalfield),which is limite...A Generative Adversarial Neural(GAN)network is designed based on deep learning for the Super-Resolution(SR)reconstruction task of temperaturefields(comparable to downscaling in the meteorologicalfield),which is limited by the small number of ground stations and the sparse distribution of observations,resulting in a lack offineness of data.To improve the network’s generalization performance,the residual structure,and batch normalization are used.Applying the nearest interpolation method to avoid over-smoothing of the climate element values instead of the conventional Bicubic interpolation in the computer visionfield.Sub-pixel convolution is used instead of transposed convolution or interpolation methods for up-sampling to speed up network inference.The experimental dataset is the European Centre for Medium-Range Weather Forecasts Reanalysis v5(ERA5)with a bidirectional resolution of 0:1°×0:1°.On the other hand,the task aims to scale up the size by a factor of 8,which is rare compared to conventional methods.The comparison methods include traditional interpolation methods and a more widely used GAN-based network such as the SRGAN.Thefinal experimental results show that the proposed scheme advances the performance of Root Mean Square Error(RMSE)by 37.25%,the Peak Signal-to-noise Ratio(PNSR)by 14.4%,and the Structural Similarity(SSIM)by 10.3%compared to the Bicubic Interpolation.For the traditional SRGAN network,a relatively obvious performance improvement is observed by experimental demonstration.Meanwhile,the GAN network can converge stably and reach the approximate Nash equilibrium for various initialization parameters to empirically illustrate the effectiveness of the method in the temperature fields.展开更多
The prime purpose for the image reconstruction of a multi-frame super-resolution is to reconstruct a higher-resolution image through incorporating the knowledge obtained from a series of relevant low-resolution images...The prime purpose for the image reconstruction of a multi-frame super-resolution is to reconstruct a higher-resolution image through incorporating the knowledge obtained from a series of relevant low-resolution images,which is useful in numerousfields.Nevertheless,super-resolution image reconstruction methods are usually damaged by undesirable restorative artifacts,which include blurring distortion,noises,and stair-casing effects.Consequently,it is always challenging to achieve balancing between image smoothness and preservation of the edges inside the image.In this research work,we seek to increase the effectiveness of multi-frame super-resolution image reconstruction by increasing the visual information and improving the automated machine perception,which improves human analysis and interpretation processes.Accordingly,we propose a new approach to the image reconstruction of multi-frame super-resolution,so that it is created through the use of the regularization framework.In the proposed approach,the bilateral edge preserving and bilateral total variation regularizations are employed to approximate a high-resolution image generated from a sequence of corresponding images with low-resolution to protect significant features of an image,including sharp image edges and texture details while preventing artifacts.The experimental results of the synthesized image demonstrate that the new proposed approach has improved efficacy both visually and numerically more than other approaches.展开更多
Background Recurrent recovery is a common method for video super-resolution(VSR)that models the correlation between frames via hidden states.However,the application of this structure in real-world scenarios can lead t...Background Recurrent recovery is a common method for video super-resolution(VSR)that models the correlation between frames via hidden states.However,the application of this structure in real-world scenarios can lead to unsatisfactory artifacts.We found that in real-world VSR training,the use of unknown and complex degradation can better simulate the degradation process in the real world.Methods Based on this,we propose the RealFuVSR model,which simulates real-world degradation and mitigates artifacts caused by the VSR.Specifically,we propose a multiscale feature extraction module(MSF)module that extracts and fuses features from multiple scales,thereby facilitating the elimination of hidden state artifacts.To improve the accuracy of the hidden state alignment information,RealFuVSR uses an advanced optical flow-guided deformable convolution.Moreover,a cascaded residual upsampling module was used to eliminate noise caused by the upsampling process.Results The experiment demonstrates that RealFuVSR model can not only recover high-quality videos but also outperforms the state-of-the-art RealBasicVSR and RealESRGAN models.展开更多
Background:This study aims to predict the extraction difficulty of mandibular third molars based on panoramic images using transfer learning while employing super-resolution(SR)technology to enhance the feasibility an...Background:This study aims to predict the extraction difficulty of mandibular third molars based on panoramic images using transfer learning while employing super-resolution(SR)technology to enhance the feasibility and validity of the prediction.Methods:We reviewed a total of 608 preoperative mandibular third molar panoramic radiographs from two medical facilities:the First Affiliated Hospital of Zhengzhou University(n=509;456 in the training set and 53 in the test set)and the Henan Provincial Dental Hospital(n=99 in the validation set).We conducted a deep-transfer learning network on high-resolution(HR)panoramic radiographs to improve the longitudinal resolution of the images and obtained the SR images.Subsequently,we constructed models named Model-HR and Model-SR using high-dimensional quantitative features extracted through the Least Absolute Shrinkage and Selection Operator method.The models’performances were evaluated using the receiver operating characteristic curve(ROC).To assess the reliability of the model,we compared the results from the test set with those of three dentists.Results:Model-SR outperformed Model-HR(area under the curve(AUC):0.779,sensitivity:85.5%,specificity:60.9%,and accuracy:79.8%vs.AUC:0.753,sensitivity:73.7%,specificity:73.9%,and accuracy:73.7%)in predicting the difficulty of extracting mandibular third molars.Both Model-HR(AUC=0.821,95%CI 0.687–0.956)and Model-SR(AUC=0.963,95%CI 0.921–0.999)demonstrated superior performance compared to expert dentists(highest AUC=0.799,95%CI 0.671–0.927).Conclusions:Model-SR yielded superior predictive performance in determining the difficulty of extracting mandibular third molars when compared with Model-HR and expert dentists’visual assessments.展开更多
The tradeoff between efficiency and model size of the convolutional neural network(CNN)is an essential issue for applications of CNN-based algorithms to diverse real-world tasks.Although deep learning-based methods ha...The tradeoff between efficiency and model size of the convolutional neural network(CNN)is an essential issue for applications of CNN-based algorithms to diverse real-world tasks.Although deep learning-based methods have achieved significant improvements in image super-resolution(SR),current CNNbased techniques mainly contain massive parameters and a high computational complexity,limiting their practical applications.In this paper,we present a fast and lightweight framework,named weighted multi-scale residual network(WMRN),for a better tradeoff between SR performance and computational efficiency.With the modified residual structure,depthwise separable convolutions(DS Convs)are employed to improve convolutional operations’efficiency.Furthermore,several weighted multi-scale residual blocks(WMRBs)are stacked to enhance the multi-scale representation capability.In the reconstruction subnetwork,a group of Conv layers are introduced to filter feature maps to reconstruct the final high-quality image.Extensive experiments were conducted to evaluate the proposed model,and the comparative results with several state-of-the-art algorithms demonstrate the effectiveness of WMRN.展开更多
Color image super-resolution reconstruction based on the sparse representation model usually adopts the regularization norm(e.g.,L1 or L2).These methods have limited ability to keep image texture detail to some extent...Color image super-resolution reconstruction based on the sparse representation model usually adopts the regularization norm(e.g.,L1 or L2).These methods have limited ability to keep image texture detail to some extent and are easy to cause the problem of blurring details and color artifacts in color reconstructed images.This paper presents a color super-resolution reconstruction method combining the L2/3 sparse regularization model with color channel constraints.The method converts the low-resolution color image from RGB to YCbCr.The L2/3 sparse regularization model is designed to reconstruct the brightness channel of the input low-resolution color image.Then the color channel-constraint method is adopted to remove artifacts of the reconstructed highresolution image.The method not only ensures the reconstruction quality of the color image details,but also improves the removal ability of color artifacts.The experimental results on natural images validate that our method has improved both subjective and objective evaluation.展开更多
Recently, neighbor embedding based face super-resolution(SR) methods have shown the ability for achieving high-quality face images, those methods are based on the assumption that the same neighborhoods are preserved i...Recently, neighbor embedding based face super-resolution(SR) methods have shown the ability for achieving high-quality face images, those methods are based on the assumption that the same neighborhoods are preserved in both low-resolution(LR) training set and high-resolution(HR) training set. However, due to the "one-to-many" mapping between the LR image and HR ones in practice, the neighborhood relationship of the LR patch in LR space is quite different with that of the HR counterpart, that is to say the neighborhood relationship obtained is not true. In this paper, we explore a novel and effective re-identified K-nearest neighbor(RIKNN) method to search neighbors of LR patch. Compared with other methods, our method uses the geometrical information of LR manifold and HR manifold simultaneously. In particular, it searches K-NN of LR patch in the LR space and refines the searching results by re-identifying in the HR space, thus giving rise to accurate K-NN and improved performance. A statistical analysis of the influence of the training set size and nearest neighbor number is given, experimental results on some public face databases show the superiority of our proposed scheme over state-of-the-art face hallucination approaches in terms of subjective and objective results as well as computational complexity.展开更多
We use the label-free microsphere-assisted microscopy to image low-contrast hexagonally close-packed polystyrene nanoparticle arrays with diameters of 300 and 250 nm.When a nanoparticle array is directly placed on a g...We use the label-free microsphere-assisted microscopy to image low-contrast hexagonally close-packed polystyrene nanoparticle arrays with diameters of 300 and 250 nm.When a nanoparticle array is directly placed on a glass slide,it cannot be distinguished.If a 30-nm-thick Ag film is deposited on the surface of a nanoparticle array,the nanoparticle array with nanoparticle diameters of 300 and 250 nm can be distinguished.In addition,the Talbot effect of the 300-nm-diameter nanoparticle array is also observed.If a nanoparticle sample is assembled on a glass slide deposited with a 30-nm-thick Ag film,an array of 300-nm-diameter nanoparticles can be discerned.We propose that in microsphere-assisted microscopy imaging,the resolution can be improved by the excitation of surface plasmon polaritons(SPPs) on the sample surface or at the sample/substrate interface,and a higher near-field intensity due to the excited SPPs would benefit the resolution improvement.Our study of label-free super-resolution imaging of low-contrast objects will promote the applications of microsphere-assisted microscopy in life sciences.展开更多
Microvascular structure and hemodynamics are important indicators for the diagnosis and assessment of many diseases and pathologies.The structural and functional imaging of tissue microvasculature in vivo is a clinica...Microvascular structure and hemodynamics are important indicators for the diagnosis and assessment of many diseases and pathologies.The structural and functional imaging of tissue microvasculature in vivo is a clinically significant objective for the development of many imaging modalities.Contrast-enhanced ultrasound(CEUS)is a popular clinical tool for characterizing tissue microvasculature,due to the moderate cost,wide accessibility,and absence of ionizing radiation of ultrasound.展开更多
基金supported by the National Natural Science Foundation of China(61971165)the Key Research and Development Program of Hubei Province(2020BAB113)。
文摘Previous deep learning-based super-resolution(SR)methods rely on the assumption that the degradation process is predefined(e.g.,bicubic downsampling).Thus,their performance would suffer from deterioration if the real degradation is not consistent with the assumption.To deal with real-world scenarios,existing blind SR methods are committed to estimating both the degradation and the super-resolved image with an extra loss or iterative scheme.However,degradation estimation that requires more computation would result in limited SR performance due to the accumulated estimation errors.In this paper,we propose a contrastive regularization built upon contrastive learning to exploit both the information of blurry images and clear images as negative and positive samples,respectively.Contrastive regularization ensures that the restored image is pulled closer to the clear image and pushed far away from the blurry image in the representation space.Furthermore,instead of estimating the degradation,we extract global statistical prior information to capture the character of the distortion.Considering the coupling between the degradation and the low-resolution image,we embed the global prior into the distortion-specific SR network to make our method adaptive to the changes of distortions.We term our distortion-specific network with contrastive regularization as CRDNet.The extensive experiments on synthetic and realworld scenes demonstrate that our lightweight CRDNet surpasses state-of-the-art blind super-resolution approaches.
基金supported by the General Project of Natural Science Foundation of Hebei Province of China(H2019201378)the Foundation of the President of Hebei University(XZJJ201917)the Special Project for Cultivating Scientific and Technological Innovation Ability of University and Middle School Students of Hebei Province(2021H060306).
文摘The diagnosis of COVID-19 requires chest computed tomography(CT).High-resolution CT images can provide more diagnostic information to help doctors better diagnose the disease,so it is of clinical importance to study super-resolution(SR)algorithms applied to CT images to improve the reso-lution of CT images.However,most of the existing SR algorithms are studied based on natural images,which are not suitable for medical images;and most of these algorithms improve the reconstruction quality by increasing the network depth,which is not suitable for machines with limited resources.To alleviate these issues,we propose a residual feature attentional fusion network for lightweight chest CT image super-resolution(RFAFN).Specifically,we design a contextual feature extraction block(CFEB)that can extract CT image features more efficiently and accurately than ordinary residual blocks.In addition,we propose a feature-weighted cascading strategy(FWCS)based on attentional feature fusion blocks(AFFB)to utilize the high-frequency detail information extracted by CFEB as much as possible via selectively fusing adjacent level feature information.Finally,we suggest a global hierarchical feature fusion strategy(GHFFS),which can utilize the hierarchical features more effectively than dense concatenation by progressively aggregating the feature information at various levels.Numerous experiments show that our method performs better than most of the state-of-the-art(SOTA)methods on the COVID-19 chest CT dataset.In detail,the peak signal-to-noise ratio(PSNR)is 0.11 dB and 0.47 dB higher on CTtest1 and CTtest2 at×3 SR compared to the suboptimal method,but the number of parameters and multi-adds are reduced by 22K and 0.43G,respectively.Our method can better recover chest CT image quality with fewer computational resources and effectively assist in COVID-19.
基金support from CAS West Light Grant (xbzgzdsys-202206)National Key Research and Development Program of China (2021YFA1401003).
文摘Super-resolution(SR)microscopy has dramatically enhanced our understanding of biological processes.However,scattering media in thick specimens severely limits the spatial resolution,often rendering the images unclear or indistinguishable.Additionally,live-cell imaging faces challenges in achieving high temporal resolution for fast-moving subcellular structures.Here,we present the principles of a synthetic wave microscopy(SWM)to extract three-dimensional information from thick unlabeled specimens,where photobleaching and phototoxicity are avoided.SWM exploits multiple-wave interferometry to reveal the specimen’s phase information in the area of interest,which is not affected by the scattering media in the optical path.SWM achieves~0.42λ/NA resolution at an imaging speed of up to 106 pixels/s.SWM proves better temporal resolution and sensitivity than the most conventional microscopes currently available while maintaining exceptional SR and anti-scattering capabilities.Penetrating through the scattering media is challenging for conventional imaging techniques.Remarkably,SWM retains its efficacy even in conditions of low signal-to-noise ratios.It facilitates the visualization of dynamic subcellular structures in live cells,encompassing tubular endoplasmic reticulum(ER),lipid droplets,mitochondria,and lysosomes.
基金supported in part by the National Natural Science Foundation of China(62276192)。
文摘Hyperspectral image super-resolution,which refers to reconstructing the high-resolution hyperspectral image from the input low-resolution observation,aims to improve the spatial resolution of the hyperspectral image,which is beneficial for subsequent applications.The development of deep learning has promoted significant progress in hyperspectral image super-resolution,and the powerful expression capabilities of deep neural networks make the predicted results more reliable.Recently,several latest deep learning technologies have made the hyperspectral image super-resolution method explode.However,a comprehensive review and analysis of the latest deep learning methods from the hyperspectral image super-resolution perspective is absent.To this end,in this survey,we first introduce the concept of hyperspectral image super-resolution and classify the methods from the perspectives with or without auxiliary information.Then,we review the learning-based methods in three categories,including single hyperspectral image super-resolution,panchromatic-based hyperspectral image super-resolution,and multispectral-based hyperspectral image super-resolution.Subsequently,we summarize the commonly used hyperspectral dataset,and the evaluations for some representative methods in three categories are performed qualitatively and quantitatively.Moreover,we briefly introduce several typical applications of hyperspectral image super-resolution,including ground object classification,urban change detection,and ecosystem monitoring.Finally,we provide the conclusion and challenges in existing learning-based methods,looking forward to potential future research directions.
基金supported by Balochistan University of Engineering and Technology,Khuzdar,Balochistan,Pakistan.
文摘Medical image super-resolution is a fundamental challenge due to absorption and scattering in tissues.These challenges are increasing the interest in the quality of medical images.Recent research has proven that the rapid progress in convolutional neural networks(CNNs)has achieved superior performance in the area of medical image super-resolution.However,the traditional CNN approaches use interpolation techniques as a preprocessing stage to enlarge low-resolution magnetic resonance(MR)images,adding extra noise in the models and more memory consumption.Furthermore,conventional deep CNN approaches used layers in series-wise connection to create the deeper mode,because this later end layer cannot receive complete information and work as a dead layer.In this paper,we propose Inception-ResNet-based Network for MRI Image Super-Resolution known as IRMRIS.In our proposed approach,a bicubic interpolation is replaced with a deconvolution layer to learn the upsampling filters.Furthermore,a residual skip connection with the Inception block is used to reconstruct a high-resolution output image from a low-quality input image.Quantitative and qualitative evaluations of the proposed method are supported through extensive experiments in reconstructing sharper and clean texture details as compared to the state-of-the-art methods.
基金supported by the National Natural Science Foundation of China(NSFC)under Grant No.62001057by Beijing University of Posts and Telecommunications Basic Research Fund,2021RC26by the National Natural Science Foundation of China(NSFC)under Grant Nos.61871048 and 61872253.
文摘Image Super-Resolution(SR)research has achieved great success with powerful neural networks.The deeper networks with more parameters improve the restoration quality but add the computation complexity,which means more inference time would be cost,hindering image SR from practical usage.Noting the spatial distribution of the objects or things in images,a twostage local objects SR system is proposed,which consists of two modules,the object detection module and the SR module.Firstly,You Only Look Once(YOLO),which is efficient in generic object detection tasks,is selected to detect the input images for obtaining objects of interest,then put them into the SR module and output corresponding High-Resolution(HR)subimages.The computational power consumption of image SR is optimized by reducing the resolution of input images.In addition,we establish a dataset,TrafficSign500,for our experiment.Finally,the performance of the proposed system is evaluated under several State-Of-The-Art(SOTA)YOLOv5 and SISR models.Results show that our system can achieve a tremendous computation improvement in image SR.
文摘The angular resolution of radar is of crucial signifi-cance to its tracking performance.In this paper,a super-resolu-tion parameter estimation algorithm based on wide-narrowband joint processing is proposed to improve the angular resolution of wideband monopulse radar.The range cells containing resolv-able scattering points are detected in the wideband mode,and these range cells are adopted to estimate part of the target parameters by algorithms of low computational requirement.Then,the likelihood function of the echo is constructed in the narrow-band mode to estimate the rest of the parameters,and the parameters estimated in the wideband mode are employed to reduce computation and enhance estimation accuracy.Simu-lation results demonstrate that the proposed algorithm has higher estimation accuracy and lower computational complexity than the current algorithm and can avoid the risk of model mis-match.
基金Deanship of Scientific Research at King Khalid University for funding this work through large group Research Project under Grant Number RGP2/80/44.
文摘Hyperspectral images can easily discriminate different materials due to their fine spectral resolution.However,obtaining a hyperspectral image(HSI)with a high spatial resolution is still a challenge as we are limited by the high computing requirements.The spatial resolution of HSI can be enhanced by utilizing Deep Learning(DL)based Super-resolution(SR).A 3D-CNNHSR model is developed in the present investigation for 3D spatial super-resolution for HSI,without losing the spectral content.The 3DCNNHSR model was tested for the Hyperion HSI.The pre-processing of the HSI was done before applying the SR model so that the full advantage of hyperspectral data can be utilized with minimizing the errors.The key innovation of the present investigation is that it used 3D convolution as it simultaneously applies convolution in both the spatial and spectral dimensions and captures spatial-spectral features.By clustering contiguous spectral content together,a cube is formed and by convolving the cube with the 3D kernel a 3D convolution is realized.The 3D-CNNHSR model was compared with a 2D-CNN model,additionally,the assessment was based on higherresolution data from the Sentinel-2 satellite.Based on the evaluation metrics it was observed that the 3D-CNNHSR model yields better results for the SR of HSI with efficient computational speed,which is significantly less than previous studies.
文摘Super-resolution techniques are used to reconstruct an image with a high resolution from one or more low-resolution image(s).In this paper,we proposed a single image super-resolution algorithm.It uses the nonlocal mean filter as a prior step to produce a denoised image.The proposed algorithm is based on curvelet transform.It converts the denoised image into low and high frequencies(sub-bands).Then we applied a multi-dimensional interpolation called Lancozos interpolation over both sub-bands.In parallel,we applied sparse representation with over complete dictionary for the denoised image.The proposed algorithm then combines the dictionary learning in the sparse representation and the interpolated sub-bands using inverse curvelet transform to have an image with a higher resolution.The experimental results of the proposed super-resolution algorithm show superior performance and obviously better-recovering images with enhanced edges.The comparison study shows that the proposed super-resolution algorithm outperforms the state-of-the-art.The mean absolute error is 0.021±0.008 and the structural similarity index measure is 0.89±0.08.
基金supported by the National Natural Science Foundation of China under Grant Nos.61772280 and 62072249.
文摘A Generative Adversarial Neural(GAN)network is designed based on deep learning for the Super-Resolution(SR)reconstruction task of temperaturefields(comparable to downscaling in the meteorologicalfield),which is limited by the small number of ground stations and the sparse distribution of observations,resulting in a lack offineness of data.To improve the network’s generalization performance,the residual structure,and batch normalization are used.Applying the nearest interpolation method to avoid over-smoothing of the climate element values instead of the conventional Bicubic interpolation in the computer visionfield.Sub-pixel convolution is used instead of transposed convolution or interpolation methods for up-sampling to speed up network inference.The experimental dataset is the European Centre for Medium-Range Weather Forecasts Reanalysis v5(ERA5)with a bidirectional resolution of 0:1°×0:1°.On the other hand,the task aims to scale up the size by a factor of 8,which is rare compared to conventional methods.The comparison methods include traditional interpolation methods and a more widely used GAN-based network such as the SRGAN.Thefinal experimental results show that the proposed scheme advances the performance of Root Mean Square Error(RMSE)by 37.25%,the Peak Signal-to-noise Ratio(PNSR)by 14.4%,and the Structural Similarity(SSIM)by 10.3%compared to the Bicubic Interpolation.For the traditional SRGAN network,a relatively obvious performance improvement is observed by experimental demonstration.Meanwhile,the GAN network can converge stably and reach the approximate Nash equilibrium for various initialization parameters to empirically illustrate the effectiveness of the method in the temperature fields.
基金the Institute for Research and Consulting Studies at King Khalid University through Corona Research(Fast Track)[Grant Number 3-103S-2020].
文摘The prime purpose for the image reconstruction of a multi-frame super-resolution is to reconstruct a higher-resolution image through incorporating the knowledge obtained from a series of relevant low-resolution images,which is useful in numerousfields.Nevertheless,super-resolution image reconstruction methods are usually damaged by undesirable restorative artifacts,which include blurring distortion,noises,and stair-casing effects.Consequently,it is always challenging to achieve balancing between image smoothness and preservation of the edges inside the image.In this research work,we seek to increase the effectiveness of multi-frame super-resolution image reconstruction by increasing the visual information and improving the automated machine perception,which improves human analysis and interpretation processes.Accordingly,we propose a new approach to the image reconstruction of multi-frame super-resolution,so that it is created through the use of the regularization framework.In the proposed approach,the bilateral edge preserving and bilateral total variation regularizations are employed to approximate a high-resolution image generated from a sequence of corresponding images with low-resolution to protect significant features of an image,including sharp image edges and texture details while preventing artifacts.The experimental results of the synthesized image demonstrate that the new proposed approach has improved efficacy both visually and numerically more than other approaches.
基金Supported by Open Project of the Ministry of Industry and Information Technology Key Laboratory of Performance and Reliability Testing and Evaluation for Basic Software and Hardware。
文摘Background Recurrent recovery is a common method for video super-resolution(VSR)that models the correlation between frames via hidden states.However,the application of this structure in real-world scenarios can lead to unsatisfactory artifacts.We found that in real-world VSR training,the use of unknown and complex degradation can better simulate the degradation process in the real world.Methods Based on this,we propose the RealFuVSR model,which simulates real-world degradation and mitigates artifacts caused by the VSR.Specifically,we propose a multiscale feature extraction module(MSF)module that extracts and fuses features from multiple scales,thereby facilitating the elimination of hidden state artifacts.To improve the accuracy of the hidden state alignment information,RealFuVSR uses an advanced optical flow-guided deformable convolution.Moreover,a cascaded residual upsampling module was used to eliminate noise caused by the upsampling process.Results The experiment demonstrates that RealFuVSR model can not only recover high-quality videos but also outperforms the state-of-the-art RealBasicVSR and RealESRGAN models.
基金supported by the National Natural Science Foundation of China(U1904145)the Joint Funds for the Innovation of Science and Technology of Fujian province(2019Y9128).
文摘Background:This study aims to predict the extraction difficulty of mandibular third molars based on panoramic images using transfer learning while employing super-resolution(SR)technology to enhance the feasibility and validity of the prediction.Methods:We reviewed a total of 608 preoperative mandibular third molar panoramic radiographs from two medical facilities:the First Affiliated Hospital of Zhengzhou University(n=509;456 in the training set and 53 in the test set)and the Henan Provincial Dental Hospital(n=99 in the validation set).We conducted a deep-transfer learning network on high-resolution(HR)panoramic radiographs to improve the longitudinal resolution of the images and obtained the SR images.Subsequently,we constructed models named Model-HR and Model-SR using high-dimensional quantitative features extracted through the Least Absolute Shrinkage and Selection Operator method.The models’performances were evaluated using the receiver operating characteristic curve(ROC).To assess the reliability of the model,we compared the results from the test set with those of three dentists.Results:Model-SR outperformed Model-HR(area under the curve(AUC):0.779,sensitivity:85.5%,specificity:60.9%,and accuracy:79.8%vs.AUC:0.753,sensitivity:73.7%,specificity:73.9%,and accuracy:73.7%)in predicting the difficulty of extracting mandibular third molars.Both Model-HR(AUC=0.821,95%CI 0.687–0.956)and Model-SR(AUC=0.963,95%CI 0.921–0.999)demonstrated superior performance compared to expert dentists(highest AUC=0.799,95%CI 0.671–0.927).Conclusions:Model-SR yielded superior predictive performance in determining the difficulty of extracting mandibular third molars when compared with Model-HR and expert dentists’visual assessments.
基金the National Natural Science Foundation of China(61772149,61866009,61762028,U1701267,61702169)Guangxi Science and Technology Project(2019GXNSFFA245014,ZY20198016,AD18281079,AD18216004)+1 种基金the Natural Science Foundation of Hunan Province(2020JJ3014)Guangxi Colleges and Universities Key Laboratory of Intelligent Processing of Computer Images and Graphics(GIIP202001).
文摘The tradeoff between efficiency and model size of the convolutional neural network(CNN)is an essential issue for applications of CNN-based algorithms to diverse real-world tasks.Although deep learning-based methods have achieved significant improvements in image super-resolution(SR),current CNNbased techniques mainly contain massive parameters and a high computational complexity,limiting their practical applications.In this paper,we present a fast and lightweight framework,named weighted multi-scale residual network(WMRN),for a better tradeoff between SR performance and computational efficiency.With the modified residual structure,depthwise separable convolutions(DS Convs)are employed to improve convolutional operations’efficiency.Furthermore,several weighted multi-scale residual blocks(WMRBs)are stacked to enhance the multi-scale representation capability.In the reconstruction subnetwork,a group of Conv layers are introduced to filter feature maps to reconstruct the final high-quality image.Extensive experiments were conducted to evaluate the proposed model,and the comparative results with several state-of-the-art algorithms demonstrate the effectiveness of WMRN.
基金supported by the National Natural Science Foundation of China(61761028)。
文摘Color image super-resolution reconstruction based on the sparse representation model usually adopts the regularization norm(e.g.,L1 or L2).These methods have limited ability to keep image texture detail to some extent and are easy to cause the problem of blurring details and color artifacts in color reconstructed images.This paper presents a color super-resolution reconstruction method combining the L2/3 sparse regularization model with color channel constraints.The method converts the low-resolution color image from RGB to YCbCr.The L2/3 sparse regularization model is designed to reconstruct the brightness channel of the input low-resolution color image.Then the color channel-constraint method is adopted to remove artifacts of the reconstructed highresolution image.The method not only ensures the reconstruction quality of the color image details,but also improves the removal ability of color artifacts.The experimental results on natural images validate that our method has improved both subjective and objective evaluation.
基金supported by the National Natural Science Foundation of China(61172173,61303114,61271256,61272544,U1304615,U1404618)the National High Technology Research and Development Program of China(863 Program)No.2013AA014602
文摘Recently, neighbor embedding based face super-resolution(SR) methods have shown the ability for achieving high-quality face images, those methods are based on the assumption that the same neighborhoods are preserved in both low-resolution(LR) training set and high-resolution(HR) training set. However, due to the "one-to-many" mapping between the LR image and HR ones in practice, the neighborhood relationship of the LR patch in LR space is quite different with that of the HR counterpart, that is to say the neighborhood relationship obtained is not true. In this paper, we explore a novel and effective re-identified K-nearest neighbor(RIKNN) method to search neighbors of LR patch. Compared with other methods, our method uses the geometrical information of LR manifold and HR manifold simultaneously. In particular, it searches K-NN of LR patch in the LR space and refines the searching results by re-identifying in the HR space, thus giving rise to accurate K-NN and improved performance. A statistical analysis of the influence of the training set size and nearest neighbor number is given, experimental results on some public face databases show the superiority of our proposed scheme over state-of-the-art face hallucination approaches in terms of subjective and objective results as well as computational complexity.
基金Project supported by the National Natural Science Foundation of China(Grant No.61673287)。
文摘We use the label-free microsphere-assisted microscopy to image low-contrast hexagonally close-packed polystyrene nanoparticle arrays with diameters of 300 and 250 nm.When a nanoparticle array is directly placed on a glass slide,it cannot be distinguished.If a 30-nm-thick Ag film is deposited on the surface of a nanoparticle array,the nanoparticle array with nanoparticle diameters of 300 and 250 nm can be distinguished.In addition,the Talbot effect of the 300-nm-diameter nanoparticle array is also observed.If a nanoparticle sample is assembled on a glass slide deposited with a 30-nm-thick Ag film,an array of 300-nm-diameter nanoparticles can be discerned.We propose that in microsphere-assisted microscopy imaging,the resolution can be improved by the excitation of surface plasmon polaritons(SPPs) on the sample surface or at the sample/substrate interface,and a higher near-field intensity due to the excited SPPs would benefit the resolution improvement.Our study of label-free super-resolution imaging of low-contrast objects will promote the applications of microsphere-assisted microscopy in life sciences.
基金This study was supported by the National Cancer Institute(NCI)of the National Institutes of Health(NIH)under Award(No.R00CA214523).
文摘Microvascular structure and hemodynamics are important indicators for the diagnosis and assessment of many diseases and pathologies.The structural and functional imaging of tissue microvasculature in vivo is a clinically significant objective for the development of many imaging modalities.Contrast-enhanced ultrasound(CEUS)is a popular clinical tool for characterizing tissue microvasculature,due to the moderate cost,wide accessibility,and absence of ionizing radiation of ultrasound.
