By automatically learning the priors embedded in images with powerful modelling ca-pabilities,deep learning-based algorithms have recently made considerable progress in reconstructing the high-resolution hyperspectral...By automatically learning the priors embedded in images with powerful modelling ca-pabilities,deep learning-based algorithms have recently made considerable progress in reconstructing the high-resolution hyperspectral(HR-HS)image.With previously collected large-amount of external data,these methods are intuitively realised under the full supervision of the ground-truth data.Thus,the database construction in merging the low-resolution(LR)HS(LR-HS)and HR multispectral(MS)or RGB image research paradigm,commonly named as HSI SR,requires collecting corresponding training triplets:HR-MS(RGB),LR-HS and HR-HS image simultaneously,and often faces dif-ficulties in reality.The learned models with the training datasets collected simultaneously under controlled conditions may significantly degrade the HSI super-resolved perfor-mance to the real images captured under diverse environments.To handle the above-mentioned limitations,the authors propose to leverage the deep internal and self-supervised learning to solve the HSI SR problem.The authors advocate that it is possible to train a specific CNN model at test time,called as deep internal learning(DIL),by on-line preparing the training triplet samples from the observed LR-HS/HR-MS(or RGB)images and the down-sampled LR-HS version.However,the number of the training triplets extracted solely from the transformed data of the observation itself is extremely few particularly for the HSI SR tasks with large spatial upscale factors,which would result in limited reconstruction performance.To solve this problem,the authors further exploit deep self-supervised learning(DSL)by considering the observations as the unlabelled training samples.Specifically,the degradation modules inside the network were elaborated to realise the spatial and spectral down-sampling procedures for transforming the generated HR-HS estimation to the high-resolution RGB/LR-HS approximation,and then the reconstruction errors of the observations were formulated for measuring the network modelling performance.By consolidating the DIL and DSL into a unified deep framework,the authors construct a more robust HSI SR method without any prior training and have great potential of flexible adaptation to different settings per obser-vation.To verify the effectiveness of the proposed approach,extensive experiments have been conducted on two benchmark HS datasets,including the CAVE and Harvard datasets,and demonstrate the great performance gain of the proposed method over the state-of-the-art methods.展开更多
Idiopathic rapid eye movement sleep behavior disorder(iRBD) is often a precursor to neurodegenerative disease. However, voxel-based morphological studies evaluating structural abnormalities in the brains of iRBD patie...Idiopathic rapid eye movement sleep behavior disorder(iRBD) is often a precursor to neurodegenerative disease. However, voxel-based morphological studies evaluating structural abnormalities in the brains of iRBD patients are relatively rare. This study aimed to explore cerebral structural alterations using magnetic resonance imaging and to determine their association with clinical parameters in iRBD patients. Brain structural T1-weighted MRI scans were acquired from 19 polysomnogram-confirmed iRBD patients(male:female 16:3; mean age 66.6 ± 7.0 years) and 20 age-matched healthy controls(male:female 5:15; mean age 63.7 ± 5.9 years). Gray matter volume(GMV) data were analyzed based on Statistical Parametric Mapping 8, using a voxel-based morphometry method and two-sample t-test and multiple regression analysis. Compared with controls, iRBD patients had increased GMV in the middle temporal gyrus and cerebellar posterior lobe, but decreased GMV in the Rolandic operculum, postcentral gyrus, insular lobe, cingulate gyrus, precuneus, rectus gyrus, and superior frontal gyrus. iRBD duration was positively correlated with GMV in the precuneus, cuneus, superior parietal gyrus, postcentral gyrus, posterior cingulate gyrus, hippocampus, lingual gyrus, middle occipital gyrus, middle temporal gyrus, and cerebellum posterior lobe. Furthermore, phasic chin electromyographic activity was positively correlated with GMV in the hippocampus, precuneus, fusiform gyrus, precentral gyrus, superior frontal gyrus, cuneus, inferior parietal lobule, angular gyrus, superior parietal gyrus, paracentral lobule, and cerebellar posterior lobe. There were no significant negative correlations of brain GMV with disease duration or electromyographic activity in iRBD patients. These findings expand the spectrum of known gray matter modifications in iRBD patients and provide evidence of a correlation between brain dysfunction and clinical manifestations in such patients. The protocol was approved by the Ethics Committee of Huashan Hospital(approval No. KY2013-336) on January 6, 2014. This trial was registered in the ISRCTN registry(ISRCTN18238599).展开更多
基金Ministry of Education,Culture,Sports,Science and Technology,Grant/Award Number:20K11867。
文摘By automatically learning the priors embedded in images with powerful modelling ca-pabilities,deep learning-based algorithms have recently made considerable progress in reconstructing the high-resolution hyperspectral(HR-HS)image.With previously collected large-amount of external data,these methods are intuitively realised under the full supervision of the ground-truth data.Thus,the database construction in merging the low-resolution(LR)HS(LR-HS)and HR multispectral(MS)or RGB image research paradigm,commonly named as HSI SR,requires collecting corresponding training triplets:HR-MS(RGB),LR-HS and HR-HS image simultaneously,and often faces dif-ficulties in reality.The learned models with the training datasets collected simultaneously under controlled conditions may significantly degrade the HSI super-resolved perfor-mance to the real images captured under diverse environments.To handle the above-mentioned limitations,the authors propose to leverage the deep internal and self-supervised learning to solve the HSI SR problem.The authors advocate that it is possible to train a specific CNN model at test time,called as deep internal learning(DIL),by on-line preparing the training triplet samples from the observed LR-HS/HR-MS(or RGB)images and the down-sampled LR-HS version.However,the number of the training triplets extracted solely from the transformed data of the observation itself is extremely few particularly for the HSI SR tasks with large spatial upscale factors,which would result in limited reconstruction performance.To solve this problem,the authors further exploit deep self-supervised learning(DSL)by considering the observations as the unlabelled training samples.Specifically,the degradation modules inside the network were elaborated to realise the spatial and spectral down-sampling procedures for transforming the generated HR-HS estimation to the high-resolution RGB/LR-HS approximation,and then the reconstruction errors of the observations were formulated for measuring the network modelling performance.By consolidating the DIL and DSL into a unified deep framework,the authors construct a more robust HSI SR method without any prior training and have great potential of flexible adaptation to different settings per obser-vation.To verify the effectiveness of the proposed approach,extensive experiments have been conducted on two benchmark HS datasets,including the CAVE and Harvard datasets,and demonstrate the great performance gain of the proposed method over the state-of-the-art methods.
基金supported by the China-US Biomedical Collaborative Research Program,No.81361120393(to CTZ)the National Natural Science Foundation of China,No.81401135(to PW),81671239(to CTZ)the Shanghai Sailing Program,No.18YF1403100(to JJG)
文摘Idiopathic rapid eye movement sleep behavior disorder(iRBD) is often a precursor to neurodegenerative disease. However, voxel-based morphological studies evaluating structural abnormalities in the brains of iRBD patients are relatively rare. This study aimed to explore cerebral structural alterations using magnetic resonance imaging and to determine their association with clinical parameters in iRBD patients. Brain structural T1-weighted MRI scans were acquired from 19 polysomnogram-confirmed iRBD patients(male:female 16:3; mean age 66.6 ± 7.0 years) and 20 age-matched healthy controls(male:female 5:15; mean age 63.7 ± 5.9 years). Gray matter volume(GMV) data were analyzed based on Statistical Parametric Mapping 8, using a voxel-based morphometry method and two-sample t-test and multiple regression analysis. Compared with controls, iRBD patients had increased GMV in the middle temporal gyrus and cerebellar posterior lobe, but decreased GMV in the Rolandic operculum, postcentral gyrus, insular lobe, cingulate gyrus, precuneus, rectus gyrus, and superior frontal gyrus. iRBD duration was positively correlated with GMV in the precuneus, cuneus, superior parietal gyrus, postcentral gyrus, posterior cingulate gyrus, hippocampus, lingual gyrus, middle occipital gyrus, middle temporal gyrus, and cerebellum posterior lobe. Furthermore, phasic chin electromyographic activity was positively correlated with GMV in the hippocampus, precuneus, fusiform gyrus, precentral gyrus, superior frontal gyrus, cuneus, inferior parietal lobule, angular gyrus, superior parietal gyrus, paracentral lobule, and cerebellar posterior lobe. There were no significant negative correlations of brain GMV with disease duration or electromyographic activity in iRBD patients. These findings expand the spectrum of known gray matter modifications in iRBD patients and provide evidence of a correlation between brain dysfunction and clinical manifestations in such patients. The protocol was approved by the Ethics Committee of Huashan Hospital(approval No. KY2013-336) on January 6, 2014. This trial was registered in the ISRCTN registry(ISRCTN18238599).
