Augmented Reality(AR)tries to seamlessly integrate virtual content into the real world of the user.Ideally,the virtual content would behave exactly like real objects.This necessitates a correct and precise estimation ...Augmented Reality(AR)tries to seamlessly integrate virtual content into the real world of the user.Ideally,the virtual content would behave exactly like real objects.This necessitates a correct and precise estimation of the user’s viewpoint(or that of a camera)with regard to the virtual content’s coordinate sys-tem.Therefore,the real-time establishment of 3-dimension(3D)maps in real scenes is particularly important for augmented reality technology.So in this paper,we integrate Simultaneous Localization and Mapping(SLAM)technology into augmented reality.Our research is to implement an augmented reality system without markers using the ORB-SLAM2 framework algorithm.In this paper we propose an improved method for Oriented FAST and Rotated BRIEF(ORB)feature extraction and optimized key frame selection,as well as the use of the Progressive Sample Consensus(PROSAC)algorithm for planar estimation of augmented reality implementations,thus solving the problem of increased sys-tem runtime because of the loss of large amounts of texture information in images.In this paper,we get better results by comparing experiments and data analysis.However,there are some improved methods of PROSAC algorithm which are more suitable for the detection of plane feature points.展开更多
The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Ki...The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Kinect v1. Our approach introduces three new aspects for three-dimensional surface tracking in radiotherapy treatment. The first aspect is a new algorithm for noise reduction of depth values. Ueda’s algorithm was implemented and enabling a fast least square regression of depth values. The second aspect is an application for detection of patient’s motion at multiple points in thracoabdominal regions. The third aspect is an estimation of three-dimensional surface from multiple depth values. For evaluation of noise reduction by Ueda’s algorithm, two respiratory patterns are measured by the Kinect as well as a laser range meter. The resulting cross correlation coefficients between the laser range meter and the Kinect were 0.982 for abdominal respiration and 0.995 for breath holding. Moreover, the mean cross correlation coefficients between the signals of our system and the signals of Anzai with respect to participant’s respiratory motion were 0.90 for thoracic respiration and 0.93 for abdominal respiration, respectively. These results proved that the performance of the developed system was comparable to existing motion monitoring devices. Reconstruction of three-dimensional surface also enabled us to detect the irregular motion and breathing arrest by comparing the averaged depth with predefined threshold values.展开更多
An automatic markerless knee tracking and registration algorithm has been proposed in the literature to avoid the marker insertion required by conventional computer-assisted knee surgery,resulting in a shorter and les...An automatic markerless knee tracking and registration algorithm has been proposed in the literature to avoid the marker insertion required by conventional computer-assisted knee surgery,resulting in a shorter and less invasive surgical workflow.However,such an algorithm considers intact femur geometry only.The bone surface modification is inevitable due to intra-operative intervention.The mismatched correspondences will degrade the reliability of registered target pose.To solve this problem,this work proposed a supervised deep neural network to automatically restore the surface of processed bone.The network was trained on a synthetic dataset that consists of real depth captures of a model leg and simulated realistic femur cutting.According to the evaluation on both synthetic data and real-time captures,the registration quality can be effectively improved by surface reconstruction.The improvement in tracking accuracy is only evident over test data,indicating the need for future enhancement of the dataset and network.展开更多
OpenPose(OP)and DeepLabCut(DLC)are applications that use deep learning to estimate posture,but there are few reports on the reliability,validity,and accuracy of their 2D lower limb joint motion analysis.This study com...OpenPose(OP)and DeepLabCut(DLC)are applications that use deep learning to estimate posture,but there are few reports on the reliability,validity,and accuracy of their 2D lower limb joint motion analysis.This study compared OP and DLC estimates of lower extremity joint angles in standing movements with those of conventional software.A total of nine healthy men participated.The trial task was to stand up from a chair.The motion was recorded by a digital camera,and the joint angles of the hip and knee joints were calculated from the video using OP,DLC,and Kinovea.To confirm reliability and validity,ICC was calculated using the Kinovea value as the validity criterion and the correlation coefficient between OP and DLC.In addition,the agreement between those data was evaluated by the Bland-Altman plot.To evaluate the accuracy of the data,root means square error(RMSE)was calculated and compared for each joint.Although the correlation coefficients and ICC(2,1)were in almost perfect agreement,fixed and proportional errors were found for most joints.The RMSE was smaller for OP than for DLC.Compared to Kinovea,OP and DLC can estimate the joint angles of the hip and knee joints during the stand-up movement with an estimation error of fewer than 10,but since they are affected by the resolution of the analysis video and other factors,they need to be validated in a variety of environments and with a variety of movements.展开更多
Respiratory motion induces the limit in delivery accuracy due to the lack of the consideration of the anatomy motion in the treatment planning. Therefore, image-guided radiation therapy (IGRT) system plays an essentia...Respiratory motion induces the limit in delivery accuracy due to the lack of the consideration of the anatomy motion in the treatment planning. Therefore, image-guided radiation therapy (IGRT) system plays an essential role in respiratory motion management and real-time tumor tracking in external beam radiation therapy. The objective of this research is the prediction of dynamic time-series images considering the motion and the deformation of the tumor and to compensate the delay that occurs between the motion of the tumor and the beam delivery. For this, we propose a prediction algorithm for dynamic time-series images. Prediction is performed using principal component analysis (PCA) and multi-channel singular spectral analysis (MSSA). Using PCA, the motion can be denoted as a vector function and it can be estimated by its principal component which is the linear combination of eigen vectors corresponding to the largest eigen values. Time-series set of 320-detector-row CT images from lung cancer patient and kilovolt (kV) fluoroscopic images from a moving phantom were used for the evaluation of the algorithm, and both image sets were successfully predicted by the proposed algorithm. The accuracy of prediction was quite high, more than 0.999 for CT images, whereas 0.995 for kV fluoroscopic images in cross-correlation coefficient value. This algorithm for image prediction makes it possible to predict the tumor images over the next breathing period with significant accuracy.展开更多
基金supported by the Hainan Provincial Natural Science Foundation of China(project number:621QN269)the Sanya Science and Information Bureau Foundation(project number:2021GXYL251).
文摘Augmented Reality(AR)tries to seamlessly integrate virtual content into the real world of the user.Ideally,the virtual content would behave exactly like real objects.This necessitates a correct and precise estimation of the user’s viewpoint(or that of a camera)with regard to the virtual content’s coordinate sys-tem.Therefore,the real-time establishment of 3-dimension(3D)maps in real scenes is particularly important for augmented reality technology.So in this paper,we integrate Simultaneous Localization and Mapping(SLAM)technology into augmented reality.Our research is to implement an augmented reality system without markers using the ORB-SLAM2 framework algorithm.In this paper we propose an improved method for Oriented FAST and Rotated BRIEF(ORB)feature extraction and optimized key frame selection,as well as the use of the Progressive Sample Consensus(PROSAC)algorithm for planar estimation of augmented reality implementations,thus solving the problem of increased sys-tem runtime because of the loss of large amounts of texture information in images.In this paper,we get better results by comparing experiments and data analysis.However,there are some improved methods of PROSAC algorithm which are more suitable for the detection of plane feature points.
文摘The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Kinect v1. Our approach introduces three new aspects for three-dimensional surface tracking in radiotherapy treatment. The first aspect is a new algorithm for noise reduction of depth values. Ueda’s algorithm was implemented and enabling a fast least square regression of depth values. The second aspect is an application for detection of patient’s motion at multiple points in thracoabdominal regions. The third aspect is an estimation of three-dimensional surface from multiple depth values. For evaluation of noise reduction by Ueda’s algorithm, two respiratory patterns are measured by the Kinect as well as a laser range meter. The resulting cross correlation coefficients between the laser range meter and the Kinect were 0.982 for abdominal respiration and 0.995 for breath holding. Moreover, the mean cross correlation coefficients between the signals of our system and the signals of Anzai with respect to participant’s respiratory motion were 0.90 for thoracic respiration and 0.93 for abdominal respiration, respectively. These results proved that the performance of the developed system was comparable to existing motion monitoring devices. Reconstruction of three-dimensional surface also enabled us to detect the irregular motion and breathing arrest by comparing the averaged depth with predefined threshold values.
文摘An automatic markerless knee tracking and registration algorithm has been proposed in the literature to avoid the marker insertion required by conventional computer-assisted knee surgery,resulting in a shorter and less invasive surgical workflow.However,such an algorithm considers intact femur geometry only.The bone surface modification is inevitable due to intra-operative intervention.The mismatched correspondences will degrade the reliability of registered target pose.To solve this problem,this work proposed a supervised deep neural network to automatically restore the surface of processed bone.The network was trained on a synthetic dataset that consists of real depth captures of a model leg and simulated realistic femur cutting.According to the evaluation on both synthetic data and real-time captures,the registration quality can be effectively improved by surface reconstruction.The improvement in tracking accuracy is only evident over test data,indicating the need for future enhancement of the dataset and network.
文摘OpenPose(OP)and DeepLabCut(DLC)are applications that use deep learning to estimate posture,but there are few reports on the reliability,validity,and accuracy of their 2D lower limb joint motion analysis.This study compared OP and DLC estimates of lower extremity joint angles in standing movements with those of conventional software.A total of nine healthy men participated.The trial task was to stand up from a chair.The motion was recorded by a digital camera,and the joint angles of the hip and knee joints were calculated from the video using OP,DLC,and Kinovea.To confirm reliability and validity,ICC was calculated using the Kinovea value as the validity criterion and the correlation coefficient between OP and DLC.In addition,the agreement between those data was evaluated by the Bland-Altman plot.To evaluate the accuracy of the data,root means square error(RMSE)was calculated and compared for each joint.Although the correlation coefficients and ICC(2,1)were in almost perfect agreement,fixed and proportional errors were found for most joints.The RMSE was smaller for OP than for DLC.Compared to Kinovea,OP and DLC can estimate the joint angles of the hip and knee joints during the stand-up movement with an estimation error of fewer than 10,but since they are affected by the resolution of the analysis video and other factors,they need to be validated in a variety of environments and with a variety of movements.
文摘Respiratory motion induces the limit in delivery accuracy due to the lack of the consideration of the anatomy motion in the treatment planning. Therefore, image-guided radiation therapy (IGRT) system plays an essential role in respiratory motion management and real-time tumor tracking in external beam radiation therapy. The objective of this research is the prediction of dynamic time-series images considering the motion and the deformation of the tumor and to compensate the delay that occurs between the motion of the tumor and the beam delivery. For this, we propose a prediction algorithm for dynamic time-series images. Prediction is performed using principal component analysis (PCA) and multi-channel singular spectral analysis (MSSA). Using PCA, the motion can be denoted as a vector function and it can be estimated by its principal component which is the linear combination of eigen vectors corresponding to the largest eigen values. Time-series set of 320-detector-row CT images from lung cancer patient and kilovolt (kV) fluoroscopic images from a moving phantom were used for the evaluation of the algorithm, and both image sets were successfully predicted by the proposed algorithm. The accuracy of prediction was quite high, more than 0.999 for CT images, whereas 0.995 for kV fluoroscopic images in cross-correlation coefficient value. This algorithm for image prediction makes it possible to predict the tumor images over the next breathing period with significant accuracy.
