Background Gesture is a basic interaction channel that is frequently used by humans to communicate in daily life. In this paper, we explore to use gesture-based approaches for target acquisition in virtual and augment...Background Gesture is a basic interaction channel that is frequently used by humans to communicate in daily life. In this paper, we explore to use gesture-based approaches for target acquisition in virtual and augmented reality. A typical process of gesture-based target acquisition is: when a user intends to acquire a target, she performs a gesture with her hands, head or other parts of the body, the computer senses and recognizes the gesture and infers the most possible target. Methods We build mental model and behavior model of the user to study two key parts of the interaction process. Mental model describes how user thinks up a gesture for acquiring a target, and can be the intuitive mapping between gestures and targets. Behavior model describes how user moves the body parts to perform the gestures, and the relationship between the gesture that user intends to perform and signals that computer senses. Results In this paper, we present and discuss three pieces of research that focus on the mental model and behavior model of gesture-based target acquisition in VR and AR. Conclusions We show that leveraging these two models, interaction experience and performance can be improved in VR and AR environments.展开更多
Background The combination of an augmented reality(AR)headset and a smartphone can simultaneously provide a wider display and a precise touch input;it can redefine the way we use applications today.However,users are d...Background The combination of an augmented reality(AR)headset and a smartphone can simultaneously provide a wider display and a precise touch input;it can redefine the way we use applications today.However,users are deprived of such benefits owing to the independence of the two devices.There is a lack of intuitive and direct interactions between them.Methods In this study,we conduct a formative investigation to understand the window management requirements and interaction preferences of using an AR headset and a smartphone simultaneously and report the insights we gained.In addition,we introduce an example vocabulary of window management operations in the AR headset and smartphone interface.Results This allows users to manipulate windows in a virtual space and shift windows between devices efficiently and seamlessly.展开更多
With the rise of the concept of the metaverse,mixed reality continues to receive keen attention from all over the world.Academia and industry worldwide are constantly innovating the key technologies of mixed reality s...With the rise of the concept of the metaverse,mixed reality continues to receive keen attention from all over the world.Academia and industry worldwide are constantly innovating the key technologies of mixed reality software and hardware.For a long time,the field of mixed reality mainly focused on display technology,but with the increase of mixed reality applications,people gradually found that the lack of interactive technology and solutions has become a bottleneck restricting the development of mixed reality technology.On the mixed reality platform represented by AR/VR helmets,people cannot completely get rid of the remote control to complete human-computer interaction,which greatly limits the development of applications on it.How to enable users to exchange information with computers based on the natural interaction modality has become an urgent problem to be solved.展开更多
Decision-making plays an essential role in various real-world systems like automatic driving,traffic dispatching,information system management,and emergency command and control.Recent breakthroughs in computer game sc...Decision-making plays an essential role in various real-world systems like automatic driving,traffic dispatching,information system management,and emergency command and control.Recent breakthroughs in computer game scenarios using deep reinforcement learning for intelligent decision-making have paved decision-making intelligence as a burgeoning research direction.In complex practical systems,however,factors like coupled distracting features,long-term interact links,and adversarial environments and opponents,make decision-making in practical applications challenging in modeling,computing,and explaining.This work proposes game interactive learning,a novel paradigm as a new approach towards intelligent decision-making in complex and adversarial environments.This novel paradigm highlights the function and role of a human in the process of intelligent decision-making in complex systems.It formalizes a new learning paradigm for exchanging information and knowledge between humans and the machine system.The proposed paradigm first inherits methods in game theory to model the agents and their preferences in the complex decision-making process.It then optimizes the learning objectives from equilibrium analysis using reformed machine learning algorithms to compute and pursue promising decision results for practice.Human interactions are involved when the learning process needs guidance from additional knowledge and instructions,or the human wants to understand the learning machine better.We perform preliminary experimental verification of the proposed paradigm on two challenging decision-making tasks in tactical-level War-game scenarios.Experimental results demonstrate the effectiveness of the proposed learning paradigm.展开更多
With ever greater amounts of data stored in cloud servers, data security and privacy issues have become increasingly important. Public cloud storage providers are semi-trustworthy because they may not have adequate se...With ever greater amounts of data stored in cloud servers, data security and privacy issues have become increasingly important. Public cloud storage providers are semi-trustworthy because they may not have adequate security mechanisms to protect user data from being stolen or misused. Therefore, it is crucial for cloud users to evaluate the security of cloud storage providers. However, existing security assessment methods mainly focus on external security risks without considering the trustworthiness of cloud providers. In addition, the widely used thirdparty mediators are assumed to be trusted and we are not aware of any work that considers the security of these mediators. This study fills these gaps by assessing the security of public cloud storage providers and third-party mediators through equilibrium analysis. More specifically, we conduct evaluations on a series of game models between public cloud storage providers and users to thoroughly analyze the security of different service scenarios.Using our proposed security assessment, users can determine the risk of whether their privacy data is likely to be hacked by the cloud service providers;the cloud service providers can also decide on strategies to make their services more trustworthy. An experimental study of 32 users verified our method and indicated its potential for real service improvement.展开更多
文摘Background Gesture is a basic interaction channel that is frequently used by humans to communicate in daily life. In this paper, we explore to use gesture-based approaches for target acquisition in virtual and augmented reality. A typical process of gesture-based target acquisition is: when a user intends to acquire a target, she performs a gesture with her hands, head or other parts of the body, the computer senses and recognizes the gesture and infers the most possible target. Methods We build mental model and behavior model of the user to study two key parts of the interaction process. Mental model describes how user thinks up a gesture for acquiring a target, and can be the intuitive mapping between gestures and targets. Behavior model describes how user moves the body parts to perform the gestures, and the relationship between the gesture that user intends to perform and signals that computer senses. Results In this paper, we present and discuss three pieces of research that focus on the mental model and behavior model of gesture-based target acquisition in VR and AR. Conclusions We show that leveraging these two models, interaction experience and performance can be improved in VR and AR environments.
基金Supported by Key Basic Research Projects of the Foundation Strengthening Program (2020-JCJQ-ZD-014-12)
文摘Background The combination of an augmented reality(AR)headset and a smartphone can simultaneously provide a wider display and a precise touch input;it can redefine the way we use applications today.However,users are deprived of such benefits owing to the independence of the two devices.There is a lack of intuitive and direct interactions between them.Methods In this study,we conduct a formative investigation to understand the window management requirements and interaction preferences of using an AR headset and a smartphone simultaneously and report the insights we gained.In addition,we introduce an example vocabulary of window management operations in the AR headset and smartphone interface.Results This allows users to manipulate windows in a virtual space and shift windows between devices efficiently and seamlessly.
文摘With the rise of the concept of the metaverse,mixed reality continues to receive keen attention from all over the world.Academia and industry worldwide are constantly innovating the key technologies of mixed reality software and hardware.For a long time,the field of mixed reality mainly focused on display technology,but with the increase of mixed reality applications,people gradually found that the lack of interactive technology and solutions has become a bottleneck restricting the development of mixed reality technology.On the mixed reality platform represented by AR/VR helmets,people cannot completely get rid of the remote control to complete human-computer interaction,which greatly limits the development of applications on it.How to enable users to exchange information with computers based on the natural interaction modality has become an urgent problem to be solved.
文摘Decision-making plays an essential role in various real-world systems like automatic driving,traffic dispatching,information system management,and emergency command and control.Recent breakthroughs in computer game scenarios using deep reinforcement learning for intelligent decision-making have paved decision-making intelligence as a burgeoning research direction.In complex practical systems,however,factors like coupled distracting features,long-term interact links,and adversarial environments and opponents,make decision-making in practical applications challenging in modeling,computing,and explaining.This work proposes game interactive learning,a novel paradigm as a new approach towards intelligent decision-making in complex and adversarial environments.This novel paradigm highlights the function and role of a human in the process of intelligent decision-making in complex systems.It formalizes a new learning paradigm for exchanging information and knowledge between humans and the machine system.The proposed paradigm first inherits methods in game theory to model the agents and their preferences in the complex decision-making process.It then optimizes the learning objectives from equilibrium analysis using reformed machine learning algorithms to compute and pursue promising decision results for practice.Human interactions are involved when the learning process needs guidance from additional knowledge and instructions,or the human wants to understand the learning machine better.We perform preliminary experimental verification of the proposed paradigm on two challenging decision-making tasks in tactical-level War-game scenarios.Experimental results demonstrate the effectiveness of the proposed learning paradigm.
文摘With ever greater amounts of data stored in cloud servers, data security and privacy issues have become increasingly important. Public cloud storage providers are semi-trustworthy because they may not have adequate security mechanisms to protect user data from being stolen or misused. Therefore, it is crucial for cloud users to evaluate the security of cloud storage providers. However, existing security assessment methods mainly focus on external security risks without considering the trustworthiness of cloud providers. In addition, the widely used thirdparty mediators are assumed to be trusted and we are not aware of any work that considers the security of these mediators. This study fills these gaps by assessing the security of public cloud storage providers and third-party mediators through equilibrium analysis. More specifically, we conduct evaluations on a series of game models between public cloud storage providers and users to thoroughly analyze the security of different service scenarios.Using our proposed security assessment, users can determine the risk of whether their privacy data is likely to be hacked by the cloud service providers;the cloud service providers can also decide on strategies to make their services more trustworthy. An experimental study of 32 users verified our method and indicated its potential for real service improvement.
