We prove that non-recursive base conversion can always be implemented by using a deterministic Markov process. Our paper discusses the pros and cons of recursive and non-recursive methods, in general. And we include a...We prove that non-recursive base conversion can always be implemented by using a deterministic Markov process. Our paper discusses the pros and cons of recursive and non-recursive methods, in general. And we include a comparison between non-recursion and a deterministic Markov process, proving that the Markov process is twice as efficient.展开更多
The exponential stability is investigated for a class of continuous time linear systems with a finite state Markov chain form process and the impulsive jump at switching moments. The conditions, based on the average d...The exponential stability is investigated for a class of continuous time linear systems with a finite state Markov chain form process and the impulsive jump at switching moments. The conditions, based on the average dwell time and the ratio of expectation of the total time running on all unstable subsystems to the expectation of the total time running on all stable subsystems,assure the exponential stability with a desired stability degree of the system irrespective of the impact of impulsive jump. The uniformly bounded result is realized for the case in which switched system is subjected to the impulsive effect of the excitation signal at some switching moments.展开更多
Let X=(Omega,F,F-t,X(t),theta(t),P-x) be a jump Markov process with q-pair q(x)-q(x, A). In this paper, the equilibrium principle is established and equilibrium functions, energy, capacity and related problems is inve...Let X=(Omega,F,F-t,X(t),theta(t),P-x) be a jump Markov process with q-pair q(x)-q(x, A). In this paper, the equilibrium principle is established and equilibrium functions, energy, capacity and related problems is investigated in terms of the q-pair q(x)-q(x, A).展开更多
By using Lamperti＇s bijection between self-similar Markov processes and L@vy processes~ we prove finiteness of moments and asymptotic behavior of passage times for increasing self-similar Markov processes valued in ...By using Lamperti＇s bijection between self-similar Markov processes and L@vy processes~ we prove finiteness of moments and asymptotic behavior of passage times for increasing self-similar Markov processes valued in （0, ~）. We Mso investigate the behavior of the process when it crosses a level. A limit theorem concerning the distribution of the process immediately before it crosses some level is proved. Some useful examples are given.展开更多
With the rapid development of urban rail transit,the existing track detection has some problems such as low efficiency and insufficient detection coverage,so an intelligent and automatic track detectionmethod based on...With the rapid development of urban rail transit,the existing track detection has some problems such as low efficiency and insufficient detection coverage,so an intelligent and automatic track detectionmethod based onUAV is urgently needed to avoid major safety accidents.At the same time,the geographical distribution of IoT devices results in the inefficient use of the significant computing potential held by a large number of devices.As a result,the Dispersed Computing(DCOMP)architecture enables collaborative computing between devices in the Internet of Everything(IoE),promotes low-latency and efficient cross-wide applications,and meets users’growing needs for computing performance and service quality.This paper focuses on examining the resource allocation challenge within a dispersed computing environment that utilizes UAV inspection tracks.Furthermore,the system takes into account both resource constraints and computational constraints and transforms the optimization problem into an energy minimization problem with computational constraints.The Markov Decision Process(MDP)model is employed to capture the connection between the dispersed computing resource allocation strategy and the system environment.Subsequently,a method based on Double Deep Q-Network(DDQN)is introduced to derive the optimal policy.Simultaneously,an experience replay mechanism is implemented to tackle the issue of increasing dimensionality.The experimental simulations validate the efficacy of the method across various scenarios.展开更多
Water distribution and gas supply systems are among the infrastructure systems that have many buried steel pipelines. Corrosion gradually appears inside and outside of the pipe walls over the service life of these pip...Water distribution and gas supply systems are among the infrastructure systems that have many buried steel pipelines. Corrosion gradually appears inside and outside of the pipe walls over the service life of these pipelines, the corrosion is primarily caused by the surrounding soil and the materials that flow through the pipelines. However, due to the uncertainty of the characteristics of the soil and materials, the size of the corrosion region is a stochastic variable. In this paper, using a homogeneous Markov process, a model is presented to simulate the occurrence of corrosion. Then, in combinations with a linear corrosion development model, the probability density function of the pipeline area corrosion percentage is derived. Based on the corrosion model, the pipeline seismic displacements and stresses are predicted. Furthermore, using the random perturbation approach, the mean and variance of the pipeline seismic response are given. To illustrate the validity of the proposed approach, a 200-meter long pipeline is numerically investigated and its random seismic response is obtained.展开更多
Effective control of time-sensitive industrial applications depends on the real-time transmission of data from underlying sensors.Quantifying the data freshness through age of information(AoI),in this paper,we jointly...Effective control of time-sensitive industrial applications depends on the real-time transmission of data from underlying sensors.Quantifying the data freshness through age of information(AoI),in this paper,we jointly design sampling and non-slot based scheduling policies to minimize the maximum time-average age of information(MAoI)among sensors with the constraints of average energy cost and finite queue stability.To overcome the intractability involving high couplings of such a complex stochastic process,we first focus on the single-sensor time-average AoI optimization problem and convert the constrained Markov decision process(CMDP)into an unconstrained Markov decision process(MDP)by the Lagrangian method.With the infinite-time average energy and AoI expression expended as the Bellman equation,the singlesensor time-average AoI optimization problem can be approached through the steady-state distribution probability.Further,we propose a low-complexity sub-optimal sampling and semi-distributed scheduling scheme for the multi-sensor scenario.The simulation results show that the proposed scheme reduces the MAoI significantly while achieving a balance between the sampling rate and service rate for multiple sensors.展开更多
In this work, for a control consumption-investment process with the discounted reward optimization criteria, a numerical estimate of the stability index is made. Using explicit formulas for the optimal stationary poli...In this work, for a control consumption-investment process with the discounted reward optimization criteria, a numerical estimate of the stability index is made. Using explicit formulas for the optimal stationary policies and for the value functions, the stability index is explicitly calculated and through statistical techniques its asymptotic behavior is investigated (using numerical experiments) when the discount coefficient approaches 1. The results obtained define the conditions under which an approximate optimal stationary policy can be used to control the original process.展开更多
One of the important problems of stochastic process theory is to define the Laplace transforms for the distribution of semi-markov random processes. With this purpose, we will investigate the semimarkov random process...One of the important problems of stochastic process theory is to define the Laplace transforms for the distribution of semi-markov random processes. With this purpose, we will investigate the semimarkov random processes with positive tendency and negative jump in this article. The first passage of the zero level of the process will be included as a random variable. The Laplace transforms for the distribution of this random variable is defined. The parameters of the distribution will be calculated on the basis of the final results.展开更多
We investigate the approximating capability of Markov modulated Poisson processes (MMPP) for modeling multifractal Internet traffic. The choice of MMPP is motivated by its ability to capture the variability and correl...We investigate the approximating capability of Markov modulated Poisson processes (MMPP) for modeling multifractal Internet traffic. The choice of MMPP is motivated by its ability to capture the variability and correlation in moderate time scales while being analytically tractable. Important statistics of traffic burstiness are described and a customized moment-based fitting procedure of MMPP to traffic traces is presented. Our methodology of doing this is to examine whether the MMPP can be used to predict the performance of a queue to which MMPP sample paths and measured traffic traces are fed for comparison respectively, in addition to the goodness-of-fit test of MMPP. Numerical results and simulations show that the fitted MMPP can approximate multifractal traffic quite well, i.e. accurately predict the queueing performance.展开更多
To ensure the correctness of train control system design and development,the ways of simulation,test and formalization were compared.According to the safe critical attribute of train control system,the characters rela...To ensure the correctness of train control system design and development,the ways of simulation,test and formalization were compared.According to the safe critical attribute of train control system,the characters related to system safety were propounded such as real time,hybrid,distribution(concurrence)and reactivity,and the specific formal methods associated with every character were introduced in details.The analysis and classification of the methods were done based on their mathematical basis and applications,and their advantages and disadvantages were given.Analysis result indicates that every method has determinate limitations,which is determined by the essences of model checking and theory proving.It is pointed out that the presentation of new method,the expansion of existing methods and the integration of many methods will be the development trend of formalization in train control system.展开更多
The Markov property of Markov process functionals which are frequently used in economy, finance, engineering and statistic analysis is studied. The conditions to judge Markov property of some important Markov process ...The Markov property of Markov process functionals which are frequently used in economy, finance, engineering and statistic analysis is studied. The conditions to judge Markov property of some important Markov process functionals are presented, the following conclusions are obtained： the multidimensional process with independent increments is a multidimensional Markov process; the functional in the form of path integral of process with independent increments is a Markov process; the surplus process with the doubly stochastic Poisson process is a vector Markov process. The conditions for linear transformation of vector Markov process being still a Markov process are given.展开更多
A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load cur...A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.展开更多
This paper presents a small perturbation Cramer method for obtaining the large deviation principle of a family of measures (β,ε> 0) on a topological vector space. As an application, we obtain the moderate deviati...This paper presents a small perturbation Cramer method for obtaining the large deviation principle of a family of measures (β,ε> 0) on a topological vector space. As an application, we obtain the moderate deviation estimations for uniformly ergodic Markov processes.展开更多
In order to solve the problem the existing vertical handoff algorithms of vehicle heterogeneous wireless network do not consider the diversification of network's status, an optimized vertical handoff algorithm bas...In order to solve the problem the existing vertical handoff algorithms of vehicle heterogeneous wireless network do not consider the diversification of network's status, an optimized vertical handoff algorithm based on markov process is proposed and discussed in this paper. This algorithm takes into account that the status transformation of available network will affect the quality of service(Qo S) of vehicle terminal's communication service. Firstly, Markov process is used to predict the transformation of wireless network's status after the decision via transition probability. Then the weights of evaluating parameters will be determined by fuzzy logic method. Finally, by comparing the total incomes of each wireless network, including handoff decision incomes, handoff execution incomes and communication service incomes after handoff, the optimal network to handoff will be selected. Simulation results show that: the algorithm proposed, compared to the existing algorithm, is able to receive a higher level of load balancing and effectively improves the average blocking rate, packet loss rate and ping-pang effect.展开更多
This paper studies the robust stochastic stabilization and robust H∞ control for linear time-delay systems with both Markovian jump parameters and unknown norm-bounded parameter uncertainties. This problem can be sol...This paper studies the robust stochastic stabilization and robust H∞ control for linear time-delay systems with both Markovian jump parameters and unknown norm-bounded parameter uncertainties. This problem can be solved on the basis of stochastic Lyapunov approach and linear matrix inequality (LMI) technique. Sufficient conditions for the existence of stochastic stabilization and robust H∞ state feedback controller are presented in terms of a set of solutions of coupled LMIs. Finally, a numerical example is included to demonstrate the practicability of the proposed methods.展开更多
文摘We prove that non-recursive base conversion can always be implemented by using a deterministic Markov process. Our paper discusses the pros and cons of recursive and non-recursive methods, in general. And we include a comparison between non-recursion and a deterministic Markov process, proving that the Markov process is twice as efficient.
基金the National Natural Science Foundation of China (60674027, 60574007)Doctoral Foundation of Education Ministry of China (20050446001).
文摘The exponential stability is investigated for a class of continuous time linear systems with a finite state Markov chain form process and the impulsive jump at switching moments. The conditions, based on the average dwell time and the ratio of expectation of the total time running on all unstable subsystems to the expectation of the total time running on all stable subsystems,assure the exponential stability with a desired stability degree of the system irrespective of the impact of impulsive jump. The uniformly bounded result is realized for the case in which switched system is subjected to the impulsive effect of the excitation signal at some switching moments.
文摘Let X=(Omega,F,F-t,X(t),theta(t),P-x) be a jump Markov process with q-pair q(x)-q(x, A). In this paper, the equilibrium principle is established and equilibrium functions, energy, capacity and related problems is investigated in terms of the q-pair q(x)-q(x, A).
基金supported in part by the National Natural Science Foundation of China(1117126211171263)
文摘By using Lamperti＇s bijection between self-similar Markov processes and L@vy processes~ we prove finiteness of moments and asymptotic behavior of passage times for increasing self-similar Markov processes valued in （0, ~）. We Mso investigate the behavior of the process when it crosses a level. A limit theorem concerning the distribution of the process immediately before it crosses some level is proved. Some useful examples are given.
文摘With the rapid development of urban rail transit,the existing track detection has some problems such as low efficiency and insufficient detection coverage,so an intelligent and automatic track detectionmethod based onUAV is urgently needed to avoid major safety accidents.At the same time,the geographical distribution of IoT devices results in the inefficient use of the significant computing potential held by a large number of devices.As a result,the Dispersed Computing(DCOMP)architecture enables collaborative computing between devices in the Internet of Everything(IoE),promotes low-latency and efficient cross-wide applications,and meets users’growing needs for computing performance and service quality.This paper focuses on examining the resource allocation challenge within a dispersed computing environment that utilizes UAV inspection tracks.Furthermore,the system takes into account both resource constraints and computational constraints and transforms the optimization problem into an energy minimization problem with computational constraints.The Markov Decision Process(MDP)model is employed to capture the connection between the dispersed computing resource allocation strategy and the system environment.Subsequently,a method based on Double Deep Q-Network(DDQN)is introduced to derive the optimal policy.Simultaneously,an experience replay mechanism is implemented to tackle the issue of increasing dimensionality.The experimental simulations validate the efficacy of the method across various scenarios.
基金Natural Science Funds for the Innovative Research Group of China Under Grant No. 50621062
文摘Water distribution and gas supply systems are among the infrastructure systems that have many buried steel pipelines. Corrosion gradually appears inside and outside of the pipe walls over the service life of these pipelines, the corrosion is primarily caused by the surrounding soil and the materials that flow through the pipelines. However, due to the uncertainty of the characteristics of the soil and materials, the size of the corrosion region is a stochastic variable. In this paper, using a homogeneous Markov process, a model is presented to simulate the occurrence of corrosion. Then, in combinations with a linear corrosion development model, the probability density function of the pipeline area corrosion percentage is derived. Based on the corrosion model, the pipeline seismic displacements and stresses are predicted. Furthermore, using the random perturbation approach, the mean and variance of the pipeline seismic response are given. To illustrate the validity of the proposed approach, a 200-meter long pipeline is numerically investigated and its random seismic response is obtained.
基金supported in part by the National Key R&D Program of China(No.2021YFB3300100)the National Natural Science Foundation of China(No.62171062)。
文摘Effective control of time-sensitive industrial applications depends on the real-time transmission of data from underlying sensors.Quantifying the data freshness through age of information(AoI),in this paper,we jointly design sampling and non-slot based scheduling policies to minimize the maximum time-average age of information(MAoI)among sensors with the constraints of average energy cost and finite queue stability.To overcome the intractability involving high couplings of such a complex stochastic process,we first focus on the single-sensor time-average AoI optimization problem and convert the constrained Markov decision process(CMDP)into an unconstrained Markov decision process(MDP)by the Lagrangian method.With the infinite-time average energy and AoI expression expended as the Bellman equation,the singlesensor time-average AoI optimization problem can be approached through the steady-state distribution probability.Further,we propose a low-complexity sub-optimal sampling and semi-distributed scheduling scheme for the multi-sensor scenario.The simulation results show that the proposed scheme reduces the MAoI significantly while achieving a balance between the sampling rate and service rate for multiple sensors.
文摘In this work, for a control consumption-investment process with the discounted reward optimization criteria, a numerical estimate of the stability index is made. Using explicit formulas for the optimal stationary policies and for the value functions, the stability index is explicitly calculated and through statistical techniques its asymptotic behavior is investigated (using numerical experiments) when the discount coefficient approaches 1. The results obtained define the conditions under which an approximate optimal stationary policy can be used to control the original process.
文摘One of the important problems of stochastic process theory is to define the Laplace transforms for the distribution of semi-markov random processes. With this purpose, we will investigate the semimarkov random processes with positive tendency and negative jump in this article. The first passage of the zero level of the process will be included as a random variable. The Laplace transforms for the distribution of this random variable is defined. The parameters of the distribution will be calculated on the basis of the final results.
文摘We investigate the approximating capability of Markov modulated Poisson processes (MMPP) for modeling multifractal Internet traffic. The choice of MMPP is motivated by its ability to capture the variability and correlation in moderate time scales while being analytically tractable. Important statistics of traffic burstiness are described and a customized moment-based fitting procedure of MMPP to traffic traces is presented. Our methodology of doing this is to examine whether the MMPP can be used to predict the performance of a queue to which MMPP sample paths and measured traffic traces are fed for comparison respectively, in addition to the goodness-of-fit test of MMPP. Numerical results and simulations show that the fitted MMPP can approximate multifractal traffic quite well, i.e. accurately predict the queueing performance.
基金This work is supported by Research projects of Beijing National Railway Research and Design Institute of Signal and Communication Co Ltd(No.2300-K1220007)。
文摘To ensure the correctness of train control system design and development,the ways of simulation,test and formalization were compared.According to the safe critical attribute of train control system,the characters related to system safety were propounded such as real time,hybrid,distribution(concurrence)and reactivity,and the specific formal methods associated with every character were introduced in details.The analysis and classification of the methods were done based on their mathematical basis and applications,and their advantages and disadvantages were given.Analysis result indicates that every method has determinate limitations,which is determined by the essences of model checking and theory proving.It is pointed out that the presentation of new method,the expansion of existing methods and the integration of many methods will be the development trend of formalization in train control system.
基金Supported by the National Natural Science Foundation of China (10671197)
文摘The Markov property of Markov process functionals which are frequently used in economy, finance, engineering and statistic analysis is studied. The conditions to judge Markov property of some important Markov process functionals are presented, the following conclusions are obtained： the multidimensional process with independent increments is a multidimensional Markov process; the functional in the form of path integral of process with independent increments is a Markov process; the surplus process with the doubly stochastic Poisson process is a vector Markov process. The conditions for linear transformation of vector Markov process being still a Markov process are given.
文摘A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.
文摘This paper presents a small perturbation Cramer method for obtaining the large deviation principle of a family of measures (β,ε> 0) on a topological vector space. As an application, we obtain the moderate deviation estimations for uniformly ergodic Markov processes.
基金supported in part by the National Natural Science Foundation of China under grant No. 61271259, No. 61301123, No. 61471076Scientific and Technological Research Program of Chongqing Municipal Education Commission of Chongqing of China under Grant No.KJ130536
文摘In order to solve the problem the existing vertical handoff algorithms of vehicle heterogeneous wireless network do not consider the diversification of network's status, an optimized vertical handoff algorithm based on markov process is proposed and discussed in this paper. This algorithm takes into account that the status transformation of available network will affect the quality of service(Qo S) of vehicle terminal's communication service. Firstly, Markov process is used to predict the transformation of wireless network's status after the decision via transition probability. Then the weights of evaluating parameters will be determined by fuzzy logic method. Finally, by comparing the total incomes of each wireless network, including handoff decision incomes, handoff execution incomes and communication service incomes after handoff, the optimal network to handoff will be selected. Simulation results show that: the algorithm proposed, compared to the existing algorithm, is able to receive a higher level of load balancing and effectively improves the average blocking rate, packet loss rate and ping-pang effect.
文摘This paper studies the robust stochastic stabilization and robust H∞ control for linear time-delay systems with both Markovian jump parameters and unknown norm-bounded parameter uncertainties. This problem can be solved on the basis of stochastic Lyapunov approach and linear matrix inequality (LMI) technique. Sufficient conditions for the existence of stochastic stabilization and robust H∞ state feedback controller are presented in terms of a set of solutions of coupled LMIs. Finally, a numerical example is included to demonstrate the practicability of the proposed methods.
