A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load cur...A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.展开更多
Let Q be the Q-matrix of an irreducible, positive recurrent Markov process on a countable state space. We show that, under a number of conditions, the stationary distributions of the n × n north-west corner augme...Let Q be the Q-matrix of an irreducible, positive recurrent Markov process on a countable state space. We show that, under a number of conditions, the stationary distributions of the n × n north-west corner augmentations of Q converge in total variation to the stationary distribution of the process. Two conditions guaranteeing such convergence include exponential ergodicity and stochastic monotonicity of the process. The same also holds for processes dominated by a stochastically monotone Markov process. In addition, we shall show that finite perturbations of stochastically monotone processes may be viewed as being dominated by a stochastically monotone process, thus extending the scope of these results to a larger class of processes. Consequently, the augmentation method provides an attractive, intuitive method for approximating the stationary distributions of a large class of Markov processes on countably infinite state spaces from a finite amount of known information.展开更多
The authors investigate the hitting probability, polarity and the relationship between the polarity and Hausdorff dimension for self-similar Markov processes with state space (0, ∞) and increasing path.
The Markov property of Markov process functionals which are frequently used in economy, finance, engineering and statistic analysis is studied. The conditions to judge Markov property of some important Markov process ...The Markov property of Markov process functionals which are frequently used in economy, finance, engineering and statistic analysis is studied. The conditions to judge Markov property of some important Markov process functionals are presented, the following conclusions are obtained: the multidimensional process with independent increments is a multidimensional Markov process; the functional in the form of path integral of process with independent incre- ments is a Markov process; the surplus process with the doubly stochastic Poisson process is a vector Markov process. The conditions for linear transformation of vector Markov process being still a Markov process are given.展开更多
The simulation of wind power time series is a key process in renewable power allocation planning,operation mode calculation,and safety assessment.Traditional single-point modeling methods discretely generate wind powe...The simulation of wind power time series is a key process in renewable power allocation planning,operation mode calculation,and safety assessment.Traditional single-point modeling methods discretely generate wind power at each moment;however,they ignore the daily output characteristics and are unable to consider both modeling accuracy and efficiency.To resolve this problem,a wind power time series simulation model based on typical daily output processes and Markov algorithm is proposed.First,a typical daily output process classification method based on time series similarity and modified K-means clustering algorithm is presented.Second,considering the typical daily output processes as status variables,a wind power time series simulation model based on Markov algorithm is constructed.Finally,a case is analyzed based on the measured data of a wind farm in China.The proposed model is then compared with traditional methods to verify its effectiveness and applicability.The comparison results indicate that the statistical characteristics,probability distributions,and autocorrelation characteristics of the wind power time series generated by the proposed model are better than those of the traditional methods.Moreover,modeling efficiency considerably improves.展开更多
Optimal policies in Markov decision problems may be quite sensitive with regard to transition probabilities.In practice,some transition probabilities may be uncertain.The goals of the present study are to find the rob...Optimal policies in Markov decision problems may be quite sensitive with regard to transition probabilities.In practice,some transition probabilities may be uncertain.The goals of the present study are to find the robust range for a certain optimal policy and to obtain value intervals of exact transition probabilities.Our research yields powerful contributions for Markov decision processes(MDPs)with uncertain transition probabilities.We first propose a method for estimating unknown transition probabilities based on maximum likelihood.Since the estimation may be far from accurate,and the highest expected total reward of the MDP may be sensitive to these transition probabilities,we analyze the robustness of an optimal policy and propose an approach for robust analysis.After giving the definition of a robust optimal policy with uncertain transition probabilities represented as sets of numbers,we formulate a model to obtain the optimal policy.Finally,we define the value intervals of the exact transition probabilities and construct models to determine the lower and upper bounds.Numerical examples are given to show the practicability of our methods.展开更多
The exponential stability is investigated for a class of continuous time linear systems with a finite state Markov chain form process and the impulsive jump at switching moments. The conditions, based on the average d...The exponential stability is investigated for a class of continuous time linear systems with a finite state Markov chain form process and the impulsive jump at switching moments. The conditions, based on the average dwell time and the ratio of expectation of the total time running on all unstable subsystems to the expectation of the total time running on all stable subsystems, assure the exponential stability with a desired stability degree of the system irrespective of the impact of impulsive jump. The uniformly bounded result is realized for the case in which switched system is subjected to the impulsive effect of the excitation signal at some switching moments.展开更多
In order to solve the problem the existing vertical handoff algorithms of vehicle heterogeneous wireless network do not consider the diversification of network's status, an optimized vertical handoff algorithm bas...In order to solve the problem the existing vertical handoff algorithms of vehicle heterogeneous wireless network do not consider the diversification of network's status, an optimized vertical handoff algorithm based on markov process is proposed and discussed in this paper. This algorithm takes into account that the status transformation of available network will affect the quality of service(Qo S) of vehicle terminal's communication service. Firstly, Markov process is used to predict the transformation of wireless network's status after the decision via transition probability. Then the weights of evaluating parameters will be determined by fuzzy logic method. Finally, by comparing the total incomes of each wireless network, including handoff decision incomes, handoff execution incomes and communication service incomes after handoff, the optimal network to handoff will be selected. Simulation results show that: the algorithm proposed, compared to the existing algorithm, is able to receive a higher level of load balancing and effectively improves the average blocking rate, packet loss rate and ping-pang effect.展开更多
In this article,we investigate Programming Evaluation and Review Technique networks with independently and generally distributed activity durations.For any path in this network,we select all the activities related to ...In this article,we investigate Programming Evaluation and Review Technique networks with independently and generally distributed activity durations.For any path in this network,we select all the activities related to this path such that the completion time of the sub-network(only consisting of all the related activities) is equal to the completion time of this path.We use the elapsed time as the supplementary variables and model this sub-network as a Markov skeleton process,the state space is related to the subnetwork structure.Then use the backward equation to compute the distribution of the sub-network's completion time,which is an important rule in project management and scheduling.展开更多
This article concerns a class of Ornstein-Uhlenbeck type Markov processes and for which the level sets will be approached. By constructing a new class f processes,we shall obtain an inequality on the Hausdorff dimensi...This article concerns a class of Ornstein-Uhlenbeck type Markov processes and for which the level sets will be approached. By constructing a new class f processes,we shall obtain an inequality on the Hausdorff dimensions of the level sets for the Ornstein-Uhlenbeck type Markov processes. Based on this result, we finally verify that any two independent O-U.M.P with α-stable processes could collide with probability one.展开更多
Let {Xt,t≥ 0} be an Ornstein-Uhlenbeck type Markov process with Levyprocess At, the authors conaider the fractal properties of its ranges, give the upper andlower bounds of the Hausdorff dimensions of the ranges and ...Let {Xt,t≥ 0} be an Ornstein-Uhlenbeck type Markov process with Levyprocess At, the authors conaider the fractal properties of its ranges, give the upper andlower bounds of the Hausdorff dimensions of the ranges and the estimate of the dimensionsof the level sets for the process. The dstence of lood times and occnption times of Xtare considered in some special展开更多
A method is proposed to calculate the railway transport capacity in an emergency to support the dispatching work of control centers.The effect of an emergency on section transport capacity is analyzed and the basic me...A method is proposed to calculate the railway transport capacity in an emergency to support the dispatching work of control centers.The effect of an emergency on section transport capacity is analyzed and the basic method to calculate the transport capacity is presented.The results show that the situation-changing process in emergency is actually a Markov process.The calculation rule is presented based on division of the time segment during which the emergency lasts.The algorithm is designed to calculate the transport capacity of each time segment.The pessimistic strategy and the fuzzy strategy are proposed to determine the computing value of the transport capacity of each time segment,to satisfy the calculating requirements in different occasions.Our study shows that the method is reasonable and practical and the method can be embedded in the train dispatching system to support the operation work in an emergency.展开更多
This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance mi...This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.展开更多
Drought conditions at a given location evolve randomly through time and are typically characterized by severity and duration. Researchers interested in modeling the economic effects of drought on agriculture or other ...Drought conditions at a given location evolve randomly through time and are typically characterized by severity and duration. Researchers interested in modeling the economic effects of drought on agriculture or other water users often capture the stochastic nature of drought and its conditions via multiyear, stochastic economic models. Three major sources of uncertainty in application of a multiyear discrete stochastic model to evaluate user preparedness and response to drought are: (1) the assumption of independence of yearly weather conditions, (2) linguistic vagueness in the definition of drought itself, and (3) the duration of drought. One means of addressing these uncertainties is to re-cast drought as a stochastic, multiyear process using a “fuzzy” semi-Markov process. In this paper, we review “crisp” versus “fuzzy” representations of drought and show how fuzzy semi-Markov processes can aid researchers in developing more robust multiyear, discrete stochastic models.展开更多
The assembly process of aerospace products such as satellites and rockets has the characteristics of single-or small-batch production,a long development period,high reliability,and frequent disturbances.How to predict...The assembly process of aerospace products such as satellites and rockets has the characteristics of single-or small-batch production,a long development period,high reliability,and frequent disturbances.How to predict and avoid quality abnormalities,quickly locate their causes,and improve product assembly quality and efficiency are urgent engineering issues.As the core technology to realize the integration of virtual and physical space,digital twin(DT)technology can make full use of the low cost,high efficiency,and predictable advantages of digital space to provide a feasible solution to such problems.Hence,a quality management method for the assembly process of aerospace products based on DT is proposed.Given that traditional quality control methods for the assembly process of aerospace products are mostly post-inspection,the Grey-Markov model and T-K control chart are used with a small sample of assembly quality data to predict the value of quality data and the status of an assembly system.The Apriori algorithm is applied to mine the strong association rules related to quality data anomalies and uncontrolled assembly systems so as to solve the issue that the causes of abnormal quality are complicated and difficult to trace.The implementation of the proposed approach is described,taking the collected centroid data of an aerospace product’s cabin,one of the key quality data in the assembly process of aerospace products,as an example.A DT-based quality management system for the assembly process of aerospace products is developed,which can effectively improve the efficiency of quality management for the assembly process of aerospace products and reduce quality abnormalities.展开更多
This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space...This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.展开更多
文摘A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.
文摘Let Q be the Q-matrix of an irreducible, positive recurrent Markov process on a countable state space. We show that, under a number of conditions, the stationary distributions of the n × n north-west corner augmentations of Q converge in total variation to the stationary distribution of the process. Two conditions guaranteeing such convergence include exponential ergodicity and stochastic monotonicity of the process. The same also holds for processes dominated by a stochastically monotone Markov process. In addition, we shall show that finite perturbations of stochastically monotone processes may be viewed as being dominated by a stochastically monotone process, thus extending the scope of these results to a larger class of processes. Consequently, the augmentation method provides an attractive, intuitive method for approximating the stationary distributions of a large class of Markov processes on countably infinite state spaces from a finite amount of known information.
基金the National Natural Science Foundation of China and the StateEducation of Commission Ph.D. Station Foundation
文摘The authors investigate the hitting probability, polarity and the relationship between the polarity and Hausdorff dimension for self-similar Markov processes with state space (0, ∞) and increasing path.
基金Supported by the National Natural Science Foundation of China (10671197)
文摘The Markov property of Markov process functionals which are frequently used in economy, finance, engineering and statistic analysis is studied. The conditions to judge Markov property of some important Markov process functionals are presented, the following conclusions are obtained: the multidimensional process with independent increments is a multidimensional Markov process; the functional in the form of path integral of process with independent incre- ments is a Markov process; the surplus process with the doubly stochastic Poisson process is a vector Markov process. The conditions for linear transformation of vector Markov process being still a Markov process are given.
基金supported by the China Datang Corporation project“Study on the performance improvement scheme of in-service wind farms”,the Fundamental Research Funds for the Central Universities(2020MS021)the Foundation of State Key Laboratory“Real-time prediction of offshore wind power and load reduction control method”(LAPS2020-07).
文摘The simulation of wind power time series is a key process in renewable power allocation planning,operation mode calculation,and safety assessment.Traditional single-point modeling methods discretely generate wind power at each moment;however,they ignore the daily output characteristics and are unable to consider both modeling accuracy and efficiency.To resolve this problem,a wind power time series simulation model based on typical daily output processes and Markov algorithm is proposed.First,a typical daily output process classification method based on time series similarity and modified K-means clustering algorithm is presented.Second,considering the typical daily output processes as status variables,a wind power time series simulation model based on Markov algorithm is constructed.Finally,a case is analyzed based on the measured data of a wind farm in China.The proposed model is then compared with traditional methods to verify its effectiveness and applicability.The comparison results indicate that the statistical characteristics,probability distributions,and autocorrelation characteristics of the wind power time series generated by the proposed model are better than those of the traditional methods.Moreover,modeling efficiency considerably improves.
基金Supported by the National Natural Science Foundation of China(71571019).
文摘Optimal policies in Markov decision problems may be quite sensitive with regard to transition probabilities.In practice,some transition probabilities may be uncertain.The goals of the present study are to find the robust range for a certain optimal policy and to obtain value intervals of exact transition probabilities.Our research yields powerful contributions for Markov decision processes(MDPs)with uncertain transition probabilities.We first propose a method for estimating unknown transition probabilities based on maximum likelihood.Since the estimation may be far from accurate,and the highest expected total reward of the MDP may be sensitive to these transition probabilities,we analyze the robustness of an optimal policy and propose an approach for robust analysis.After giving the definition of a robust optimal policy with uncertain transition probabilities represented as sets of numbers,we formulate a model to obtain the optimal policy.Finally,we define the value intervals of the exact transition probabilities and construct models to determine the lower and upper bounds.Numerical examples are given to show the practicability of our methods.
基金the National Natural Science Foundation of China (60674027, 60574007)Doctoral Foundation of Education Ministry of China (20050446001).
文摘The exponential stability is investigated for a class of continuous time linear systems with a finite state Markov chain form process and the impulsive jump at switching moments. The conditions, based on the average dwell time and the ratio of expectation of the total time running on all unstable subsystems to the expectation of the total time running on all stable subsystems, assure the exponential stability with a desired stability degree of the system irrespective of the impact of impulsive jump. The uniformly bounded result is realized for the case in which switched system is subjected to the impulsive effect of the excitation signal at some switching moments.
基金supported in part by the National Natural Science Foundation of China under grant No. 61271259, No. 61301123, No. 61471076Scientific and Technological Research Program of Chongqing Municipal Education Commission of Chongqing of China under Grant No.KJ130536
文摘In order to solve the problem the existing vertical handoff algorithms of vehicle heterogeneous wireless network do not consider the diversification of network's status, an optimized vertical handoff algorithm based on markov process is proposed and discussed in this paper. This algorithm takes into account that the status transformation of available network will affect the quality of service(Qo S) of vehicle terminal's communication service. Firstly, Markov process is used to predict the transformation of wireless network's status after the decision via transition probability. Then the weights of evaluating parameters will be determined by fuzzy logic method. Finally, by comparing the total incomes of each wireless network, including handoff decision incomes, handoff execution incomes and communication service incomes after handoff, the optimal network to handoff will be selected. Simulation results show that: the algorithm proposed, compared to the existing algorithm, is able to receive a higher level of load balancing and effectively improves the average blocking rate, packet loss rate and ping-pang effect.
基金supported by the National Natural Science Foundation of China(10671212,10901164,90820302)the Graduate Research Innovation Projects in Hunan Province(CX2009B020)the Graduate Degree Thesis Innovation Foundation of Central Sourth University(2009ybfz11)
文摘In this article,we investigate Programming Evaluation and Review Technique networks with independently and generally distributed activity durations.For any path in this network,we select all the activities related to this path such that the completion time of the sub-network(only consisting of all the related activities) is equal to the completion time of this path.We use the elapsed time as the supplementary variables and model this sub-network as a Markov skeleton process,the state space is related to the subnetwork structure.Then use the backward equation to compute the distribution of the sub-network's completion time,which is an important rule in project management and scheduling.
文摘This article concerns a class of Ornstein-Uhlenbeck type Markov processes and for which the level sets will be approached. By constructing a new class f processes,we shall obtain an inequality on the Hausdorff dimensions of the level sets for the Ornstein-Uhlenbeck type Markov processes. Based on this result, we finally verify that any two independent O-U.M.P with α-stable processes could collide with probability one.
文摘Let {Xt,t≥ 0} be an Ornstein-Uhlenbeck type Markov process with Levyprocess At, the authors conaider the fractal properties of its ranges, give the upper andlower bounds of the Hausdorff dimensions of the ranges and the estimate of the dimensionsof the level sets for the process. The dstence of lood times and occnption times of Xtare considered in some special
基金Supported by the State Key Laboratory of Rail Traffic Control and Safety (RCS2011K004)Beijing Jiaotong University,Fundamental Research Funds of Gansu Province (620030)+1 种基金the National Natural Science Foundation of China(61074151)Project of Research and Development on Train Control Center Operation Expressing System of TubeRail(306086)
文摘A method is proposed to calculate the railway transport capacity in an emergency to support the dispatching work of control centers.The effect of an emergency on section transport capacity is analyzed and the basic method to calculate the transport capacity is presented.The results show that the situation-changing process in emergency is actually a Markov process.The calculation rule is presented based on division of the time segment during which the emergency lasts.The algorithm is designed to calculate the transport capacity of each time segment.The pessimistic strategy and the fuzzy strategy are proposed to determine the computing value of the transport capacity of each time segment,to satisfy the calculating requirements in different occasions.Our study shows that the method is reasonable and practical and the method can be embedded in the train dispatching system to support the operation work in an emergency.
基金supported by the National Natural Science Foundation of China(10801056)the Natural Science Foundation of Ningbo(2010A610094)
文摘This paper studies the limit average variance criterion for continuous-time Markov decision processes in Polish spaces. Based on two approaches, this paper proves not only the existence of solutions to the variance minimization optimality equation and the existence of a variance minimal policy that is canonical, but also the existence of solutions to the two variance minimization optimality inequalities and the existence of a variance minimal policy which may not be canonical. An example is given to illustrate all of our conditions.
文摘Drought conditions at a given location evolve randomly through time and are typically characterized by severity and duration. Researchers interested in modeling the economic effects of drought on agriculture or other water users often capture the stochastic nature of drought and its conditions via multiyear, stochastic economic models. Three major sources of uncertainty in application of a multiyear discrete stochastic model to evaluate user preparedness and response to drought are: (1) the assumption of independence of yearly weather conditions, (2) linguistic vagueness in the definition of drought itself, and (3) the duration of drought. One means of addressing these uncertainties is to re-cast drought as a stochastic, multiyear process using a “fuzzy” semi-Markov process. In this paper, we review “crisp” versus “fuzzy” representations of drought and show how fuzzy semi-Markov processes can aid researchers in developing more robust multiyear, discrete stochastic models.
基金National Key Research and Development Program of China(Grant No.2020YFB1710300)National Natural Science Foundation of China(Grant No.52005042)+2 种基金National Defense Fundamental Research Foundation of China(Grant No.JCKY2020203B039)Equipment Pre-research Foundation of China(Grant No.80923010101)Beijing Institute of Technology Research Fund Program for Young Scholars.
文摘The assembly process of aerospace products such as satellites and rockets has the characteristics of single-or small-batch production,a long development period,high reliability,and frequent disturbances.How to predict and avoid quality abnormalities,quickly locate their causes,and improve product assembly quality and efficiency are urgent engineering issues.As the core technology to realize the integration of virtual and physical space,digital twin(DT)technology can make full use of the low cost,high efficiency,and predictable advantages of digital space to provide a feasible solution to such problems.Hence,a quality management method for the assembly process of aerospace products based on DT is proposed.Given that traditional quality control methods for the assembly process of aerospace products are mostly post-inspection,the Grey-Markov model and T-K control chart are used with a small sample of assembly quality data to predict the value of quality data and the status of an assembly system.The Apriori algorithm is applied to mine the strong association rules related to quality data anomalies and uncontrolled assembly systems so as to solve the issue that the causes of abnormal quality are complicated and difficult to trace.The implementation of the proposed approach is described,taking the collected centroid data of an aerospace product’s cabin,one of the key quality data in the assembly process of aerospace products,as an example.A DT-based quality management system for the assembly process of aerospace products is developed,which can effectively improve the efficiency of quality management for the assembly process of aerospace products and reduce quality abnormalities.
文摘This paper considers the variance optimization problem of average reward in continuous-time Markov decision process (MDP). It is assumed that the state space is countable and the action space is Borel measurable space. The main purpose of this paper is to find the policy with the minimal variance in the deterministic stationary policy space. Unlike the traditional Markov decision process, the cost function in the variance criterion will be affected by future actions. To this end, we convert the variance minimization problem into a standard (MDP) by introducing a concept called pseudo-variance. Further, by giving the policy iterative algorithm of pseudo-variance optimization problem, the optimal policy of the original variance optimization problem is derived, and a sufficient condition for the variance optimal policy is given. Finally, we use an example to illustrate the conclusion of this paper.
