## Elements of the theory of Markov processes and their applicationsGraduate-level text and reference in probability, with numerous scientific applications. Nonmeasure-theoretic introduction to theory of Markov processes and to mathematical models based on the theory. Appendixes. Bibliographies. 1960 edition. |

### From inside the book

Results 1-3 of 36

Page 17

Hence Qjt = 1 implies QH = 1. If the stated is nonrecurrent, Qfi = 0 by (1.16).

Hence (1.17) must converge. Additional concepts we now introduce are those of

denned as follows: Tt = m if Xn+k ^ t for 1 ^ k $£. m and Xn+m = i. From (1.13) we

see that *£ &{Ti = n} = K\f The random variable Tt is called the

the state t.

Hence Qjt = 1 implies QH = 1. If the stated is nonrecurrent, Qfi = 0 by (1.16).

Hence (1.17) must converge. Additional concepts we now introduce are those of

**recurrence**and first-passage times for a given state. Suppose a state i is**recurrent**, and &{Xn — i) > 0. Given that X n = i, we introduce a random variable Ttdenned as follows: Tt = m if Xn+k ^ t for 1 ^ k $£. m and Xn+m = i. From (1.13) we

see that *£ &{Ti = n} = K\f The random variable Tt is called the

**recurrence**time forthe state t.

Page 93

The integral = \dHit(t) Jo is the probability that, if the system starts in state i, it

leaves i and then returns to i in a finite period of time. We can now define the

states as follows: Definition : The tth state is

If » is a

with nonzero elements on the lower diagonal, diagonal, and upper diagonal only.

2 Cf. Hellinger and Wall [34].

I t ...

The integral = \dHit(t) Jo is the probability that, if the system starts in state i, it

leaves i and then returns to i in a finite period of time. We can now define the

states as follows: Definition : The tth state is

**recurrent**if / = 1 and transient if / # 1.If » is a

**recurrent**state, it can be classified asergrodic or 1 A continuant is a matrixwith nonzero elements on the lower diagonal, diagonal, and upper diagonal only.

2 Cf. Hellinger and Wall [34].

**recurrent**null according as its mean**recurrence**timeI t ...

Page 94

Similarly, for processes we have the following. Definition: A process is called

corresponding property. Karlin and McGregor have given the following criteria for

the classification of birth-and-death processes: 1. A birth-and-death process is

and only if ...

**recurrent**null according as its mean**recurrence**time I t dHH(t) is finite or infinite.Similarly, for processes we have the following. Definition: A process is called

**recurrent**, ergodic,**recurrent**null, or transient if every one of its states has thecorresponding property. Karlin and McGregor have given the following criteria for

the classification of birth-and-death processes: 1. A birth-and-death process is

**recurrent**if and only if 00 J 2 — = oo 2a. A birth-and-death process is ergodic ifand only if ...

### What people are saying - Write a review

We haven't found any reviews in the usual places.

### Contents

Preface | 1 |

Processes Continuous In Space and Time | 3 |

Processes Discrete in Space and Time | 9 |

Copyright | |

10 other sections not shown

### Other editions - View all

Elements of the Theory of Markov Processes and Their Applications A. T. Bharucha-Reid Limited preview - 2012 |

Elements of the Theory of Markov Processes and Their Applications Albert T. Bharucha-Reid Limited preview - 1997 |

### Common terms and phrases

absorber Acad applications assume assumptions asymptotic birth process birth-and-death process cascade process cascade theory coefficient collision consider counter defined denote the number denote the probability derive determined deterministic differential equation diffusion equations diffusion processes discrete branching process distribution function electron-photon cascades epidemic exists expression extinction Feller finite fluctuation functional equation gambler's ruin given Hence independent initial condition integral equation interval introduce ionization Kendall Kolmogorov equations Laplace transform Laplace-Stieltjes transform Let the random limit theorems machine Markov chain Markov processes Math mathematical matrix Mellin transform method Monte Carlo methods neutron nonnegative nucleon nucleon cascades number of individuals o(At obtain photon Poisson process population positive probability distribution problem Proc Px(t queueing process queueing system radiation Ramakrishnan random variable random variable X(t random walk recurrent refer satisfies sequence Statist stochastic model Stochastic Processes tion transition probabilities zero