| |

| |

Preface to the Fourth Edition | |

| |

| |

Preface to the Third Edition | |

| |

| |

Preface to the First Edition | |

| |

| |

To the Instructor | |

| |

| |

Acknowledgments | |

| |

| |

| |

Introduction | |

| |

| |

| |

Stochastic Modeling | |

| |

| |

| |

Stochastic Processes | |

| |

| |

| |

Probability Review | |

| |

| |

| |

Events and Probabilities | |

| |

| |

| |

Random Variables | |

| |

| |

| |

Moments and Expected Values | |

| |

| |

| |

Joint Distribution Functions | |

| |

| |

| |

Sums and Convolutions | |

| |

| |

| |

Change of Variable | |

| |

| |

| |

Conditional Probability | |

| |

| |

| |

Review of Axiomatic Probability Theory | |

| |

| |

| |

The Major Discrete Distributions | |

| |

| |

| |

Bernoulli Distribution | |

| |

| |

| |

Binomial Distribution | |

| |

| |

| |

Geometric and Negative Binominal Distributions | |

| |

| |

| |

The Poisson Distribution | |

| |

| |

| |

The Multinomial Distribution | |

| |

| |

| |

Important Continuous Distributions | |

| |

| |

| |

The Normal Distribution | |

| |

| |

| |

The Exponential Distribution | |

| |

| |

| |

The Uniform Distribution | |

| |

| |

| |

The Gamma Distribution | |

| |

| |

| |

The Beta Distribution | |

| |

| |

| |

The Joint Normal Distribution | |

| |

| |

| |

Some Elementary Exercises | |

| |

| |

| |

Tail Probabilities | |

| |

| |

| |

The Exponential Distribution | |

| |

| |

| |

Useful Functions, Integrals, and Sums | |

| |

| |

| |

Conditional Probability and Conditional Expectation | |

| |

| |

| |

The Discrete Case | |

| |

| |

| |

The Dice Game Craps | |

| |

| |

| |

Random Sums | |

| |

| |

| |

Conditional Distributions: The Mixed Case | |

| |

| |

| |

The Moments of a Random Sum | |

| |

| |

| |

The Distribution of a Random Sum | |

| |

| |

| |

Conditioning on a Continuous Random Variable | |

| |

| |

| |

Martingales | |

| |

| |

| |

The Definition | |

| |

| |

| |

The Markov Inequality | |

| |

| |

| |

The Maximal Inequality for Nonnegative Martingales | |

| |

| |

| |

Markov Chains: Introduction | |

| |

| |

| |

Definitions | |

| |

| |

| |

Transition Probability Matrices of a Markov Chain | |

| |

| |

| |

Some Markov Chain Models | |

| |

| |

| |

An Inventory Model | |

| |

| |

| |

The Ehrenfest Urn Model | |

| |

| |

| |

Markov Chains in Genetics | |

| |

| |

| |

A Discrete Queueing Markov Chain | |

| |

| |

| |

First Step Analysis | |

| |

| |

| |

Simple First Step Analyses | |

| |

| |

| |

The General Absorbing Markov Chain | |

| |

| |

| |

Some Special Markov Chains | |

| |

| |

| |

The Two-State Markov Chain | |

| |

| |

| |

Markov Chains Defined by Independent Random Variables | |

| |

| |

| |

One-Dimensional Random Walks | |

| |

| |

| |

Success Runs | |

| |

| |

| |

Functionals of Random Walks and Success Runs | |

| |

| |

| |

The General Random Walk | |

| |

| |

| |

Cash Management | |

| |

| |

| |

The Success Runs Markov Chain | |

| |

| |

| |

Another Look at First Step Analysis | |

| |

| |

| |

Branching Processes | |

| |

| |

| |

Examples of Branching Processes | |

| |

| |

| |

The Mean and Variance of a Branching Process | |

| |

| |

| |

Extinction Probabilities | |

| |

| |

| |

Branching Processes and Generating Functions | |

| |

| |

| |

Generating Functions and Extinction Probabilities | |

| |

| |

| |

Probability Generating Functions and Sums of Independent Random Variables | |

| |

| |

| |

Multiple Branching Processes | |

| |

| |

| |

The Long Run Behavior of Markov Chains | |

| |

| |

| |

Regular Transition Probability Matrices | |

| |

| |

| |

Doubly Stochastic Matrices | |

| |

| |

| |

Interpretation of the Limiting Distribution | |

| |

| |

| |

Examples | |

| |

| |

| |

Including History in the State Description | |

| |

| |

| |

Reliability and Redundancy | |

| |

| |

| |

A Continuous Sampling Plan | |

| |

| |

| |

Age Replacement Policies | |

| |

| |

| |

Optimal Replacement Rules | |

| |

| |

| |

The Classification of States | |

| |

| |

| |

Irreducible Markov Chains | |

| |

| |

| |

Periodicity of a Markov Chain | |

| |

| |

| |

Recurrent and Transient States | |

| |

| |

| |

The Basic Limit Theorem of Markov Chains | |

| |

| |

| |

Reducible Markov Chains | |

| |

| |

| |

Poisson Processes | |

| |

| |

| |

The Poisson Distribution and the Poisson Process | |

| |

| |

| |

The Poisson Distribution | |

| |

| |

| |

The Poisson Process | |

| |

| |

| |

Nonhomogeneous Processes | |

| |

| |

| |

Cox Processes | |

| |

| |

| |

The Law of Rare Events | |

| |

| |

| |

The Law of Rare Events and the Poisson Process | |

| |

| |

| |

Proof of Theorem 5.3 | |

| |

| |

| |

Distributions Associated with the Poisson Process | |

| |

| |

| |

The Uniform Distribution and Poisson Processes | |

| |

| |

| |

Shot Noise | |

| |

| |

| |

Sum Quota Sampling | |

| |

| |

| |

Spatial Poisson Processes | |

| |

| |

| |

Compound and Marked Poisson Processes | |

| |

| |

| |

Compound Poisson Processes | |

| |

| |

| |

Marked Poisson Processes | |

| |

| |

| |

Continuous Time Markov Chains | |

| |

| |

| |

Pure Birth Processes | |

| |

| |

| |

Postulates for the Poisson Process | |

| |

| |

| |

Pure Birth Process | |

| |

| |

| |

The Yule Process | |

| |

| |

| |

Pure Death Processes | |

| |

| |

| |

The Linear Death Process | |

| |

| |

| |

Cable Failure Under Static Fatigue | |

| |

| |

| |

Birth and Death Processes | |

| |

| |

| |

Postulates | |

| |

| |

| |

Sojourn Times | |

| |

| |

| |

Differential Equations of Birth and Death Processes | |

| |

| |

| |

The Limiting Behavior of Birth and Death Processes | |

| |

| |

| |

Birth and Death Processes with Absorbing States | |

| |

| |

| |

Probability of Absorption into State 0 | |

| |

| |

| |

Mean Time Until Absorption | |

| |

| |

| |

Finite-State Continuous Time Markov Chains | |

| |

| |

| |

A Poisson Process with a Markov Intensity | |

| |

| |

| |

Renewal Phenomena | |

| |

| |

| |

Definition of a Renewal Process and Related Concepts | |

| |

| |

| |

Some Examples of Renewal Processes | |

| |

| |

| |

Brief Sketches of Renewal Situations | |

| |

| |

| |

Block Replacement | |

| |

| |

| |

The Poisson Process Viewed as a Renewal Process | |

| |

| |

| |

The Asymptotic Behavior of Renewal Processes | |

| |

| |

| |

The Elementary Renewal Theorem | |

| |

| |

| |

The Renewal Theorem for Continuous Lifetimes | |

| |

| |

| |

The Asymptotic Distribution of N(t) | |

| |

| |

| |

The Limiting Distribution of Age and Excess Life | |

| |

| |

| |

Generalizations and Variations on Renewal Processes | |

| |

| |

| |

Delayed Renewal Processes | |

| |

| |

| |

Stationary Renewal Processes | |

| |

| |

| |

Cumulative and Related Processes | |

| |

| |

| |

Discrete Renewal Theory | |

| |

| |

| |

The Discrete Renewal Theorem | |

| |

| |

| |

Deterministic Population Growth with Age Distribution | |

| |

| |

| |

Brownian Motion and Related Processes | |

| |

| |

| |

Brownian Motion and Gaussian Processes | |

| |

| |

| |

A Little History | |

| |

| |

| |

The Brownian Motion Stochastic Process | |

| |

| |

| |

The Central Limit Theorem and the Invariance Principle | |

| |

| |

| |

Gaussian Processes | |

| |

| |

| |

The Maximum Variable and the Reflection Principle | |

| |

| |

| |

The Reflection Principle | |

| |

| |

| |

The Time to First Reach a Level | |

| |

| |

| |

The Zeros of Brownian Motion | |

| |

| |

| |

Variations and Extensions | |

| |

| |

| |

Reflected Brownian Motion | |

| |

| |

| |

Absorbed Brownian Motion | |

| |

| |

| |

The Brownian Bridge | |

| |

| |

| |

Brownian Meander | |

| |

| |

| |

Brownian Motion with Drift | |

| |

| |

| |

The Gambler's Ruin Problem | |

| |

| |

| |

Geometric Brownian Motion | |

| |

| |

| |

The Ornstein-Uhlenbeck Process | |

| |

| |

| |

A Second Approach to Physical Brownian Motion | |

| |

| |

| |

The Position Process | |

| |

| |

| |

The Long Run Behavior | |

| |

| |

| |

Brownian Measure and Integration | |

| |

| |

| |

Queueing Systems | |

| |

| |

| |

Queueing Processes | |

| |

| |

| |

The Queueing Formula L = X W | |

| |

| |

| |

A Sampling of Queueing Models | |

| |

| |

| |

Poisson Arrivals, Exponential Service Times | |

| |

| |

| |

The M/M/1 System | |

| |

| |

| |

The M/M/$ System | |

| |

| |

| |

The M/M/s System | |

| |

| |

| |

General Service Time Distributions | |

| |

| |

| |

The M/G/1 System | |

| |

| |

| |

The M/G/$ System | |

| |

| |

| |

Variations and Extensions | |

| |

| |

| |

Systems with Balking | |

| |

| |

| |

Variable Service Rates | |

| |

| |

| |

A System with Feedback | |

| |

| |

| |

A Two-Server Overflow Queue | |

| |

| |

| |

Preemptive Priority Queues | |

| |

| |

| |

Open Acyclic Queueing Networks | |

| |

| |

| |

The Basic Theorem | |

| |

| |

| |

Two Queues in Tandem | |

| |

| |

| |

Open Acyclic Networks | |

| |

| |

| |

Appendix: Time Reversibility | |

| |

| |

| |

Proof of Theorem 9.1 | |

| |

| |

| |

General Open Networks | |

| |

| |

| |

The General Open Network | |

| |

| |

| |

Random Evolutions | |

| |

| |

| |

Two-State Velocity Model | |

| |

| |

| |

Two-State Random Evolution | |

| |

| |

| |

The Telegraph Equation | |

| |

| |

| |

Distribution Functions and Densities in the Two-State Model | |

| |

| |

| |

Passage Time Distributions | |

| |

| |

| |

JV-State Random Evolution | |

| |

| |

| |

Finite Markov Chains and Random Velocity Models | |

| |

| |

| |

Constructive Approach of Random Velocity Models | |

| |

| |

| |

Random Evolution Processes | |

| |

| |

| |

Existence-Uniqueness of the First-Order System (10.26) | |

| |

| |

| |

Single Hyperbolic Equation | |

| |

| |

| |

Spectral Properties of the Transition Matrix | |

| |

| |

| |

Recurrence Properties of Random Evolution | |

| |

| |

| |

Weak Law and Central Limit Theorem | |

| |

| |

| |

Isotropic Transport in Higher Dimensions | |

| |

| |

| |

The Rayleigh Problem of Random Flights | |

| |

| |

| |

Three-Dimensional Rayleigh Model | |

| |

| |

| |

Characteristic Functions and Their Applications | |

| |

| |

| |

Definition of the Characteristic Function | |

| |

| |

| |

Two Basic Properties of the Characteristic Function | |

| |

| |

| |

Inversion Formulas for Characteristic Functions | |

| |

| |

| |

Fourier Reciprocity/Local Non-Uniqueness | |

| |

| |

| |

Fourier Inversion and Parseval's Identity Inversion | |

| |

| |

| |

Formula for General Random Variables | |

| |

| |

| |

The Continuity Theorem | |

| |

| |

| |

Proof of the Continuity Theorem | |

| |

| |

| |

Proof of the Central Limit Theorem | |

| |

| |

| |

Stirling's Formula and Applications | |

| |

| |

| |

Poisson Representation of n! | |

| |

| |

| |

Proof of Stirling's Formula | |

| |

| |

| |

Local deMoivre-Laplace Theorem | |

| |

| |

Further Reading | |

| |

| |

Answers to Exercises | |

| |

| |

Index | |