| |
| |
Preface to the Fourth Edition | |
| |
| |
Preface to the Third Edition | |
| |
| |
Preface to the First Edition | |
| |
| |
To the Instructor | |
| |
| |
Acknowledgments | |
| |
| |
| |
Introduction | |
| |
| |
| |
Stochastic Modeling | |
| |
| |
| |
Stochastic Processes | |
| |
| |
| |
Probability Review | |
| |
| |
| |
Events and Probabilities | |
| |
| |
| |
Random Variables | |
| |
| |
| |
Moments and Expected Values | |
| |
| |
| |
Joint Distribution Functions | |
| |
| |
| |
Sums and Convolutions | |
| |
| |
| |
Change of Variable | |
| |
| |
| |
Conditional Probability | |
| |
| |
| |
Review of Axiomatic Probability Theory | |
| |
| |
| |
The Major Discrete Distributions | |
| |
| |
| |
Bernoulli Distribution | |
| |
| |
| |
Binomial Distribution | |
| |
| |
| |
Geometric and Negative Binominal Distributions | |
| |
| |
| |
The Poisson Distribution | |
| |
| |
| |
The Multinomial Distribution | |
| |
| |
| |
Important Continuous Distributions | |
| |
| |
| |
The Normal Distribution | |
| |
| |
| |
The Exponential Distribution | |
| |
| |
| |
The Uniform Distribution | |
| |
| |
| |
The Gamma Distribution | |
| |
| |
| |
The Beta Distribution | |
| |
| |
| |
The Joint Normal Distribution | |
| |
| |
| |
Some Elementary Exercises | |
| |
| |
| |
Tail Probabilities | |
| |
| |
| |
The Exponential Distribution | |
| |
| |
| |
Useful Functions, Integrals, and Sums | |
| |
| |
| |
Conditional Probability and Conditional Expectation | |
| |
| |
| |
The Discrete Case | |
| |
| |
| |
The Dice Game Craps | |
| |
| |
| |
Random Sums | |
| |
| |
| |
Conditional Distributions: The Mixed Case | |
| |
| |
| |
The Moments of a Random Sum | |
| |
| |
| |
The Distribution of a Random Sum | |
| |
| |
| |
Conditioning on a Continuous Random Variable | |
| |
| |
| |
Martingales | |
| |
| |
| |
The Definition | |
| |
| |
| |
The Markov Inequality | |
| |
| |
| |
The Maximal Inequality for Nonnegative Martingales | |
| |
| |
| |
Markov Chains: Introduction | |
| |
| |
| |
Definitions | |
| |
| |
| |
Transition Probability Matrices of a Markov Chain | |
| |
| |
| |
Some Markov Chain Models | |
| |
| |
| |
An Inventory Model | |
| |
| |
| |
The Ehrenfest Urn Model | |
| |
| |
| |
Markov Chains in Genetics | |
| |
| |
| |
A Discrete Queueing Markov Chain | |
| |
| |
| |
First Step Analysis | |
| |
| |
| |
Simple First Step Analyses | |
| |
| |
| |
The General Absorbing Markov Chain | |
| |
| |
| |
Some Special Markov Chains | |
| |
| |
| |
The Two-State Markov Chain | |
| |
| |
| |
Markov Chains Defined by Independent Random Variables | |
| |
| |
| |
One-Dimensional Random Walks | |
| |
| |
| |
Success Runs | |
| |
| |
| |
Functionals of Random Walks and Success Runs | |
| |
| |
| |
The General Random Walk | |
| |
| |
| |
Cash Management | |
| |
| |
| |
The Success Runs Markov Chain | |
| |
| |
| |
Another Look at First Step Analysis | |
| |
| |
| |
Branching Processes | |
| |
| |
| |
Examples of Branching Processes | |
| |
| |
| |
The Mean and Variance of a Branching Process | |
| |
| |
| |
Extinction Probabilities | |
| |
| |
| |
Branching Processes and Generating Functions | |
| |
| |
| |
Generating Functions and Extinction Probabilities | |
| |
| |
| |
Probability Generating Functions and Sums of Independent Random Variables | |
| |
| |
| |
Multiple Branching Processes | |
| |
| |
| |
The Long Run Behavior of Markov Chains | |
| |
| |
| |
Regular Transition Probability Matrices | |
| |
| |
| |
Doubly Stochastic Matrices | |
| |
| |
| |
Interpretation of the Limiting Distribution | |
| |
| |
| |
Examples | |
| |
| |
| |
Including History in the State Description | |
| |
| |
| |
Reliability and Redundancy | |
| |
| |
| |
A Continuous Sampling Plan | |
| |
| |
| |
Age Replacement Policies | |
| |
| |
| |
Optimal Replacement Rules | |
| |
| |
| |
The Classification of States | |
| |
| |
| |
Irreducible Markov Chains | |
| |
| |
| |
Periodicity of a Markov Chain | |
| |
| |
| |
Recurrent and Transient States | |
| |
| |
| |
The Basic Limit Theorem of Markov Chains | |
| |
| |
| |
Reducible Markov Chains | |
| |
| |
| |
Poisson Processes | |
| |
| |
| |
The Poisson Distribution and the Poisson Process | |
| |
| |
| |
The Poisson Distribution | |
| |
| |
| |
The Poisson Process | |
| |
| |
| |
Nonhomogeneous Processes | |
| |
| |
| |
Cox Processes | |
| |
| |
| |
The Law of Rare Events | |
| |
| |
| |
The Law of Rare Events and the Poisson Process | |
| |
| |
| |
Proof of Theorem 5.3 | |
| |
| |
| |
Distributions Associated with the Poisson Process | |
| |
| |
| |
The Uniform Distribution and Poisson Processes | |
| |
| |
| |
Shot Noise | |
| |
| |
| |
Sum Quota Sampling | |
| |
| |
| |
Spatial Poisson Processes | |
| |
| |
| |
Compound and Marked Poisson Processes | |
| |
| |
| |
Compound Poisson Processes | |
| |
| |
| |
Marked Poisson Processes | |
| |
| |
| |
Continuous Time Markov Chains | |
| |
| |
| |
Pure Birth Processes | |
| |
| |
| |
Postulates for the Poisson Process | |
| |
| |
| |
Pure Birth Process | |
| |
| |
| |
The Yule Process | |
| |
| |
| |
Pure Death Processes | |
| |
| |
| |
The Linear Death Process | |
| |
| |
| |
Cable Failure Under Static Fatigue | |
| |
| |
| |
Birth and Death Processes | |
| |
| |
| |
Postulates | |
| |
| |
| |
Sojourn Times | |
| |
| |
| |
Differential Equations of Birth and Death Processes | |
| |
| |
| |
The Limiting Behavior of Birth and Death Processes | |
| |
| |
| |
Birth and Death Processes with Absorbing States | |
| |
| |
| |
Probability of Absorption into State 0 | |
| |
| |
| |
Mean Time Until Absorption | |
| |
| |
| |
Finite-State Continuous Time Markov Chains | |
| |
| |
| |
A Poisson Process with a Markov Intensity | |
| |
| |
| |
Renewal Phenomena | |
| |
| |
| |
Definition of a Renewal Process and Related Concepts | |
| |
| |
| |
Some Examples of Renewal Processes | |
| |
| |
| |
Brief Sketches of Renewal Situations | |
| |
| |
| |
Block Replacement | |
| |
| |
| |
The Poisson Process Viewed as a Renewal Process | |
| |
| |
| |
The Asymptotic Behavior of Renewal Processes | |
| |
| |
| |
The Elementary Renewal Theorem | |
| |
| |
| |
The Renewal Theorem for Continuous Lifetimes | |
| |
| |
| |
The Asymptotic Distribution of N(t) | |
| |
| |
| |
The Limiting Distribution of Age and Excess Life | |
| |
| |
| |
Generalizations and Variations on Renewal Processes | |
| |
| |
| |
Delayed Renewal Processes | |
| |
| |
| |
Stationary Renewal Processes | |
| |
| |
| |
Cumulative and Related Processes | |
| |
| |
| |
Discrete Renewal Theory | |
| |
| |
| |
The Discrete Renewal Theorem | |
| |
| |
| |
Deterministic Population Growth with Age Distribution | |
| |
| |
| |
Brownian Motion and Related Processes | |
| |
| |
| |
Brownian Motion and Gaussian Processes | |
| |
| |
| |
A Little History | |
| |
| |
| |
The Brownian Motion Stochastic Process | |
| |
| |
| |
The Central Limit Theorem and the Invariance Principle | |
| |
| |
| |
Gaussian Processes | |
| |
| |
| |
The Maximum Variable and the Reflection Principle | |
| |
| |
| |
The Reflection Principle | |
| |
| |
| |
The Time to First Reach a Level | |
| |
| |
| |
The Zeros of Brownian Motion | |
| |
| |
| |
Variations and Extensions | |
| |
| |
| |
Reflected Brownian Motion | |
| |
| |
| |
Absorbed Brownian Motion | |
| |
| |
| |
The Brownian Bridge | |
| |
| |
| |
Brownian Meander | |
| |
| |
| |
Brownian Motion with Drift | |
| |
| |
| |
The Gambler's Ruin Problem | |
| |
| |
| |
Geometric Brownian Motion | |
| |
| |
| |
The Ornstein-Uhlenbeck Process | |
| |
| |
| |
A Second Approach to Physical Brownian Motion | |
| |
| |
| |
The Position Process | |
| |
| |
| |
The Long Run Behavior | |
| |
| |
| |
Brownian Measure and Integration | |
| |
| |
| |
Queueing Systems | |
| |
| |
| |
Queueing Processes | |
| |
| |
| |
The Queueing Formula L = X W | |
| |
| |
| |
A Sampling of Queueing Models | |
| |
| |
| |
Poisson Arrivals, Exponential Service Times | |
| |
| |
| |
The M/M/1 System | |
| |
| |
| |
The M/M/$ System | |
| |
| |
| |
The M/M/s System | |
| |
| |
| |
General Service Time Distributions | |
| |
| |
| |
The M/G/1 System | |
| |
| |
| |
The M/G/$ System | |
| |
| |
| |
Variations and Extensions | |
| |
| |
| |
Systems with Balking | |
| |
| |
| |
Variable Service Rates | |
| |
| |
| |
A System with Feedback | |
| |
| |
| |
A Two-Server Overflow Queue | |
| |
| |
| |
Preemptive Priority Queues | |
| |
| |
| |
Open Acyclic Queueing Networks | |
| |
| |
| |
The Basic Theorem | |
| |
| |
| |
Two Queues in Tandem | |
| |
| |
| |
Open Acyclic Networks | |
| |
| |
| |
Appendix: Time Reversibility | |
| |
| |
| |
Proof of Theorem 9.1 | |
| |
| |
| |
General Open Networks | |
| |
| |
| |
The General Open Network | |
| |
| |
| |
Random Evolutions | |
| |
| |
| |
Two-State Velocity Model | |
| |
| |
| |
Two-State Random Evolution | |
| |
| |
| |
The Telegraph Equation | |
| |
| |
| |
Distribution Functions and Densities in the Two-State Model | |
| |
| |
| |
Passage Time Distributions | |
| |
| |
| |
JV-State Random Evolution | |
| |
| |
| |
Finite Markov Chains and Random Velocity Models | |
| |
| |
| |
Constructive Approach of Random Velocity Models | |
| |
| |
| |
Random Evolution Processes | |
| |
| |
| |
Existence-Uniqueness of the First-Order System (10.26) | |
| |
| |
| |
Single Hyperbolic Equation | |
| |
| |
| |
Spectral Properties of the Transition Matrix | |
| |
| |
| |
Recurrence Properties of Random Evolution | |
| |
| |
| |
Weak Law and Central Limit Theorem | |
| |
| |
| |
Isotropic Transport in Higher Dimensions | |
| |
| |
| |
The Rayleigh Problem of Random Flights | |
| |
| |
| |
Three-Dimensional Rayleigh Model | |
| |
| |
| |
Characteristic Functions and Their Applications | |
| |
| |
| |
Definition of the Characteristic Function | |
| |
| |
| |
Two Basic Properties of the Characteristic Function | |
| |
| |
| |
Inversion Formulas for Characteristic Functions | |
| |
| |
| |
Fourier Reciprocity/Local Non-Uniqueness | |
| |
| |
| |
Fourier Inversion and Parseval's Identity Inversion | |
| |
| |
| |
Formula for General Random Variables | |
| |
| |
| |
The Continuity Theorem | |
| |
| |
| |
Proof of the Continuity Theorem | |
| |
| |
| |
Proof of the Central Limit Theorem | |
| |
| |
| |
Stirling's Formula and Applications | |
| |
| |
| |
Poisson Representation of n! | |
| |
| |
| |
Proof of Stirling's Formula | |
| |
| |
| |
Local deMoivre-Laplace Theorem | |
| |
| |
Further Reading | |
| |
| |
Answers to Exercises | |
| |
| |
Index | |