| |
| |
| Preface to the Fourth Edition | |
| |
| |
| Preface to the Third Edition | |
| |
| |
| Preface to the First Edition | |
| |
| |
| To the Instructor | |
| |
| |
| Acknowledgments | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Stochastic Modeling | |
| |
| |
| |
| Stochastic Processes | |
| |
| |
| |
| Probability Review | |
| |
| |
| |
| Events and Probabilities | |
| |
| |
| |
| Random Variables | |
| |
| |
| |
| Moments and Expected Values | |
| |
| |
| |
| Joint Distribution Functions | |
| |
| |
| |
| Sums and Convolutions | |
| |
| |
| |
| Change of Variable | |
| |
| |
| |
| Conditional Probability | |
| |
| |
| |
| Review of Axiomatic Probability Theory | |
| |
| |
| |
| The Major Discrete Distributions | |
| |
| |
| |
| Bernoulli Distribution | |
| |
| |
| |
| Binomial Distribution | |
| |
| |
| |
| Geometric and Negative Binominal Distributions | |
| |
| |
| |
| The Poisson Distribution | |
| |
| |
| |
| The Multinomial Distribution | |
| |
| |
| |
| Important Continuous Distributions | |
| |
| |
| |
| The Normal Distribution | |
| |
| |
| |
| The Exponential Distribution | |
| |
| |
| |
| The Uniform Distribution | |
| |
| |
| |
| The Gamma Distribution | |
| |
| |
| |
| The Beta Distribution | |
| |
| |
| |
| The Joint Normal Distribution | |
| |
| |
| |
| Some Elementary Exercises | |
| |
| |
| |
| Tail Probabilities | |
| |
| |
| |
| The Exponential Distribution | |
| |
| |
| |
| Useful Functions, Integrals, and Sums | |
| |
| |
| |
| Conditional Probability and Conditional Expectation | |
| |
| |
| |
| The Discrete Case | |
| |
| |
| |
| The Dice Game Craps | |
| |
| |
| |
| Random Sums | |
| |
| |
| |
| Conditional Distributions: The Mixed Case | |
| |
| |
| |
| The Moments of a Random Sum | |
| |
| |
| |
| The Distribution of a Random Sum | |
| |
| |
| |
| Conditioning on a Continuous Random Variable | |
| |
| |
| |
| Martingales | |
| |
| |
| |
| The Definition | |
| |
| |
| |
| The Markov Inequality | |
| |
| |
| |
| The Maximal Inequality for Nonnegative Martingales | |
| |
| |
| |
| Markov Chains: Introduction | |
| |
| |
| |
| Definitions | |
| |
| |
| |
| Transition Probability Matrices of a Markov Chain | |
| |
| |
| |
| Some Markov Chain Models | |
| |
| |
| |
| An Inventory Model | |
| |
| |
| |
| The Ehrenfest Urn Model | |
| |
| |
| |
| Markov Chains in Genetics | |
| |
| |
| |
| A Discrete Queueing Markov Chain | |
| |
| |
| |
| First Step Analysis | |
| |
| |
| |
| Simple First Step Analyses | |
| |
| |
| |
| The General Absorbing Markov Chain | |
| |
| |
| |
| Some Special Markov Chains | |
| |
| |
| |
| The Two-State Markov Chain | |
| |
| |
| |
| Markov Chains Defined by Independent Random Variables | |
| |
| |
| |
| One-Dimensional Random Walks | |
| |
| |
| |
| Success Runs | |
| |
| |
| |
| Functionals of Random Walks and Success Runs | |
| |
| |
| |
| The General Random Walk | |
| |
| |
| |
| Cash Management | |
| |
| |
| |
| The Success Runs Markov Chain | |
| |
| |
| |
| Another Look at First Step Analysis | |
| |
| |
| |
| Branching Processes | |
| |
| |
| |
| Examples of Branching Processes | |
| |
| |
| |
| The Mean and Variance of a Branching Process | |
| |
| |
| |
| Extinction Probabilities | |
| |
| |
| |
| Branching Processes and Generating Functions | |
| |
| |
| |
| Generating Functions and Extinction Probabilities | |
| |
| |
| |
| Probability Generating Functions and Sums of Independent Random Variables | |
| |
| |
| |
| Multiple Branching Processes | |
| |
| |
| |
| The Long Run Behavior of Markov Chains | |
| |
| |
| |
| Regular Transition Probability Matrices | |
| |
| |
| |
| Doubly Stochastic Matrices | |
| |
| |
| |
| Interpretation of the Limiting Distribution | |
| |
| |
| |
| Examples | |
| |
| |
| |
| Including History in the State Description | |
| |
| |
| |
| Reliability and Redundancy | |
| |
| |
| |
| A Continuous Sampling Plan | |
| |
| |
| |
| Age Replacement Policies | |
| |
| |
| |
| Optimal Replacement Rules | |
| |
| |
| |
| The Classification of States | |
| |
| |
| |
| Irreducible Markov Chains | |
| |
| |
| |
| Periodicity of a Markov Chain | |
| |
| |
| |
| Recurrent and Transient States | |
| |
| |
| |
| The Basic Limit Theorem of Markov Chains | |
| |
| |
| |
| Reducible Markov Chains | |
| |
| |
| |
| Poisson Processes | |
| |
| |
| |
| The Poisson Distribution and the Poisson Process | |
| |
| |
| |
| The Poisson Distribution | |
| |
| |
| |
| The Poisson Process | |
| |
| |
| |
| Nonhomogeneous Processes | |
| |
| |
| |
| Cox Processes | |
| |
| |
| |
| The Law of Rare Events | |
| |
| |
| |
| The Law of Rare Events and the Poisson Process | |
| |
| |
| |
| Proof of Theorem 5.3 | |
| |
| |
| |
| Distributions Associated with the Poisson Process | |
| |
| |
| |
| The Uniform Distribution and Poisson Processes | |
| |
| |
| |
| Shot Noise | |
| |
| |
| |
| Sum Quota Sampling | |
| |
| |
| |
| Spatial Poisson Processes | |
| |
| |
| |
| Compound and Marked Poisson Processes | |
| |
| |
| |
| Compound Poisson Processes | |
| |
| |
| |
| Marked Poisson Processes | |
| |
| |
| |
| Continuous Time Markov Chains | |
| |
| |
| |
| Pure Birth Processes | |
| |
| |
| |
| Postulates for the Poisson Process | |
| |
| |
| |
| Pure Birth Process | |
| |
| |
| |
| The Yule Process | |
| |
| |
| |
| Pure Death Processes | |
| |
| |
| |
| The Linear Death Process | |
| |
| |
| |
| Cable Failure Under Static Fatigue | |
| |
| |
| |
| Birth and Death Processes | |
| |
| |
| |
| Postulates | |
| |
| |
| |
| Sojourn Times | |
| |
| |
| |
| Differential Equations of Birth and Death Processes | |
| |
| |
| |
| The Limiting Behavior of Birth and Death Processes | |
| |
| |
| |
| Birth and Death Processes with Absorbing States | |
| |
| |
| |
| Probability of Absorption into State 0 | |
| |
| |
| |
| Mean Time Until Absorption | |
| |
| |
| |
| Finite-State Continuous Time Markov Chains | |
| |
| |
| |
| A Poisson Process with a Markov Intensity | |
| |
| |
| |
| Renewal Phenomena | |
| |
| |
| |
| Definition of a Renewal Process and Related Concepts | |
| |
| |
| |
| Some Examples of Renewal Processes | |
| |
| |
| |
| Brief Sketches of Renewal Situations | |
| |
| |
| |
| Block Replacement | |
| |
| |
| |
| The Poisson Process Viewed as a Renewal Process | |
| |
| |
| |
| The Asymptotic Behavior of Renewal Processes | |
| |
| |
| |
| The Elementary Renewal Theorem | |
| |
| |
| |
| The Renewal Theorem for Continuous Lifetimes | |
| |
| |
| |
| The Asymptotic Distribution of N(t) | |
| |
| |
| |
| The Limiting Distribution of Age and Excess Life | |
| |
| |
| |
| Generalizations and Variations on Renewal Processes | |
| |
| |
| |
| Delayed Renewal Processes | |
| |
| |
| |
| Stationary Renewal Processes | |
| |
| |
| |
| Cumulative and Related Processes | |
| |
| |
| |
| Discrete Renewal Theory | |
| |
| |
| |
| The Discrete Renewal Theorem | |
| |
| |
| |
| Deterministic Population Growth with Age Distribution | |
| |
| |
| |
| Brownian Motion and Related Processes | |
| |
| |
| |
| Brownian Motion and Gaussian Processes | |
| |
| |
| |
| A Little History | |
| |
| |
| |
| The Brownian Motion Stochastic Process | |
| |
| |
| |
| The Central Limit Theorem and the Invariance Principle | |
| |
| |
| |
| Gaussian Processes | |
| |
| |
| |
| The Maximum Variable and the Reflection Principle | |
| |
| |
| |
| The Reflection Principle | |
| |
| |
| |
| The Time to First Reach a Level | |
| |
| |
| |
| The Zeros of Brownian Motion | |
| |
| |
| |
| Variations and Extensions | |
| |
| |
| |
| Reflected Brownian Motion | |
| |
| |
| |
| Absorbed Brownian Motion | |
| |
| |
| |
| The Brownian Bridge | |
| |
| |
| |
| Brownian Meander | |
| |
| |
| |
| Brownian Motion with Drift | |
| |
| |
| |
| The Gambler's Ruin Problem | |
| |
| |
| |
| Geometric Brownian Motion | |
| |
| |
| |
| The Ornstein-Uhlenbeck Process | |
| |
| |
| |
| A Second Approach to Physical Brownian Motion | |
| |
| |
| |
| The Position Process | |
| |
| |
| |
| The Long Run Behavior | |
| |
| |
| |
| Brownian Measure and Integration | |
| |
| |
| |
| Queueing Systems | |
| |
| |
| |
| Queueing Processes | |
| |
| |
| |
| The Queueing Formula L = X W | |
| |
| |
| |
| A Sampling of Queueing Models | |
| |
| |
| |
| Poisson Arrivals, Exponential Service Times | |
| |
| |
| |
| The M/M/1 System | |
| |
| |
| |
| The M/M/$ System | |
| |
| |
| |
| The M/M/s System | |
| |
| |
| |
| General Service Time Distributions | |
| |
| |
| |
| The M/G/1 System | |
| |
| |
| |
| The M/G/$ System | |
| |
| |
| |
| Variations and Extensions | |
| |
| |
| |
| Systems with Balking | |
| |
| |
| |
| Variable Service Rates | |
| |
| |
| |
| A System with Feedback | |
| |
| |
| |
| A Two-Server Overflow Queue | |
| |
| |
| |
| Preemptive Priority Queues | |
| |
| |
| |
| Open Acyclic Queueing Networks | |
| |
| |
| |
| The Basic Theorem | |
| |
| |
| |
| Two Queues in Tandem | |
| |
| |
| |
| Open Acyclic Networks | |
| |
| |
| |
| Appendix: Time Reversibility | |
| |
| |
| |
| Proof of Theorem 9.1 | |
| |
| |
| |
| General Open Networks | |
| |
| |
| |
| The General Open Network | |
| |
| |
| |
| Random Evolutions | |
| |
| |
| |
| Two-State Velocity Model | |
| |
| |
| |
| Two-State Random Evolution | |
| |
| |
| |
| The Telegraph Equation | |
| |
| |
| |
| Distribution Functions and Densities in the Two-State Model | |
| |
| |
| |
| Passage Time Distributions | |
| |
| |
| |
| JV-State Random Evolution | |
| |
| |
| |
| Finite Markov Chains and Random Velocity Models | |
| |
| |
| |
| Constructive Approach of Random Velocity Models | |
| |
| |
| |
| Random Evolution Processes | |
| |
| |
| |
| Existence-Uniqueness of the First-Order System (10.26) | |
| |
| |
| |
| Single Hyperbolic Equation | |
| |
| |
| |
| Spectral Properties of the Transition Matrix | |
| |
| |
| |
| Recurrence Properties of Random Evolution | |
| |
| |
| |
| Weak Law and Central Limit Theorem | |
| |
| |
| |
| Isotropic Transport in Higher Dimensions | |
| |
| |
| |
| The Rayleigh Problem of Random Flights | |
| |
| |
| |
| Three-Dimensional Rayleigh Model | |
| |
| |
| |
| Characteristic Functions and Their Applications | |
| |
| |
| |
| Definition of the Characteristic Function | |
| |
| |
| |
| Two Basic Properties of the Characteristic Function | |
| |
| |
| |
| Inversion Formulas for Characteristic Functions | |
| |
| |
| |
| Fourier Reciprocity/Local Non-Uniqueness | |
| |
| |
| |
| Fourier Inversion and Parseval's Identity Inversion | |
| |
| |
| |
| Formula for General Random Variables | |
| |
| |
| |
| The Continuity Theorem | |
| |
| |
| |
| Proof of the Continuity Theorem | |
| |
| |
| |
| Proof of the Central Limit Theorem | |
| |
| |
| |
| Stirling's Formula and Applications | |
| |
| |
| |
| Poisson Representation of n! | |
| |
| |
| |
| Proof of Stirling's Formula | |
| |
| |
| |
| Local deMoivre-Laplace Theorem | |
| |
| |
| Further Reading | |
| |
| |
| Answers to Exercises | |
| |
| |
| Index | |