| |
| |
| |
| Review Of Probability Theory | |
| |
| |
| Experiments and Outcomes | |
| |
| |
| Sample Space and Events | |
| |
| |
| Probabilities Defined on Events | |
| |
| |
| Probability Models | |
| |
| |
| Probabilities of Combined Events | |
| |
| |
| Conditional Probabilities | |
| |
| |
| Conditional Arguments | |
| |
| |
| Independent Events | |
| |
| |
| Random Variables | |
| |
| |
| Discrete Random Variables and Their Expected Values | |
| |
| |
| Bernoulli Random Variables | |
| |
| |
| Geometric Distribution | |
| |
| |
| Binomial Distribution | |
| |
| |
| Negative Binomial Distribution | |
| |
| |
| Continuous and Mixed Random Variables | |
| |
| |
| Continuous Random Variables and Their Expected Values | |
| |
| |
| Uniform Distribution | |
| |
| |
| Normal Distribution | |
| |
| |
| Conditional Arguments for a Random Variable | |
| |
| |
| Conditional Expectations | |
| |
| |
| Random Sum | |
| |
| |
| Summary | |
| |
| |
| Problems | |
| |
| |
| |
| Discrete-Time Markov Chains | |
| |
| |
| Stochastic Processes | |
| |
| |
| Independent and Identically Distributed Random Variables | |
| |
| |
| Markov Chains of Order i | |
| |
| |
| First-Order Markov Chains | |
| |
| |
| Matrix Notation for Markov Chains | |
| |
| |
| Transition Diagrams | |
| |
| |
| The Importance of First-Order Markov Chains | |
| |
| |
| Time-Homogeneity | |
| |
| |
| Seasonal Chains | |
| |
| |
| The Rat in the Maze | |
| |
| |
| Empirical Transition Matrices | |
| |
| |
| Population-Homogeneity | |
| |
| |
| Enlarging the State Space | |
| |
| |
| Using Supplementary Variables | |
| |
| |
| Refining the State Space | |
| |
| |
| What Do a Little Rat and a Big Company Have in Common? | |
| |
| |
| Lumpable States | |
| |
| |
| Discrete-Time Chains In a Continuous-Time Chain | |
| |
| |
| The Sojourn Times | |
| |
| |
| Summary | |
| |
| |
| Problems | |
| |
| |
| |
| Transient And Limiting Results | |
| |
| |
| Paths | |
| |
| |
| k-Step Transition Probabilities | |
| |
| |
| Path Analysis | |
| |
| |
| Matrix Multiplication Method | |
| |
| |
| Chapman-Kolmogorov Equation | |
| |
| |
| Initial Conditions | |
| |
| |
| Transient Results | |
| |
| |
| Validating the Model | |
| |
| |
| Limiting Distributions | |
| |
| |
| Expected Numbers of Visits | |
| |
| |
| Summary | |
| |
| |
| Problems | |
| |
| |
| |
| Classification Of Finite Chains | |
| |
| |
| Classification of Finite Chains | |
| |
| |
| Transient and Recurrent States | |
| |
| |
| Reachability Between States | |
| |
| |
| Communication of Two States | |
| |
| |
| Classes | |
| |
| |
| Transient and Recurrent Classes | |
| |
| |
| Canonical Forms | |
| |
| |
| Absorbing Chains | |
| |
| |
| Irreducible Chains | |
| |
| |
| Periodic States | |
| |
| |
| Periodic Classes | |
| |
| |
| Finite, Periodic Chains | |
| |
| |
| Summary | |
| |
| |
| Problems | |
| |
| |
| |
| Finite Absorbing Chains | |
| |
| |
| Absorbing Chains | |
| |
| |
| First-Step Analysis | |
| |
| |
| Expected Number of Visits to a Transient State | |
| |
| |
| Expected Absorption Time | |
| |
| |
| Absorbing Probabilities | |
| |
| |
| Limiting Distributions | |
| |
| |
| Summary | |
| |
| |
| Problems | |
| |
| |
| |
| Finite Non-Absorbing Chains | |
| |
| |
| Non-Absorbing Chains | |
| |
| |
| The Taxicab Example | |
| |
| |
| Limiting Probabilities | |
| |
| |
| Long-term Visiting Rates | |
| |
| |
| Expected First Reaching Times | |
| |
| |
| Expected Return Times | |
| |
| |
| Regenerative Property | |
| |
| |
| Periodic Chains | |
| |
| |
| Finite Reducible Chains | |
| |
| |
| Transient Behavior | |
| |
| |
| Class-Absorption Probabilities | |
| |
| |
| Long-Term Visiting Rate | |
| |
| |
| Limiting Distribution | |
| |
| |
| Summary | |
| |
| |
| Problems | |
| |
| |
| |
| Infinite Chains | |
| |
| |
| Infinite Random Walks | |
| |
| |
| Infinite Chains | |
| |
| |
| Infinite Absorbing Chains | |
| |
| |
| Infinite Irreducible Chains | |
| |
| |
| Positive-Recurrence | |
| |
| |
| Null-Recurrence | |
| |
| |
| Summary | |
| |
| |
| Problems | |
| |
| |
| |
| Poisson Streams Of Events | |
| |
| |
| Continuous-Time Chains | |
| |
| |
| Markovian Streams | |
| |
| |
| Regular Streams | |
| |
| |
| Poisson Streams | |
| |
| |
| Instantaneous and Average Rates | |
| |
| |
| Rates | |
| |
| |
| Remaining Lives | |
| |
| |
| Exponential and Hyperexponential Distributions | |
| |
| |
| Lives | |
| |
| |
| The Shorter Remaining Life | |
| |
| |
| Poisson (Counting) Process | |
| |
| |
| Erlang Distributions | |
| |
| |
| Random Streams | |
| |
| |
| Superposition of Two Poisson Streams | |
| |
| |
| Decomposition of a Poisson Stream | |
| |
| |
| Compound Poisson Process | |
| |
| |
| Summary | |
| |
| |
| Problems | |
| |
| |
| |
| Renewal Streams Of Events | |
| |
| |
| Renewal Streams and Renewal (Counting) Processes | |
| |
| |
| Elementary Renewal Theorem | |
| |
| |
| Renewal Rates | |
| |
| |
| Renewal-Reward Processes | |
| |
| |
| Gradual-Reward Processes | |
| |
| |
| Regenerative Processes | |
| |
| |
| Remaining Lives | |
| |
| |
| Lattice Random Variables | |
| |
| |
| Spread-Out Random Variables | |
| |
| |
| Limiting Distributions | |
| |
| |
| Summary | |
| |
| |
| Problems | |
| |
| |
| |
| Semi-Markov Chains | |
| |
| |
| Semi-Markov Chains | |
| |
| |
| The Rat Maze Example | |
| |
| |
| A Marketing Example | |
| |
| |
| Holding Times | |
| |
| |
| Sojourn Times | |
| |
| |
| Classification of States | |
| |
| |
| Absorbing Semi-Markov Chains | |
| |
| |
| Irreducible Semi-Markov Chains | |
| |
| |
| Expected First Reaching Times | |
| |
| |
| Expected Return Times | |
| |
| |
| Visiting Rates | |
| |
| |
| Limiting Results | |
| |
| |
| Summary | |
| |
| |
| Problems | |
| |
| |
| |
| Continuous-Time Markov Chains | |
| |
| |
| Continuous-Time Markov Chain | |
| |
| |
| Memoryless Property | |
| |
| |
| Transition Rates | |
| |
| |
| Transition Diagrams | |
| |
| |
| The Q-Matrix | |
| |
| |
| Equally-Spaced Discrete-Time Chains | |
| |
| |
| Why Th | |