| |
| |
| Preface | |
| |
| |
| |
| Stochastic Processes: Description and Definition | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Description and Definition | |
| |
| |
| |
| Probability Distributions | |
| |
| |
| |
| The Markov Process | |
| |
| |
| |
| The Renewal Process | |
| |
| |
| |
| The Stationary Process | |
| |
| |
| |
| A Plan for the Remaining Chapters | |
| |
| |
| References | |
| |
| |
| Exercises | |
| |
| |
| Elementary Review Exercises | |
| |
| |
| Advanced Review Exercises | |
| |
| |
| |
| Markov Chains | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| The n-Step Transition Probability Matrix | |
| |
| |
| |
| Classification of States | |
| |
| |
| |
| A Canonical Representation of the Transition Probability Matrix | |
| |
| |
| |
| Classification of States in Practice | |
| |
| |
| |
| Finite Markov Chains with Transient States | |
| |
| |
| References | |
| |
| |
| Exercises | |
| |
| |
| |
| Irreducible Markov Chains with Ergodic States | |
| |
| |
| |
| Transient Behavior | |
| |
| |
| |
| Limiting Behavior | |
| |
| |
| |
| First Passage and Related Results | |
| |
| |
| References | |
| |
| |
| Exercises | |
| |
| |
| |
| Branching Processes and other Special Topics | |
| |
| |
| |
| Branching Processes | |
| |
| |
| |
| Markov Chains of Order Higher than 1 | |
| |
| |
| |
| Lumpable Markov Chains | |
| |
| |
| |
| Reversed Markov Chains | |
| |
| |
| References | |
| |
| |
| Exercises | |
| |
| |
| |
| Statistical Inference for Markov Chains | |
| |
| |
| |
| Estimation of the Elements in a Transition Probability Matrix | |
| |
| |
| |
| Hypothesis Testing Issues for Markov Chains | |
| |
| |
| |
| Inference From Partially Observable Markov Chains | |
| |
| |
| |
| Statistical Inference for Branching Processes | |
| |
| |
| |
| Additional Comments | |
| |
| |
| References | |
| |
| |
| Exercises | |
| |
| |
| |
| Applied Markov Chains | |
| |
| |
| |
| Queueing Models | |
| |
| |
| |
| Inventory Systems | |
| |
| |
| |
| Storage Models | |
| |
| |
| |
| Industrial Mobility of Labor | |
| |
| |
| |
| Educational Advancement | |
| |
| |
| |
| Human Resource Management | |
| |
| |
| |
| Term Structure | |
| |
| |
| |
| Income Determination under Uncertainty | |
| |
| |
| |
| A Markov Decision Process | |
| |
| |
| References | |
| |
| |
| |
| Simple Markov Processes | |
| |
| |
| |
| Examples | |
| |
| |
| |
| Markov Processes: General Properties | |
| |
| |
| |
| The Poisson Process | |
| |
| |
| |
| The Pure Birth Process | |
| |
| |
| |
| The Pure Death Process | |
| |
| |
| |
| Birth and Death Processes | |
| |
| |
| |
| Limiting Distributions | |
| |
| |
| |
| Markovian Networks | |
| |
| |
| |
| Additional Examples | |
| |
| |
| References | |
| |
| |
| Exercises | |
| |
| |
| |
| Statistical Inference for Simple Markov Processes | |
| |
| |
| |
| Estimation of Parameters | |
| |
| |
| |
| Hypothesis Testing for Simple Markov Processes | |
| |
| |
| |
| Statistical Inference for Queues | |
| |
| |
| |
| Additional Examples | |
| |
| |
| References | |
| |
| |
| Exercises | |
| |
| |
| |
| Applied Markov Processes | |
| |
| |
| |
| Queueing Models | |
| |
| |
| |
| The Machine Interference Problem | |
| |
| |
| |
| Queueing Networks | |
| |
| |
| |
| Flexible Manufacturing Systems | |
| |
| |
| |
| Inventory Systems | |
| |
| |
| |
| Reliability Models | |
| |
| |
| |
| Markovian Combat Models | |
| |
| |
| |
| Stochastic Models for Social Networks | |
| |
| |
| |
| Recovery, Relapse, and Death Due to Disease | |
| |
| |
| References | |
| |
| |
| |
| Renewal Processes | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Renewal Processes when Time is Discrete | |
| |
| |
| |
| Renewal Processes when Time is Continuous | |
| |
| |
| |
| Alternating Renewal Processes | |
| |
| |
| |
| Markov Renewal Processes (Semi-Markov Processes) | |
| |
| |
| |
| Renewal Reward Processes | |
| |
| |
| |
| Statistical Inference for Renewal Processes | |
| |
| |
| |
| Additional Examples | |
| |
| |
| References | |
| |
| |
| Exercises | |
| |
| |
| |
| Stationary Processes and Time Series Analysis | |
| |
| |
| |
| Definition | |
| |
| |
| |
| Some Examples | |
| |
| |
| |
| Ergodic Theorems | |
| |
| |
| |
| Covariance Stationary Processes in the Frequency Domain | |
| |
| |
| |
| Time Series Analysis: Introduction | |
| |
| |
| |
| Stochastic Models for Time Series | |
| |
| |
| |
| The Autoregressive Process | |
| |
| |
| |
| The Moving Average Process | |
| |
| |
| |
| A Mixed Autoregressive Moving Average Process | |
| |
| |
| |
| Autoregressive Integrated Moving Average Processes | |
| |
| |
| |
| Time Series Analysis in the Time Domain | |
| |
| |
| |
| Spectral Analysis of Time Series Data | |
| |
| |
| References | |
| |
| |
| Exercises | |
| |
| |
| |
| Simulation and Markov Chain Monte Carlo | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Simulation | |
| |
| |
| |
| Markov Chain Monte Carlo | |
| |
| |
| References | |
| |
| |
| Answers to Selected Exercises | |
| |
| |
| Appendix | |
| |
| |
| Author Index | |
| |
| |
| Subject Index | |