| |
| |
| List of figures | |
| |
| |
| Prologue to the Second edition, Peter W. Glynn | |
| |
| |
| Preface to the second edition, Sean Meyn | |
| |
| |
| Preface to the first edition | |
| |
| |
| |
| Communication and Regeneration | |
| |
| |
| |
| Heuristics | |
| |
| |
| |
| A range of Markovian environments | |
| |
| |
| |
| Basic models in practice | |
| |
| |
| |
| Stochastic stability for Markov models | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Markov models | |
| |
| |
| |
| Markov models in time series | |
| |
| |
| |
| Nonlinear state space models | |
| |
| |
| |
| Models in control and systems theory | |
| |
| |
| |
| Markov models with regeneration times | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Transition probabilities | |
| |
| |
| |
| Defining a Markovian Process | |
| |
| |
| |
| Foundations on a countable space | |
| |
| |
| |
| Specific transition matrices | |
| |
| |
| |
| Foundations for general state space chains | |
| |
| |
| |
| Building transition kernels for specific models | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Irreducibility | |
| |
| |
| |
| Communication and irreducibility: Countable spaces | |
| |
| |
| |
| �-Irreducibility | |
| |
| |
| |
| �-Irreducibility for random walk models | |
| |
| |
| |
| �-Irreducible linear models | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Pseudo-atoms | |
| |
| |
| |
| Splitting �-irreducible chains | |
| |
| |
| |
| Small sets | |
| |
| |
| |
| Small sets for specific models | |
| |
| |
| |
| Cyclic behavior | |
| |
| |
| |
| Petite sets and sampled chains | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Topology and continuity | |
| |
| |
| |
| Feller properties and forms of stability | |
| |
| |
| |
| T-chains | |
| |
| |
| |
| Continuous components for specific models | |
| |
| |
| |
| e-Chains | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| The nonlinear state space model | |
| |
| |
| |
| Forward accessibility and continuous components | |
| |
| |
| |
| Minimal sets and irreducibility | |
| |
| |
| |
| Periodicity for nonlinear state space models | |
| |
| |
| |
| Forward accessible examples | |
| |
| |
| |
| Equicontinuity and the nonlinear state space model | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Stability Structures | |
| |
| |
| |
| Transience and recurrence | |
| |
| |
| |
| Classifying chains on countable spaces | |
| |
| |
| |
| Classifying �-irreducible chains | |
| |
| |
| |
| Recurrence and transience relationships | |
| |
| |
| |
| Classification using drift criteria | |
| |
| |
| |
| Classifying random walk on R+ | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Harris and topological recurrence | |
| |
| |
| |
| Harris recurrence | |
| |
| |
| |
| Non-evanescent and recurrent chains | |
| |
| |
| |
| Topologically recurrent and transient states | |
| |
| |
| |
| Criteria for stability on a topological space | |
| |
| |
| |
| Stochastic comparison and increment analysis | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| The existence of � | |
| |
| |
| |
| Stationarity and invariance | |
| |
| |
| |
| The existence of �: chains with atoms | |
| |
| |
| |
| Invariant measures for countable space models | |
| |
| |
| |
| The existence of �: �-irreducible chains | |
| |
| |
| |
| Invariant measures for general models | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Drift and regularity | |
| |
| |
| |
| Regular chains | |
| |
| |
| |
| Drift, hitting times and deterministic models | |
| |
| |
| |
| Drift, criteria for regularity | |
| |
| |
| |
| Using the regularity criteria | |
| |
| |
| |
| Evaluating non-positivity | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Invariance and tightness | |
| |
| |
| |
| Chains bounded in probability | |
| |
| |
| |
| Generalized sampling and invariant measures | |
| |
| |
| |
| The existence of a �-finite invariant measure | |
| |
| |
| |
| Invariant measures for e-chains | |
| |
| |
| |
| Establishing boundedness in probability | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Convergence | |
| |
| |
| |
| Ergodicity | |
| |
| |
| |
| Ergodic chains on countable spaces | |
| |
| |
| |
| Renewal and regeneration | |
| |
| |
| |
| Ergodicity of positive Harris chains | |
| |
| |
| |
| Sums of transition probabilities | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| f-Ergodicity and f-regularity | |
| |
| |
| |
| f-Properties: chains with atoms | |
| |
| |
| |
| f-Regularity and drift | |
| |
| |
| |
| f-Ergodicity for general chains | |
| |
| |
| |
| f-Ergodicity of specific models | |
| |
| |
| |
| A key renewal theorem | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Geometric ergodicity | |
| |
| |
| |
| Geometric properties: chains with atoms | |
| |
| |
| |
| Kendall sets and drift criteria | |
| |
| |
| |
| f-Geometric regularity of � and its skeleton | |
| |
| |
| |
| f-Geometric ergodicity for general chains | |
| |
| |
| |
| Simple random walk and linear models | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| V-Uniform ergodicity | |
| |
| |
| |
| Operator norm convergence | |
| |
| |
| |
| Uniform ergodicity | |
| |
| |
| |
| Geometric ergodicity and increment analysis | |
| |
| |
| |
| Models from queueing theory | |
| |
| |
| |
| Autoregressive and state space models | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Sample paths and limit theorems | |
| |
| |
| |
| Invariant �-fields and the LLN | |
| |
| |
| |
| Ergodic theorems for chains possessing an atom | |
| |
| |
| |
| General Harris chains | |
| |
| |
| |
| The functional CLT | |
| |
| |
| |
| Criteria for the CLT and the LIL | |
| |
| |
| |
| Applications | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Positivity | |
| |
| |
| |
| Null recurrent chains | |
| |
| |
| |
| Characterizing positivity using Pn | |
| |
| |
| |
| Positivity and T-chains | |
| |
| |
| |
| Positivity and e-chains | |
| |
| |
| |
| The LLN for e-chains | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Generalized classification criteria | |
| |
| |
| |
| State-dependent drifts | |
| |
| |
| |
| History-dependent drift criteria | |
| |
| |
| |
| Mixed drift conditions | |
| |
| |
| |
| Commentary | |
| |
| |
| |
| Epilogue to the second edition | |
| |
| |
| |
| Geometric ergodicity and spectral theory | |
| |
| |
| |
| Simulation and MCMC | |
| |
| |
| |
| Continuous time models | |
| |
| |
| |
| Appendices | |
| |
| |
| |
| Mud maps | |
| |
| |
| |
| Recurrence versus transience | |
| |
| |
| |
| Positivity versus nullity | |
| |
| |
| |
| Convergence properties | |
| |
| |
| |
| Testing for stability | |
| |
| |
| |
| Glossary of drift conditions | |
| |
| |
| |
| The Scalar SETAR model: a complete classification | |
| |
| |
| |
| Glossary of models assumptions | |
| |
| |
| |
| Regenerative models | |
| |
| |
| |
| State space models | |
| |
| |
| |
| Some mathematical background | |
| |
| |
| |
| Some measure theory | |
| |
| |
| |
| Some probability theory | |
| |
| |
| |
| Some topology | |
| |
| |
| |
| Some real analysis | |
| |
| |
| |
| Convergence concepts for measures | |
| |
| |
| |
| Some martingale theory | |
| |
| |
| |
| Some results on sequences and numbers | |
| |
| |
| Bibliography | |
| |
| |
| Indexes | |
| |
| |
| General index | |
| |
| |
| Symbols | |