| |
| |
List of figures | |
| |
| |
Prologue to the Second edition, Peter W. Glynn | |
| |
| |
Preface to the second edition, Sean Meyn | |
| |
| |
Preface to the first edition | |
| |
| |
| |
Communication and Regeneration | |
| |
| |
| |
Heuristics | |
| |
| |
| |
A range of Markovian environments | |
| |
| |
| |
Basic models in practice | |
| |
| |
| |
Stochastic stability for Markov models | |
| |
| |
| |
Commentary | |
| |
| |
| |
Markov models | |
| |
| |
| |
Markov models in time series | |
| |
| |
| |
Nonlinear state space models | |
| |
| |
| |
Models in control and systems theory | |
| |
| |
| |
Markov models with regeneration times | |
| |
| |
| |
Commentary | |
| |
| |
| |
Transition probabilities | |
| |
| |
| |
Defining a Markovian Process | |
| |
| |
| |
Foundations on a countable space | |
| |
| |
| |
Specific transition matrices | |
| |
| |
| |
Foundations for general state space chains | |
| |
| |
| |
Building transition kernels for specific models | |
| |
| |
| |
Commentary | |
| |
| |
| |
Irreducibility | |
| |
| |
| |
Communication and irreducibility: Countable spaces | |
| |
| |
| |
�-Irreducibility | |
| |
| |
| |
�-Irreducibility for random walk models | |
| |
| |
| |
�-Irreducible linear models | |
| |
| |
| |
Commentary | |
| |
| |
| |
Pseudo-atoms | |
| |
| |
| |
Splitting �-irreducible chains | |
| |
| |
| |
Small sets | |
| |
| |
| |
Small sets for specific models | |
| |
| |
| |
Cyclic behavior | |
| |
| |
| |
Petite sets and sampled chains | |
| |
| |
| |
Commentary | |
| |
| |
| |
Topology and continuity | |
| |
| |
| |
Feller properties and forms of stability | |
| |
| |
| |
T-chains | |
| |
| |
| |
Continuous components for specific models | |
| |
| |
| |
e-Chains | |
| |
| |
| |
Commentary | |
| |
| |
| |
The nonlinear state space model | |
| |
| |
| |
Forward accessibility and continuous components | |
| |
| |
| |
Minimal sets and irreducibility | |
| |
| |
| |
Periodicity for nonlinear state space models | |
| |
| |
| |
Forward accessible examples | |
| |
| |
| |
Equicontinuity and the nonlinear state space model | |
| |
| |
| |
Commentary | |
| |
| |
| |
Stability Structures | |
| |
| |
| |
Transience and recurrence | |
| |
| |
| |
Classifying chains on countable spaces | |
| |
| |
| |
Classifying �-irreducible chains | |
| |
| |
| |
Recurrence and transience relationships | |
| |
| |
| |
Classification using drift criteria | |
| |
| |
| |
Classifying random walk on R+ | |
| |
| |
| |
Commentary | |
| |
| |
| |
Harris and topological recurrence | |
| |
| |
| |
Harris recurrence | |
| |
| |
| |
Non-evanescent and recurrent chains | |
| |
| |
| |
Topologically recurrent and transient states | |
| |
| |
| |
Criteria for stability on a topological space | |
| |
| |
| |
Stochastic comparison and increment analysis | |
| |
| |
| |
Commentary | |
| |
| |
| |
The existence of � | |
| |
| |
| |
Stationarity and invariance | |
| |
| |
| |
The existence of �: chains with atoms | |
| |
| |
| |
Invariant measures for countable space models | |
| |
| |
| |
The existence of �: �-irreducible chains | |
| |
| |
| |
Invariant measures for general models | |
| |
| |
| |
Commentary | |
| |
| |
| |
Drift and regularity | |
| |
| |
| |
Regular chains | |
| |
| |
| |
Drift, hitting times and deterministic models | |
| |
| |
| |
Drift, criteria for regularity | |
| |
| |
| |
Using the regularity criteria | |
| |
| |
| |
Evaluating non-positivity | |
| |
| |
| |
Commentary | |
| |
| |
| |
Invariance and tightness | |
| |
| |
| |
Chains bounded in probability | |
| |
| |
| |
Generalized sampling and invariant measures | |
| |
| |
| |
The existence of a �-finite invariant measure | |
| |
| |
| |
Invariant measures for e-chains | |
| |
| |
| |
Establishing boundedness in probability | |
| |
| |
| |
Commentary | |
| |
| |
| |
Convergence | |
| |
| |
| |
Ergodicity | |
| |
| |
| |
Ergodic chains on countable spaces | |
| |
| |
| |
Renewal and regeneration | |
| |
| |
| |
Ergodicity of positive Harris chains | |
| |
| |
| |
Sums of transition probabilities | |
| |
| |
| |
Commentary | |
| |
| |
| |
f-Ergodicity and f-regularity | |
| |
| |
| |
f-Properties: chains with atoms | |
| |
| |
| |
f-Regularity and drift | |
| |
| |
| |
f-Ergodicity for general chains | |
| |
| |
| |
f-Ergodicity of specific models | |
| |
| |
| |
A key renewal theorem | |
| |
| |
| |
Commentary | |
| |
| |
| |
Geometric ergodicity | |
| |
| |
| |
Geometric properties: chains with atoms | |
| |
| |
| |
Kendall sets and drift criteria | |
| |
| |
| |
f-Geometric regularity of � and its skeleton | |
| |
| |
| |
f-Geometric ergodicity for general chains | |
| |
| |
| |
Simple random walk and linear models | |
| |
| |
| |
Commentary | |
| |
| |
| |
V-Uniform ergodicity | |
| |
| |
| |
Operator norm convergence | |
| |
| |
| |
Uniform ergodicity | |
| |
| |
| |
Geometric ergodicity and increment analysis | |
| |
| |
| |
Models from queueing theory | |
| |
| |
| |
Autoregressive and state space models | |
| |
| |
| |
Commentary | |
| |
| |
| |
Sample paths and limit theorems | |
| |
| |
| |
Invariant �-fields and the LLN | |
| |
| |
| |
Ergodic theorems for chains possessing an atom | |
| |
| |
| |
General Harris chains | |
| |
| |
| |
The functional CLT | |
| |
| |
| |
Criteria for the CLT and the LIL | |
| |
| |
| |
Applications | |
| |
| |
| |
Commentary | |
| |
| |
| |
Positivity | |
| |
| |
| |
Null recurrent chains | |
| |
| |
| |
Characterizing positivity using Pn | |
| |
| |
| |
Positivity and T-chains | |
| |
| |
| |
Positivity and e-chains | |
| |
| |
| |
The LLN for e-chains | |
| |
| |
| |
Commentary | |
| |
| |
| |
Generalized classification criteria | |
| |
| |
| |
State-dependent drifts | |
| |
| |
| |
History-dependent drift criteria | |
| |
| |
| |
Mixed drift conditions | |
| |
| |
| |
Commentary | |
| |
| |
| |
Epilogue to the second edition | |
| |
| |
| |
Geometric ergodicity and spectral theory | |
| |
| |
| |
Simulation and MCMC | |
| |
| |
| |
Continuous time models | |
| |
| |
| |
Appendices | |
| |
| |
| |
Mud maps | |
| |
| |
| |
Recurrence versus transience | |
| |
| |
| |
Positivity versus nullity | |
| |
| |
| |
Convergence properties | |
| |
| |
| |
Testing for stability | |
| |
| |
| |
Glossary of drift conditions | |
| |
| |
| |
The Scalar SETAR model: a complete classification | |
| |
| |
| |
Glossary of models assumptions | |
| |
| |
| |
Regenerative models | |
| |
| |
| |
State space models | |
| |
| |
| |
Some mathematical background | |
| |
| |
| |
Some measure theory | |
| |
| |
| |
Some probability theory | |
| |
| |
| |
Some topology | |
| |
| |
| |
Some real analysis | |
| |
| |
| |
Convergence concepts for measures | |
| |
| |
| |
Some martingale theory | |
| |
| |
| |
Some results on sequences and numbers | |
| |
| |
Bibliography | |
| |
| |
Indexes | |
| |
| |
General index | |
| |
| |
Symbols | |