| |
| |
| Introduction | |
| |
| |
| |
| Review of Continuous Time Models | |
| |
| |
| |
| Martingales and Martingale Inequalities | |
| |
| |
| |
| Stochastic Integration | |
| |
| |
| |
| Stochastic Differential Equations: Diffusions | |
| |
| |
| |
| Reflected Diffusions | |
| |
| |
| |
| Processes with Jumps | |
| |
| |
| |
| Controlled Markov Chains | |
| |
| |
| |
| Recursive Equations for the Cost | |
| |
| |
| |
| Optimal Stopping Problems | |
| |
| |
| |
| Discounted Cost | |
| |
| |
| |
| Control to a Target Set and Contraction Mappings | |
| |
| |
| |
| Finite Time Control Problems | |
| |
| |
| |
| Dynamic Programming Equations | |
| |
| |
| |
| Functionals of Uncontrolled Processes | |
| |
| |
| |
| The Optimal Stopping Problem | |
| |
| |
| |
| Control Until a Target Set Is Reached | |
| |
| |
| |
| A Discounted Problem with a Target Set and Reflection | |
| |
| |
| |
| Average Cost Per Unit Time | |
| |
| |
| |
| Markov Chain Approximation Method: Introduction | |
| |
| |
| |
| Markov Chain Approximation | |
| |
| |
| |
| Continuous Time Interpolation | |
| |
| |
| |
| A Markov Chain Interpolation | |
| |
| |
| |
| A Random Walk Approximation | |
| |
| |
| |
| A Deterministic Discounted Problem | |
| |
| |
| |
| Deterministic Relaxed Controls | |
| |
| |
| |
| Construction of the Approximating Markov Chains | |
| |
| |
| |
| One Dimensional Examples | |
| |
| |
| |
| Numerical Simplifications | |
| |
| |
| |
| The General Finite Difference Method | |
| |
| |
| |
| A Direct Construction | |
| |
| |
| |
| Variable Grids | |
| |
| |
| |
| Jump Diffusion Processes | |
| |
| |
| |
| Reflecting Boundaries | |
| |
| |
| |
| Dynamic Programming Equations | |
| |
| |
| |
| Controlled and State Dependent Variance | |
| |
| |
| |
| Computational Methods for Controlled Markov Chains | |
| |
| |
| |
| The Problem Formulation | |
| |
| |
| |
| Classical Iterative Methods | |
| |
| |
| |
| Error Bounds | |
| |
| |
| |
| Accelerated Jacobi and Gauss-Seidel Methods | |
| |
| |
| |
| Domain Decomposition | |
| |
| |
| |
| Coarse Grid-Fine Grid Solutions | |
| |
| |
| |
| A Multigrid Method | |
| |
| |
| |
| Linear Programming | |
| |
| |
| |
| The Ergodic Cost Problem: Formulation and Algorithms | |
| |
| |
| |
| Formulation of the Control Problem | |
| |
| |
| |
| A Jacobi Type Iteration | |
| |
| |
| |
| Approximation in Policy Space | |
| |
| |
| |
| Numerical Methods | |
| |
| |
| |
| The Control Problem | |
| |
| |
| |
| The Interpolated Process | |
| |
| |
| |
| Computations | |
| |
| |
| |
| Boundary Costs and Controls | |
| |
| |
| |
| Heavy Traffic and Singular Control | |
| |
| |
| |
| Motivating Examples | |
| |
| |
| |
| The Heavy Traffic Problem | |
| |
| |
| |
| Singular Control | |
| |
| |
| |
| Weak Convergence and the Characterization of Processes | |
| |
| |
| |
| W | |