Approximate Dynamic Programming Solving the Curses of Dimensionality

ISBN-10: 047060445X
ISBN-13: 9780470604458
Edition: 2nd 2011
Authors: Warren B. Powell
List price: $145.00
30 day, 100% satisfaction guarantee

If an item you ordered from TextbookRush does not meet your expectations due to an error on our part, simply fill out a return request and then return it by mail within 30 days of ordering it for a full refund of item cost.

Learn more about our returns policy

Description: Understanding approximate dynamic programming (ADP) in large industrial settings helps develop practical and high-quality solutions to problems that involve making decisions in the presence of uncertainty. With a focus on modeling and algorithms in  More...

what's this?
Rush Rewards U
Members Receive:
coins
coins
You have reached 400 XP and carrot coins. That is the daily max!
You could win $10,000

Get an entry for every item you buy, rent, or sell.

Study Briefs

Limited time offer: Get the first one free! (?)

All the information you need in one place! Each Study Brief is a summary of one specific subject; facts, figures, and explanations to help you learn faster.

Add to cart
Study Briefs
Italian Grammar Online content $4.95 $1.99
Add to cart
Study Briefs
Portuguese Grammar Online content $4.95 $1.99
Add to cart
Study Briefs
Spanish Grammar Online content $4.95 $1.99
Add to cart
Study Briefs
German Grammar Online content $4.95 $1.99

Customers also bought

Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading

Book details

List price: $145.00
Edition: 2nd
Copyright year: 2011
Publisher: John Wiley & Sons, Incorporated
Publication date: 9/27/2011
Binding: Hardcover
Pages: 656
Size: 6.50" wide x 9.50" long x 1.75" tall
Weight: 2.618
Language: English

Understanding approximate dynamic programming (ADP) in large industrial settings helps develop practical and high-quality solutions to problems that involve making decisions in the presence of uncertainty. With a focus on modeling and algorithms in conjunction with the language of mainstream operations research, artificial intelligence, and control theory, this second edition of Approximate Dynamic Programming Solving the Curses of Dimensionality uniquely integrates four distinct disciplines#xC2;#x14;Markov design processes, mathematical programming, simulation, and statistics#xC2;#x14;to show students, practitioners, and researchers how to successfully model and solve a wide range of real-life problems using ADP.

Preface to the Second Edition
Preface to the First Edition
Acknowledgments
The Challenges of Dynamic Programming
A Dynamic Programming Example: A Shortest Path Problem
The Three Curses of Dimensionality
Some Real Applications
Problem Classes
The Many Dialects of Dynamic Programming
What Is New in This Book?
Pedagogy
Bibliographic Notes
Some Illustrative Models
Deterministic Problems
Stochastic Problems
Information Acquisition Problems
A Simple Modeling Framework for Dynamic Programs
Bibliographic Notes
Problems
Introduction to Markov Decision Processes
The Optimality Equations
Finite Horizon Problems
Infinite Horizon Problems
Value Iteration
Policy Iteration
Hybrid Value-Policy Iteration
Average Reward Dynamic Programming
The Linear Programming Method for Dynamic Programs
Monotone Policies*
Why Does It Work?**
Bibliographic Notes
Problems
Introduction to Approximate Dynamic Programming
The Three Curses of Dimensionality (Revisited)
The Basic Idea
Q-Learning and SARSA
Real-Time Dynamic Programming
Approximate Value Iteration
The Post-Decision State Variable
Low-Dimensional Representations of Value Functions
So Just What Is Approximate Dynamic Programming?
Experimental Issues
But Does It Work?
Bibliographic Notes
Problems
Modeling Dynamic Programs
Notational Style
Modeling Time
Modeling Resources
The States of Our System
Modeling Decisions
The Exogenous Information Process
The Transition Function
The Objective Function
A Measure-Theoretic View of Information**
Bibliographic Notes
Problems
Policies
Myopic Policies
Lookahead Policies
Policy Function Approximations
Value Function Approximations
Hybrid Strategies
Randomized Policies
How to Choose a Policy?
Bibliographic Notes
Problems
Policy Search
Background
Gradient Search
Direct Policy Search for Finite Alternatives
The Knowledge Gradient Algorithm for Discrete Alternatives
Simulation Optimization
Why Does It Work?**
Bibliographic Notes
Problems
Approximating Value Functions
Lookup Tables and Aggregation
Parametric Models
Regression Variations
Nonparametric Models
Approximations and the Curse of Dimensionality
Why Does It Work?**
Bibliographic Notes
Problems
Learning Value Function Approximations
Sampling the Value of a Policy
Stochastic Approximation Methods
Recursive Least Squares for Linear Models
Temporal Difference Learning with a Linear Model
Bellman's Equation Using a Linear Model
Analysis of TD(0), LSTD, and LSPE Using a Single State
Gradient-Based Methods for Approximate Value Iteration*
Least Squares Temporal Differencing with Kernel Regression*
Value Function Approximations Based on Bayesian Learning*
Why Does It Work*
Bibliographic Notes
Problems
Optimizing While Learning
Overview of Algorithmic Strategies
Approximate Value Iteration and Q-Learning Using Lookup Tables
Statistical Bias in the Max Operator
Approximate Value Iteration and Q-Learning Using Linear Models
Approximate Policy Iteration
The Actor-Critic Paradigm
Policy Gradient Methods
The Linear Programming Method Using Basis Functions
Approximate Policy Iteration Using Kernel Regression*
Finite Horizon Approximations for Steady-State Applications
Bibliographic Notes
Problems
Adaptive Estimation and Stepsizes
Learning Algorithms and Stepsizes
Deterministic Stepsize Recipes
Stochastic Stepsizes
Optimal Stepsizes for Nonstationary Time Series
Optimal Stepsizes for Approximate Value Iteration
Convergence
Guidelines for Choosing Stepsize Formulas
Bibliographic Notes
Problems
Exploration Versus Exploitation
A Learning Exercise: The Nomadic Trucker
An Introduction to Learning
Heuristic Learning Policies
Gittins Indexes for Online Learning
The Knowledge Gradient Policy
Learning with a Physical State
Bibliographic Notes
Problems
Value Function Approximations for Resource Allocation Problems
Value Functions versus Gradients
Linear Approximations
Piecewise-Linear Approximations
Solving a Resource Allocation Problem Using Piecewise-Linear Functions
The SHAPE Algorithm
Regression Methods
Cutting Planes*
Why Does It Work?**
Bibliographic Notes
Problems
Dynamic Resource Allocation Problems
An Asset Acquisition Problem
The Blood Management Problem
A Portfolio Optimization Problem
A General Resource Allocation Problem
A Fleet Management Problem
A Driver Management Problem
Bibliographic Notes
Problems
Implementation Challenges
Will ADP Work for Your Problem?
Designing an ADP Algorithm for Complex Problems
Debugging an ADP Algorithm
Practical Issues
Modeling Your Problem
Online versus Offline Models
If It Works, Patent It!
Bibliography
Index

×
Free shipping on orders over $35*

*A minimum purchase of $35 is required. Shipping is provided via FedEx SmartPost® and FedEx Express Saver®. Average delivery time is 1 – 5 business days, but is not guaranteed in that timeframe. Also allow 1 - 2 days for processing. Free shipping is eligible only in the continental United States and excludes Hawaii, Alaska and Puerto Rico. FedEx service marks used by permission."Marketplace" orders are not eligible for free or discounted shipping.

Learn more about the TextbookRush Marketplace.

×