Dynamic programming and Markov processes

✍ Scribed by Ronald A. Howard

• Publisher: Technology Press of Massachusetts Institute of Technology
• Year: 1960
• Tongue: English
• Leaves: 148
• Category: Library

✦ Table of Contents

Front cover
Dynamic Programming and Markov Processes
Copyright
Preface
Contents
Introduction
1 Markov Processes
The Toymaker Example-State Probabilities
The z- Transformation
z- Transform Analysis of Markov Processes
Transient, Multichain; and Periodic Behavior
2 Markov Processes with Rewards
Solution by Recurrence Relation
The Toymaker Example
z-Transform Analysis of the Markov Process with Rewards
Asymptotic Behavior
3 The Solution of the Sequential Decision Process by Value Iteration
Introduction of Alternatives
The Toymaker's Problem Solved by Value Iteration
Evaluation of the Value-Iteration Approach
4 The Policy-Iteration Method for the Solution of Sequential Decision Processes
The Value-Determination Operation
The Policy-Improvement Routine
The Iteration Cycle
The Toymaker's Problem
A Proof of the Properties of the Policy-Iteration Method
5 Use of the Policy-Iteration Method in Problems of Taxicab Operation, Baseball, and Automobile Replacement
An Example-Taxicab Operation
A Baseball Problem
The Policy-Improvement Routine
6 The Policy-Iteration Method for Multiple-Chain Processes
The Value-Determination Operation
The Policy-Improvement Routine
A Multichain Example
Properties of the Iteration Cycle
7 The Sequential Decision Process with Discounting
The Sequential Decision Process with Discounting Solved by Value Iteration
The Value-Determination Operation
The Policy-Improvement Routine
An Example
Proof of the Properties of the Iteration Cycle
The Sensitivity of the Optimal Policy to the Discount Factor
The Automobile Problem with Discounting
Summary
8 The Continuous- Time Decision Process
The Continuous-Time Markov Process
The Solution of Continuous-Time Markov Processes by Laplace Transformation
The Continuous-Time Markov Process with Rewards
The Continuous-Time Decision Problem
The Value-Determination Operation
The Policy-Improvement Routine
Completely Ergodic Processes
The Foreman's Dilemma
Computational Considerations
The Continuous-Time Decision Process with Discounting
Policy Improvement
An Example
Comparison with Discrete-Time Case
9 Conclusion
Appendix: The Relationship of Transient to Recurrent Behavior
References
General References
Index
Back cover