✍ Scribed by Bertsekas (editor)
No coin nor oath required. For personal study only.
This text evolved from an introductory course on optimization under uncertainty that I taught at Stanford University in the spring of 1973 and at the University of Illinois in the fall of 1974. It is aimed at graduate students and practicing analysts in engineering, operations research, economics, statistics, and business administration. As a textbook it could be used, for example, in a one-semester first-year graduate course, which could cover primarily the first five chapters, the first half of Chapter 6, and parts of Chapter 8. It could also be used in a two-quarter graduate course, which would probably cover the whole text. Depending on the students’ backgrounds and interests, some material could be omitted or added by the instructor.
Front Cover
Dynamic Programming and Stochastic Control
Copyright Page
Contents
Preface
Acknowledgments
Chapter 1. Introduction
1.1 The Problems of Decision under Uncertainty
1.2 Expected Utility Theory and Risk
1.3 Some Nonsequential Decision Problems
1.4 A Model for Sequential Decision Making
1.5 Notes
Problems
Part I: CONTROL OF UNCERTAIN SYSTEMS OVER A FINITE HORIZON
Chapter 2. The Dynamic Programming Algorithm
2.1 The Basic Problem
2.2 The Dynamic Programming Algorithm
2.3 Time Lags, Correlated Disturbances, and Forecasts
2.4 Notes
Problems
Chapter 3. Applications in Specific Areas
3.1 Linear Systems with Quadratic Cost Functional—The Certainty Equivalence Principle
3.2 Inventory Control
3.3 Dynamic Portfolio Analysis
3.4 Optimal Stopping Problems—Examples
3.5 Notes
Problems
Chapter 4. Problems with Imperfect State Information
4.1 Reduction to the Perfect State Information Case
4.2 Sufficient Statistics
4.3 Linear Systems with Quadratic Cost Functionals—Separation of Estimation and Control
4.4 Finite State Markov Chains—A Problem of Instruction
4.5 Hypothesis Testing—Sequential Probability Ratio Test
4.6 Sequential Sampling of a Large Batch
4.7 Notes
Problems
Appendix. Least-Squares Estimation—The Kalman Filter
Chapter 5. Computational Aspects of Dynamic Programming—Suboptimal Control
5.1 The Curse of Dimensionality
5.2 Discretization Procedures and Their Convergence
5.3 Suboptimal Controllers and the Notion of Adaptivity
5.4 Naive Feedback and Open-Loop Feedback Controllers
5.5 Partial Open-Loop Feedback Controllers and the Efficient Utilization of Forecasts
5.6 Control of Systems with Unknown Parameters—Self-Tuning Regulators
5.7 Notes
Problems
Part II: CONTROL OF UNCERTAIN SYSTEMS OVER AN INFINITE HORIZON
Chapter 6. Minimization of Total Expected Value—Discounted Cost
6.1 Convergence and Existence Results
6.2 Computational Methods—Successive Approximation, Policy Iteration, Linear Programming
6.3 Contraction Mappings
6.4 Unbounded Costs per Stage
6.5 Linear Systems and Quadratic Cost Functionals
6.6 Inventory Control
6.7 Nonstationary and Periodic Problems
6.8 Notes
Problems
Chapter 7. Minimization of Total Expected Value—Undiscounted Cost
7.1 Convergence and Existence Results
7.2 Optimal Stopping
7.3 Optimal Gambling Strategies
7.4 The First Passage Problem
7.5 Notes
Problems
Chapter 8. Minimization of Average Expected Value
8.1 Existence Results
8.2 Successive Approximation
8.3 Policy Iteration
8.4 Infinite State Space—Linear Systems with Quadratic Cost Functionals
8.5 Notes
Problems
Appendix. Existence Analysis under the Weak Accessibility Condition
APPENDIXES
Appendix A. Mathematical Review
Appendix B. On Optimization Theory
Appendix C. On Probability Theory
Appendix D. On Finite State Markov Chains
References
Index
📜 SIMILAR VOLUMES