Stochastic Controls: Hamiltonian Systems and HJB Equations (Stochastic Modelling and Applied Probability 43)

✍ Scribed by Jiongmin Yong, Xun Yu Zhou

• Publisher: Springer
• Year: 1999
• Tongue: English
• Leaves: 465
• Series: Stochastic Modelling and Applied Probability 43
• Edition: 1
• Category: Library

✦ Synopsis

The maximum principle and dynamic programming are the two most commonly used approaches in solving optimal control problems. These approaches have been developed independently. The theme of this book is to unify these two approaches, and to demonstrate that the viscosity solution theory provides the framework to unify them.

✦ Table of Contents

STOCHASTIC CONTROLS: HAMILTONIAN SYSTEMS AND HJB EQUATIONS......Page 1
Applications of Mathematics......Page 4
Title Page......Page 5
Copyright Page......Page 6
Dedication......Page 7
Preface......Page 9
Contents......Page 17
Notation......Page 21
Assumption Index......Page 23
Problem Index......Page 24
1.1. Probability spaces......Page 25
1.2. Random variables......Page 28
1.3. Conditional expectation......Page 32
1.4. Convergence of probabilities......Page 37
2.1. General considerations......Page 39
2.2. Brownian motions......Page 45
3. Stopping Times......Page 47
4. Martingales......Page 51
5.1. Nondifferentiability of Brownian motion......Page 54
5.2. Definition of Itô's integral and basic properties......Page 56
5.3. Itô's formula......Page 60
5.4. Martingale representation theorems......Page 62
6. Stochastic Differential Equations......Page 64
6.1. Strong solutions......Page 65
6.2. Weak solutions......Page 68
6.3. Linear SDEs......Page 71
6.4. Other types of SDEs......Page 72
1. Introduction......Page 75
2. Deterministic Cases Revisited......Page 76
3.1. Production planning......Page 79
3.2. Investment vs. consumption......Page 80
3.3. Reinsurance and dividend management......Page 82
3.4. Technology diffusion......Page 83
3.5. Queueing systems in heavy traffic......Page 84
4.1. Strong formulation......Page 86
4.2. Weak formulation......Page 88
5.1. A deterministic result......Page 89
5.2. Existence under strong formulation......Page 91
5.3. Existence under weak formulation......Page 93
6. Reachable Sets of Stochastic Control Systems......Page 99
6.1. Nonconvexity of the reachable sets......Page 100
6.2. Noncloseness of the reachable sets......Page 105
7.1. Random duration......Page 109
7.3. Singular and impulse controls......Page 110
7.4. Risk-sensitive controls......Page 112
7.6. Partially observable systems......Page 113
8. Historical Remarks......Page 116
1. Introduction......Page 125
2. The Deterministic Case Revisited......Page 126
3. Statement of the Stochastic Maximum Principle......Page 137
3.1. Adjoint equations......Page 139
3.2. The maximum principle and stochastic Hamiltonian systems......Page 141
3.3. A worked-out example......Page 144
4. A Proof of the Maximum Principle......Page 147
4.1. A moment estimate......Page 148
4.2. Taylor expansions......Page 150
4.3. Duality analysis and completion of the proof......Page 158
5. Sufficient Conditions of Optimality......Page 161
6.1. Formulation of the problem and the maximum principle......Page 165
6.2. Some preliminary lemmas......Page 169
6.3. A proof of Theorem 6.1......Page 173
7. Historical Remarks......Page 177
1. Introduction......Page 181
2. The Deterministic Case Revisited......Page 182
3.1. A stochastic framework for dynamic programming......Page 199
3.2. Principle of optimality......Page 204
3.3. The HJB equation......Page 206
4.1. Continuous dependence on parameters......Page 208
4.2. Semiconcavity......Page 210
5.1. Definitions......Page 213
5.2. Some properties......Page 220
6.1. A uniqueness theorem......Page 222
6.2. Proofs of Lemmas 6.6 and 6.7......Page 232
7. Historical Remarks......Page 236
1. Introduction......Page 241
2. Classical Hamilton–Jacobi Theory......Page 243
3. Relationship for Deterministic Systems......Page 251
3.1. Adjoint variable and value function: Smooth case......Page 253
3.2. Economic interpretation......Page 255
3.3. Methods of characteristics and the Feynman–Kac formula......Page 256
3.4. Adjoint variable and value function: Nonsmooth case......Page 259
3.5. Verification theorems......Page 265
4. Relationship for Stochastic Systems......Page 271
4.1. Smooth case......Page 274
4.2. Nonsmooth case: Differentials in the spatial variable......Page 279
4.3. Nonsmooth case: Differentials in the time variable......Page 287
5.1. Smooth case......Page 292
5.2. Nonsmooth case......Page 293
6. Optimal Feedback Controls......Page 299
7. Historical Remarks......Page 302
1. Introduction......Page 305
2.1. Formulation......Page 308
2.2. A minimization problem of a quadratic functional......Page 310
2.3. A linear Hamiltonian system......Page 313
2.4. The Riccati equation and feedback optimal control......Page 317
3.1. Statement of the problems......Page 324
3.2. Examples......Page 325
4. Finiteness and Solvability......Page 328
5. A Necessary Condition and a Hamiltonian System......Page 332
6. Stochastic Riccati Equations......Page 337
7. Global Solvability of Stochastic Riccati Equations......Page 343
7.1. Existence: The standard case......Page 344
7.2. Existence: The case C = 0, S = 0, and Q, G ≥ 0......Page 348
7.3. Existence: The one-dimensional case......Page 353
8. A Mean-variance Portfolio Selection Problem......Page 359
9. Historical Remarks......Page 366
1. Introduction......Page 369
2. Linear Backward Stochastic Differential Equations......Page 371
3.1. BSDEs in finite deterministic durations: Method of contraction mapping......Page 378
3.2. BSDEs in random durations: Method of continuation......Page 384
4.1. Representation via SDEs......Page 396
4.2. Representation via BSDEs......Page 401
5. Forward—Backward Stochastic Differential Equations......Page 405
5.1. General formulation and nonsolvability......Page 406
5.2. The four-step scheme, a heuristic derivation......Page 407
5.3. Several solvable classes of FBSDEs......Page 411
6.1. European call options and the Black–Scholes formula......Page 416
6.2. Other options......Page 420
7. Historical Remarks......Page 422
References......Page 425
Index......Page 457
Applications of Mathematics (continued from page ii)......Page 463
Back Cover......Page 465