Sequential Stochastic Optimization (Wiley Series in Probability and Statistics)

✍ Scribed by Renzo Cairoli, Robert C. Dalang

• Publisher: Wiley
• Year: 1996
• Tongue: English
• Leaves: 348
• Edition: 1
• Category: Library

✦ Synopsis

Sequential Stochastic Optimization provides mathematicians and applied researchers with a well-developed framework in which stochastic optimization problems can be formulated and solved. Offering much material that is either new or has never before appeared in book form, it lucidly presents a unified theory of optimal stopping and optimal sequential control of stochastic processes. This book has been carefully organized so that little prior knowledge of the subject is assumed; its only prerequisites are a standard graduate course in probability theory and some familiarity with discrete-parameter martingales.Major topics covered in Sequential Stochastic Optimization include: * Fundamental notions, such as essential supremum, stopping points, accessibility, martingales and supermartingales indexed by INd * Conditions which ensure the integrability of certain suprema of partial sums of arrays of independent random variables * The general theory of optimal stopping for processes indexed by Ind * Structural properties of information flows * Sequential sampling and the theory of optimal sequential control * Multi-armed bandits, Markov chains and optimal switching between random walks

✦ Table of Contents

Sequential Stochastic Optimization......Page 5
Contents......Page 7
Preface......Page 11
Notation and Conventions......Page 15
1.1 Filtered Probability Spaces......Page 18
1.2 Random Variables......Page 20
1.3 Stopping Points......Page 25
1.4 Increasing Paths and Accessible Stopping Points......Page 27
1.5 Some Operations on Accessible Stopping Points......Page 32
1.6 Stochastic Processes and Martingales......Page 35
Exercises......Page 42
Historical Notes......Page 45
2.1 Maximal Inequalities......Page 47
2.2 Integrability Criteria for the Supremum......Page 51
2.3 The Strong Law of Large Numbers......Page 55
2.4 Case Where the Random Variables Are Identically Distributed......Page 60
Exercises......Page 69
Historical Notes......Page 72
3.1 Stating the Problem......Page 73
3.2 Snell's Envelope......Page 75
3.3 Solving the Problem......Page 80
3.4 A Related Problem......Page 84
3.5 Maximal Accessible Stopping Points......Page 90
3.6 Case Where the Index Set is Finite......Page 93
3.7 An Application to Normalized Partial Sums......Page 97
3.8 Complements......Page 102
Exercises......Page 105
Historical Notes......Page 107
4.1 Linear Representation of Accessible Stopping Points......Page 109
4.2 Applications......Page 115
4.3 Linear Representation in the Setting of Inaccessible Stopping Points......Page 117
Exercises......Page 122
Historical Notes......Page 125
5.1 Conditions for Accessibility......Page 126
5.2 Consequences for the Structure of the Filtration......Page 130
5.3 The Bidimensional Case......Page 136
5.4 Predictability of Optional Increasing Paths......Page 141
5.5 The Combinatorial Structure of a Filtration......Page 142
5.6 The Combinatorial Structure of a Filtration Satisfying COl......Page 147
5.7 Optimal Stopping and Linear Optimization......Page 154
Exercises......Page 158
Historical Notes......Page 161
6. Sequential Sampling......Page 162
6.1 Stating the Problem......Page 163
6.2 Constructing the Model......Page 167
6.3 The Reward Process and Snell's Envelope......Page 173
6.4 Describing the Optimal Strategy......Page 178
6.5 The Likelihood-Ratio Test......Page 185
6.6 Applications......Page 196
6.7 Complement......Page 199
Exercises......Page 204
Historical Notes......Page 205
7.1 An Example......Page 206
7.2 Preliminaries......Page 207
7.3 Controls......Page 209
7.4 Optimization......Page 214
7.5 Optimization Over Finite Controls......Page 222
7.6 Case Where the Index Set Is Finite......Page 224
7.7 Extension to General Index Sets......Page 227
Exercises......Page 229
Historical Notes......Page 231
8.1 Formulating the Problem......Page 232
8.2 Index Controls......Page 237
8.3 Gittins Indices......Page 243
8.4 Characterizing Optimal Controls......Page 248
8.5 Examples......Page 256
Exercises......Page 266
Historical Notes......Page 268
9.1 Markov Chains and Superharmonic Functions......Page 270
9.2 Optimal Control of a Markov Chain......Page 274
9.3 The Special Case of a Random Walk......Page 276
9.4 Control and Stopping at the Time of First Visit to a Set of States......Page 279
9.5 Markov Structures......Page 282
Exercises......Page 288
Historical Notes......Page 291
10.1 Formulating and Solving the Problem......Page 292
10.2 Some Properties of the Solution......Page 295
10.3 The Structure of the Solution......Page 300
10.4 Constructing the Switching Curves......Page 303
10.5 Characterizing the Type of the Solution......Page 307
10.6 Determining the Type of the Solution......Page 313
Exercises......Page 322
Historical Notes......Page 327
Bibliography......Page 328
Index of Notation......Page 334
Index of Terms......Page 337