Adaptive dynamic programming for control: algorithms and stability

✍ Scribed by Zhang, Huaguang

• Publisher: Springer-Verlag
• Year: 2013
• Tongue: English
• Leaves: 431
• Series: Communications and control engineering
• Category: Library

✦ Synopsis

There are many methods of stable controller design for nonlinear systems. In seeking to go beyond the minimum requirement of stability, Adaptive Dynamic Programming in Discrete Time approaches the challenging topic of optimal control for nonlinear systems using the tools of adaptive dynamic programming (ADP). The range of systems treated is extensive; affine, switched, singularly perturbed and time-delay nonlinear systems are discussed as are the uses of neural networks and techniques of value and policy iteration.

✦ Table of Contents

Adaptive Dynamic Programming for Control......Page 2
Background of This Book......Page 4
The Content of This Book......Page 5
Acknowledgments......Page 8
Contents......Page 10
1.1 Challenges of Dynamic Programming......Page 15
1.2 Background and Development of Adaptive Dynamic Programming......Page 17
1.2.1.1 Heuristic Dynamic Programming (HDP)......Page 18
1.2.1.2 Dual Heuristic Programming (DHP)......Page 19
1.2.2.1 Development of ADP Structures......Page 20
1.2.2.2 Development of Algorithms and Convergence Analysis......Page 23
1.2.2.3 Applications of ADP Algorithms......Page 24
1.3 Feedback Control Based on Adaptive Dynamic Programming......Page 25
1.4 Non-linear Games Based on Adaptive Dynamic Programming......Page 31
References......Page 33
2.2 Infinite-Horizon Optimal State Feedback Control Based on DHP......Page 40
2.2.1 Problem Formulation......Page 41
2.2.2 Infinite-Horizon Optimal State Feedback Control via DHP......Page 43
2.2.3 Simulations......Page 57
2.3.1 Problem Formulation......Page 65
2.3.2.1 NN Identification of the Unknown Nonlinear System......Page 67
2.3.2.2 Derivation of the Iterative ADP Algorithm......Page 70
2.3.2.3 Convergence Analysis of the Iterative ADP Algorithm......Page 71
2.3.2.4 NN Implementation of the Iterative ADP Algorithm Using GDHP Technique......Page 77
2.3.3 Simulations......Page 80
2.4.1 Problem Formulation......Page 84
2.4.2 Constrained Optimal Control Based on GHJB Equation......Page 86
2.4.3 Simulations......Page 91
2.5 Finite-Horizon Optimal State Feedback Control Based on HDP......Page 93
2.5.1 Problem Formulation......Page 95
2.5.2.1 Derivation and Properties of the Iterative ADP Algorithm......Page 97
2.5.2.2 The epsilon-Optimal Control Algorithm......Page 104
2.5.3 Simulations......Page 115
References......Page 119
3.2 Infinite-Horizon Optimal Tracking Control Based on HDP......Page 121
3.2.1 Problem Formulation......Page 122
3.2.2.1 System Transformation......Page 123
3.2.2.2 Derivation of the Iterative HDP Algorithm......Page 124
3.2.2.3 Summary of the Algorithm......Page 129
3.2.3 Simulations......Page 130
3.3 Infinite-Horizon Optimal Tracking Control Based on GDHP......Page 132
3.3.1 Problem Formulation......Page 135
3.3.2.1 Design and Implementation of Feedforward Controller......Page 138
3.3.2.2 Design and Implementation of Optimal Feedback Controller......Page 139
3.3.2.3 Convergence Characteristics of the Neural-Network Approximation Process......Page 147
3.3.3 Simulations......Page 149
3.4 Finite-Horizon Optimal Tracking Control Based on ADP......Page 150
3.4.1 Problem Formulation......Page 153
3.4.2.1 Derivation of the Iterative ADP Algorithm......Page 156
3.4.2.2 Convergence Analysis of the Iterative ADP Algorithm......Page 158
3.4.2.3 The epsilon-Optimal Control Algorithm......Page 162
3.4.2.5 Neural-Network Implementation of the Iterative ADP Algorithm via HDP Technique......Page 163
3.4.3 Simulations......Page 166
3.5 Summary......Page 170
References......Page 171
4.1 Introduction......Page 173
4.2.1 Problem Formulation......Page 174
4.2.2 Optimal State Feedback Control Using Delay Matrix......Page 175
4.2.2.1 Model Network......Page 184
4.2.2.3 Critic Network......Page 185
4.2.2.4 Action Network......Page 186
4.2.3 Simulations......Page 187
4.3.1 Problem Formulation......Page 189
4.3.2 Optimal Control Based on Iterative HDP......Page 192
4.3.3 Simulations......Page 198
4.4.1 Problem Formulation......Page 200
4.4.2 Optimal Control Based on Improved Iterative ADP......Page 202
4.4.3 Simulations......Page 208
4.5 Summary......Page 209
References......Page 210
5.2 Problem Formulation......Page 212
5.3 Optimal Tracking Control Based on Improved Iterative ADP Algorithm......Page 213
5.4 Simulations......Page 224
References......Page 231
6.2 Optimal Robust Feedback Control for Unknown General Nonlinear Systems......Page 233
6.2.2 Data-Based Robust Approximate Optimal Tracking Control......Page 234
6.2.3 Simulations......Page 246
6.3.1 Problem Formulation......Page 252
6.3.2 Robust Approximate Optimal Control Based on ADP Algorithm......Page 253
6.3.3 Simulations......Page 260
6.4 Summary......Page 263
References......Page 264
7.1 Introduction......Page 266
7.2.1 Problem Description......Page 267
7.2.2 Optimal Feedback Control Based on Two-Stage ADP Algorithm......Page 268
7.2.3 Simulations......Page 277
7.3.1 Problem Formulation......Page 280
7.3.2 Optimal Controller Design for a Class of Descriptor Systems......Page 282
7.3.3 Simulations......Page 288
7.4.1 Problem Formulation......Page 290
7.4.2.1 Algorithm Design......Page 292
7.4.2.2 Neural Network Approximation......Page 295
7.5.1 Problem Formulation......Page 297
7.5.2 Optimal Controller Design for Constrained Systems via SNAC......Page 301
7.5.3 Simulations......Page 308
References......Page 315
8.2 Zero-Sum Differential Games for a Class of Discrete-Time 2-D Systems......Page 317
8.2.1 Problem Formulation......Page 318
8.2.2 Data-Based Optimal Control via Iterative ADP Algorithm......Page 325
8.2.2.1 The Derivation of Data-Based Iterative ADP Algorithm......Page 326
8.2.2.2 Properties of Data-Based Iterative ADP Algorithm......Page 327
8.2.2.4 Critic Network......Page 334
8.2.2.5 Action Networks......Page 335
8.2.3 Simulations......Page 336
8.3 Zero-Sum Games for a Class of Discrete-Time Systems via Model-Free ADP......Page 339
8.3.1 Problem Formulation......Page 340
8.3.2 Data-Based Optimal Output Feedback Control via ADP Algorithm......Page 342
8.3.3 Simulations......Page 349
References......Page 351
9.1 Introduction......Page 353
9.2.1 Problem Formulation......Page 354
9.2.2.1 Derivation of the Iterative ADP Method......Page 355
9.2.2.2 The Iterative ADP Algorithm......Page 357
9.2.2.3 Properties of the Iterative ADP Algorithm......Page 358
9.2.3 Simulations......Page 363
9.3 Finite Horizon Zero-Sum Games for a Class of Nonlinear Systems......Page 366
9.3.1 Problem Formulation......Page 368
9.3.2 Finite Horizon Optimal Control of Nonaffine Nonlinear Zero-Sum Games......Page 370
9.3.3 Simulations......Page 378
9.4 Non-Zero-Sum Games for a Class of Nonlinear Systems Based on ADP......Page 380
9.4.1 Problem Formulation of Non-Zero-Sum Games......Page 381
9.4.2 Optimal Control of Nonlinear Non-Zero-Sum Games Based on ADP......Page 384
9.4.3 Simulations......Page 395
9.5 Summary......Page 399
References......Page 400
10.1 Introduction......Page 402
10.2.1 Problem Formulation......Page 403
10.2.2.1 Adaptive Critic Designs for Problems with Finite Action Space......Page 405
10.2.2.2 Self-learning Call Admission Control for CDMA Cellular Networks......Page 409
10.2.3 Simulations......Page 413
10.3.1 Problem Formulation......Page 419
10.3.2 Self-learning Neural Network Control for Both Engine Torque and Exhaust Air-Fuel Ratio......Page 420
10.3.3.1 Critic Network......Page 422
10.3.3.3 Simulation Results......Page 424
10.4 Summary......Page 426
References......Page 427
Index......Page 430

✦ Subjects

Ingénierie;MATHEMATICS--Applied;MATHEMATICS--Probability & Statistics--General;Adaptive control systems;Computer algorithms;Dynamic programming;Electronic books;MATHEMATICS -- Applied;MATHEMATICS -- Probability & Statistics -- General;Ingénierie