Preface......Page 6
Contents......Page 8
About the Authors......Page 13
Symbols......Page 15
1.1.1 Continuous-Time LQR......Page 16
1.1.2 Discrete-Time LQR......Page 17
1.2 Adaptive Dynamic Programming......Page 18
1.3 Review of Matrix Algebra......Page 20
References......Page 21
2.1 Introduction......Page 22
2.3 The Data-Based Identifier......Page 24
2.4 Derivation of the Iterative ADP Algorithm with Convergence Analysis......Page 26
2.5 Neural Network Implementation of the Iterative Control Algorithm......Page 32
2.6 Simulation Study......Page 33
2.7 Conclusions......Page 35
References......Page 37
3.1 Introduction......Page 39
3.2 Problem Statement......Page 40
3.3.1 The Novel ADP Iteration Algorithm......Page 44
3.3.2 Convergence Analysis of the Improved Iteration Algorithm......Page 47
3.3.3 Neural Network Implementation of the Iteration ADP Algorithm......Page 52
3.4 Simulation Study......Page 54
3.5 Conclusion......Page 59
References......Page 62
4.1 Introduction......Page 63
4.2 Problem Formulation......Page 64
4.3 Derivation of the ADP Algorithm for Time-Delay Systems......Page 65
4.4 Neural Network Implementation for the Multi-objective Optimal Control Problem of Time-Delay Systems......Page 68
4.5 Simulation Results......Page 69
4.6 Conclusions......Page 75
References......Page 76
5.1 Introduction......Page 77
5.2 Problem Statement......Page 79
5.3 SIANN Architecture-Based Classification......Page 80
5.4 Optimal Control Based on ADP......Page 83
5.4.1 Model Neural Network......Page 84
5.4.2 Critic Network and Action Network......Page 88
5.5 Simulation Study......Page 96
References......Page 105
6.1 Introduction......Page 108
6.2 Motivations and Preliminaries......Page 109
6.3.1 Critic Network......Page 112
6.3.2 Action Network......Page 114
6.3.3 Design of the Compensation Controller......Page 115
6.3.4 Stability Analysis......Page 116
6.4 Simulation Study......Page 120
References......Page 123
7.1 Introduction......Page 125
7.2 Problem Statement......Page 126
7.3 Off-Policy Optimal Control Method......Page 127
7.3.1 Convergence Analysis of Off-Policy PI Algorithm......Page 129
7.3.2 Implementation Method of Off-Policy Iteration Algorithm......Page 131
7.4 Simulation Study......Page 134
References......Page 137
8.1 Introduction......Page 139
8.2 Problem Formulation and Preliminaries......Page 140
8.3.1 Description of Approximation-Error ADP Algorithm......Page 142
8.3.2 Convergence Analysis of The Iterative ADP Algorithm......Page 144
8.4 Simulation Study......Page 148
References......Page 156
9.1 Introduction......Page 158
9.2 Problem Statement......Page 159
9.3.1 On-Policy IRL for Nonzero Disturbance......Page 162
9.3.2 Off-Policy IRL for Nonzero Disturbance......Page 163
9.3.3 NN Approximation for Actor-Critic Structure......Page 165
9.4.1 Disturbance Compensation Off-Policy Controller Design......Page 168
9.4.2 Stability Analysis......Page 169
9.5 Simulation Study......Page 172
References......Page 174
10.1 Introduction......Page 176
10.2 Preliminaries and Assumptions......Page 177
10.3.1 Derivation of The Iterative ADP Method......Page 180
10.3.2 The Procedure of the Method......Page 185
10.3.3 The Properties of the Iterative ADP Method......Page 187
10.4 Neural Network Implementation......Page 201
10.4.1 The Model Network......Page 202
10.4.2 The Critic Network......Page 203
10.4.3 The Action Network......Page 204
10.5 Simulation Study......Page 206
References......Page 215
11.1 Introduction......Page 217
11.2 Motivations and Preliminaries......Page 218
11.3.1 Derivation of Off-Policy Algorithm......Page 223
11.3.2 Implementation Method for Off-Policy Algorithm......Page 224
11.3.3 Stability Analysis......Page 228
11.4 Simulation Study......Page 229
References......Page 234
12.1 Introduction......Page 236
12.2 Problem Statement......Page 237
12.3 Multi-player Learning PI Solution for NZS Games......Page 238
12.4.1 Derivation of Off-Policy Algorithm......Page 243
12.4.2 Implementation Method for Off-Policy Algorithm......Page 245
12.4.3 Stability Analysis......Page 247
12.5 Simulation Study......Page 251
References......Page 257
13.1 Introduction......Page 259
13.2.1 Graph Theory......Page 260
13.2.2 Synchronization and Tracking Error Dynamic Systems......Page 261
13.3 Optimal Distributed Cooperative Control for Multi-agent โฆ......Page 263
13.3.1 Cooperative Performance Index Function......Page 264
13.3.2 Nash Equilibrium......Page 265
13.4.1 Derivation of the Heterogeneous Multi-agent Differential Graphical Games......Page 267
13.4.2 Properties of the Developed Policy Iteration Algorithm......Page 268
13.4.3 Heterogeneous Multi-agent Policy Iteration Algorithm......Page 272
13.5 Simulation Study......Page 273
References......Page 277
Index......Page 278