Adaptive Dynamic Programming for Control
Preface
Background of This Book
Why This Book?
The Content of This Book
Acknowledgments
Contents
Chapter 1: Overview
1.1 Challenges of Dynamic Programming
1.2 Background and Development of Adaptive Dynamic Programming
1.2.1 Basic Structures of ADP
1.2.1.1 Heuristic Dynamic Programming (HDP)
1.2.1.2 Dual Heuristic Programming (DHP)
1.2.2 Recent Developments of ADP
1.2.2.1 Development of ADP Structures
1.2.2.2 Development of Algorithms and Convergence Analysis
1.2.2.3 Applications of ADP Algorithms
1.3 Feedback Control Based on Adaptive Dynamic Programming
1.4 Non-linear Games Based on Adaptive Dynamic Programming
1.5 Summary
References
Chapter 2: Optimal State Feedback Control for Discrete-Time Systems
2.1 Introduction
2.2 Infinite-Horizon Optimal State Feedback Control Based on DHP
2.2.1 Problem Formulation
2.2.2 Infinite-Horizon Optimal State Feedback Control via DHP
2.2.3 Simulations
2.3 Infinite-Horizon Optimal State Feedback Control Based on GDHP
2.3.1 Problem Formulation
2.3.2 Infinite-Horizon Optimal State Feedback Control Based on GDHP
2.3.2.1 NN Identification of the Unknown Nonlinear System
2.3.2.2 Derivation of the Iterative ADP Algorithm
2.3.2.3 Convergence Analysis of the Iterative ADP Algorithm
2.3.2.4 NN Implementation of the Iterative ADP Algorithm Using GDHP Technique
2.3.3 Simulations
2.4 Infinite-Horizon Optimal State Feedback Control Based on GHJB Algorithm
2.4.1 Problem Formulation
2.4.2 Constrained Optimal Control Based on GHJB Equation
2.4.3 Simulations
2.5 Finite-Horizon Optimal State Feedback Control Based on HDP
2.5.1 Problem Formulation
2.5.2 Finite-Horizon Optimal State Feedback Control Based on HDP
2.5.2.1 Derivation and Properties of the Iterative ADP Algorithm
2.5.2.2 The epsilon-Optimal Control Algorithm
2.5.3 Simulations
2.6 Summary
References
Chapter 3: Optimal Tracking Control for Discrete-Time Systems
3.1 Introduction
3.2 Infinite-Horizon Optimal Tracking Control Based on HDP
3.2.1 Problem Formulation
3.2.2 Infinite-Horizon Optimal Tracking Control Based on HDP
3.2.2.1 System Transformation
3.2.2.2 Derivation of the Iterative HDP Algorithm
3.2.2.3 Summary of the Algorithm
3.2.2.4 Neural-Network Implementation for the Tracking Control Scheme
3.2.3 Simulations
3.3 Infinite-Horizon Optimal Tracking Control Based on GDHP
3.3.1 Problem Formulation
3.3.2 Infinite-Horizon Optimal Tracking Control Based on GDHP
3.3.2.1 Design and Implementation of Feedforward Controller
3.3.2.2 Design and Implementation of Optimal Feedback Controller
3.3.2.3 Convergence Characteristics of the Neural-Network Approximation Process
3.3.3 Simulations
3.4 Finite-Horizon Optimal Tracking Control Based on ADP
3.4.1 Problem Formulation
3.4.2 Finite-Horizon Optimal Tracking Control Based on ADP
3.4.2.1 Derivation of the Iterative ADP Algorithm
3.4.2.2 Convergence Analysis of the Iterative ADP Algorithm
3.4.2.3 The epsilon-Optimal Control Algorithm
3.4.2.4 Summary of the Algorithm
3.4.2.5 Neural-Network Implementation of the Iterative ADP Algorithm via HDP Technique
3.4.3 Simulations
3.5 Summary
References
Chapter 4: Optimal State Feedback Control of Nonlinear Systems with Time Delays
4.1 Introduction
4.2 Infinite-Horizon Optimal State Feedback Control via Delay Matrix
4.2.1 Problem Formulation
4.2.2 Optimal State Feedback Control Using Delay Matrix
4.2.2.1 Model Network
4.2.2.2 The M Network
4.2.2.3 Critic Network
4.2.2.4 Action Network
4.2.3 Simulations
4.3 Infinite-Horizon Optimal State Feedback Control via HDP
4.3.1 Problem Formulation
4.3.2 Optimal Control Based on Iterative HDP
4.3.3 Simulations
4.4 Finite-Horizon Optimal State Feedback Control for a Class of Nonlinear Systems with Time Delays
4.4.1 Problem Formulation
4.4.2 Optimal Control Based on Improved Iterative ADP
4.4.3 Simulations
4.5 Summary
References
Chapter 5: Optimal Tracking Control of Nonlinear Systems with Time Delays
5.1 Introduction
5.2 Problem Formulation
5.3 Optimal Tracking Control Based on Improved Iterative ADP Algorithm
5.4 Simulations
5.5 Summary
References
Chapter 6: Optimal Feedback Control for Continuous-Time Systems via ADP
6.1 Introduction
6.2 Optimal Robust Feedback Control for Unknown General Nonlinear Systems
6.2.1 Problem Formulation
6.2.2 Data-Based Robust Approximate Optimal Tracking Control
6.2.3 Simulations
6.3 Optimal Feedback Control for Nonaffine Nonlinear Systems
6.3.1 Problem Formulation
6.3.2 Robust Approximate Optimal Control Based on ADP Algorithm
6.3.3 Simulations
6.4 Summary
References
Chapter 7: Several Special Optimal Feedback Control Designs Based on ADP
7.1 Introduction
7.2 Optimal Feedback Control for a Class of Switched Systems
7.2.1 Problem Description
7.2.2 Optimal Feedback Control Based on Two-Stage ADP Algorithm
7.2.3 Simulations
7.3 Optimal Feedback Control for a Class of Descriptor Systems
7.3.1 Problem Formulation
7.3.2 Optimal Controller Design for a Class of Descriptor Systems
7.3.3 Simulations
7.4 Optimal Feedback Control for a Class of Singularly Perturbed Systems
7.4.1 Problem Formulation
7.4.2 Optimal Controller Design for Singularly Perturbed Systems
7.4.2.1 Algorithm Design
7.4.2.2 Neural Network Approximation
7.4.3 Simulations
7.5 Optimal Feedback Control for a Class of Constrained Systems Via SNAC
7.5.1 Problem Formulation
7.5.2 Optimal Controller Design for Constrained Systems via SNAC
7.5.3 Simulations
7.6 Summary
References
Chapter 8: Zero-Sum Games for Discrete-Time Systems Based on Model-Free ADP
8.1 Introduction
8.2 Zero-Sum Differential Games for a Class of Discrete-Time 2-D Systems
8.2.1 Problem Formulation
8.2.2 Data-Based Optimal Control via Iterative ADP Algorithm
8.2.2.1 The Derivation of Data-Based Iterative ADP Algorithm
8.2.2.2 Properties of Data-Based Iterative ADP Algorithm
8.2.2.3 Neural Network Implementation
8.2.2.4 Critic Network
8.2.2.5 Action Networks
8.2.3 Simulations
8.3 Zero-Sum Games for a Class of Discrete-Time Systems via Model-Free ADP
8.3.1 Problem Formulation
8.3.2 Data-Based Optimal Output Feedback Control via ADP Algorithm
8.3.3 Simulations
8.4 Summary
References
Chapter 9: Nonlinear Games for a Class of Continuous-Time Systems Based on ADP
9.1 Introduction
9.2 Infinite Horizon Zero-Sum Games for a Class of Affine Nonlinear Systems
9.2.1 Problem Formulation
9.2.2 Zero-Sum Differential Games Based on Iterative ADP Algorithm
9.2.2.1 Derivation of the Iterative ADP Method
9.2.2.2 The Iterative ADP Algorithm
9.2.2.3 Properties of the Iterative ADP Algorithm
9.2.3 Simulations
9.3 Finite Horizon Zero-Sum Games for a Class of Nonlinear Systems
9.3.1 Problem Formulation
9.3.2 Finite Horizon Optimal Control of Nonaffine Nonlinear Zero-Sum Games
9.3.3 Simulations
9.4 Non-Zero-Sum Games for a Class of Nonlinear Systems Based on ADP
9.4.1 Problem Formulation of Non-Zero-Sum Games
9.4.2 Optimal Control of Nonlinear Non-Zero-Sum Games Based on ADP
9.4.3 Simulations
9.5 Summary
References
Chapter 10: Other Applications of ADP
10.1 Introduction
10.2 Self-Learning Call Admission Control for CDMA Cellular Networks Using ADP
10.2.1 Problem Formulation
10.2.2 A Self-Learning Call Admission Control Scheme for CDMA Cellular Networks
10.2.2.1 Adaptive Critic Designs for Problems with Finite Action Space
10.2.2.2 Self-learning Call Admission Control for CDMA Cellular Networks
10.2.3 Simulations
10.3 Engine Torque and Air-Fuel Ratio Control Based on ADP
10.3.1 Problem Formulation
10.3.2 Self-learning Neural Network Control for Both Engine Torque and Exhaust Air-Fuel Ratio
10.3.3 Simulations
10.3.3.1 Critic Network
10.3.3.2 Controller/Action Network
10.3.3.3 Simulation Results
10.4 Summary
References
Index