Reinforcement Learning Aided Performance Optimization of Feedback Control Systems

✍ Scribed by Changsheng Hua

• Publisher: Springer Vieweg
• Year: 2021
• Tongue: English
• Leaves: 139
• Category: Library

✦ Synopsis

Changsheng Hua proposes two approaches, an input/output recovery approach and a performance index-based approach for robustness and performance optimization of feedback control systems. For their data-driven implementation in deterministic and stochastic systems, the author develops Q-learning and natural actor-critic (NAC) methods, respectively. Their effectiveness has been demonstrated by an experimental study on a brushless direct current motor test rig.

The author:

Changsheng Hua received the Ph.D. degree at the Institute of Automatic Control and Complex Systems (AKS), University of Duisburg-Essen, Germany, in 2020. His research interests include model-based and data-driven fault diagnosis and fault-tolerant techniques.

✦ Table of Contents

Acknowledgments
Contents
Abbreviations and Notation
Abbreviations
Notation
List of Figures
1 Introduction
1.1 Motivation
1.2 Scope of the Work
1.3 Objective of the Work
1.4 Outline of the Thesis
2 The Basics of Feedback Control Systems
[DELETE]
2.1 Design of Feedback Controllers
2.1.1 Description of a Nominal System
2.1.2 A Coprime Factorization Design Tool
2.1.3 Well-posedness and Internal Stability
2.1.4 Parameterization of Stabilizing Controllers
2.2 Model Uncertainty and Robustness
2.2.1 Small Gain Theorem
2.2.2 Coprime Factor Representation of Model Uncertainty
2.2.3 Dual YK Representation of Model Uncertainty
2.3 Concluding Remarks
3 Reinforcement Learning and Feedback Control
3.1 An Overview of RL Methods
3.2 Infinite Horizon Linear Quadratic Regulator
3.2.1 An Overview of the Infinite Horizon LQR Problem and DP
3.2.2 Policy Iteration and Value Iteration
3.2.3 Q-learning
3.2.4 SARSA
3.2.5 Simulation Results
3.3 Infinite Horizon Linear Quadratic Gaussian
3.3.1 An Overview of Infinite Horizon LQG Problem and Stochastic DP
3.3.2 On-policy Natural Actor-critic
3.3.3 Simulation Results
3.4 Concluding Remarks
4 Q-learning Aided Performance Optimization of Deterministic Systems
4.1 Problem Formulation
4.2 Robustness Optimization
4.2.1 Existing Robustness Optimization Approaches
4.2.2 Input and Output Recovery
4.2.3 Robustness Optimization Using Q-learning
4.2.4 Simulation Results
4.3 Performance Optimization Using a Prescribed Performance Index
4.3.1 Performance Optimization Using Q-learning
4.3.2 An Extension to Tracking Performance Optimization
4.3.3 Simulation Results
4.4 Concluding Remarks
5 NAC Aided Performance Optimization of Stochastic Systems
[DELETE]
5.1 Problem Formulation
5.2 Robustness Optimization
5.2.1 Noise Characteristics
5.2.2 Conditions of Closed-loop Internal Stability
5.2.3 Robustness Optimization using NAC
5.2.4 Simulation Results
5.3 Performance Optimization using a Prescribed Performance Index
5.3.1 Efficient NAC Learning using Both Nominal Plant Model and Data
5.3.2 Data-driven Performance Optimization
5.3.3 Simulation Results
5.4 Performance Optimization of Plants with General Output Feedback Controllers
5.4.1 Problem Formulation
5.4.2 Performance Optimization using NAC
5.4.3 Experimental Results
5.5 Concluding Remarks
6 Conclusion and Future Work
6.1 Conclusion
6.2 Future Work
Bibliography
Bibliography