Optimal Control Theory: The Variational Method

✍ Scribed by Zhongjing Ma, Suli Zou

• Publisher: Springer
• Year: 2021
• Tongue: English
• Leaves: 355
• Category: Library

✦ Synopsis

This book focuses on how to implement optimal control problems via the variational method. It studies how to implement the extrema of functional by applying the variational method and covers the extrema of functional with different boundary conditions, involving multiple functions and with certain constraints etc. It gives the necessary and sufficient condition for the (continuous-time) optimal control solution via the variational method, solves the optimal control problems with different boundary conditions, analyzes the linear quadratic regulator & tracking problems respectively in detail, and provides the solution of optimal control problems with state constraints by applying the Pontryagin’s minimum principle which is developed based upon the calculus of variations. And the developed results are applied to implement several classes of popular optimal control problems and say minimum-time, minimum-fuel and minimum-energy problems and so on.

As another key branch of optimal control methods, it also presents how to solve the optimal control problems via dynamic programming and discusses the relationship between the variational method and dynamic programming for comparison.

Concerning the system involving individual agents, it is also worth to study how to implement the decentralized solution for the underlying optimal control problems in the framework of differential games. The equilibrium is implemented by applying both Pontryagin’s minimum principle and dynamic programming.

The book also analyzes the discrete-time version for all the above materials as well since the discrete-time optimal control problems are very popular in many fields.

✦ Table of Contents

Preface
Acknowledgements
Contents
List of Figures
List of Tables
1 Introduction
1.1 Backgrounds and Motivation
1.2 Optimal Control Theory
1.3 Examples of Optimal Control Problems
1.4 Formulation of Continuous-Time Optimal Control Problems
1.5 Formulation of Discrete-Time Optimal Control Problems
1.6 Organization
References
2 Extrema of a Functional via the Variational Method
2.1 Fundamental Notations
2.1.1 Linearity of Function and Functional
2.1.2 Norm in Euclidean Space and Functional
2.1.3 Increment of Function and Functional
2.1.4 Differential of Function and Variation of Functional
2.2 Extrema of Functional
2.2.1 Extrema with Fixed Final Time and Fixed Final State
2.2.2 Specific Forms of Euler Equation in Different Cases
2.2.3 Sufficient Condition for Extrema
2.2.4 Extrema with Fixed Final Time and Free Final State
2.2.5 Extrema with Free Final Time and Fixed Final State
2.2.6 Extrema with Free Final Time and Free Final State
2.3 Extrema of Functional with Multiple Independent Functions
2.4 Extrema of Function with Constraints
2.4.1 Elimination/Direct Method
2.4.2 Lagrange Multiplier Method
2.5 Extrema of Functional with Constraints
2.5.1 Extrema of Functional with Differential Constraints
2.5.2 Extrema of Functional with Isoperimetric Constraints
2.6 Summary
2.7 Exercises
3 Optimal Control via Variational Method
3.1 Necessary and Sufficient Condition for Optimal Control
3.2 Optimal Control Problems with Different Boundary Conditions
3.2.1 Optimal Control with Fixed Final Time and Fixed Final State
3.2.2 Optimal Control with Fixed Final Time and Free Final State
3.2.3 Optimal Control with Free Final Time and Fixed Final State
3.2.4 Optimal Control with Free Final Time and Free Final State
3.3 Linear-Quadratic Regulation Problems
3.3.1 Infinite-Interval Time-Invariant LQR Problems
3.4 Linear-Quadratic Tracking Problems
3.5 Summary
3.6 Exercises
4 Pontryagin's Minimum Principle
4.1 Pontryagin's Minimum Principle with Constrained Control
4.2 Pontryagin's Minimum Principle with Constrained State Variable
4.3 Minimum Time Problems
4.3.1 Optimal Control Solution for Minimum Time Problems
4.3.2 Minimum Time Problems for Linear Time-Invariant Systems
4.4 Minimum Fuel Problems
4.5 Performance Cost Composed of Elapsed Time and Consumed Fuel
4.6 Minimum Energy Problems
4.7 Performance Cost Composed of Elapsed Time and Consumed Energy
4.8 Summary
4.9 Exercises
5 Dynamic Programming
5.1 The Hamilton–Jacobi–Bellman Equation
5.2 Analysis on Optimal Control
5.3 Linear-Quadratic Regulation Problems
5.4 Affine-Quadratic Regulation Problems
5.5 Affine-Quadratic Tracking Problems
5.6 Development of Pontryagin's Minimum Principle via Dynamic Programming
5.7 Summary
5.8 Exercises
6 Differential Games
6.1 Noncooperative Differential Games
6.1.1 Formulation of Noncooperative Differential Games
6.1.2 Nash Equilibrium of Noncooperative Differential Games
6.1.3 Affine-Quadratic Noncooperative Differential Games
6.2 Two-Person Zero-Sum Differential Games
6.2.1 Formulation of Two-Person Zero-Sum Differential Games
6.2.2 Saddle Point of Two-Person Zero-Sum Differential Games
6.2.3 Implementation of Saddle Point of Two-Person Zero-Sum Differential Games via Dynamic Programming
6.2.4 Linear-Quadratic Two-Person Zero-Sum Differential Games
6.3 Summary
6.4 Exercises
7 Discrete-Time Optimal Control Problems
7.1 Variational Calculus for Discrete-Time Systems
7.1.1 Optimum of Performance Functions with Fixed Final Time and Fixed Final Value
7.1.2 Optimum with Fixed Final Time and Free Final Value
7.2 Discrete-Time Optimal Control via Variational Method
7.2.1 Optimal Control with Fixed Final Time and Fixed Final State
7.2.2 Optimal Control with Fixed Final Time and Free Final State
7.3 Discrete-Time Linear-Quadratic Regulation Problems
7.3.1 Linear-Quadratic Regulation Problems with Fixed Final Time and Fixed Final State
7.3.2 Linear-Quadratic Regulation Problems with Fixed Final Time and Free Final State
7.3.3 Optimal Control with Respect to State
7.3.4 Optimal Cost Function
7.3.5 Infinite-Interval Time-Invariant Linear-Quadratic Regulation Problems
7.4 Discrete-Time Linear-Quadratic Tracking Problems
7.5 Discrete-Time Pontryagin's Minimum Principle
7.6 Discrete-Time Dynamic Programming
7.6.1 Optimal Control Problems with Discrete State Values
7.6.2 Optimal Control Problems with Continuous State Values
7.6.3 Discrete-Time Linear-Quadratic Problems
7.7 Discrete-Time Noncooperative Dynamic Games
7.7.1 Formulation of Discrete-Time Noncooperative Dynamic Games
7.7.2 NE of Discrete-Time Noncooperative Dynamic Games
7.7.3 Discrete-Time Linear-Quadratic Noncooperative Dynamic Games
7.8 Discrete-Time Two-Person Zero-Sum Dynamic Games
7.8.1 Formulation of Discrete-Time Two-Person Zero-Sum Dynamic Games
7.8.2 Saddle Point of Discrete-Time Two-Person Zero-Sum Dynamic Games
7.8.3 Discrete-Time Linear-Quadratic Two-Person Zero-Sum Dynamic Games
7.9 Summary
7.10 Exercises
8 Conclusions