Control and System Theory of Discrete-Time Stochastic Systems

✍ Scribed by Jan H. van Schuppen

• Publisher: Springer Nature
• Year: 2021
• Tongue: English
• Leaves: 940
• Series: Communications and Control Engineering
• Category: Library

✦ Synopsis

This book helps students, researchers, and practicing engineers to understand the theoretical framework of control and system theory for discrete-time stochastic systems so that they can then apply its principles to their own stochastic control systems and to the solution of control, filtering, and realization problems for such systems. Applications of the theory in the book include the control of ships, shock absorbers, traffic and communications networks, and power systems with fluctuating power flows. The focus of the book is a stochastic control system defined for a spectrum of probability distributions including Bernoulli, finite, Poisson, beta, gamma, and Gaussian distributions. The concepts of observability and controllability of a stochastic control system are defined and characterized. Each output process considered is, with respect to conditions, represented by a stochastic system called a stochastic realization. The existence of a control law is related to stochastic controllability while the existence of a filter system is related to stochastic observability. Stochastic control with partial observations is based on the existence of a stochastic realization of the filtration of the observed process.​

✦ Table of Contents

Preface
Motivation of Control and Filtering of Stochastic Systems
About the Book
Chapter Relations
To the Reader
To the Teacher
Concluding Remarks
Acknowledgements
Contents
Abbreviations and Symbols
Abbreviations
Symbols
1 Control Problems
1.1 Control of a Mooring Tanker
1.2 Control of Freeway Traffic Flow
1.3 Control of a Shock Absorber
1.4 Further Reading
References
2 Probability
2.1 Probability Distribution Functions
2.2 Motivation of the Concept of a Probability Measure
2.3 Sets and σ-Algebras
2.3.1 Operations on Sets
2.3.2 σ-Algebras
2.3.3 The Borel σ-Algebra of the Real Numbers
2.4 Probability Measures
2.4.1 Probability Measures on the Real Numbers
2.4.2 Finite Probability Spaces
2.4.3 Independence
2.5 Random Variables
2.5.1 Real-Valued Measurable Functions
2.5.2 Transformation of a Probability Space by a Random Variable
2.5.3 Support of a Measure of a Random Variable
2.5.4 Finite-Valued Random Variables
2.5.5 Random Variables and σ-Algebras
2.6 Expectation and the Characteristic Function
2.6.1 The Characteristic Function
2.6.2 Expectations of Several Probability Distributions
2.7 Gaussian Random Variables
2.8 Conditional Expectation
2.8.1 Properties of Conditional Expectation
2.8.2 Special Cases of Conditional Expectation
2.9 Conditional Independence
2.10 Computations
2.11 Exercises
2.12 Further Reading
References
3 Stochastic Processes
3.1 Concepts
3.1.1 Construction of a Stochastic Process
3.1.2 Equivalent Processes
3.2 Special Subsets of Stochastic Processes
3.3 Properties of Stochastic Processes
3.3.1 Integrability of Stochastic Processes
3.3.2 Stationarity and Time-Reversibility
3.3.3 Markov Processes
3.4 Gaussian Processes
3.4.1 Covariance Functions
3.4.2 Stationarity and Time-Reversibility of Gaussian Processes
3.4.3 Gauss–Markov Processes
3.5 Finite-Valued Stochastic Processes
3.6 Exercises
3.7 Further Reading
References
4 Gaussian Stochastic Systems
4.1 Modeling of Phenomena as a Stochastic System
4.2 The Concept of a Stochastic System
4.3 Time-Varying Gaussian Systems
4.4 Time-Invariant Gaussian Systems
4.4.1 The Invariant Measure of a Forward Gaussian System Representation
4.4.2 The Invariant Measure of a Backward Gaussian System
4.4.3 Transformations of a Gaussian System
4.5 Relation of Forward and Backward Gaussian System Representations
4.6 Stochastic Observability and Stochastic Co-Observability
4.6.1 Observability and Co-Observability
4.6.2 Stochastic Observability and Stochastic Co-Observability
4.6.3 Stochastic Observability and Stochastic Co-Observability of Gaussian Systems
4.7 Interconnections of Gaussian Systems
4.8 Stochastic Stability
4.9 Gaussian Factor Models and Gaussian Factor Systems
4.10 Computations
4.11 Exercises
4.12 Further Reading
References
5 Stochastic Systems
5.1 Stochastic Systems and Probability Distributions
5.1.1 Output-State Conditional Probability Distribution
5.1.2 Next-State–Current-State Conditional Probability Distributions
5.2 Output in Binary Set
5.3 Output in the Natural Numbers
5.4 Output in a Bounded Interval
5.5 Output in the Positive Real Numbers
5.6 Output in the Real Numbers
5.7 Output-Finite–State-Polytopic Stochastic Systems
5.7.1 Time-Varying Stochastic Systems
5.7.2 Time-Invariant Stochastic Systems
5.7.3 State Set a Polytope
5.7.4 Decompositions of the State Set
5.7.5 Forward and Backward System Representations
5.7.6 Stochastic Observability
5.8 σ-Algebraic Stochastic System
5.9 The Multiple Conditional Independence Relation
5.10 Technicalities
5.11 Further Reading
References
6 Stochastic Realization of Gaussian Systems
6.1 Introduction to Realization Theory
6.2 Motivation
6.3 Weak Gaussian Stochastic Realization Problem
6.4 The Theorem
6.5 Explanation
6.6 The Proof
6.7 Realization Procedures
6.8 State-Space Reduction of a Gaussian System
6.9 Special Stochastic Realizations-1
6.10 Special Stochastic Realizations-2
6.11 A Canonical Form
6.12 Exercises
6.13 Further Reading
References
7 Stochastic Realization
7.1 The Conceptual Framework of Stochastic Realization
7.2 Stochastic Realization of a Tuple of Gaussian Random Variables
7.2.1 Concepts
7.2.2 The Problem
7.2.3 Characterization of Minimality
7.2.4 Classification
7.2.5 Strong Stochastic Realization of a Tuple of Gaussian Random Variables
7.2.6 States in the Frame σ-algebra
7.2.7 Classification
7.3 Stochastic Realization of a Tuple of σ-Algebras
7.3.1 Problem of Stochastic Realization
7.3.2 Concepts
7.3.3 Existence of Minimal State σ-Algebras
7.3.4 State σ-Algebras in the Frame σ-Algebra
7.3.5 Introduction to Characterization
7.3.6 Characterization of Minimal State σ-Algebras-1
7.3.7 Characterization of Minimal State σ-Algebras-2
7.3.8 Relations of Tuples of Minimal State σ-Algebras
7.4 Stochastic Realization of a σ-Algebra Family
7.4.1 Problem
7.4.2 Concepts
7.4.3 Characterization of a Stochastic Realization
7.4.4 Stochastic Realization as a Filter System
7.4.5 Minimality of a Stochastic Realization
7.5 Stochastic Realization of Output-Finite Stochastic Systems
7.6 Further Reading
References
8 Filtering of Gaussian Systems
8.1 Problems of Filtering, Prediction, Smoothing, and Interpolation
8.2 Problem of Filtering
8.3 Time-Varying Kalman Filter
8.4 Time-Varying Kalman Filter and Stochastic Realization
8.4.1 The Kalman Filter and the Wiener Filter
8.4.2 The Kalman Filter and Stochastic Realization
8.4.3 Derivation of the Kalman Filter via Stochastic Realization
8.4.4 Levinson Filter
8.5 Time-Invariant Kalman Filter
8.6 Approximations of a Time-Invariant Kalman Filter
8.7 Sensor Allocation
8.8 Prediction
8.9 Interpolation
8.10 Conditional Kalman Filter
8.11 Exercises
8.12 Further Reading
References
9 Filtering of Stochastic Systems
9.1 Problems of Estimation, Sequential Estimation, and of Filtering
9.2 Finite-Dimensional Filter Systems
9.3 Estimation Theory
9.3.1 Estimator Binomial–Beta
9.3.2 Estimator Poisson–Gamma
9.3.3 Estimator Gamma–Gamma
9.3.4 Estimator Gauss–Gauss
9.3.5 Estimation of a Finite-Valued Random Variable
9.4 Sequential Estimation
9.4.1 Sequential Estimator Binomial–Beta
9.4.2 Sequential Estimator Gamma–Gamma
9.5 Filtering Theory
9.6 Filter of a Poisson–Gamma System
9.7 Filter of an Output-Finite–State-Finite Stochastic System
9.8 Further Reading
References
10 Stochastic Control Systems
10.1 Stochastic Control System
10.2 Gaussian Stochastic Control Systems
10.3 Stochastic Controllability and Stochastic Co-Controllability
10.3.1 Controllability of a Deterministic System
10.3.2 The Concept of Stochastic Controllability
10.3.3 Stochastic Controllability of a Gaussian Control System
10.4 State-Finite Stochastic Control Systems
10.4.1 Definition
10.4.2 Stochastic Controllability
10.5 Further Reading
References
11 Stochastic Control Problems
11.1 Control Problems of Stochastic Control
11.2 Control Laws
11.3 Closed-Loop Stochastic Control Systems
11.4 Stochastic Control Problems
11.5 Control Synthesis and Control Design
11.6 Statistical Decision Problems
11.7 Exercises
11.8 Further Reading
References
12 Stochastic Control with Complete Observations on a Finite Horizon
12.1 Control Problems
12.2 Problem Formulation
12.3 Explanation of Dynamic Programming
12.4 Digression on Optimization
12.5 Digression on Measurable Control Laws
12.6 Dynamic Programming for Additive Cost Functions
12.7 Control of a Gaussian Control System
12.8 Control of a State-Finite Stochastic Control System
12.9 Invariance of a Subset of Value Functions
12.10 Relation of Optimal Control Law and State
12.11 Dynamic Programming for Multiplicative Cost Functions
12.12 Stochastic Control Problems of Economics and of Finance
12.13 Control via System Approximation
12.14 Exercises
12.15 Further Reading
References
13 Stochastic Control with Complete Observations on an Infinite Horizon
13.1 Introduction to Control on an Infinite Horizon
13.2 Average Cost
13.2.1 Problem Formulation
13.2.2 Positive Cost
13.2.3 Control of a Gaussian Control System
13.2.4 Control of a State-Finite Stochastic Control System
13.2.5 Derivation of Dynamic Programming Equation
13.2.6 Dynamic Programming
13.2.7 Computation of an Optimal Control Law
13.3 Discounted Cost
13.3.1 Positive Cost
13.3.2 Control of a Gaussian Stochastic Control System
13.3.3 Control of a State-Finite Stochastic Control System
13.3.4 Procedures
13.4 Minimum-Variance Control with Complete Observations
13.5 Exercises
13.6 Further Reading
References
14 Stochastic Control with Partial Observations on a Finite Horizon
14.1 Motivation
14.2 Problem Formulation
14.3 Stochastic Realization of a Stochastic Control System
14.4 Stochastic Control of a Gaussian Stochastic Control System
14.4.1 Stochastic Realization by Filtering
14.4.2 Stochastic Realization by the Conditional Kalman Filter
14.4.3 Dynamic Programming
14.4.4 Quadratic Cost Rate
14.4.5 Examples
14.4.6 Cost Function Is Exponential–Quadratic
14.4.7 A Tracking Problem
14.5 Control of a State-Finite Stochastic Control System
14.5.1 Problem Formulation
14.5.2 Filtering
14.5.3 Dynamic Programming
14.5.4 Example
14.5.5 Alternative Control Problem
14.6 Exercises
14.7 Further Reading
References
15 Stochastic Control with Partial Observations on an Infinite Horizon
15.1 Problem Issues
15.2 Control of a Gaussian Stochastic Control System
15.2.1 Problem Formulation
15.2.2 Stochastic Realization
15.2.3 Dynamic Programming
15.2.4 Quadratic Cost Rate
15.3 Minimum-Variance Control with Partial Observations
15.4 Further Reading
References
16 Stochastic Control Theory
16.1 Research Problems of Control of Stochastic Systems
16.1.1 Control for the Effective Interaction of Control and Observation
16.1.2 Control of Partially Observed Stochastic Control Systems
16.1.3 Control of Communication Systems
16.1.4 Control of a Networked Stochastic Control System
16.1.5 Multiple Conditional Independence
16.2 General Optimality Conditions
16.3 Stochastic Control Via a Measure Transformation
16.4 Further Reading
References
17 Appendix: Mathematics
17.1 Algebra of Sets
17.1.1 Relations and Canonical Forms
17.1.2 Order Relation
17.1.3 Functions
17.2 Algebraic Structures
17.3 Linear Algebra and Linear Dependence
17.4 Matrices
17.4.1 Linear Transformation and Their Matrix Representations
17.4.2 Multiplicative Factorization of a Matrix
17.4.3 Square Matrices
17.4.4 Determinant, Trace, and Norm
17.4.5 Spectral Theory
17.4.6 Inverses of Nonsingular Square Matrices
17.4.7 Symmetric Square Matrices
17.4.8 Contragradient Transform
17.5 Analysis
17.6 Geometry
17.7 Optimization
17.8 Further Reading
References
18 Appendix: Positive Matrices
18.1 Problems
18.2 The Positive Real Numbers and a Positive Vector Space
18.3 Definitions of Positive Matrices
18.4 Geometry and Cones
18.5 Units
18.6 Similarity
18.7 Eigenvalues and Eigenvectors of Positive Matrices
18.8 Eigenvalues and Eigenvectors of Stochastic Matrices
18.8.1 A Partition of the Set of Stochastic Matrices
18.8.2 Overview of Convergence Results
18.8.3 Irreducible Matrices
18.8.4 Reducible Matrices
18.8.5 Weighted Average of Powers
18.9 Multiplicative Factorization
18.9.1 Problem
18.9.2 Equivalence
18.9.3 Permutation Matrices
18.9.4 Circulant Doubly Stochastic Matrices
18.9.5 Doubly Stochastic Matrices
18.9.6 Multiplicative Factorizations
18.9.7 Positive Matrices
18.9.8 Positive Matrices—Extremal Cones
18.9.9 Embedding in an Extremal Cone
18.10 Computations
18.11 Further Reading
References
19 Appendix: Probability
19.1 Sets and the Monotone Class Theorems
19.2 Probability Measures
19.3 Stable Subsets of Probability Distribution Functions
19.4 Gaussian Random Variables
19.5 Spaces and Sequences of Random Variables
19.5.1 Sequences of Random Variables
19.5.2 Convergence of Sequences of Random Variables
19.6 Conditional Expectation and Conditional Probability
19.7 Conditionally Gaussian Random Variables
19.8 Conditional Independence Continued
19.8.1 Conditional Independence and Finite Probability Spaces
19.9 Measure Transformations
19.10 The Family of Exponential Probability Distributions
19.11 Pseudo-distances on the Set of Probability Measures
19.12 P-Essential Infima
19.13 Further Reading
References
20 Appendix: Stochastic Processes
20.1 Stochastic Processes and Filtrations
20.2 Martingale Theory
20.3 Stochastic Processes and Stopping Times
20.4 Supermartingale Convergence
20.5 Ergodicity
20.6 Further Reading
References
21 Appendix: Control and System Theory of Deterministic Systems
21.1 Deterministic Control Systems
21.2 Controllability
21.2.1 Stabilizability
21.2.2 Controllability After Feedback
21.2.3 Controllability of a Time-Varying Linear Control System
21.3 Observability
21.4 Geometric Approach to Linear Systems
21.5 Zero-Output Dynamics
21.6 Inverse of a Linear System
21.7 Canonical Factorization of a Deterministic Map
21.8 Realization Theory for Linear Systems
21.8.1 Realization Theorem and Its Proof
21.8.2 Reduction of a Nonminimal Realization
21.9 Stability
21.10 Further Reading
References
22 Appendix: Matrix Equations
22.1 Lyapunov Equation
22.2 Algebraic Riccati Equations of Filtering and of Control
22.2.1 Existence and Characterization of Solution
22.2.2 Proof for the Filter Algebraic Riccati Equation
22.2.3 Dependence of the Solution on Matrices
22.2.4 Computation of the Solution
22.2.5 The Hamiltonian Approach
22.3 Algebraic Riccati Equation of Gaussian Stochastic Realization
22.3.1 Existence and Characterization of the Solution
22.3.2 An Iterative Algorithm for the Algebraic Riccati Equation of Stochastic Realization
22.3.3 The Hamiltonian Approach to the Algebraic Riccati Equation of Stochastic Realization
22.4 Further Reading
References
23 Appendix: Covariance Functions and Dissipative Systems
23.1 Definitions
23.2 Storage Functions
23.3 Relations
23.4 Algebraic Characterization of Dissipative Linear Systems
23.5 Further Reading
References
24 Appendix: State-Variance Matrices
24.1 Definition and Problem Formulation
24.2 Transformations
24.3 The Geometric Structure
24.4 Regularity
24.5 The Boundary of the Set of State-Variance Matrices
24.6 Singular Boundary Matrices
24.7 The Classification of State-Variance Matrices
24.8 Further Reading
References
Index