Data Driven Model Learning for Engineers: With Applications to Univariate Time Series

✍ Scribed by Guillaume Mercère

• Publisher: Springer
• Year: 2023
• Tongue: English
• Leaves: 218
• Category: Library

✦ Synopsis

The main goal of this comprehensive textbook is to cover the core techniques required to understand some of the basic and most popular model learning algorithms available for engineers, then illustrate their applicability directly with stationary time series. A multi-step approach is introduced for modeling time series which differs from the mainstream in the literature. Singular spectrum analysis of univariate time series, trend and seasonality modeling with least squares and residual analysis, and modeling with ARMA models are discussed in more detail.
As applications of data-driven model learning become widespread in society, engineers need to understand its underlying principles, then the skills to develop and use the resulting data-driven model learning solutions. After reading this book, the users will have acquired the background, the knowledge and confidence to (i) read other model learning textbooks more easily, (ii) use linear algebra and statistics for data analysis and modeling, (iii) explore other fields of applications where model learning from data plays a central role. Thanks to numerous illustrations and simulations, this textbook will appeal to undergraduate and graduate students who need a first course in data-driven model learning. It will also be useful for practitioners, thanks to the introduction of easy-to-implement recipes dedicated to stationary time series model learning. Only a basic familiarity with advanced calculus, linear algebra and statistics is assumed, making the material accessible to students at the advanced undergraduate level.

✦ Table of Contents

Preface
References
Contents
1 Basic Concepts of Time Series Modeling
References
2 Singular Spectrum Analysis of Univariate Time Series
2.1 The Main Ingredients
2.2 The Basic Singular Spectrum Analysis for Low Rank Approximation
2.2.1 Times Series Decomposition
2.2.2 Times Series Reconstruction
2.3 The Impact of the SVD for the Basic SSA
2.3.1 SVD and Separability
2.3.2 SVD and Noise Effect Filtering
2.3.3 In a Nutshell
2.3.4 Another Example
2.4 Take Home Messages
References
3 Trend and Seasonality Model Learning with Least Squares
3.1 Linear Least Squares Solutions
3.1.1 Problem Formulation
3.1.2 Analytic Solution from a Projection Viewpoint
3.1.3 Numerical Solution with a QR Factorization
3.1.4 Numerical Solution with a Singular ValueDecomposition
3.1.5 Linear Least Squares and Condition Number
3.2 A Digression to Nonlinear Least Squares
3.3 Linear Model Complexity Selection and Validation
3.4 Hints for Least Squares Solutions Refinement
3.5 Take Home Messages
References
4 Least Squares Estimators and Residuals Analysis
4.1 Residual Components
4.2 Stochastic Description of the Residuals
4.2.1 Stochastic Process
4.2.2 Stationarity
4.2.3 Ergodicity
4.3 Basic Tests of the Residuals
4.3.1 Autocorrelation Function Test
4.3.2 Portmanteau Test
4.3.3 Turning Point Test
4.3.4 Normality Test
4.4 Statistical Properties of the Least Squares Estimates
4.4.1 Bias, Variance, and Consistency of the Linear Least Squares Estimators
4.4.2 Bias, Variance, and Consistency of the Nonlinear Least Squares Estimators
4.4.3 Least Squares Statistical Properties Validation with the Bootstrap Method
4.4.4 From Least Squares Statistical Properties to Confidence Intervals
Estimated Parameters Confidence Intervals
Estimated Outputs Confidence Intervals
Predicted Outputs Confidence Intervals
4.5 Wold Decomposition
4.6 Take Home Messages
References
5 Residuals Modeling with AR and ARMA Representations
5.1 From Transfer Functions to Linear Difference Equations
5.2 AR and ARMA Model Learning
5.2.1 AR Model Parameters Estimation
Parameters Estimation with Linear Least Squares
Parameters Estimation with the Yule-Walker Algorithm
5.2.2 ARMA Model Parameters Estimation
Parameters Estimation with Pseudolinear Least Squares
Parameters Estimation with Nonlinear Least Squares
5.3 Partial Autocorrelation Function
5.4 Forecasting with AR and ARMA Models
5.5 Take Home Messages
References
6 A Last Illustration to Conclude
A Vectors and Matrices
A.1 Vector Space
A.1.1 Linear Space and Subspace
A.1.2 Inner Product, Induced Norm and Inner Product Space
A.1.3 Complementary Subspaces
A.1.4 Orthogonal Complement and Projection
A.2 Vector
A.2.1 First Definitions
A.2.2 Basic Operations
A.2.3 Span, Vector Space, and Subspace
A.2.4 Linear Dependency and Basis
A.2.5 Euclidean Inner Product and Norm
A.2.6 Orthogonality and Orthonormality
A.3 Matrix
A.3.1 First Definitions
A.3.2 Basic Operations
A.3.3 Symmetry
A.3.4 Hankel and Toeplitz Matrices
A.3.5 Gram, Normal, and Orthogonal Matrices
A.3.6 Vectorization and Frobenius Matrix Norm
A.3.7 Quadratic Form and Positive Definiteness
A.4 Matrix Fundamental Subspaces
A.4.1 Range and Nullspace
A.4.2 Rank
A.5 Matrix Inverses
A.5.1 Square Matrix Inverse
A.5.2 Matrix Inversion Lemmas
A.5.3 Matrix Pseudo-inverse
A.6 Some Useful Matrix Decompositions
A.6.1 QR Decomposition
A.6.2 Singular Value Decomposition
A.6.3 Condition Number and Norms
A.6.4 SVD and Eckart–Young–Mirsky Theorem
A.6.5 Moore–Penrose Pseudo-inverse
A.7 Orthogonal Projector
A.7.1 First Definitions
A.7.2 Orthogonal Projector and Singular ValueDecomposition
References
B Random Variables and Vectors
B.1 Probability Space and Random Variable
B.1.1 Probability Space
B.1.2 Random Variable
B.2 Univariate Random Variable
B.2.1 Cumulative Distribution Function
B.2.2 Probability Density Function
B.2.3 Cumulative Distribution Function and Quantile
B.2.4 Uniform Random Variable
B.2.5 Normal Random Variable
B.2.6 Student's Random Variable
B.2.7 Chi-squared Random Variable
B.2.8 Moments
B.3 Multivariate Random Variable
B.3.1 Basic Idea
B.3.2 2D Joint Distribution Function
B.3.3 2D Joint Probability Density Function
B.3.4 Marginal Distribution and Density Functions
B.3.5 Statistical Independence
B.3.6 Generalized Mean and Moments
B.3.7 Covariance
B.3.8 Correlation Coefficient
B.3.9 Correlation
B.3.10 nxD Joint Cumulative Distribution and DensityFunction
B.3.11 nxD Marginal Distribution and Density Functions
B.3.12 Independence
B.3.13 Uncorrelatedness and Orthogonality
B.3.14 Random Vector
B.3.15 Mean Vector, Covariance, and Correlation Matrices
B.3.16 Pay Attention to the Definition!!!
B.3.17 White Random Vector
B.4 Sum of Random Variables
B.4.1 Sample Mean and Variance
B.4.2 Sample vs. Expected Values
B.4.3 Central Limit Theorem
B.4.4 Sample Mean with Gaussian Assumption
B.4.5 Laws of Large Numbers
B.4.6 Weak Law of Large Numbers
B.4.7 Strong Law of Large Numbers
References
C Data