Introduction to Random Signals, Estimation Theory, and Kalman Filtering

✍ Scribed by M. Sami Fadali

• Publisher: Springer
• Year: 2024
• Tongue: English
• Leaves: 489
• Category: Library

✦ Synopsis

This book provides first-year graduate engineering students and practicing engineers with a solid introduction to random signals and estimation. It includes a statistical background that is often omitted in other textbooks but is essential for a clear understanding of estimators and their properties. The book emphasizes applicability rather than mathematical theory. It includes many examples and exercises to demonstrate and learn the theory that makes extensive use of MATLAB and its toolboxes. Although there are several excellent books on random signals and Kalman filtering, this book fulfills the need for a book that is suitable for a single-semester course that covers both random signals and Kalman filters and is used for a two-semester course for students that need remedial background. For students interested in more advanced studies in the area, the book provides a bridge between typical undergraduate engineering education and more advanced graduate-level courses.

✦ Table of Contents

Preface
Contents
About the Author
Acronyms
List of Figures
List of Tables
1 Review of Probability Theory
1.1 Interpretations of Probability
1.2 Axiomatic Definition of Probability
1.3 Marginal Probability
1.4 Conditional Probability
1.5 Independence
1.6 Bayes’ Theorem
Bibliography
2 Random Variables
2.1 Mathematical Characterization of a Random Variable
2.2 Expectation of a Random Variable
2.3 Moments
2.3.1 Moment Generating Function and Characteristic Function
2.4 Normal or Gaussian Density
2.4.1 Right Tail Probability
2.5 Multiple Random Variables
2.5.1 Marginal Distributions
2.5.2 Conditional Probability Density
2.6 Correlation RX and Covariance CX
2.7 Multivariate Normal Distribution
2.7.1 Properties of Multivariate Normal
2.8 Transformation of Random Variables
2.8.1 Linear Transformation
2.8.2 Diagonalizing Transformation
2.8.3 Nonlinear Transformation
2.9 Pseudorandom Number Generators
2.9.1 True Random Number Generators
2.10 The Method of Moments
Bibliography
3 Random Signals
3.1 Random Processes
3.1.1 Joint Densities
3.1.2 Gaussian Random Process
3.2 Autocorrelation
3.3 Stationary Random Process
3.4 Ergodic Random Processes
3.5 Properties of Autocorrelation
3.6 Cross-Correlation Function
3.6.1 Time Delay Estimation
3.7 Power Spectral Density Function (PSD)
3.7.1 Properties of the Power Spectral Density (PSD)
3.7.2 Cross-Spectral Density Function
3.8 Spectral Factorization
3.8.1 Continuous-Time Processes
3.8.2 Discrete-Time Processes
3.9 Examples of Stochastic Processes
3.9.1 Markov Processes
Appendix 3.1 Brief Review of the Two-Sided Z-Transform
Bibliography
4 Linear System Response to Random Inputs
4.1 Calculus for Random Signals
4.1.1 Continuity
4.2 Response to Random Input
4.3 Continuous-Time (CT) Random Signals
4.3.1 Mean Response
4.3.2 Stationary Steady-State Analysis for Continuous-Time Systems
4.3.3 Shaping (Innovations) Filter
4.4 Nonstationary Analysis for Continuous-Time Systems
4.4.1 Zero-Input Response
4.4.2 Forced (Zero-State) Response MIMO Time-Varying Case
4.4.3 Covariance Computation
4.5 Discrete-Time (DT) Random Signals
4.5.1 Mean Response
4.5.2 Stationary Steady-State Analysis for Discrete-Time Systems
4.5.3 Nonstationary Analysis for Discrete-Time Systems
Bibliography
5 Estimation and Estimator Properties
5.1 Small Sample Properties
5.1.1 Unbiased Estimators
5.1.2 Efficiency
5.2 Large Sample Properties
5.2.1 Consistent Estimators
5.2.2 Asymptotic Efficiency
5.2.3 Asymptotic Normality
5.3 Random Sample
5.3.1 Sufficient Statistics
5.4 Estimation for the Autocorrelation and the Power Spectral Density
5.4.1 Autocorrelation Standard Estimate (ACS)
5.4.2 Periodogram
References
6 Least-Squares Estimation
6.1 Linear Model
6.2 Properties of the WLS Estimator
6.3 Best Linear Unbiased Estimator (BLUE)
Bibliography
7 The Likelihood Function and Signal Detection
7.1 The Likelihood Function
7.2 Likelihood Ratio
7.3 Signal Detection
7.4 Matched Filters
Bibliography
8 Maximum-Likelihood Estimation
8.1 Maximum-Likelihood Estimator (MLE)
8.2 Properties of Maximum-Likelihood Estimators
8.3 Comparison of Estimators
8.4 Maximum a Posteriori (MAP)
8.5 Numerical Computation of the ML Estimate
8.5.1 MATLAB MLE
Bibliography
9 Minimum Mean-Square Error Estimation
9.1 Minimum Mean-Square Error
9.1.1 Orthogonality
9.1.2 Bayesian Estimation
9.2 Batch Versus Recursive Computation
9.3 The Discrete Kalman Filter
9.4 Expressions for the Error Covariance
9.4.1 Deterministic Input
9.4.2 Separation Principle
9.5 Information Filter
9.6 Steady-State Kalman Filter and Stability
9.6.1 Discrete Lyapunov Equation
Bibliography
10 Generalizing the Basic Discrete Kalman Filter
10.1 Correlated Noise
10.1.1 Equivalent Model with Uncorrelated Noise
10.1.2 Delayed Process Noise
10.2 Colored Noise
10.3 Reduced-Order Estimator for Perfect Measurements
10.4 Schmidt–Kalman Filter
10.5 Sequential DKF Computation
10.6 Square Root Filtering
Bibliography
11 Prediction and Smoothing
11.1 Prediction
11.2 Smoothing
11.3 Fixed-Point Smoothing
11.3.1 Properties of Fixed-Point Smoother
11.4 Fixed-Lag Smoother
11.4.1 Properties of Fixed-Lag Smoother
11.5 Fixed-Interval Smoothing
Bibliography
12 Nonlinear Filtering
12.1 The Extended and Linearized Kalman Filters
12.2 Unscented Transformation and the Unscented Kalman Filter
12.2.1 Unscented Kalman Filter
12.3 Ensemble Kalman Filter
12.4 Bayesian Filter
12.5 Particle Filters
12.6 Degeneracy
Bibliography
13 The Expectation Maximization Algorithm
13.1 Maximum Likelihood Estimation with Incomplete Data
13.2 Exponential Family
13.3 EM for the Multivariate Normal Distribution
13.4 Distribution Mixture
13.5 Gaussian Mixture
Bibliography
14 Hidden Markov Models
14.1 Markov Chains
14.2 Hidden Markov Model
14.3 The Forward Algorithm
14.4 Hidden Markov Modeling
14.5 The Backward Algorithm
14.6 The Baum–Welch Algorithm: Application of EM to HMM
14.7 Minimum Path Problem
14.8 MATLAB Commands
Bibliography
Appendix A Table of Integrals
Appendix B Table of Fourier Transforms
Appendix C Table of Two-Sided Laplace and Z-Transforms
Appendix D Computation and Computational Errors
Bibliography
Appendix E The Continuous-Time Kalman Filter
E.1 The Optimal Gain
E.2 Autocorrelation of the State Vector
E.3 The Lyapunov and Riccati Equations
E.3.a The Lyapunov Equation
E.3.b The Riccati Equation
E.4 Steady-State Filter
Bibliography
Appendix F Modes of Convergence
Topics
F.1 Deterministic Convergence
F.2 Stochastic Convergence
F.2.i Convergence in Law
F.2.ii Convergence in Probability
F.2.iii Convergence in rth Mean
F.2.iv Almost Sure Convergence
Bibliography
Appendix G State-Space Models and Their Properties
G.1 Continuous-Time State-Space Models
G.2 Discrete-Time Systems
G.3 State-Space Realizations
G.4 Stability
G.5 Controllability and Observability
G.6 Similarity Transformation
Appendix H Review of Linear Algebra
H.1 Determinant of a Matrix
H.2 Inverse of a Matrix
H.3 Combinations of Operations
H.4 Trace of a Matrix
H.5 Linearly Independent Vectors
H.6 Eigenvalues and Eigenvectors
H.7 Eigenvalues and Eigenvectors
H.8 Partitioned Matrix
H.9 Norms
H.10 Quadratic Forms
H.11 Singular Value Decomposition and Pseudoinverses
Eigenvalues and Singular Values
Singular Value Inequalities
H.12 Matrix Differentiation/integration
H.13 Kronecker Product
Index