Kalman Filtering and Neural Networks

✍ Scribed by Simon Haykin

• Publisher: Wiley-Interscience
• Year: 2001
• Tongue: English
• Leaves: 299
• Edition: 1
• Category: Library

✦ Synopsis

State-of-the-art coverage of Kalman filter methods for the design of neural networks

This self-contained book consists of seven chapters by expert contributors that discuss Kalman filtering as applied to the training and use of neural networks. Although the traditional approach to the subject is almost always linear, this book recognizes and deals with the fact that real problems are most often nonlinear.

The first chapter offers an introductory treatment of Kalman filters with an emphasis on basic Kalman filter theory, Rauch-Tung-Striebel smoother, and the extended Kalman filter. Other chapters cover:

• An algorithm for the training of feedforward and recurrent multilayered perceptrons, based on the decoupled extended Kalman filter (DEKF)
• Applications of the DEKF learning algorithm to the study of image sequences and the dynamic reconstruction of chaotic processes
• The dual estimation problem
• Stochastic nonlinear dynamics: the expectation-maximization (EM) algorithm and the extended Kalman smoothing (EKS) algorithm
• The unscented Kalman filter

Each chapter, with the exception of the introduction, includes illustrative applications of the learning algorithms described here, some of which involve the use of simulated and real-life data. Kalman Filtering and Neural Networks serves as an expert resource for researchers in neural networks and nonlinear dynamical systems.

✦ Table of Contents

Cover
Kalman Filter and Neural Networks
Adaptive and Learning Systems for Signal Processing, Communications, and Control (Series)
Title Page
Copyright Page
CONTENTS
PREFACE
Contributors
1 Kalman Filters (Simon Haykin)
1.1 Introduction
1.2 Optimum Estimates
1.3 Kalman Filter
1.4 Divergence Phenomenon: Square-Root Filtering
1.5 Rauch–Tung–Striebel Smoother
1.6 Extended Kalman Filter
1.7 Summary
References
2 Parameter-Based Kalman Filter Training: Theory and Implementation (Gintaras V. Puskorius and Lee A. Feldkamp)
2.1 Introduction
2.2 Network Architectures
2.3 The EKF Procedure
2.3.1 Global EKF Training
2.3.2 Learning Rate and Scaled Cost Function
2.3.3 Parameter Settings
2.4 Decoupled EKF (DEKF)
2.5 Multistream Training
2.5.1 Some Insight into the Multistream Technique
2.5.2 Advantages and Extensions of Multistream Training
2.6 Computational Considerations
2.6.1 Derivative Calculations
2.6.2 Computationally Efficient Formulations for Multiple-Output Problems
2.6.3 Avoiding Matrix Inversions
2.6.4 Square-Root Filtering
2.7 Other Extensions and Enhancements
2.7.1 EKF Training with Constrained Weights
2.7.2 EKF Training with an Entropic Cost Function
2.7.3 EKF Training with Scalar Errors
2.8 Automotive Applications of EKF Training
2.8.1 Air=Fuel Ratio Control
2.8.2 Idle Speed Control
2.8.3 Sensor-Catalyst Modeling
2.8.4 Engine Misfire Detection
2.8.5 Vehicle Emissions Estimation
2.9 Discussion
2.9.1 Virtues of EKF Training
2.9.2 Limitations of EKF Training
2.9.3 Guidelines for Implementation and Use
References
3 Learning Shape and Motion from Image Sequences (Gaurav S. Patel, Sue Becker, and Ron Racine)
3.1 Introduction
3.2 Neurobiological and Perceptual Foundations of our Model
3.3 Network Description
3.4 Experiment 1
3.5 Experiment 2
3.6 Experiment 3
3.7 Discussion
References
4 Chaotic Dynamics (Gaurav S. Patel and Simon Haykin)
4.1 Introduction
4.2 Chaotic (Dynamic) Invariants
4.3 Dynamic Reconstruction
4.4 Modeling Numerically Generated Chaotic Time Series
4.4.1 Logistic Map
4.4.2 Ikeda Map
4.4.3 Lorenz Attractor
4.5 Nonlinear Dynamic Modeling of Real-World Time Series
4.5.1 Laser Intensity Pulsations
4.5.2 Sea Clutter Data
4.6 Discussion
References
5 Dual Extended Kalman Filter Methods (Eric A. Wan and Alex T. Nelson)
5.1 Introduction
5.2 Dual EKF – Prediction Error
5.2.1 EKF – State Estimation
5.2.2 EKF –Weight Estimation
5.2.3 Dual Estimation
5.3 A Probabilistic Perspective
5.3.1 Joint Estimation Methods
5.3.2 Marginal Estimation Methods
5.3.3 Dual EKF Algorithms
5.3.4 Joint EKF
5.4 Dual EKF Variance Estimation
5.5 Applications
5.5.1 Noisy Time-Series Estimation and Prediction
5.5.2 Economic Forecasting – Index of Industrial Production
5.5.3 Speech Enhancement
5.6 Conclusions
Acknowledgments
Appendix A: Recurrent Derivative of the Kalman Gain
Appendix B: Dual EKF with Colored Measurement Noise
References
6 Learning Nonlinear Dynamical System Using the Expectation-Maximization Algorithm (Sam T. Roweis and Zoubin Ghahramani)
6.1 Learning Stochastic Nonlinear Dynamics
6.1.1 State Inference and Model Learning
6.1.2 The Kalman Filter
6.1.3 The EM Algorithm
6.2 Combining EKS and EM
6.2.1 Extended Kalman Smoothing (E-step)
6.2.2 Learning Model Parameters (M-step)
6.2.3 Fitting Radial Basis Functions to Gaussian Clouds
6.2.4 Initialization of Models and Choosing Locations for RBF Kernels
6.3 Results
6.3.1 One- and Two-Dimensional Nonlinear State-Space Models
6.3.2 Weather Data
6.4 Extensions
6.4.1 Learning the Means and Widths of the RBFs
6.4.2 On-Line Learning
6.4.3 Nonstationarity
6.4.4 Using Bayesian Methods for Model Selection and Complexity Control
6.5 Discussion
6.5.1 Identifiability and Expressive Power
6.5.2 Embedded Flows
6.5.3 Stability
6.5.4 Takens’ Theorem and Hidden States
6.5.5 Should Parameters and Hidden States be Treated Differently?
6.6 Conclusions
Acknowledgments
Appendix: Expectations Required to Fit the RBFs
References
7 The Unscented Kalman Filter (Eric A. Wan and Rudolph van der Merwe)
7.1 Introduction
7.2 Optimal Recursive Estimation and the EKF
7.3 The Unscented Kalman Filter
7.3.1 State-Estimation Examples
7.3.2 The Unscented Kalman Smoother
7.4 UKF Parameter Estimation
7.4.1 Parameter-Estimation Examples
7.5 UKF Dual Estimation
7.5.1 Dual Estimation Experiments
7.6 The Unscented Particle Filter
7.6.1 The Particle Filter Algorithm
7.6.2 UPF Experiments
7.7 Conclusions
Appendix A: Accuracy of the Unscented Transformation
Appendix B: Efficient Square-Root UKF Implementations
References
Index