The Extended Kalman Filter (EKF) has become a standard
technique used in a number of nonlinear estimation and machine
learning applications. These include estimating the
state of a nonlinear dynamic system, estimating parameters
for nonlinear system identification (e.g., learning the
weights of a neural network), and dual estimation (e.g., the
Expectation Maximization (EM) algorithm) where both states
and parameters are estimated simultaneously.
This paper points out the flaws in using the EKF, and
introduces an improvement, the Unscented Kalman Filter
(UKF), proposed by Julier and Uhlman [5]. A central and
vital operation performed in the Kalman Filter is the propagation
of a Gaussian random variable (GRV) through the
system dynamics. In the EKF, the state distribution is approximated
by a GRV, which is then propagated analytically
through the first-order linearization of the nonlinear
system. This can introduce large errors in the true posterior
mean and covariance of the transformed GRV, which may
lead to sub-optimal performance and sometimes divergence
of the filter. The UKF addresses this problem by using a
deterministic sampling approach. The state distribution is
again approximated by a GRV, but is now represented using
a minimalset of carefully chosen sample points. These sample
points completely capture the true mean and covariance
of the GRV, and when propagated through the true nonlinear
system, captures the posterior mean and covariance
accurately to the 3rd order (Taylor series expansion) for any
nonlinearity. The EKF, in contrast, only achieves first-order
accuracy. Remarkably, the computational complexity of the
UKF is the same order as that of the EKF.
Julier and Uhlman demonstrated the substantial performance
gains of the UKF in the context of state-estimation
for nonlinear control. Machine learning problems were not
considered. We extend the use of the UKF to a broader class
of nonlinear estimation problems, including nonlinear system
identification, training of neural networks, and dual estimation
problems. Our preliminary results were presented
in [13]. In this paper, the algorithms are further developed
and illustrated with a number of additional examples.