Local minima escape transients by stochastic gradient descent algorithms in blind adaptive equalizers
✍ Scribed by Michael R. Frater; Robert R. Bitmead; C.Richard Johnson Jr
- Publisher
- Elsevier Science
- Year
- 1995
- Tongue
- English
- Weight
- 638 KB
- Volume
- 31
- Category
- Article
- ISSN
- 0005-1098
No coin nor oath required. For personal study only.
✦ Synopsis
Many adaptive algorithms perform stochastic gradient descent on performance surfaces that are not guaranteed to be unimodal. In some examples, it is possible to show that not only is there more than one stationary point on this performance surface, but also that there is at least one incorrect local minimum. In the past, many authors have noted the existence of these incorrect stable equilibria, and noted that transitions between the regions of attraction of these local equilibria are possible. However, very little work has been done to determine the escape times, beyond observing that if the valleys surrounding these undesirable equilibria are very small and shallow, the escape time should not be too large. In this paper, we begin with a general discussion of the escape behaviour of adaptive algorithms, and follow this with an analysis, using diffusion approximations and large deviations theory, of the escape behaviour of the Godard class of blind equalizers. From this analysis, we obtain asymptotic estimates for the expected value of the escape time when leaving the region of attraction of local equilibria. Some observations are made also on the trajectories followed during such escapes. The basis for the computation of escape time estimates is the connection between large deviations and optimal control theory. For this interesting class of adaptive estimation problems, possessing multiple equilibria, the construction and solution of the optimal control problem is approximated, and shown to yield reasonable quantifications.
1. Introduction
Adaptive systems abound in control, signal processing and telecommunications.
The parameter adaptation subsystems of these applications are effectively identical, and analysis of one use provides insight into their behaviour in the others. In this paper we analyse, in some detail, the dynamics of adaptation for a blind equalizer from telecommunications, especially in terms of its ability to escape from the region of attraction of local minima. The conclusions are then drawn to apply this description to adaptive systems in general. *