Williams and Zipser (1989)
proposed two analoglte learning algorithms for fully recurrent networks.
The first method is an exact gradient-following algorithm for problems where data consists of epochs. The second method, called the Real-Time Recurrent Learning ( RTRL ) algorithm, uses data described by a temporal stream of inputs and outputs, without time marks or epochs. In this paper we describe a new implementation of this RTRL algorithm. This improved implementation makes it possible to increase the performance of the learning algorithm during the training phase by using some a priori knowledge about the temporal necessities of the problem. The reduction of the computational expense of the training enables the use of this algorithm for more complex problems. Some simulations of a process control task demonstrate the properties of this algorithm.
Keywords--Temporal pattern recognition, Temporal data processing, Dynamic neural networks, Real-Time Recurrent Learning algorithm.
1. INTRODUCrlON
During the last 10 years some researchers started to explore the importance of time in artificial intelligence (Decortis & Cacciabue, 1988;Klopf & Morgan, 1990). In this area, dynamic neural networks offer the possibility to solve one of the most important aspects of artificial intelligence technologies for process environments: the real-time requirements of a process control system (Stock, 1989). These requirements are mainly a high computation speed, a known computational effort (O'Reilly & Cromarty, 1985), and the use of time to understand the environment (Dockx & Timmermans, 1992). One advantage of artificial neural networks is the low computational effort needed to perform one network cycle (i.e., a complete network task, a pattern matching instruction or a reasoning process) after training. The second advantage is that this effort is known. With expert systems this effort can be quite high and is mostly not known a priori. Another problem in expert systems is the implementation of time dependencies between the input data. With dynamic Acknowledgements: The author would like to thank James Lutsko, Kris Dockx, and Kfirt Meert for their constructive comments during the preparation of this paper.
Requests for reprints should be sent to Thierry Catfolis,