Some informal remarks are offered on the present status and future direction of stochastic control.
Summary--It is indicated that optimal stochastic control is still in its infancy, and that at the present time it has little use in practice although a wide class of problems can be precisely stated. A brief survey of the problem involved in attempting to formulate and to solve optimal stochastic control problems is discussed along with the corresponding results and thoughts about future research. Mathematical niceties are not presented but a list of references where they may be found is included. "STOCHASTIC CONTROL" is a convenient misnomer for the control of systems subject to stochastic disturbances. This branch of control theory has evolved, since 1960, largely within the framework of dynamic programming [2]. A few simple, academic problems have been solved more-or-less explicitly. A larger class can be, at least, precisely stated, and for some of these existence theorems for optimal controls have been obtained. The subject is intricate enough to provide a number of fascinating analytical and computational problems. It has not yet proved to be of much technological importance.
In this presentation, certain known results will be briefly surveyed together with some unsolved problems. Some statements will be tendentious, and others should be suitably hedged. For brevity all mathematical niceties will be left out.
I believe it was Norbert Wiener who first had the insight to observe that all cybernetic situations-i.e. those involving control and communication-are essentially stochastic [1]. There is no need of feedback and thus, in principle, no problem, if no chance phenomena are present. I shall discuss here system models where stochastic elements are featured explicitly. However, we shall be concerned only with small dynamic systems, described by random differential equations, and shall ignore the "large" systems of operations research.
First, some results. There is a solvable stochastic version of the well-known linear-quadratic regulator problem. One considers * Edited transcript of a lecture presented at the 1968 Joint Automatic Control Conference.