Penalty function and adaptive control of constrained finite Markov chains

✍ Scribed by K. Najim; A. S. Poznyak

• Publisher: John Wiley and Sons
• Year: 1998
• Tongue: English
• Weight: 161 KB
• Volume: 12
• Category: Article
• ISSN: 0890-6327
• DOI: 10.1002/(sici)1099-1115(199811)12:7<545::aid-acs511>3.0.co;2-j

✦ Synopsis

In this paper we consider the adaptive control of constrained finite ergodic controller Markov chains whose transition probabilities are unknown. The control policy is designed to achieve the minimization of a loss function under a set of inequality constraints. The average values of conditional mathematical expectations of this loss function and constraints are also assumed to be unknown. A regularized penalty function is introduced to derive an adaptive control algorithm. In this algorithm the transition probabilities of the Markov chain and the average values of the constraints are estimated at each time n. The control policy is adjusted using the Bush-Mosteller reinforcement scheme as a stochastic approximation procedure. Its asymptotic properties are stated. We establish that the optimal convergence rate is equal to n>B ( is any small positive parameter).