✦ LIBER ✦

Linear stochastic approximation driven by slowly varying Markov chains

✍ Scribed by Vijay R. Konda; John N. Tsitsiklis

Publisher: Elsevier Science
Year: 2003
Tongue: English
Weight: 241 KB
Volume: 50
Category: Article
ISSN: 0167-6911
DOI: 10.1016/s0167-6911(03)00132-4

No coin nor oath required. For personal study only.

✦ Synopsis

We study a linear stochastic approximation algorithm that arises in the context of reinforcement learning. The algorithm employs a decreasing step-size, and is driven by Markov noise with time-varying statistics. We show that under suitable conditions, the algorithm can track the changes in the statistics of the Markov noise, as long as these changes are slower than the rate at which the step-size of the algorithm goes to zero.