Foundations of Average-Cost Nonhomogeneous Controlled Markov Chains

✍ Scribed by Xi-Ren Cao

Publisher: Springer International Publishing;Springer
Year: 2021
Tongue: English
Leaves: 128
Series: SpringerBriefs in Electrical and Computer Engineering
Edition: 1st ed.
Category: Library

No coin nor oath required. For personal study only.

✦ Synopsis

This Springer brief addresses the challenges encountered in the study of the optimization of time-nonhomogeneous Markov chains. It develops new insights and new methodologies for systems in which concepts such as stationarity, ergodicity, periodicity and connectivity do not apply.

This brief introduces the novel concept of confluencity and applies a relative optimization approach. It develops a comprehensive theory for optimization of the long-run average of time-nonhomogeneous Markov chains. The book shows that confluencity is the most fundamental concept in optimization, and that relative optimization is more suitable for treating the systems under consideration than standard ideas of dynamic programming. Using confluencity and relative optimization, the author classifies states as confluent or branching and shows how the under-selectivity issue of the long-run average can be easily addressed, multi-class optimization implemented, and Nth biases and Blackwell optimality conditions derived. These results are presented in a book for the first time and so may enhance the understanding of optimization and motivate new research ideas in the area.

✦ Table of Contents

Front Matter ....Pages i-viii
Introduction (Xi-Ren Cao)....Pages 1-12
Confluencity and State Classification (Xi-Ren Cao)....Pages 13-27
Optimization of Average Rewards and Bias: Single Class (Xi-Ren Cao)....Pages 29-58
Optimization of Average Rewards: Multi-Chains (Xi-Ren Cao)....Pages 59-78
The Nth-Bias and Blackwell Optimality (Xi-Ren Cao)....Pages 79-108
Back Matter ....Pages 109-120

✦ Subjects

Engineering; Control; Statistics and Computing/Statistics Programs; Probability Theory and Stochastic Processes; Optimization

📜 SIMILAR VOLUMES

Markov Processes and Controlled Markov C

📁 Markov Processes and Controlled Markov Chains

✍ E. B. Dynkin (auth.), Zhenting Hou, Jerzy A. Filar, Anyue Chen (eds.) 📂 Library 📅 2002 🏛 Springer US 🌐 English

<p><P>The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations r

Foundations of Cost Control

📁 Foundations of Cost Control

✍ Daniel Traster 📂 Library 📅 2012 🏛 Pearson 🌐 English

Cost Control: A Fundamental Approach will inspire you to learn cost control as an essential skill for any future chef or foodservice manager. The text begins by helping you master key culinary math basics, making subsequent cost control equations easier to understand and compute. Balancing real-worl

Continuous Average Control of Piecewise

📁 Continuous Average Control of Piecewise Deterministic Markov Processes

✍ Oswaldo Luiz do Valle Costa, Francois Dufour 📂 Library 📅 2013 🏛 Springer 🌐 English

The intent of this book is to present recent results in the control theory for the long run average continuous control problem of piecewise deterministic Markov processes (PDMPs). The book focuses mainly on the long run average cost criteria and extends to the PDMPs some well-known techniques relat

Continuous Average Control of Piecewise

📁 Continuous Average Control of Piecewise Deterministic Markov Processes

✍ Oswaldo Luiz do Valle Costa, Francois Dufour (auth.) 📂 Library 📅 2013 🏛 Springer-Verlag New York 🌐 English

<p>The intent of this book is to present recent results in the control theory for the long run average continuous control problem of piecewise deterministic Markov processes (PDMPs). The book focuses mainly on the long run average cost criteria and extends to the PDMPs some well-known techniques rel

Continuous Average Control of Piecewise

📁 Continuous Average Control of Piecewise Deterministic Markov Processes

✍ Oswaldo Luiz do Valle Costa, Francois Dufour 📂 Library 📅 2013 🏛 Springer 🌐 English

Self-learning Control of Finite Markov C

📁 Self-learning Control of Finite Markov Chains

✍ Alexander S. Poznyak, K. Najim, E. Gómez-Ramírez 📂 Library 📅 2000 🏛 Marcel Dekker /CRC 🌐 English

Presents a number of new and potentially useful self-learning (adaptive) control algorithms and theoretical as well as practical results for both unconstrained and constrained finite Markov chains-efficiently processing new information by adjusting the control strategies directly or indirectly.