Continuous-Time Markov Decision Processes: Theory and Applications

✍ Scribed by Xianping Guo, Onésimo Hernández-Lerma (auth.)

Publisher: Springer-Verlag Berlin Heidelberg
Year: 2009
Tongue: English
Leaves: 239
Series: Stochastic Modelling and Applied Probability 62
Edition: 1
Category: Library

No coin nor oath required. For personal study only.

✦ Synopsis

Continuous-time Markov decision processes (MDPs), also known as controlled Markov chains, are used for modeling decision-making problems that arise in operations research (for instance, inventory, manufacturing, and queueing systems), computer science, communications engineering, control of populations (such as fisheries and epidemics), and management science, among many other fields. This volume provides a unified, systematic, self-contained presentation of recent developments on the theory and applications of continuous-time MDPs. The MDPs in this volume include most of the cases that arise in applications, because they allow unbounded transition and reward/cost rates. Much of the material appears for the first time in book form.

✦ Table of Contents

Front Matter....Pages I-XVII
Introduction and Summary....Pages 1-8
Continuous-Time Markov Decision Processes....Pages 9-18
Average Optimality for Finite Models....Pages 19-53
Discount Optimality for Nonnegative Costs....Pages 55-70
Average Optimality for Nonnegative Costs....Pages 71-86
Discount Optimality for Unbounded Rewards....Pages 87-103
Average Optimality for Unbounded Rewards....Pages 105-125
Average Optimality for Pathwise Rewards....Pages 127-142
Advanced Optimality Criteria....Pages 143-162
Variance Minimization....Pages 163-173
Constrained Optimality for Discount Criteria....Pages 175-186
Constrained Optimality for Average Criteria....Pages 187-194
Back Matter....Pages 195-234

✦ Subjects

Operations Research, Mathematical Programming; Probability Theory and Stochastic Processes

📜 SIMILAR VOLUMES

Continuous-Time Markov Decision Processe

📁 Continuous-Time Markov Decision Processes

✍ Alexey Piunovskiy, Yi Zhang 📂 Library 📅 2020 🏛 Springer 🌐 English

This book offers a systematic and rigorous treatment of continuous-time Markov decision processes, covering both theory and possible applications to queueing systems, epidemiology, finance, and other fields. Unlike most books on the subject, much attention is paid to problems with functional constra

Continuous Time Markov Processes

📁 Continuous Time Markov Processes

✍ Thomas M. Liggett 📂 Library 📅 2010 🏛 American Mathematical Society 🌐 English

Markov processes are among the most important stochastic processes for both theory and applications. This book develops the general theory of these processes and applies this theory to various special examples. The initial chapter is devoted to the most important classical example--one-dimensional B

Continuous Time Markov Processes

📁 Continuous Time Markov Processes

✍ Thomas M. Liggett 📂 Library 📅 2010 🏛 American Mathematical Society 🌐 English

Continuous time Markov processes

📁 Continuous time Markov processes

✍ Liggett T.M. 📂 Library 📅 2010 🏛 AMS 🌐 English

Handbook of Markov Decision Processes: M

📁 Handbook of Markov Decision Processes: Methods and Applications

✍ Eugene A. Feinberg, Adam Shwartz (auth.), Eugene A. Feinberg, Adam Shwartz (eds. 📂 Library 📅 2002 🏛 Springer US 🌐 English

<p>Eugene A. Feinberg Adam Shwartz This volume deals with the theory of Markov Decision Processes (MDPs) and their applications. Each chapter was written by a leading expert in the re spective area. The papers cover major research areas and methodologies, and discuss open questions and future resea

Handbook of Markov Decision Processes: M

📁 Handbook of Markov Decision Processes: Methods and Applications

✍ Eugene A. Feinberg, Adam Shwartz (eds.) 📂 Library 📅 2002,2012 🏛 Springer 🌐 English