Markov Decision Processes in Practice

✍ Scribed by Richard J. Boucherie, Nico M. van Dijk (eds.)

• Publisher: Springer International Publishing
• Year: 2017
• Tongue: English
• Leaves: 563
• Series: International Series in Operations Research & Management Science 248
• Edition: 1
• Category: Library

✦ Synopsis

This book presents classical Markov Decision Processes (MDP) for real-life applications and optimization. MDP allows users to develop and formally support approximate and simple decision rules, and this book showcases state-of-the-art applications in which MDP was key to the solution approach. The book is divided into six parts. Part 1 is devoted to the state-of-the-art theoretical foundation of MDP, including approximate methods such as policy improvement, successive approximation and infinite state spaces as well as an instructive chapter on Approximate Dynamic Programming. It then continues with five parts of specific and non-exhaustive application areas. Part 2 covers MDP healthcare applications, which includes different screening procedures, appointment scheduling, ambulance scheduling and blood management. Part 3 explores MDP modeling within transportation. This ranges from public to private transportation, from airports and traffic lights to car parking or charging your electric car . Part 4 contains three chapters that illustrates the structure of approximate policies for production or manufacturing structures. In Part 5, communications is highlighted as an important application area for MDP. It includes Gittins indices, down-to-earth call centers and wireless sensor networks. Finally Part 6 is dedicated to financial modeling, offering an instructive review to account for financial portfolios and derivatives under proportional transactional costs. The MDP applications in this book illustrate a variety of both standard and non-standard aspects of MDP modeling and its practical use. This book should appeal to readers for practitioning, academic research and educational purposes, with a background in, among others, operations research, mathematics, computer science, and industrial engineering.

✦ Table of Contents

Front Matter....Pages i-xxxii
Front Matter....Pages 1-1
One-Step Improvement Ideas and Computational Aspects....Pages 3-32
Value Function Approximation in Complex Queueing Systems....Pages 33-62
Approximate Dynamic Programming by Practical Examples....Pages 63-101
Server Optimization of Infinite Queueing Systems....Pages 103-129
Structures of Optimal Policies in MDPs with Unbounded Jumps: The State of Our Art....Pages 131-186
Front Matter....Pages 187-187
Markov Decision Processes for Screening and Treatment of Chronic Diseases....Pages 189-222
Stratified Breast Cancer Follow-Up Using a Partially Observable MDP....Pages 223-244
Advance Patient Appointment Scheduling....Pages 245-268
Optimal Ambulance Dispatching....Pages 269-291
Blood Platelet Inventory Management....Pages 293-317
Front Matter....Pages 319-319
Stochastic Dynamic Programming for Noise Load Management....Pages 321-335
Allocation in a Vertical Rotary Car Park....Pages 337-370
Dynamic Control of Traffic Lights....Pages 371-386
Smart Charging of Electric Vehicles....Pages 387-404
Front Matter....Pages 405-405
Analysis of a Stochastic Lot Scheduling Problem with Strict Due-Dates....Pages 407-423
Optimal Fishery Policies....Pages 425-438
Near-Optimal Switching Strategies for a Tandem Queue....Pages 439-459
Front Matter....Pages 461-461
Wireless Channel Selection with Restless Bandits....Pages 463-485
Flexible Staffing for Call Centers with Non-stationary Arrival Rates....Pages 487-503
MDP for Query-Based Wireless Sensor Networks....Pages 505-519
Front Matter....Pages 521-521
Optimal Portfolios and Pricing of Financial Derivatives Under Proportional Transaction Costs....Pages 523-546
Back Matter....Pages 547-550

✦ Subjects

Operation Research/Decision Theory;Operations Research, Management Science;Probability Theory and Stochastic Processes