Applying Reinforcement Learning on Real-World Data with Practical Examples in Python

✍ Scribed by Philip Osborne, Kajal Singh, Matthew E. Taylor

• Publisher: Springer
• Year: 2022
• Tongue: English
• Leaves: 105
• Series: Synthesis Lectures on Artificial Intelligence and Machine Learning
• Edition: 1
• Category: Library

✦ Synopsis

Reinforcement learning is a powerful tool in artificial intelligence in which virtual or physical agents learn to optimize their decision making to achieve long-term goals. In some cases, this machine learning approach can save programmers time, outperform existing controllers, reach super-human performance, and continually adapt to changing conditions. This book argues that these successes show reinforcement learning can be adopted successfully in many different situations, including robot control, stock trading, supply chain optimization, and plant control. However, reinforcement learning has traditionally been limited to applications in virtual environments or simulations in which the setup is already provided. Furthermore, experimentation may be completed for an almost limitless number of attempts risk-free. In many real-life tasks, applying reinforcement learning is not as simple as (1) data is not in the correct form for reinforcement learning, (2) data is scarce, and (3) automation has limitations in the real-world. Therefore, this book is written to help academics, domain specialists, and data enthusiast alike to understand the basic principles of applying reinforcement learning to real-world problems. This is achieved by focusing on the process of taking practical examples and modeling standard data into the correct form required to then apply basic agents. To further assist with readers gaining a deep and grounded understanding of the approaches, the book shows hand-calculated examples in full and then how this can be achieved in a more automated manner with code. For decision makers who are interested in reinforcement learning as a solution but are not technically proficient we include simple, non-technical examples in the introduction and case studies section. These provide context of what reinforcement learning offer but also the challenges and risks associated with applying it in practice. Specifically, the book illustrates the differences between reinforcement learning and other machine learning approaches as well as how well-known companies have found success using the approach to their problems.

✦ Table of Contents

Cover
Copyright Page
Title Page
Contents
Preface
Acknowledgments
Background and Definitions
What is Real-World Reinforcement Learning?
Reinforcement Learning vs. Other Machine Learning Techniques
What About Deep Learning?
Notable Real-World Examples
Supporting Code
What Form is the Code Published in?
Summary of Learning Procedure
The Markov Decision Process Environment
The Learning Components
Model-Based vs. Model-Free Approaches
Returns and Episodes
Reinforcement Learning Theory
Core Principles of Reinforcement Learning
Model-Based Methods
Policy Evaluation
Policy Improvement
Value Iteration
Model-Free Methods
Q-Learning: Off-Policy TD Control
SARSA: On-Policy TD Control
Temporal Difference
Online vs. Offline Reinforcement Learning
A Robot Cleaner Example
The Task
Applying Dynamic Programming Methods
Implementing Value Iteration
Value Iteration Update Example
Applying Model-Free Methods
Implementing the Q-Learning Algorithm
Understanding the Agent's Parameters
What Remains Unaddressed?
Understanding the Environment Generation
The Pre-Built Markov Decision Process
The Robot's Sensory Data
Defining the MDP
Conclusion of the Robot Cleaner Example
The Classroom Environment
Description of the Real-World Problem
The Agent's Role in This Problem
Defining the Reward Signal
Defining Episodes
Exploration vs. Exploitation
Environmental Definition
Mimicking the Real-World with a Simulation
Data Collection Methods
An Online Setup
A Batch-Offline Setup
Conclusion of the Classroom Example
Industry Applications
Google Data Center
Salesforce Text Summarization
IBM Trading
PepsiCo Cheetos
Siam Cement Group Public Company Limited
Conclusion
Bibliography
Authors' Biographies