Bandit algorithms for website optimization

✍ Scribed by John Myles White

• Publisher: O'Reilly
• Year: 2013
• Tongue: English
• Leaves: 87
• Series: O'REILLY
• Category: Library

✦ Table of Contents

Copyright
Table of Contents
Preface
Finding the Code for This Book
Dealing with Jargon: A Glossary
Conventions Used in This Book
Using Code Examples
Safari® Books Online
How to Contact Us
Acknowledgments
Chapter 1. Two Characters: Exploration and Exploitation
The Scientist and the Businessman
Cynthia the Scientist
Bob the Businessman
Oscar the Operations Researcher
The Explore-Exploit Dilemma
Chapter 2. Why Use Multiarmed Bandit Algorithms?
What Are We Trying to Do?
The Business Scientist: Web-Scale A/B Testing
Chapter 3. The epsilon-Greedy Algorithm
Introducing the epsilon-Greedy Algorithm
Describing Our Logo-Choosing Problem Abstractly
What’s an Arm?
What’s a Reward?
What’s a Bandit Problem?
Implementing the epsilon-Greedy Algorithm
Thinking Critically about the epsilon-Greedy Algorithm
Chapter 4. Debugging Bandit Algorithms
Monte Carlo Simulations Are Like Unit Tests for Bandit Algorithms
Simulating the Arms of a Bandit Problem
Analyzing Results from a Monte Carlo Study
Approach 1: Track the Probability of Choosing the Best Arm
Approach 2: Track the Average Reward at Each Point in Time
Approach 3: Track the Cumulative Reward at Each Point in Time
Exercises
Chapter 5. The Softmax Algorithm
Introducing the Softmax Algorithm
Implementing the Softmax Algorithm
Measuring the Performance of the Softmax Algorithm
The Annealing Softmax Algorithm
Exercises
Chapter 6. UCB – The Upper Confidence Bound Algorithm
Introducing the UCB Algorithm
Implementing UCB
Comparing Bandit Algorithms Side-by-Side
Exercises
Chapter 7. Bandits in the Real World: Complexity and Complications
A/A Testing
Running Concurrent Experiments
Continuous Experimentation vs. Periodic Testing
Bad Metrics of Success
Scaling Problems with Good Metrics of Success
Intelligent Initialization of Values
Running Better Simulations
Moving Worlds
Correlated Bandits
Contextual Bandits
Implementing Bandit Algorithms at Scale
Chapter 8. Conclusion
Learning Life Lessons from Bandit Algorithms
A Taxonomy of Bandit Algorithms
Learning More and Other Topics
About the Author