Learning for Decision and Control in Stochastic Networks (Synthesis Lectures on Learning, Networks, and Algorithms)

✍ Scribed by Longbo Huang

• Publisher: Springer
• Year: 2023
• Tongue: English
• Leaves: 80
• Edition: 2023
• Category: Library

✦ Synopsis

This book introduces the Learning-Augmented Network Optimization (LANO) paradigm, which interconnects network optimization with the emerging AI theory and algorithms and has been receiving a growing attention in network research. The authors present the topic based on a general stochastic network optimization model, and review several important theoretical tools that are widely adopted in network research, including convex optimization, the drift method, and mean-field analysis. The book then covers several popular learning-based methods, i.e., learning-augmented drift, multi-armed bandit and reinforcement learning, along with applications in networks where the techniques have been successfully applied. The authors also provide a discussion on potential future directions and challenges.

✦ Table of Contents

Preface
Acknowledgments
Contents
About the Author
1 Introduction
1.1 The Stochastic Network Model
1.2 Network Optimization Techniques
1.3 Organization
2 The Stochastic Network Model
2.1 The Network Model
2.1.1 Network State
2.1.2 Traffic, Service and Resource
2.1.3 Queueing and Resource
2.2 Optimization Objective
2.3 Application Examples
2.3.1 Wireless Communication
2.3.2 Edge Computing
2.3.3 Virtual Reality
2.3.4 Quantum Network
2.3.5 Smart Grids
2.3.6 Supply Chain Management
2.4 Summary
3 Network Optimization Techniques
3.1 Convex Optimization
3.2 The Drift Method
3.3 Mean-Field Methods
3.4 Summary and Discussion
4 Learning Network Decisions
4.1 Learning-Augmented Drift Method
4.1.1 Dual Learning
4.1.2 Learning-Augmented Drift Algorithms
4.1.3 The Augmented Problem and Preliminaries
4.1.4 Performance of OLAC and OLAC2
4.1.5 Convergence Time Analysis
4.1.6 Proofs
4.2 Online-Learning Based Control
4.2.1 Multi-armed Bandit Based Algorithms
4.2.2 Learning with Queues
4.3 Reinforcement Learning
4.3.1 Markov Decision Process
4.3.2 Reinforcement Learning
4.3.3 Deep RL Based Control
4.4 Summary and Discussion
5 Summary and Discussions