Lifelong Machine Learning

✍ Scribed by Zhiyuan Chen, Bing Liu

• Publisher: Springer
• Year: 2018
• Tongue: English
• Leaves: 199
• Series: Synthesis Lectures on Artificial Intelligence and Machine Learning
• Edition: 2
• Category: Library

✦ Synopsis

Lifelong Machine Learning, Second Edition is an introduction to an advanced machine learning paradigm that continuously learns by accumulating past knowledge that it then uses in future learning and problem solving. In contrast, the current dominant machine learning paradigm learns in isolation: given a training dataset, it runs a machine learning algorithm on the dataset to produce a model that is then used in its intended application. It makes no attempt to retain the learned knowledge and use it in subsequent learning. Unlike this isolated system, humans learn effectively with only a few examples precisely because our learning is very knowledge-driven: the knowledge learned in the past helps us learn new things with little data or effort. Lifelong learning aims to emulate this capability, because without it, an AI system cannot be considered truly intelligent.

Research in lifelong learning has developed significantly in the relatively short time since the first edition of this book was published. The purpose of this second edition is to expand the definition of lifelong learning, update the content of several chapters, and add a new chapter about continual learning in deep neural networks—which has been actively researched over the past two or three years. A few chapters have also been reorganized to make each of them more coherent for the reader. Moreover, the authors want to propose a unified framework for the research area. Currently, there are several research topics in machine learning that are closely related to lifelong learning—most notably, multi-task learning, transfer learning, and meta-learning—because they also employ the idea of knowledge sharing and transfer. This book brings all these topics under one roof and discusses their similarities and differences. Its goal is to introduce this emerging machine learning paradigm and present a comprehensive survey and review of the important research results and latest ideas in the area. This book is thus suitable for students, researchers, and practitioners who are interested in machine learning, data mining, natural language processing, or pattern recognition. Lecturers can readily use the book for courses in any of these related fields.

✦ Table of Contents

Cover
Copyright Page
Title Page
Dedication
Contents
Preface
Acknowledgments
Introduction
Classic Machine Learning Paradigm
Motivating Examples
A Brief History of Lifelong Learning
Definition of Lifelong Learning
Types of Knowledge and Key Challenges
Evaluation Methodology and Role of Big Data
Outline of the Book
Related Learning Paradigms
Transfer Learning
Structural Correspondence Learning
Naïve Bayes Transfer Classifier
Deep Learning in Transfer Learning
Difference from Lifelong Learning
Multi-Task Learning
Task Relatedness in Multi-Task Learning
GO-MTL: Multi-Task Learning using Latent Basis
Deep Learning in Multi-Task Learning
Difference from Lifelong Learning
Online Learning
Difference from Lifelong Learning
Reinforcement Learning
Difference from Lifelong Learning
Meta Learning
Difference from Lifelong Learning
Summary
Lifelong Supervised Learning
Definition and Overview
Lifelong Memory-Based Learning
Two Memory-Based Learning Methods
Learning a New Representation for Lifelong Learning
Lifelong Neural Networks
MTL Net
Lifelong EBNN
ELLA: An Efficient Lifelong Learning Algorithm
Problem Setting
Objective Function
Dealing with the First Inefficiency
Dealing with the Second Inefficiency
Active Task Selection
Lifelong Naive Bayesian Classification
Naïve Bayesian Text Classification
Basic Ideas of LSC
LSC Technique
Discussions
Domain Word Embedding via Meta-Learning
Summary and Evaluation Datasets
Continual Learning and Catastrophic Forgetting
Catastrophic Forgetting
Continual Learning in Neural Networks
Learning without Forgetting
Progressive Neural Networks
Elastic Weight Consolidation
iCaRL: Incremental Classifier and Representation Learning
Incremental Training
Updating Representation
Constructing Exemplar Sets for New Classes
Performing Classification in iCaRL
Expert Gate
Autoencoder Gate
Measuring Task Relatedness for Training
Selecting the Most Relevant Expert for Testing
Encoder-Based Lifelong Learning
Continual Learning with Generative Replay
Generative Adversarial Networks
Generative Replay
Evaluating Catastrophic Forgetting
Summary and Evaluation Datasets
Open-World Learning
Problem Definition and Applications
Center-Based Similarity Space Learning
Incrementally Updating a CBS Learning Model
Testing a CBS Learning Model
CBS Learning for Unseen Class Detection
DOC: Deep Open Classification
Feed-Forward Layers and the 1-vs.-Rest Layer
Reducing Open-Space Risk
DOC for Image Classification
Unseen Class Discovery
Summary and Evaluation Datasets
Lifelong Topic Modeling
Main Ideas of Lifelong Topic Modeling
LTM: A Lifelong Topic Model
LTM Model
Topic Knowledge Mining
Incorporating Past Knowledge
Conditional Distribution of Gibbs Sampler
AMC: A Lifelong Topic Model for Small Data
Overall Algorithm of AMC
Mining Must-link Knowledge
Mining Cannot-link Knowledge
Extended Pólya Urn Model
Sampling Distributions in Gibbs Sampler
Summary and Evaluation Datasets
Lifelong Information Extraction
NELL: A Never-Ending Language Learner
NELL Architecture
Extractors and Learning in NELL
Coupling Constraints in NELL
Lifelong Opinion Target Extraction
Lifelong Learning through Recommendation
AER Algorithm
Knowledge Learning
Recommendation using Past Knowledge
Learning on the Job
Conditional Random Fields
General Dependency Feature
The L-CRF Algorithm
Lifelong-RL: Lifelong Relaxation Labeling
Relaxation Labeling
Lifelong Relaxation Labeling
Summary and Evaluation Datasets
Continuous Knowledge Learning in Chatbots
LiLi: Lifelong Interactive Learning and Inference
Basic Ideas of LiLi
Components of LiLi
A Running Example
Summary and Evaluation Datasets
Lifelong Reinforcement Learning
Lifelong Reinforcement Learning through Multiple Environments
Acquiring and Incorporating Bias
Hierarchical Bayesian Lifelong Reinforcement Learning
Motivation
Hierarchical Bayesian Approach
MTRL Algorithm
Updating Hierarchical Model Parameters
Sampling an MDP
PG-ELLA: Lifelong Policy Gradient Reinforcement Learning
Policy Gradient Reinforcement Learning
Policy Gradient Lifelong Learning Setting
Objective Function and Optimization
Safe Policy Search for Lifelong Learning
Cross-domain Lifelong Reinforcement Learning
Summary and Evaluation Datasets
Conclusion and Future Directions
Bibliography
Authors' Biographies