Foreword
Preface
Acknowledgments
Contents
About the Authors
Acronyms
1 Introduction
1.1 The Basic Concepts
1.2 Main Tasks of Text Data Mining
1.3 Existing Challenges in Text Data Mining
1.4 Overview and Organization of This Book
1.5 Further Reading
Exercises
2 Data Annotation and Preprocessing
2.1 Data Acquisition
2.2 Data Preprocessing
2.3 Data Annotation
2.4 Basic Tools of NLP
2.4.1 Tokenization and POS Tagging
2.4.2 Syntactic Parser
2.4.3 N-gram Language Model
2.5 Further Reading
Exercises
3 Text Representation
3.1 Vector Space Model
3.1.1 Basic Concepts
3.1.2 Vector Space Construction
3.1.3 Text Length Normalization
3.1.4 Feature Engineering
3.1.5 Other Text Representation Methods
3.2 Distributed Representation of Words
3.2.1 Neural Network Language Model
3.2.2 C&W Model
3.2.3 CBOW and Skip-Gram Model
3.2.4 Noise Contrastive Estimation and Negative Sampling
3.2.5 Distributed Representation Based on the Hybrid Character-Word Method
3.3 Distributed Representation of Phrases
3.3.1 Distributed Representation Based on the Bag-of-Words Model
3.3.2 Distributed Representation Based on Autoencoder
3.4 Distributed Representation of Sentences
3.4.1 General Sentence Representation
3.4.2 Task-Oriented Sentence Representation
3.5 Distributed Representation of Documents
3.5.1 General Distributed Representation of Documents
3.5.2 Task-Oriented Distributed Representation of Documents
3.6 Further Reading
Exercises
4 Text Representation with Pretraining and Fine-Tuning
4.1 ELMo: Embeddings from Language Models
4.1.1 Pretraining Bidirectional LSTM Language Models
4.1.2 Contextualized ELMo Embeddings for Downstream Tasks
4.2 GPT: Generative Pretraining
4.2.1 Transformer
4.2.2 Pretraining the Transformer Decoder
4.2.3 Fine-Tuning the Transformer Decoder
4.3 BERT: Bidirectional Encoder Representations from Transformer
4.3.1 BERT: Pretraining
4.3.2 BERT: Fine-Tuning
4.3.3 XLNet: Generalized Autoregressive Pretraining
4.3.4 UniLM
4.4 Further Reading
Exercises
5 Text Classification
5.1 The Traditional Framework of Text Classification
5.2 Feature Selection
5.2.1 Mutual Information
5.2.2 Information Gain
5.2.3 The Chi-Squared Test Method
5.2.4 Other Methods
5.3 Traditional Machine Learning Algorithms for Text Classification
5.3.1 NaΓ―ve Bayes
5.3.2 Logistic/Softmax and Maximum Entropy
5.3.3 Support Vector Machine
5.3.4 Ensemble Methods
5.4 Deep Learning Methods
5.4.1 Multilayer Feed-Forward Neural Network
5.4.2 Convolutional Neural Network
5.4.3 Recurrent Neural Network
5.5 Evaluation of Text Classification
5.6 Further Reading
Exercises
6 Text Clustering
6.1 Text Similarity Measures
6.1.1 The Similarity Between Documents
6.1.2 The Similarity Between Clusters
6.2 Text Clustering Algorithms
6.2.1 K-Means Clustering
6.2.2 Single-Pass Clustering
6.2.3 Hierarchical Clustering
6.2.4 Density-Based Clustering
6.3 Evaluation of Clustering
6.3.1 External Criteria
6.3.2 Internal Criteria
6.4 Further Reading
Exercises
7 Topic Model
7.1 The History of Topic Modeling
7.2 Latent Semantic Analysis
7.2.1 Singular Value Decomposition of the Term-by-Document Matrix
7.2.2 Conceptual Representation and Similarity Computation
7.3 Probabilistic Latent Semantic Analysis
7.3.1 Model Hypothesis
7.3.2 Parameter Learning
7.4 Latent Dirichlet Allocation
7.4.1 Model Hypothesis
7.4.2 Joint Probability
7.4.3 Inference in LDA
7.4.4 Inference for New Documents
7.5 Further Reading
Exercises
8 Sentiment Analysis and Opinion Mining
8.1 History of Sentiment Analysis and Opinion Mining
8.2 Categorization of Sentiment Analysis Tasks
8.2.1 Categorization According to Task Output
8.2.2 According to Analysis Granularity
8.3 Methods for Document/Sentence-Level Sentiment Analysis
8.3.1 Lexicon- and Rule-Based Methods
8.3.2 Traditional Machine Learning Methods
8.3.3 Deep Learning Methods
8.4 Word-Level Sentiment Analysis and Sentiment Lexicon Construction
8.4.1 Knowledgebase-Based Methods
8.4.2 Corpus-Based Methods
8.4.3 Evaluation of Sentiment Lexicons
8.5 Aspect-Level Sentiment Analysis
8.5.1 Aspect Term Extraction
8.5.2 Aspect-Level Sentiment Classification
8.5.3 Generative Modeling of Topics and Sentiments
8.6 Special Issues in Sentiment Analysis
8.6.1 Sentiment Polarity Shift
8.6.2 Domain Adaptation
8.7 Further Reading
Exercises
9 Topic Detection and Tracking
9.1 History of Topic Detection and Tracking
9.2 Terminology and Task Definition
9.2.1 Terminology
9.2.2 Task
9.3 Story/Topic Representation and Similarity Computation
9.4 Topic Detection
9.4.1 Online Topic Detection
9.4.2 Retrospective Topic Detection
9.5 Topic Tracking
9.6 Evaluation
9.7 Social Media Topic Detection and Tracking
9.7.1 Social Media Topic Detection
9.7.2 Social Media Topic Tracking
9.8 Bursty Topic Detection
9.8.1 Burst State Detection
9.8.2 Document-Pivot Methods
9.8.3 Feature-Pivot Methods
9.9 Further Reading
Exercises
10 Information Extraction
10.1 Concepts and History
10.2 Named Entity Recognition
10.2.1 Rule-based Named Entity Recognition
10.2.2 Supervised Named Entity Recognition Method
10.2.3 Semisupervised Named Entity Recognition Method
10.2.4 Evaluation of Named Entity Recognition Methods
10.3 Entity Disambiguation
10.3.1 Clustering-Based Entity Disambiguation Method
10.3.2 Linking-Based Entity Disambiguation
10.3.3 Evaluation of Entity Disambiguation
10.4 Relation Extraction
10.4.1 Relation Classification Using Discrete Features
10.4.2 Relation Classification Using Distributed Features
10.4.3 Relation Classification Based on Distant Supervision
10.4.4 Evaluation of Relation Classification
10.5 Event Extraction
10.5.1 Event Description Template
10.5.2 Event Extraction Method
10.5.3 Evaluation of Event Extraction
10.6 Further Reading
Exercises
11 Automatic Text Summarization
11.1 Main Tasks in Text Summarization
11.2 Extraction-Based Summarization
11.2.1 Sentence Importance Estimation
11.2.2 Constraint-Based Summarization Algorithms
11.3 Compression-Based Automatic Summarization
11.3.1 Sentence Compression Method
11.3.2 Automatic Summarization Based on Sentence Compression
11.4 Abstractive Automatic Summarization
11.4.1 Abstractive Summarization Based on Information Fusion
11.4.2 Abstractive Summarization Based on the Encoder-Decoder Framework
11.5 Query-Based Automatic Summarization
11.5.1 Relevance Calculation Based on the Language Model
11.5.2 Relevance Calculation Based on Keyword Co-occurrence
11.5.3 Graph-Based Relevance Calculation Method
11.6 Crosslingual and Multilingual Automatic Summarization
11.6.1 Crosslingual Automatic Summarization
11.6.2 Multilingual Automatic Summarization
11.7 Summary Quality Evaluation and Evaluation Workshops
11.7.1 Summary Quality Evaluation Methods
11.7.2 Evaluation Workshops
11.8 Further Reading
Exercises
References