Cover
Title
Copyright
Contents
Preface
Chapter 1 Introduction of NumPy
What is NumPy?
Useful NumPy Features
What are NumPy Arrays?
Working with Loops
Appending Elements to Arrays (1)
Appending Elements to Arrays (2
)
Multiply Lists and Arrays
Doubling the Elements in a List
Lists and Exponents
Arrays and Exponents
Math Operations and Arrays
Working with “-1” Subranges with Vectors
Working with “-1” Subranges with Arrays
Other Useful NumPy Methods
Arrays and Vector Operations
NumPy and Dot Products (1)
NumPy and Dot Products (2
)
NumPy and the “Norm” of Vectors
NumPy and Other Operations
NumPy and the reshape() Method
Calculating the Mean and Standard Deviation
Trimmed Mean and Weighted Mean
Code Sample with Mean and Standard Deviation
Working with Lines in the Plane (Optional)
Plotting a Line with NumPy and Matplotlib
Plotting a Quadratic with NumPy and Matplotlib
What is Linear Regression?
What i
s Multivariate Analysis?
What about Nonlinear Datasets?
The MSE Formula
Other Error Types
Nonlinear Least Squares
Calculating the MSE Manually
Find the Best-Fitting Line with NumPy
Calculating MSE by Successive Approximation (1)
Calculating MSE by Successive Approximation (2
)
What is Jax?
Google Colaboratory
Uploading CSV Files in Google Colaboratory
Summary
Chapter 2 Introduction to Pandas
What is Pandas?
Pandas Options and Settings
Pandas Data Frames
Data Frames and Data Cleaning Tasks
Alternatives to Pandas
A Pandas Data Frame with NumPy Example
Describing a Pandas Data Frame
Pandas Boolean Data Frames
Transposing a Pandas Data Frame
Pandas Data Frames and Random Numbers
Reading CSV Files in Pandas
The loc() and iloc() Methods in Pandas
Converting Categorical Data to Numeric Data
Matching and Splitting Strings in Pandas
Converting Strings to Dates in Pandas
Merging and Splitting Columns in Pandas
Combining Pandas Data frames
Data Manipulation with Pandas Data Frames (1)
Data Manipulation with Pandas Data Frames (2
)
Data Manipulation with Pandas Data Frames (3
)
Pandas Data Frames and CSV Files
Managing Columns in Data Frames
Switching Columns
Appending Columns
Deleting Columns
Inserting Columns
Scaling Numeric Columns
Managing Rows in Pandas
Selecting a Range of Rows in Pandas
Finding Duplicate Rows in Pandas
Inserting New Rows in Pandas
Handling Missing Data in Pandas
Multiple Types of Missing Values
Test for Numeric Values in a Column
Replacing NaN Values in Pandas
Sorting Data Frames in Pandas
Working with groupby() in Pandas
Working with apply() and mapapply() in Pandas
Handling Outliers in Pandas
Pandas Data Frames and Scatterplots
Pandas Data Frames and Simple Statistics
Aggregate Operations in Pandas Data Frames
Aggregate Operations with the titanic.csv Dataset
Save Data Frames as CSV Files and Zip Files
Pandas Data Frames and Excel Spreadsheets
Working with JSON-based Data
Python Dictionary and JSON
Python, Pandas, and JSON
Pandas and Regular Expressions (Optional)
Useful One-Line Commands in Pandas
What is Method Chaining?
Pandas and Method Chaining
Pandas Profiling
What is Texthero?
Summary
Chapter 3 NLP Concepts (I)
The Origin of Languages
Language Fluency
Major Language Groups
Peak Usage of Some Languages
Languages and Regional Accents
Languages and Slang
Languages and Dialects
The Complexity of Natural Languages
Word Order in Sentences
What about Verbs?
Auxiliary Verbs
What are Case Endings?
Languages and Gender
Singular and Plural Forms of Nouns
Change in Spelling of Words
Japanese Grammar
Japanese Postpositions (Particles)
Ambiguity in Japanese Sentences
Japanese Nominalization
Google Translate and Japanese
Japanese and Korean
Vowel-Optional Languages and Word Direction
Mutating Consonant Spelling
Expressing Negative Opinions
Phonetic Languages
Phonemes and Morphemes
English Words of Greek and Latin Origin
Multiple Ways to Pronounce Consonants
The Letter “j” in Various Languages
“Hard” versus “Soft” Consonant Sounds
“Ess,” “zee,” and “sh” Sounds
Three Consecutive Consonants
Diphthongs and Triphthongs in English
Semi-Vowels in English
Challenging English Sounds
English in Canada, UK, Australia, and the United States
English Pronouns and Prepositions
What is NLP?
The Evolution of NLP
A Wide-Angle View of NLP
NLP Applications and Use Cases
NLU and NLG
What is Text Classification?
Information Extraction and Retrieval
Word Sense Disambiguation
NLP Techniques in ML
NLP Steps for Training a Model
Text Normalization and Tokenization
Word Tokenization in Japanese
Text Tokenization with Unix Commands
Handling Stop Words
What is Stemming?
Singular versus Plural Word Endings
Common Stemmers
Stemmers and Word Prefixes
Over Stemming and Under Stemming
What is Lemmatization?
Stemming/Lemmatization Caveats
Limitations of Stemming and Lemmatization
Working with Text: POS
POS Tagging
POS Tagging Techniques
Working with Text: NER
Abbreviations and Acronyms
NER Techniques
What is Topic Modeling?
Keyword Extraction, Sentiment Analysis, and Text Summarization
Summary
Chapter 4 NLP Concepts (II)
What is Word Relevance?
What is Text Similarity
?
Senence Similarity
Sentence Encoders
Working with Documents
Working with Documents
Document Similarity (doc2vec)
Techniques for Text Similarity
Similarity Queries
What is Text Encoding?
Text Encoding Techniques
Document Vectorization
One-Hot Encoding (OHE)
Index-Based Encoding
Additional Encoders
The BoW Algorithm
What are n-grams?
Calculating Probabilities with N-grams
Calculating tf, idf, and tf-idf
What is Term Frequency (TF)?
What is Inverse Document Frequency (IDF)?
What is tf-idf?
Limitations of tf-idf
Pointwise Mutual Information (PMI)
The Context of Words in a Document
What is Semantic Context?
Textual Entailment
Discrete, Distributed, and Contextual Word Representations
What is Cosine Similarity?
Text Vectorization (aka Word Embeddings)
Overview of Word Embeddings and Algorithms
Word Embeddings
Word Embedding Algorithms
What is Word2vec?
The Intuition for Word2vec
The Word2vec Architecture
Limitations of Word2vec
The CBoW Architecture
What are Skip-grams?
Skip-gram Example
The Skip-gram Architecture
Neural Network Reduction
What is GloVe?
Working with GloVe
What is FastText?
Comparison of Word Embeddings
What is Topic Modeling?
Topic Modeling Algorithms
LDA and Topic Modeling
Text Classification versus Topic Modeling
Language Models and NLP
How to Create a Language Model
Vector Space Models
Term-Document Matrix
Tradeoffs of the VSM
NLP and Text Mining
Text Extraction Preprocessing and N-Grams
Relation Extraction and Information Extraction
What is a BLEU Score?
ROUGE Score: An Alternative to BLEU
Summary
Chapter 5 Algorithms and Toolkits (I)
Cleaning Data with Regular Expressions
Handling Contracted Words
Python Code Samples of BoW
One-Hot Encoding Examples
Sklearn and Word Embedding Examples
What is BeautifulSoup?
Web Scraping with Pure Regular Expressions
What is Scrapy?
What is SpaCy?
SpaCy and Stop Words
SpaCy and Tokenization
SpaCy and Lemmatization
SpaCy and NER
SpaCy Pipelines
SpaCy and Word Vectors
The scispaCy Library (Optional)
Summary
Chapter 6 Algorithms and Toolkits (II)
What is NLTK?
NLTK and BoW
NLTK and Stemmers
NLTK and Lemmatization
NLTK and Stop Words
What is Wordnet?
Synonyms and Antonyms
NLTK, lxml, and XPath
NLTK and n-grams
NLTK and POS (1)
NLTK and POS (2
)
NLTK and Tokenizers
NLTK and Context-Free Grammars (Optional)
What is Gensim?
Gensim and tf-idf Example
Saving a Word2vec Model in Genism
An Example of Topic Modeling
A Brief Comparison of Popular Python-Based NLP Libraries
Miscellaneous Libraries
Summary
Chapter 7 Introduction to Machine Learning
What is Machine Learning?
Learning Style of Machine Learning Algorithms
Types of Machine Learning Algorithms
Machine Learning Tasks
Preparing a Dataset and Training a Model
Feature Engineering, Selection, and Extraction
Feature Engineering
Feature Selection
Feature Extraction
Model Selection
Working with Datasets
Training Data versus Test Data
What is Cross-Validation?
Overfitting versus Underfitting
What is Regularization?
ML and Feature Scaling
Data Normalization Techniques
Metrics in Machine Learning
R-Squared and its Limitations
Confusion Matrix
Precision, Recall, and Specificity
The ROC Curve and AUC
Metrics for Model Evaluation and Selection
What is Linear Regression?
Linear Regression versus Curve-Fitting
When are Solutions Exact Values?
What is Multivariate Analysis?
Other Types of Regression
Working with Lines in the Plane (Optional)
Scatter Plots with NumPy and Matplotlib (1)
Why the “Perturbation Technique” is Useful
Scatter Plots with NumPy and Matplotlib (2)
A Quadratic Scatterplot with NumPy and Matplotlib
The Mean Squared Error (MSE) Formula
A List of Error Types
Nonlinear Least Squares
Calculating the MSE Manually
Approximating Linear Data with np.linspace()
What are Ensemble Methods?
Four Types of Ensemble Methods
Bagging
Boosting
Stacked Models and Blending Models
What is Bootstrapping?
Common Boosting Algorithms
Hyperparameter Optimization
Grid Search
Randomized Search
Bayesian Optimization
AutoML, AutoML-Zero, and AutoNLP
Miscellaneous Topics
What is Causality?
What is Explainability?
What is Interpretability?
Summary
Chapter 8
Classifiers in Machine Learning
What is Classification?
What are Classifiers?
Common Classifiers
Binary versus Multiclass Classification
Multilabel Classification
What are Linear Classifiers?
What is kNN?
How to Handle a Tie in kNN
SMOTE and kNN
kNN for Data Imputation
What are Decision Trees?
Trade-offs with Decision Trees
Decision Tree Algorithms
Decision Tree Code Samples
Decision Trees, Gini Impurity, and Entropy
What are Random Forests?
What are Support Vector Machines?
Trade-offs of SVMs
What is a Bayesian Classifier?
Types of Naïve Bayes Classifiers
Training Classifiers
Evaluating Classifiers
Trade-offs for ML Algorithms
What are Activation Functions?
Why Do we Need Activation Functions?
How Do Activation Functions Work?
Common Activation Functions
Activation Functions in Python
Keras Activation Functions
The ReLU and ELU Activation Functions
The Advantages and Disadvantages of ReLU
Elu
Sigmoid, Softmax, and Hardmax Similarities
Softmax
Softplus
Tanh
Sigmoid, Softmax, and HardMax Differences
Hyperparameters for Neural Networks
The Loss Function Hyperparameter
The Optimizer Hyperparameter
The Learning Rate Hyperparameter
The Dropout Rate Hyperparameter
What is Backward Error Propagation?
What is Logistic Regression?
Setting a Threshold Value
Logistic Regression: Important Assumptions
Linearly Separable Data
Keras, Logistic Regression, and Iris Dataset
Sklearn and Linear Regression
SciPy and Linear Regression
Keras and Linear Regression
Summary
Chapter 9
NLP Applications
What is Text Summarization?
Extractive Text Summarization
Abstractive Text Summarization
Text Summarization with gensim and SpaCy
What are Recommender Systems?
Movie Recommender Systems
Factoring the Rating Matrix R
Content-Based Recommendation Systems
Analyzing only the Description of the Content
Building User Profiles and Item Profiles
Collaborative Filtering Algorithm
User–User Collaborative Filtering
Item–Item Collaborative Filtering
Recommender System with Surprise
Recommender Systems and Reinforcement Learning (Optional)
Basic Reinforcement Learning in Five Minutes
What is RecSim?
What is Sentiment Analysis?
Useful Tools for Sentiment Analysis
Aspect-Based Sentiment Analysis
Deep Learning and Sentiment Analysis
Sentiment Analysis with Naïve Bayes
Sentiment Analysis in NLTK and VADER
Sentiment Analysis with Textblob
Sentiment Analysis with Flair
Detecting Spam
Logistic Regression and Sentiment Analysis
Working with COVID-19
What are Chatbots?
Open Domain Chatbots
Chatbot Types
Logic Flow of Chatbots
Chatbot Abuses
Useful Links
Summary
Chapter 10
NLP and TF2/Keras
Term-Document Matrix
Text Classification Algorithms in Machine Learning
A Keras-Based Tokenizer
TF2 and Tokenization
TF2 and Encoding
A Keras-Based Word Embedding
An Example of BoW with TF2
The 20newsgroup Dataset
Text Classification with the kNN Algorithm
Text Classification with a Decision Tree Algorithm
Text Classification with a Random Forest Algorithm
Text Classification with the SVC Algorithm
Text Classification with the Naïve Bayes Algorithm
Text Classification with the kMeans Algorithm
TF2/Keras and Word Tokenization
TF2/Keras and Word Encodings
Text Summarization with TF2/Keras and Reuters Dataset
Summary
Chapter 11
Transformer, Bert, and Gpt
What is Attention?
Types of Word Embeddings
Types of Attention and Algorithms
An Overview of the Transformer Architecture
The Transformers Library from HuggingFace
Transformer and NER Tasks
Transformer and QnA Tasks
Transformer and Sentiment Analysis Tasks
Transformer and Mask Filling Tasks
What is T5?
What is BERT?
BERT Features
How is BERT Trained?
How BERT Differs from Earlier NLP Techniques
The Inner Workings of BERT
What is MLM?
What is NSP?
Special Tokens
BERT Encoding: Sequence of Steps
Subword Tokenization
Sentence Similarity in BERT
Word Context in BERT
Generating BERT Tokens (1)
Generating BERT Tokens (2
)
The BERT Family
Surpassing Human Accuracy: deBERTa
What is Google Smith?
Introduction to GPT
Installing the Transformers Package
Working with GPT-2
What is GPT-3?
What is the Goal?
GPT-3 Task Strengths and Mistakes
GPT-3 Architecture
GPT versus BERT
Zero-Shot, One-Shot, and Few Shot Learners
GPT Task Performance
The Switch Transformer: One Trillion Parameters
Looking Ahead
Summary
Appendix A Data and Statistics
What are Datasets?
Data Preprocessing
Data Types
Preparing Datasets
Continuous versus Discrete Data
“Binning” Continuous Data
Scaling Numeric Data via Normalization
Scaling Numeric Data via Standardization
What to Look for in Categorical Data
Mapping Categorical Data to Numeric Values
Working with Dates
Working with Currency
Missing Data, Anomalies, and Outliers
Anomalies and Outliers
Outlier Detection
Missing Data: MCAR, MAR, and MNAR
What is Data Drift?
What is Imbalanced Classification?
Undersampling and Oversampling
Limitations of Resampling
What is SMOTE?
SMOTE Extensions
Analyzing Classifiers
What is LIME?
What is ANOVA?
What is a Probability?
Calculating the Expected Value
Random Variables
Discrete versus Continuous Random Variables
Well-Known Probability Distributions
Fundamental Concepts in Statistics
The Mean
The Median
The Mode
The Variance and Standard Deviation
Population, Sample, and Population Variance
Chebyshev’s Inequality
What is p-Value?
The Moments of a Function (Optional)
Skewness
Kurtosis
Data and Statistics
The Central Limit Theorem
Correlation versus Causation
Statistical Inferences
The Bias-Variance Trade-off
Types of Bias in Data
Gini Impurity, Entropy, and Perplexity
What is Gini Impurity?
What is Entropy?
Calculating Gini Impurity and Entropy Values
Multidimensional Gini Index
What is Perplexity?
Cross-Entropy and KL Divergence
What is Cross Entropy?
What is KL Divergence?
What’s their Purpose?
Covariance and Correlation Matrices
Covariance Matrix
Covariance Matrix: An Example
Correlation Matrix
Eigenvalues and Eigenvectors
Calculating Eigenvectors: A Simple Example
Gauss Jordan Elimination (Optional)
Principal Component Analysis (PCA)
The New Matrix of Eigenvectors
Dimensionality Reduction
Dimensionality Reduction Techniques
The Curse of Dimensionality
What are Manifolds (Optional)?
Singular Value Decomposition (SVD)
Locally Linear Embedding (LLE)
UMAP
t-SNE (“tee-snee”)
PHATE
Linear Versus Nonlinear Reduction Techniques
Types of Distance Metrics
Other Well-Known Distance Metrics
Pearson Correlation Coefficient
Jaccard Index (or Similarity)
Local Sensitivity Hashing (Optional)
What is Sklearn?
Sklearn, Pandas, and the IRIS Dataset
Sklearn and Outlier Detection
What is Bayesian Inference?
Bayes Theorem
Some Bayesian Terminology
What is MAP?
Why Use Bayes Theorem?
What are Vector Spaces?
Summary
Appendix B Introduction to Python
Tools for Python
easy_install and pip
virtualenv
IPython
Python Installation
Setting the PATH Environment Variable (Windows Only)
Launching Python on Your Machine
The Python Interactive Interpreter
Python Identifiers
Lines, Indentation, and Multilines
Quotation and Comments in Python
Saving Your Code in a Module
Some Standard Modules in Python
The help() and dir() Functions
Compile Time and Runtime Code Checking
Simple Data Types in Python
Working with Numbers
Working with Other Bases
The chr () Function
The round() Function in Python
Formatting Numbers in Python
Working with Fractions
Unicode and UTF-8
Working with Unicode
Working with Strings
Comparing Strings
Formatting Strings in Python
Uninitialized Variables and the Value None in Python
Slicing and Splicing Strings
Testing for Digits and Alphabetic Characters
Search and Replace a String in Other Strings
Remove Leading and Trailing Characters
Printing Text without NewLine Characters
Text Alignment
Working with Dates
Converting Strings to Dates
Exception Handling in Python
Handling User Input
Python and Emojis (Optional)
Command-Line Arguments
Summary
Appendix C Introduction to Regular Expressions
What are Regular Expressions?
Metacharacters in Python
Character Sets in Python
Working with “^” and “\”
Character Classes in Python
Matching Character Classes with the re Module
Using the re.match() Method
Options for the re.match() Method
Matching Character Classes with the re.search() Method
Matching Character Classes with the findAll() Method
Finding Capitalized Words in a String
Additional Matching Function for Regular Expressions
Grouping with Character Classes in Regular Expressions
Using Character Classes in Regular Expressions
Matching Strings with Multiple Consecutive Digits
Reversing Words in Strings
Modifying Text Strings with the re Module
Splitting Text Strings with the re.split() Method
Splitting Text Strings Using Digits and Delimiters
Substituting Text Strings with the re.sub() Method
Matching the Beginning and the End of Text Strings
Compilation Flags
Compound Regular Expressions
Counting Character Types in a String
Regular Expressions and Grouping
Simple String Matches
Additional Topics for Regular Expressions
Summary
Exercises
Appendix D Introduction to Keras
What is Keras?
Working with Keras Namespaces in TF 2
Working with the tf.keras.layers Namespace
Working with the tf.keras.activations Namespace
Working with the keras.tf.datasets Namespace
Working with the tf.keras.experimental Namespace
Working with Other tf.keras Namespaces
TF 2 Keras versus “Standalone” Keras
Creating a Keras-Based Model
Keras and Linear Regression
Keras, MLPs, and MNIST
Keras, CNNs, and cifar10
Resizing Images in Keras
Keras and Early Stopping (1)
Keras and Early Stopping (2
)
Keras and Metrics
Saving and Restoring Keras Models
Summary
Appendix E
Introduction to TensorFlow 2
What is TF 2?
TF 2 Use Cases
TF 2 Architecture: The Short Version
TF 2 Installation
TF 2 and the Python REPL
Other TF 2-Based Toolkits
TF 2 Eager Execution
TF 2 Tensors, Data Types, and Primitive Types
TF 2 Data Types
TF 2 Primitive Types
Constants in TF 2
Variables in TF 2
The tf.rank() API
The tf.shape() API
Variables in TF 2 (Revisited)
TF 2 Variables versus Tensors
What is @tf.function in TF 2?
How Does @tf.function Work?
A Caveat about @tf.function in TF 2
The tf.print() Function and Standard Error
Working with @tf.function in TF 2
An Example without @tf.function
An Example with @tf.function
Overloading Functions with @tf.function
What is AutoGraph in TF 2?
Arithmetic Operations in TF 2
Caveats for Arithmetic Operations in TF 2
TF 2 and Built-In Functions
Calculating Trigonometric Values in TF 2
Calculating Exponential Values in TF 2
Working with Strings in TF 2
Working with Tensors and Operations in TF 2
Second-Order Tensors in TF 2 (1)
Second-Order Tensors in TF 2 (2)
Multiplying Two Second-Order Tensors in TF
Convert Python Arrays to TF Tensors
Conflicting Types in TF 2
Differentiation and tf.GradientTape in TF 2
Examples of tf.GradientTape
Using Nested Loops with tf.GradientTape
Other Tensors with tf.GradientTape
A Persistent Gradient Tape
What is Trax?
Google Colaboratory
Other Cloud Platforms
GCP SDK
TF2 and tf.data.Dataset
The TF 2 tf.data.Dataset
Creating a Pipeline
A Simple TF 2 tf.data.Dataset
What are Lambda Expressions?
Working with Generators in TF 2
Summary
Appendix F Data Visualization
What is Data Visualization?
Types of Data Visualization
What is Matplotlib?
Horizontal Lines in Matplotlib
Slanted Lines in Matplotlib
Parallel Slanted Lines in Matplotlib
A Grid of Points in Matplotlib
A Dotted Grid in Matplotlib
Lines in a Grid in Matplotlib
Plot a Best-Fitting Line in Matplotlib
A Colored Grid in Matplotlib
A Colored Square in an Unlabeled Grid in Matplotlib
Randomized Data Points in Matplotlib
A Histogram in Matplotlib
A Set of Line Segments in Matplotlib
Plotting Multiple Lines in Matplotlib
Trigonometric Functions in Matplotlib
Display IQ Scores in Matplotlib
Introduction to Sklearn (scikit-learn)
The Digits Dataset in Sklearn
The Iris Dataset in Sklearn
Sklearn, Pandas, and the Iris Dataset
The Iris Dataset in Sklearn (Optional)
The faces Dataset in Sklearn (Optional)
Working with Seaborn
Features of Seaborn
Seaborn Built-in Datasets
The Iris Dataset in Seaborn
The Titanic Dataset in Seaborn
Extracting Data from the Titanic Dataset in Seaborn (1)
Extracting Data from the Titanic Dataset in Seaborn (2
)
Visualizing a Pandas Dataset in Seaborn
Data Visualization in Pandas
Summary
Index