Contents
Foreword
Before We Start
Section I: Introduction
Chapter 1: Introduction
1.1 Terminology
1.1.1 Dataset, Variable, and Observation
1.1.2 Feature Engineering
1.1.3 Feature Extraction
1.1.4 Feature Selection
1.1.5 Cost Function
1.2 Process of Training a Machine Learning Model
1.3 Preventing Overfitting
1.4 Code Conventions
1.5 Datasets Used
1.5.1 Hotel Booking Demand Datasets
1.5.2 Car Sales
1.5.3 Coupon Recommendation
1.5.4 Raman Spectroscopy of Skimmed Milk Samples
1.5.5 Beaver Body Temperatures
1.6 References
Section II: Feature Engineering
Chapter 2: Domain Specific Feature Engineering
2.1 Introduction
2.2 Domain-Specific Feature Engineering
2.2.1 Ask Probing Questions
2.2.2 Literature Review
2.3 References
Contents
Foreword
Before We Start
Section I: Introduction
Chapter 1: Introduction
1.1 Terminology
1.1.1 Dataset, Variable, and Observation
1.1.2 Feature Engineering
1.1.3 Feature Extraction
1.1.4 Feature Selection
1.1.5 Cost Function
1.2 Process of Training a Machine Learning Model
1.3 Preventing Overfitting
1.4 Code Conventions
1.5 Datasets Used
1.5.1 Hotel Booking Demand Datasets
1.5.2 Car Sales
1.5.3 Coupon Recommendation
1.5.4 Raman Spectroscopy of Skimmed Milk Samples
1.5.5 Beaver Body Temperatures
1.6 References
Section II: Feature Engineering
Chapter 2: Domain Specific Feature Engineering
2.1 Introduction
2.2 Domain-Specific Feature Engineering
2.2.1 Ask Probing Questions
2.2.2 Literature Review
2.3 References
Chapter 3: EDA Feature Engineering
3.1 Introduction
3.2 Car Sales
3.3 Coupon Recommendation
3.4 Conclusion
Chapter 4: Higher Order Feature Engineering
4.1 Engineering Categorical Features
4.1.1 Dummy Encoding or One-Hot Encoding
4.1.2 Label Encoding
4.1.3 Count, and Percentage Encoding
4.1.4 Encoding by Rank of Counts
4.1.5 Target Encoding
4.2 Engineering Ordinal Features
4.2.1 Rank Encoding
4.2.2 Polynomial Encoding
4.2.3 Backward Difference Encoding
4.3 Engineering Numerical Features
4.3.1 Binning
4.3.2 Square and Cube
4.3.3 Regression Splines
4.3.4 Square Root and Cube Root
4.3.5 Log Transformation
4.3.6 Standardization and Normalization
4.3.7 Box-cox Transformation
4.3.8 Yeo-Johnson Transformation
4.4 Conclusion
Chapter 5: Interaction Effect Feature Engineering
5.1 Interaction Plot
5.2 SHAP
5.2.1 Car Sales
5.2.2 Coupon Recommendation
5.3 Putting Everything Together
5.3.1 Hotel Total Room Booking
5.3.2 Hotel Booking Cancellation
5.3.3 Car Sales
5.3.4 Coupon Recommendation
5.4 Conclusion
5.5 References
Section III: Feature Selection
Chapter 6: Fundamentals of Feature Selection
6.1 Introduction
6.2 Different Feature Selection Methods
6.3 Filter Method
6.4 Wrapper Method
6.4.1 Forward Selection
6.4.2 Backward Selection
6.4.3 Stepwise Selection
6.4.4 Recursive Feature Elimination
6.5 Putting Everything Together
6.5.1 Hotel Total Room Booking
6.5.2 Hotel Booking Cancellation
6.5.3 Car Sales
6.5.4 Coupon Recommendation
6.6 Conclusion
Chapter 7: Feature Selection Concerning Modeling Techniques
7.1 Lasso, Ridge, and ElasticNet
7.2 Feature Importance of Tree Models
7.3 Boruta
7.4 Using Tree-Based Feature Importance for Linear Model
7.5 Using Linear Model Feature Importance for Tree Models
7.6 Linear Regression
7.7 SVM
7.8 PCA
7.9 Putting Everything Together
7.9.1 Hotel Total Room Booking
7.9.2 Hotel Booking Cancellation
7.9.3 Car Sales
7.9.4 Coupon Recommendation
7.10 Conclusion
Chapter 8: Feature Selection Using Metaheuristic Algorithms
8.1 Exhaustive Feature Selection
8.2 Genetic Algorithm
8.3 Simulated Annealing
8.4 Ant Colony Optimization
8.5 Particle Swarm Optimization
8.6 Putting Everything Together
8.6.1 Hotel Total Room Booking
8.6.2 Hotel Booking Cancellation
8.6.3 Car Sales
8.6.4 Coupon Recommendation
8.7 Conclusion
8.8 References
Section IV: Model Explanation
Chapter 9: Explaining Model and Model Predictions to Layman
9.1 Introduction
9.2 Explainable models
9.2.1 Linear Regression
9.2.2 Logistic Regression
9.2.3 Decision Tree
9.3 Explanation Techniques
9.3.1 Explaining Overall Model
9.3.1.1 Partial Dependence Plot
9.3.1.2 Accumulated Local Effects Plot
9.3.1.3 Permutation Feature Importance
9.3.1.4 Surrogate Model
9.3.2 Explaining Individual Predictions
9.3.2.1 Individual Conditional Expectation Plots
9.3.2.2 Local interpretable model-agnostic explanations
9.3.2.3 Counterfactual Model Explanations
9.3.2.4 SHAP
9.4 Putting Everything Together
9.4.1 Hotel Total Room Booking
9.4.2 Hotel Booking Cancellation
9.5 Conclusion
9.6 References
Section V: Special Chapters
Chapter 10: Feature Engineering & Selection for Text Classification
10.1 Introduction
10.2 Feature Construction
10.2.1 N-gram
10.2.2 Syntactic N-gram
10.2.3 Domain-Specific Taxonomy Features
10.2.4 Meta Features
10.3 Feature Selection
10.3.1 Filter Method
10.3.1.1 Document Frequency
10.3.1.2 Chi-Square
10.3.1.3 Mutual Information
10.3.1.4 Proportional Difference
10.3.1.5 Information Gain
10.3.2 Metaheuristics Algorithms
10.3.3 Ensemble Feature Selection
10.4 Feature Extraction
10.4.1 Bag of Words
10.4.2 Term Frequency Inverse Document Frequency
10.4.3 Word2vec
10.5 Feature Reduction
10.5.1 Singular Value Decomposition
10.5.2 Non-Negative Matrix Factorization
10.6 Conclusion
10.7 References
Chapter 11: Things That Can Give Additional Improvement
11.1 Introduction
11.2 Hyperparameter Tuning
11.3 Ensemble Learning
11.4 Signal Processing
11.4.1 Filtering
11.4.2 Baseline Removal
11.5 Conclusion
11.6 References