Feature Engineering and Selection: A Practical Approach for Predictive Models

✍ Scribed by Max Kuhn, Kjell Johnson

• Publisher: Chapman and Hall/CRC
• Year: 2019
• Tongue: English
• Leaves: 314
• Series: Chapman & Hall/CRC Data Science Series
• Edition: 1
• Category: Library

✦ Synopsis

The process of developing predictive models includes many stages. Most resources focus on the modeling algorithms but neglect other critical aspects of the modeling process. This book describes techniques for finding the best representations of predictors for modeling and for nding the best subset of predictors for improving model performance. A variety of example data sets are used to illustrate the techniques along with R programs for reproducing the results.

✦ Table of Contents

Cover
Half Title
Title Page
Copyright Page
Dedication
Table of Contents
Preface
Author Bios
1: Introduction
1.1 A Simple Example
1.2 Important Concepts
1.3 A More Complex Example
1.4 Feature Selection
1.5 An Outline of the Book
1.6 Computing
2: Illustrative Example: Predicting Risk of Ischemic Stroke
2.1 Splitting
2.2 Preprocessing
2.3 Exploration
2.4 Predictive Modeling across Sets
2.5 Other Considerations
2.6 Computing
3: A Review of the Predictive Modeling Process
3.1 Illustrative Example: OkCupid Profile Data
3.2 Measuring Performance
3.3 Data Splitting
3.4 Resampling
3.5 Tuning Parameters and Overfitting
3.6 Model Optimization and Tuning
3.7 Comparing Models Using the Training Set
3.8 Feature Engineering without Overfitting
3.9 Summary
3.10 Computing
4: Exploratory Visualizations
4.1 Introduction to the Chicago Train Ridership Data
4.2 Visualizations for Numeric Data: Exploring Train Ridership Data
4.3 Visualizations for Categorical Data: Exploring the OkCupid Data
4.4 Postmodeling Exploratory Visualizations
4.5 Summary
4.6 Computing
5: Encoding Categorical Predictors
5.1 Creating Dummy Variables for Unordered Categories
5.2 Encoding Predictors with Many Categories
5.3 Approaches for Novel Categories
5.4 Supervised Encoding Methods
5.5 Encodings for Ordered Data
5.6 Creating Features from Text Data
5.7 Factors versus Dummy Variables in Tree-Based Models
5.8 Summary
5.9 Computing
6: Engineering Numeric Predictors
6.1 1:1 Transformations
6.2 1:Many Transformations
6.3 Many:Many Transformations
6.4 Summary
6.5 Computing
7: Detecting Interaction Effects
7.1 Guiding Principles in the Search for Interactions
7.2 Practical Considerations
7.3 The Brute-Force Approach to Identifying Predictive Interactions
7.4 Approaches when Complete Enumeration Is Practically Impossible
7.5 Other Potentially Useful Tools
7.6 Summary
7.7 Computing
8: Handling Missing Data
8.1 Understanding the Nature and Severity of Missing Information
8.2 Models that Are Resistant to Missing Values
8.3 Deletion of Data
8.4 Encoding Missingness
8.5 Imputation Methods
8.6 Special Cases
8.7 Summary
8.8 Computing
9: Working with Profile Data
9.1 Illustrative Data: Pharmaceutical Manufacturing Monitoring
9.2 What Are the Experimental Unit and the Unit of Prediction?
9.3 Reducing Background
9.4 Reducing Other Noise
9.5 Exploiting Correlation
9.6 Impacts of Data Processing on Modeling
9.7 Summary
9.8 Computing
10: Feature Selection Overview
10.1 Goals of Feature Selection
10.2 Classes of Feature Selection Methodologies
10.3 Effect of Irrelevant Features
10.4 Overfitting to Predictors and External Validation
10.5 A Case Study
10.6 Next Steps
10.7 Computing
11: Greedy Search Methods
11.1 Illustrative Data: Predicting Parkinson’s Disease
11.2 Simple Filters
11.3 Recursive Feature Elimination
11.4 Stepwise Selection
11.5 Summary
11.6 Computing
12: Global Search Methods
12.1 Naive Bayes Models
12.2 Simulated Annealing
12.3 Genetic Algorithms
12.4 Test Set Results
12.5 Summary
12.6 Computing
Bibliography
Index