Pattern Recognition and Classification : an Introduction

✍ Scribed by Geoff Dougherty

• Publisher: Springer
• Year: 2013
• Tongue: English
• Leaves: 203
• Category: Library

✦ Synopsis

The use of pattern recognition and classification is fundamental to many of the automated electronic systems in use today. However, despite the existence of a number of notable books in the field, the subject remains very challenging, especially for the beginner. Pattern Recognition and Classification presents a comprehensive introduction to the core concepts involved in automated pattern recognition. It is designed to be accessible to newcomers from varied backgrounds, but it will also be useful to researchers and professionals in image and signal processing and analysis, and in computer vision. Fundamental concepts of supervised and unsupervised classification are presented in an informal, rather than axiomatic, treatment so that the reader can quickly acquire the necessary background for applying the concepts to real problems. More advanced topics, such as estimating classifier performance and combining classifiers, and details of particular project applications are addressed in the later chapters. This book is suitable for undergraduates and graduates studying pattern recognition and machine learning. Read more... Preface -- Acknowledgments -- Chapter 1 Introduction -- 1.1 Overview -- 1.2 Classification -- 1.3 Organization of the Book -- Bibliography -- Exercises -- Chapter 2 Classification -- 2.1 The Classification Process -- 2.2 Features -- 2.3 Training and Learning -- 2.4 Supervised Learning and Algorithm Selection -- 2.5 Approaches to Classification -- 2.6 Examples -- 2.6.1 Classification by Shape -- 2.6.2 Classification by Size -- 2.6.3 More Examples -- 2.6.4 Classification of Letters -- Bibliography -- Exercises -- Chapter 3 Non-Metric Methods -- 3.1 Introduction -- 3.2 Decision Tree Classifier -- 3.2.1 Information, Entropy and Impurity -- 3.2.2 Information Gain -- 3.2.3 Decision Tree Issues -- 3.2.4 Strengths and Weaknesses -- 3.3 Rule-Based Classifier -- 3.4 Other Methods -- Bibliography -- Exercises -- Chapter 4 Statistical Pattern Recognition -- 4.1 Measured Data and Measurement Errors -- 4.2 Probability Theory -- 4.2.1 Simple Probability Theory -- 4.2.2 Conditional Probability and Bayes' Rule -- 4.2.3 Naïve Bayes classifier -- 4.3 Continuous Random Variables -- 4.3.1 The Multivariate Gaussian -- 4.3.2 The Covariance Matrix -- 4.3.3 The Mahalanobis Distance -- Bibliography -- Exercises -- Chapter 5 Supervised Learning -- 5.1 Parametric and Non-Parametric Learning -- 5.2 Parametric Learning -- 5.2.1 Bayesian Decision Theory -- 5.2.2 Discriminant Functions and Decision Boundaries -- 5.2.3 MAP (Maximum A Posteriori) Estimator -- Bibliography -- Exercises -- Chapter 6 Non-Parametric Learning -- 6.1 Histogram Estimator and Parzen Windows -- 6.2 k-Nearest Neighbor (k-NN) Classification -- 6.3 Artificial Neural Networks (ANNs) -- 6.4 Kernel Machines -- Bibliography -- Exercises -- Chapter 7 Feature Extraction and Selection -- 7.1 Reducing Dimensionality -- 7.1.1 Pre-Processing -- 7.2 Feature Selection -- 7.2.1 Inter/Intra-Class Distance -- 7.2.2 Subset Selection -- 7.3 Feature Extraction -- 7.3.1 Principal Component Analysis (PCA) -- 7.3.2 Linear Discriminant Analysis (LDA) -- Bibliography -- Exercises -- Chapter 8 Unsupervised Learning -- 8.1 Clustering -- 8.2 k-Means Clustering -- 8.2.1 Fuzzy c-Means Clustering -- 8.3 (Agglomerative) Hierarchical Clustering -- Bibliography -- Exercises -- Chapter 9 Estimating and Comparing Classifiers -- 9.1 Comparing Classifiers and the No Free Lunch Theorem -- 9.1.2 Bias and Variance -- 9.2 Cross-Validation and Resampling Methods -- 9.2.1 The Holdout Method -- 9.2.2 k-Fold Cross-Validation -- 9.2.3 Bootstrap -- 9.3 Measuring Classifier Performance -- 9.4 Comparing Classifiers -- 9.4.1 ROC curves -- 9.4.2 McNemar's Test -- 9.4.3 Other Statistical Tests -- 9.4.4 The Classification Toolbox -- 9.5 Combining classifiers -- Bibliography -- Chapter 10 Projects -- 10.1 Retinal Tortuosity as an Indicator of Disease -- 10.2 Segmentation by Texture -- 10.3 Biometric Systems -- 10.3.1 Fingerprint Recognition -- 10.3.2 Face Recognition -- Bibliography -- Index

✦ Table of Contents

Cover......Page 1
Pattern Recognition and Classification
......Page 4
Preface......Page 6
Acknowledgments......Page 8
Contents......Page 10
1.1 Overview......Page 13
1.2 Classification......Page 15
1.4 Exercises......Page 18
References......Page 19
2.1 The Classification Process......Page 20
2.2 Features......Page 22
2.3 Training and Learning......Page 27
2.4 Supervised Learning and Algorithm Selection......Page 28
2.5 Approaches to Classification......Page 29
2.6.1 Classification by Shape......Page 32
2.6.2 Classification by Size......Page 33
2.6.3 More Examples......Page 34
2.7 Exercises......Page 36
References......Page 37
3.2 Decision Tree Classifier......Page 38
3.2.1 Information, Entropy, and Impurity......Page 40
3.2.2 Information Gain......Page 42
3.2.3 Decision Tree Issues......Page 46
3.2.4 Strengths and Weaknesses......Page 49
3.4 Other Methods......Page 50
3.5 Exercises......Page 51
References......Page 52
4.2.1 Simple Probability Theory......Page 53
4.2.2 Conditional Probability and Bayes´ Rule......Page 56
4.2.3 Naïve Bayes Classifier......Page 63
4.3 Continuous Random Variables......Page 64
4.3.1 The Multivariate Gaussian......Page 67
4.3.2 The Covariance Matrix......Page 69
4.3.3 The Mahalanobis Distance......Page 79
4.4 Exercises......Page 82
References......Page 84
5.2.1.1 Single Feature (1D)......Page 85
5.2.1.2 Multiple features......Page 95
5.2.2 Discriminant Functions and Decision Boundaries......Page 97
5.2.3 MAP (Maximum A Posteriori) Estimator......Page 104
5.3 Exercises......Page 106
References......Page 108
6.1 Histogram Estimator and Parzen Windows......Page 109
6.2 k-Nearest Neighbor (k-NN) Classification......Page 110
6.3 Artificial Neural Networks......Page 114
6.4 Kernel Machines......Page 127
6.5 Exercises......Page 130
References......Page 131
7.1 Reducing Dimensionality......Page 132
7.2.1 Inter/Intraclass Distance......Page 133
7.2.2 Subset Selection......Page 135
7.3.1 Principal Component Analysis......Page 136
7.3.2 Linear Discriminant Analysis......Page 144
7.4 Exercises......Page 149
References......Page 150
8.1 Clustering......Page 151
8.2 k-Means Clustering......Page 153
8.2.1 Fuzzy c-Means Clustering......Page 156
8.3 (Agglomerative) Hierarchical Clustering......Page 158
8.4 Exercises......Page 162
References......Page 163
9.1 Comparing Classifiers and the No Free Lunch Theorem......Page 164
9.1.1 Bias and Variance......Page 166
9.2 Cross-Validation and Resampling Methods......Page 167
9.2.1 The Holdout Method......Page 168
9.2.2 k-Fold Cross-Validation......Page 169
9.2.3 Bootstrap......Page 170
9.3 Measuring Classifier Performance......Page 171
9.4.3 Other Statistical Tests......Page 176
9.4.4 The Classification Toolbox......Page 178
9.5 Combining Classifiers......Page 181
References......Page 183
10.1 Retinal Tortuosity as an Indicator of Disease......Page 184
10.2 Segmentation by Texture......Page 188
10.3 Biometric Systems......Page 190
10.3.1 Fingerprint Recognition......Page 191
References......Page 194
Index......Page 196