Kernel Based Algorithms for Mining Huge Data Sets: Supervised, Semi-supervised, and Unsupervised Learning

✍ Scribed by Te-Ming Huang, Vojislav Kecman, Ivica Kopriva

• Publisher: Springer
• Year: 2006
• Tongue: English
• Leaves: 266
• Series: Studies in Computational Intelligence
• Edition: 1
• Category: Library

✦ Synopsis

This is the first book treating the fields of supervised, semi-supervised and unsupervised machine learning collectively. The book presents both the theory and the algorithms for mining huge data sets using support vector machines (SVMs) in an iterative way. It demonstrates how kernel based SVMs can be used for dimensionality reduction and shows the similarities and differences between the two most popular unsupervised techniques.

✦ Table of Contents

Contents......Page 12
1.1 An Overview of Machine Learning......Page 16
1.2 Challenges in Machine Learning......Page 18
1.2.1 Solving Large-Scale SVMs......Page 19
1.2.2 Feature Reduction with Support Vector Machines......Page 20
1.2.3 Graph-Based Semi-supervised Learning Algorithms......Page 21
1.2.4 Unsupervised Learning Based on Principle of Redundancy Reduction......Page 22
2 Support Vector Machines in Classification and Regression – An Introduction......Page 25
2.1 Basics of Learning from Data......Page 26
2.2.1 Linear Maximal Margin Classifier for Linearly Separable Data......Page 35
2.2.2 Linear Soft Margin Classifier for Overlapping Classes......Page 46
2.2.3 The Nonlinear SVMs Classifier......Page 50
2.2.4 Regression by Support Vector Machines......Page 62
2.3 Implementation Issues......Page 71
3.1 Introduction......Page 75
3.2 Iterative Single Data Algorithm for Positive Definite Kernels without Bias Term b......Page 77
3.2.1 Kernel AdaTron in Classification......Page 78
3.2.2 SMO without Bias Term b in Classification......Page 79
3.2.3 Kernel AdaTron in Regression......Page 80
3.2.4 SMO without Bias Term b in Regression......Page 81
3.2.5 The Coordinate Ascent Based Learning for Nonlinear Classification and Regression Tasks......Page 82
3.3 Iterative Single Data Algorithm with an Explicit Bias Term b......Page 87
3.3.1 Iterative Single Data Algorithm for SVMs Classification with a Bias Term b......Page 88
3.4 Performance of the Iterative Single Data Algorithm and Comparisons......Page 94
3.5.1 Working-set Selection and Shrinking of ISDA for Classification......Page 97
3.5.2 Computation of the Kernel Matrix and Caching of ISDA for Classification......Page 103
3.5.3 Implementation Details of ISDA for Regression......Page 106
3.6 Conclusions......Page 108
4.1 Introduction......Page 110
4.2 Basics of Microarray Technology......Page 112
4.3.1 Recursive Feature Elimination with Support Vector Machines......Page 114
4.3.2 Selection Bias and How to Avoid It......Page 115
4.4 Influence of the Penalty Parameter C in RFE-SVMs......Page 116
4.5.1 Results for Various C Parameters......Page 117
4.5.2 Simulation Results with Different Preprocessing Procedures......Page 120
4.6.1 Basic Concept of Nearest Shrunken Centroid Method......Page 125
4.6.2 Results on the Colon Cancer Data Set and the Lymphoma Data Set......Page 128
4.7 Comparison of Genes' Ranking with Different Algorithms......Page 133
4.8 Conclusions......Page 135
5.1 Introduction......Page 137
5.2.1 Gaussian Random Fields Model......Page 139
5.2.2 Global Consistency Model......Page 142
5.2.3 Random Walks on Graph......Page 145
5.3.1 Background and Test Settings......Page 148
5.3.3 Possible Theoretical Explanations on the Effect of Unbalanced Labeled Data......Page 151
5.4 Classifier Output Normalization: A Novel Decision Rule for Semi-supervised Learning Algorithm......Page 154
5.5 Performance Comparison of Semi-supervised Learning Algorithms......Page 157
5.5.1 Low Density Separation: Integration of Graph-Based Distances and ∇TSVM......Page 158
5.5.2 Combining Graph-Based Distance with Manifold Approaches......Page 161
5.5.3 Test Data Sets......Page 162
5.5.4 Performance Comparison Between the LDS and the Manifold Approaches......Page 164
5.6 Implementation of the Manifold Approaches......Page 166
5.6.1 Variants of the Manifold Approaches Implemented in the Software Package SemiL......Page 167
5.6.2 Implementation Details of SemiL......Page 169
5.6.3 Conjugate Gradient Method with Box Constraints......Page 174
5.6.4 Simulation Results on the MNIST Data Set......Page 178
5.7 An Overview of Text Classification......Page 179
5.8 Conclusions......Page 183
6 Unsupervised Learning by Principal and Independent Component Analysis......Page 186
6.1 Principal Component Analysis......Page 191
6.2 Independent Component Analysis......Page 208
6.3 Concluding Remarks......Page 219
A Support Vector Machines......Page 220
A.1 L2 Soft Margin Classifier......Page 221
A.2 L2 Soft Regressor......Page 222
A.3 Geometry and the Margin......Page 224
B Matlab Code for ISDA Classification......Page 227
C Matlab Code for ISDA Regression......Page 232
D Matlab Code for Conjugate Gradient Method with Box Constraints......Page 238
E Uncorrelatedness and Independence......Page 241
F Independent Component Analysis by Empirical Estimation of Score Functions i.e., Probability Density Functions......Page 245
G.1 Installation......Page 249
G.2.1 Raw Data Format:......Page 251
G.3 Getting Started......Page 252
G.3.1 Design Stage......Page 253
References......Page 254
E......Page 263
M......Page 264
S......Page 265
W......Page 266