Contents......Page 6
1 Introduction......Page 8
2 A Primer on Neural Networks......Page 9
3 A Mathematical Formalization of Neural Networks......Page 11
4 Problem and Dataset......Page 12
5 A Neural Network in Python......Page 13
6 A Distributed Neural Network Using a Message Queue for Communication......Page 15
7 A GPU-Powered Neural Network......Page 18
8 Discussion and Homework......Page 22
References......Page 24
Deep Learning for Scene Understanding......Page 27
2 Object Recognition......Page 28
2.1 Object Recognition Pipeline......Page 29
2.2 Hand-Crafted Features for Object Recognition......Page 31
2.3 Deep Learning Techniques for Object Recognition......Page 32
3 Face Detection and Recognition......Page 33
3.1 Non-deep Learning Techniques for Face Detection and Recognition......Page 34
3.2 Deep Learning for Face Detection and Recognition......Page 35
4 Text Detection in Natural Scenes......Page 36
4.1 Classical Approaches for Text Detection......Page 37
4.2 Deep Networks for Text Detection......Page 38
5 Depth Map Estimation......Page 39
5.1 Methodology of Depth Map Estimation......Page 40
5.3 Deep Learning Networks for Depth Map Estimation......Page 42
6 Scene Classification......Page 43
6.1 Scene Classification Using Handcrafted-Features......Page 44
6.2 Scene Classification Using Deep Features......Page 45
7 Caption Generation......Page 46
7.1 Deep Networks for Caption Generation......Page 47
8.1 Deep Learning Methods for VQA......Page 49
9 Integration of Scene Understanding Components......Page 50
9.1 Non-deep Learning Works for Holistic Scene Understanding......Page 51
9.2 Deep Learning Based Works for Holistic Scene Understanding......Page 52
10 Conclusion......Page 53
References......Page 54
1 Introduction......Page 58
2.3 Automatic Creation of Company-Specific Dictionaries......Page 60
3.1 Historic Document Analysis......Page 61
3.4 Automated Traffic Monitoring, Surveillance and Security Systems......Page 62
4 Significance of Deep Learning over Machine Learning......Page 63
4.1 Deep Learning Techniques and Architecture......Page 64
5.1 Dataset......Page 65
5.4 Language and Script Peculiarities......Page 66
5.7 Implementation (Available Libraries) Can Be Hardware Dependent......Page 67
6.1 OCR for Arabic like Script......Page 68
6.2 OCR for Symbolic Script......Page 73
6.3 OCR for Latin Script......Page 77
6.5 OCR for Multiple Scripts......Page 80
7 Open Challenges and Future Directions......Page 82
References......Page 83
Deep Learning for Driverless Vehicles......Page 87
1 Introduction......Page 88
2.1 Positioning......Page 89
2.2 Path Planning and Control Systems......Page 90
2.3 Obstacle Detection......Page 91
3.1 Deep Learning Based Approaches for Positioning......Page 93
3.2 Deep Learning Based Approaches Obstacle Detection......Page 94
3.3 Control Systems/End to End Deep Learning Architecture......Page 99
References......Page 101
1 Introduction......Page 104
2 Traditional Document Representation Methods......Page 105
3.1 Word2Vec Model......Page 106
3.2 Doc2Vec Model......Page 107
3.4 Long-Short Term Memory......Page 108
3.5 Convolutional Neural Networks......Page 109
3.6 Experimental Results......Page 110
4 Combined Studies......Page 111
References......Page 112
1 Introduction......Page 114
2 Convolutional Neural Network......Page 115
3 Segmentation of Brachial Plexus from Ultrasound Images......Page 118
3.1 Dataset......Page 119
3.2 Network Architecture......Page 120
3.4 Results......Page 123
4 Classification of Diabetic Retinopathy Stages from Color Fundus Retinal Images......Page 125
4.2 Network Architecture......Page 126
4.3 Results......Page 127
References......Page 128
Deep Learning for Marine Species Recognition......Page 131
1 Introduction......Page 132
2 Deep Learning for Marine Species Classification......Page 133
2.1 Marine Species Classification Based on Deep Convolutional Neural Network (CNN) Features......Page 134
2.2 Marine Species Classification Based on Hybrid Features......Page 135
2.3 End-to-End Training for Marine Species Classification with Deep Convolutional Neural Networks......Page 139
3 Deep Learning for Marine Species Detection......Page 140
4 Future Prospects......Page 142
4.2 Deep Learning in Segmentation......Page 143
5 Conclusion......Page 144
References......Page 145
Deep Molecular Representation in Cheminformatics......Page 148
1 Introduction......Page 149
2 Molecular Representation......Page 150
2.2 Coulomb Matrix......Page 151
2.3 Bag of Bonds......Page 152
3.2 Variance Autoencoders......Page 153
4 Database......Page 155
5 Model......Page 156
6 Simulation Results and Discussion......Page 157
7 Conclusion......Page 158
References......Page 159
1 Introduction......Page 161
2 Deep Learning......Page 165
2.1 Artificial Neural Networks......Page 166
2.2 Deep Learning Software Frameworks and Libraries......Page 170
3.1 Architecture......Page 173
3.2 Training......Page 179
3.3 Some Known CNN Architectures......Page 180
4 Semantic Image Segmentation and Fully Convolutional Networks......Page 181
4.1 Semantic Image Segmentation......Page 182
4.2 Conversion from CNN to FCN......Page 183
5.1 SYNTHIA-Rand-CVPR16 Dataset......Page 186
5.2 Training and Validations of the Models......Page 187
6 Conclusions......Page 195
References......Page 197
Phase Identification and Workflow Modeling in Laparoscopy Surgeries Using Temporal Connectionism of Deep Visual Residual Abstractions......Page 201
2 Prior Art......Page 202
3 Mathematical Formulation......Page 203
4.1 Phase Recognition in Surgical Videos......Page 204
4.2 Statistical Modeling of Workflow......Page 206
5.1 Dataset Description......Page 207
5.2 Compensating Class Imbalance in Training Dataset......Page 210
5.3 Baselines......Page 211
5.5 Visualization of Surgical Workflow......Page 212
6 Discussion......Page 214
7 Conclusion......Page 216
References......Page 217
1 Introduction......Page 218
2 Multilayer Neural Network with Backpropagation......Page 220
2.1 Backpropagating the Error Across the Layers......Page 223
2.2 Updating the Parameters......Page 225
3 Convolution Neural Network (CNN)......Page 227
3.1 Feature Extraction Layer......Page 228
3.2 The Activation Function ReLU:......Page 229
4.1 Parameter Initialization......Page 230
4.2 Forward Propagation......Page 231
4.3 Backward Error Propagation......Page 232
5 CNN in Cytopathology Applications......Page 235
6 Custom Designed CNN for Malaria Detection......Page 236
6.1 Dataset Collection......Page 237
6.2 Identifying Suspected Parasite Locations......Page 238
6.3 Characterising Suspected Parasite Locations......Page 239
6.4 Results and Discussion......Page 244
7.1 Dataset Collection......Page 248
7.2 CNNs in Transfer Learning Mode......Page 249
7.3 Results and Discussion......Page 251
7.4 Comparison with the Classification on Morphometric Features Discussed in GK7......Page 252
References......Page 253
1 Introduction......Page 257
2.1 The Autoencoder......Page 260
2.2 Training Procedure of the Autoencoder......Page 262
2.3 The Stacked Autoencoder......Page 263
2.4 The Softmax Classifier......Page 264
2.6 Training of the DNN Classifier......Page 266
3 Experimental Results and Discussion......Page 267
3.3 Data Sets......Page 268
3.4 Results......Page 273
4 Conclusion......Page 286
References......Page 287
Why Dose Layer-by-Layer Pre-training Improve Deep Neural Networks Learning?......Page 291
1 Introduction......Page 292
2 Functional Analysis of Linear and Nonlinear Feed-Forward Neural Networks in High-Dimensional Spaces......Page 293
2.1 Linear Neural Network (AÂ Linear Mapping)......Page 294
2.2 Functional Analysis of Single-Layer Feed-Forward Neural Networks with a Step Nonlinear Activation Function and Bias......Page 295
2.3 Functional Analysis of Single-Layer Feed-Forward Neural Networks with a Soft Nonlinear Activation Function (Sigmoid) and Bias......Page 299
3 Analysis of Training Some Distinct Samples to a DAE with Nonlinear Activation Functions......Page 300
3.1 Analysis of Training of Some Distinct Samples to the DAE with Step Activation Functions......Page 302
3.2 Analysis of Training of Some Distinct Samples to the DAE with Continuous (Sigmoid) Activation Functions......Page 303
4 Layer-by-Layer Pre-training......Page 307
5 Experimental Results and Discussion......Page 308
5.1 Databases......Page 309
5.4 Evaluation of Locations of Hyperplanes Corresponding to Neurons of DAE Hidden Layers......Page 310
6 Conclusion......Page 314
References......Page 315
1 Introduction......Page 317
1.3 Artificial Intelligence......Page 318
2 Deep Learners......Page 319
2.1 Types of Deep Learners......Page 321
3.1 Predictive Health Care......Page 323
3.2 Medical Decision Support......Page 324
4 Challenges......Page 325
5 Future Research......Page 326
References......Page 327
Deep Learning for Brain Computer Interfaces......Page 330
1 Introduction......Page 331
2 Different Parts of a BCI......Page 332
3 Deep Neural Networks for Brain Computer Interfaces......Page 334
4 Applications of BCI......Page 337
5 Conclusion and Future Work......Page 339
References......Page 340
1 Introduction......Page 342
2.1 Problem Definition......Page 343
2.3 Simple Games with Embedded Learning......Page 344
2.4 HFDLN as Deep Learning Game......Page 347
3 Experimental Evaluations......Page 350
References......Page 357
1 Introduction and Motivations......Page 360
2 Deep Learning......Page 362
3 Deep Learning Models and Applications......Page 363
3.1 Feed Forward Networks (FFNs)......Page 364
3.2 Convolutional Neural Networks......Page 367
3.3 Recurrent Neural Networks......Page 369
4 Deep Learning Challenges......Page 370
5.2 Uses of Gene Expressions Data and Limitations......Page 371
5.3 Publically Available Data Repositories for Gene Expressions......Page 372
5.4 Deep Learning on Gene Expressions......Page 373
5.5 Deep Neural Nets......Page 374
5.6 Convolutional Neural Networks......Page 375
6 Conclusion......Page 377
References......Page 378