Neural Networks, Fuzzy Logic and Genetic Algorithms

✍ Scribed by S. Rajasekaran

• Publisher: PHI Learning Private Limited
• Year: 2004
• Tongue: English
• Leaves: 965
• Category: Library

✦ Table of Contents

NEURAL NETWORKS, FUZZY LOGIC, AND GENETIC ALGORITHMS: Synthesis and Applications
Copyright
Dedication
Table of Contents
Preface
ORGANIZATION
1. Introduction to Artificial Intelligence Systems
1.1 Neural Networks
1.2 Fuzzy Logic
1.3 Genetic Algorithms
1.4 Structure of This Book
SUMMARY
REFERENCES
Part 1: Neural Networks
2. Fundamentals of Neural Networks
2.1 Basic Concepts of Neural Networks
2.2 Human Brain
2.3 Model of an Artificial Neuron
2.4 Neural Network Architectures
2.4.1 Single Layer Feedforward Network
2.4.2 Multilayer Feedforward Network
2.4.3 Recurrent Networks
2.5 Characteristics of Neural Networks
2.6 Learning Methods
2.7 Taxonomy of Neural Network architectures
2.8 HISTORY OF NEURAL NETWORK RESEARCH
2.9 Early Neural Network Architectures
2.9.l Rosenblatt’s Perceptron
XOR Problem
Algorithm 2.1
2.9.2 ADALINE Network
2.9.3 MADALINE Network
2.10 Some Application Domains
SUMMARY
PROGRAMMING ASSIGNMENT
SUGGESTED FURTHER READING
REFERENCES
3. Backpropagation Networks
3.1 ARCHITECTURE OF A BACKPROPAGATION NETWORK
3.1.1 The Perceptron Model
3.1.2 The Solution
3.1.3 Single Layer Artificial Neural Network
3.1.4 Model for Multilayer Perceptron
3.2 BACKPROPAGATION LEARNING
3.2.1 Input Layer Computation
3.2.2 Hidden Layer Computation
3.2.3 Output Layer Computation
3.2.4 Calculation of Error
3.2.5 Training of Neural Network
3.2.6 Method of Steepest Descent
3.2.7 Effect of Learning Rate ‘$eta$$’
3.2.8 Adding a Momentum Term
3.2.9 Backpropagation Algorithm
Algorithm 3.1 (Backpropagation Learning Algorithm)
3.3 ILLUSTRATION
3.4 APPLICATIONS
3.4.1 Design of Journal Bearing
3.4.2 Classification of Soil
3.4.3 Hot Extrusion of Steel
3.5 EFFECT OF TUNING PARAMETERS OF THE BACKPROPAGATION NEURAL NETWORK
3.6 SELECTION OF VARIOUS PARAMETERS IN BPN
3.6.1 Number of Hidden Nodes
3.6.2 Momentum Coefficient $alpha$$
3.6.3 Sigmoidal Gain $lambda$$
3.6.4 Local Minima
3.6.5 Learning Coefficient $eta$$
3.7 VARIATIONS OF STANDARD BACKPROPATATION ALGORITHM
3.7.1 Decremental Iteration Procedure
3.7.2 Adaptive Backpropagation (Accelerated Learning)
3.7.3 Genetic Algorithm Based Backpropagation
3.7.4 Quick Prop Training
3.7.5 Augmented BP Networks
3.7.6 Sequential Learning Approach for Single Hidden Layer Neural Networks
3.8 RESEARCH DIRECTIONS
3.8.1 New Topologies
3.8.2 Better Learning Algorithms
3.8.3 Better Training Strategies
3.8.4 Hardware Implementation
3.8.5 Conscious Networks
SUMMARY
PROGRAMMING ASSIGNMENT
REFERENCES
4. Associative Memory
4.1 AutoCorrelators
4.2 HeteroCorrelators: Kosko’s Discrete BAM
4.2.1 Addition and Deletion of Pattern Pairs
4.2.2 Energy Function for BAM
4.3 WANG ET AL.’S MULTIPLE TRAINING ENCODING STRATEGY
Algorithm 4.1 (Wang et al.’s Multiple Training Encoding Strategy)
4.4 EXPONENTIAL BAM
4.4.1 Evolution Equations
4.5 Associative Memory for real-coded pattern pairs
4.5.1 Input Normalization
4.5.2 Evolution Equations
Algorithm 4.2 (Simplified Bi-directional Associative Memory)
4.6 Applications
4.6.1 Recognition of Characters
4.6.2 Fabric Defect Identification
4.7 RECENT TRENDS
SUMMARY
PROGRAMMING ASSIGNMENT
REFERENCES
5. Adaptive Resonance Theory
5.1 INTRODUCTION
5.1.1 Cluster Structure
5.1.2 Vector Quantization
FOR THRESHOLD DISTANCE OF 2
FOR THRESHOLD DISTANCE OF 4.5
5.1.3 Classical ART Networks
5.1.4 Simplified ART Architecture
5.2 ART1
5.2.1 Architecture of ART1
5.2.2 Special Features of ART1 Models
5.2.3 ART1 Algorithm
Algorithm 5.1 (Art1 Algorithm)
5.2.4 Illustration
5.3 ART2
5.3.1 Architecture of ART2
5.3.2 ART2 Algorithm
Algorithm 5.2 (ART2 Algorithm)
5.3.3 Illustration
5.4 APPLICATIONS
5.4.1 Character Recognition Using ART1
5.4.2 Classification of Soil (Rajasekaran et al., 2001)
5.4.3 Prediction of Load from Yield Patterns of Elastic-Plastic Clamped Square Plate
Output of the Example 5.4
5.4.4 Chinese Character Recognition—Some Remarks
5.5 Sensitiveness of Ordering of Data
SUMMARY
PROGRAMMING ASSIGNMENT
SUGGESTED FURTHER READING
REFERENCES
Part 2: FUZZY LOGIC
6. Fuzzy Set Theory
6.1 FUZZY VERSUS CRISP
6.2 CRISP SETS
6.2.1 Operations on Crisp Sets
6.2.2 Properties of Crisp Sets
6.2.3 Partition and Covering
6.3 FUZZY SETS
6.3.1 Membership Function
6.3.2 Basic Fuzzy Set Operations
6.3.3 Properties of Fuzzy Sets
6.4 CRISP RELATIONS
6.4.1 Cartesian Product
6.4.2 Other Crisp Relations
6.4.3 Operations on Relations
6.5 FUZZY RELATIONS
6.5.1 Fuzzy Cartesian Product
6.5.2 Operations on Fuzzy Relations
SUMMARY
PROGRAMMING ASSIGNMENT
SUGGESTED FURTHER READING
REFERENCE
7. Fuzzy Systems
7.1 CRISP LOGIC
7.1.1 Laws of Propositional Logic
7.1.2 Inference in Propositional Logic
7.2 PREDICATE LOGIC
7.2.1 Interpretations of Predicate Logic Formula
7.2.2 Inference in Predicate Logic
7.3 Fuzzy Logic
7.3.1 Fuzzy Quantifiers
7.3.2 Fuzzy Inference
7.4 FUZZY RULE BASED SYSTEM
7.5 Defuzzification
7.6 Applications
7.6.1 Greg Viot’s Fuzzy Cruise Controller
7.6.2 Air Conditioner Controller
SUMMARY
PROGRAMMING ASSIGNMENT
SUGGESTED FURTHER READING
REFERENCES
Part 3: GENETIC ALGORITHMS
8. Fundamentals of Genetic Algorithms
8.1 GENETIC ALGORITHMS: HISTORY
8.2 BASIC CONCEPTS
8.2.1 Biological Background
8.3 CREATION OF OFFSPRINGS
8.3.1 Search Space
8.4 WORKING PRINCIPLE
8.5 ENCODING
8.5.1 Binary Encoding
8.5.2 Octal Encoding (0 to 7)
8.5.3 Hexadecimal Encoding (0123456789ABCDEF)
8.5.4 Permutation Encoding
8.5.5 Value Encoding
8.5.6 Tree Encoding
8.6 FITNESS FUNCTION
8.7 REPRODUCTION
8.7.1 Roulette-wheel Selection
8.7.2 Boltzmann Selection
8.7.3 Tournament Selection
8.7.4 Rank Selection
8.7.5 Steady-state Selection
8.7.6 Elitism
8.7.7 Generation Gap and Steady-state Replacement
SUMMARY
PROGRAMMING ASSIGNMENT
REFERENCES
9. Genetic Modelling
9.1 INHERITANCE OPERATORS
9.2 CROSS OVER
9.2.1 Single-site Cross Over
9.2.2 Two-point Cross Over
9.2.3 Multi-point Cross Over
9.2.4 Uniform Cross Over
9.2.5 Matrix Cross Over (Two-dimensional Cross Over)
9.2.6 Cross Over Rate
9.3 INVERSION AND DELETION
9.3.1 Inversion
9.3.2 Deletion and Duplication
9.3.3 Deletion and Regeneration
9.3.4 Segregation
9.3.5 Cross Over and Inversion
9.4 MUTATION OPERATOR
9.4.1 Mutation
9.4.2 Mutation Rate Pm
9.5 BIT-WISE OPERATORS
9.5.1 One’s Complement Operator
9.5.2 Logical Bit-wise Operators
9.5.3 Shift Operators
9.6 BIT-WISE OPERATORS USED IN GA
9.7 GENERATIONAL CYCLE
9.8 CONVERGENCE OF GENETIC ALGORITHM
9.9 APPLICATIONS
9.9.1 Composite Laminates
9.9.2 Constrained Optimization
9.10 MULTI-LEVEL OPTIMIZATION
9.11 REAL LIFE PROBLEM
9.12 DIFFERENCES AND SIMILARITIES BETWEEN GA AND OTHER TRADITIONAL METHODS
9.13 ADVANCES IN GA
SUMMARY
PROGRAMMING ASSIGNMENT
SUGGESTED FURTHER READING
SOME USEFUL WEBSITES
REFERENCES
Part 4: HYBRID SYSTEMS
10. Integration of Neural Networks, Fuzzy Logic, and Genetic Algorithms
10.1 HYBRID SYSTEMS
10.1.1 Sequential Hybrid Systems
10.1.2 Auxiliary Hybrid Systems
10.1.3 Embedded Hybrid Systems
10.2 NEURAL NETWORKS, FUZZY LOGIC, AND GENETIC
10.2.1 Neuro-Fuzzy Hybrids
10.2.2 Neuro-Genetic Hybrids
10.2.3 Fuzzy-Genetic Hybrids
10.3 PREVIEW OF THE HYBRID SYSTEMS TO BE DISCUSSED
10.3.1 Genetic Algorithm based Backpropagation Network
10.3.2 Fuzzy-Backpropagation Network
10.3.3 Simplified Fuzzy ARTMAP
10.3.4 Fuzzy Associative Memories
10.3.5 Fuzzy Logic Controlled Genetic Algorithms
SUMMARY
REFERENCES
11. Genetic Algorithm Based Backpropagation Networks
11.1 GA BASED WEIGHT DETERMINATION
11.1.1 Coding
11.1.2 Weight Extraction
11.1.3 Fitness Function
11.1.4 Reproduction
11.1.5 Convergence
11.2 APPLICATIONS
11.2.1 K-factor Determination in Columns
11.2.2 Electrical Load Forecasting
SUMMARY
PROGRAMMING ASSIGNMENT
SUGGESTED FURTHER READING
REFERENCES
12. Fuzzy Backpropagation Networks
12.1 LR-TYPE FUZZY NUMBERS
12.1.1 Operations on LR-type Fuzzy Numbers
12.2 FUZZY NEURON
12.3 FUZZY BP ARCHITECTURE
12.4 LEARNING IN FUZZY BP
12.5 INFERENCE BY FUZZY BP
Algorithm 12.2
12.6 APPLICATIONS
12.6.1 Knowledge Base Evaluation
12.6.2 Earthquake Damage Evaluation
SUMMARY
PROGRAMMING ASSIGNMENT
REFERENCES
13. Simplified Fuzzy ARTMAP
13.1 FUZZY ARTMAP: A BRIEF INTRODUCTION
13.2 SIMPLIFIED FUZZY ARTMAP
13.2.1 Input Normalization
13.2.2 Output Node Activation
13.3 WORKING OF SIMPLIFIED FUZZY ARTMAP
13.4 Application: Image Recognition
13.4.1 Feature Extraction—Moment Based Invariants
13.4.2 Computation of Invariants
13.4.3 Structure of the Simplified Fuzzy ARTMAP based
13.4.4 Experimental Study
13.5 RECENT TRENDS
SUMMARY
PROGRAMMING ASSIGNMENT
REFERENCES
14. Fuzzy Associative Memories
14.1 FAM—AN INTRODUCTION
14.2 SINGLE ASSOCIATION FAM
14.2.1 Graphical Method of Inference
14.2.2 Correlation Matrix Encoding
14.3 Fuzzy Hebb FAMs
14.4 FAM INVOLVING A RULE BASE
14.5 FAM RULES WITH MULTIPLE ANTECEDENTS/CONSEQUENTS
14.5.1 Decomposition Rules
14.6 APPLICATIONS
14.6.1 Balancing an Inverted Pendulum
14.6.2 Fuzzy Truck Backer-upper System
SUMMARY
PROGRAMMING ASSIGNMENT
SUGGESTED FURTHER READING
REFERENCES
15. Fuzzy Logic Controlled Genetic Algorithms
15.1 SOFT COMPUTING TOOLS
15.1.1 Fuzzy Logic as a Soft Computing Tool
15.1.2 Genetic Algorithm as a Soft Computing Tool
15.2 PROBLEM DESCRIPTION OF OPTIMUM DESIGN
15.3 FUZZY CONSTRAINTS
15.4 ILLUSTRATIONS
15.4.1 Optimization of the Weight of A Beam
15.4.2 Optimal Mix Design for High Performance Concrete
15.5 GA IN FUZZY LOGIC CONTROLLER DESIGN
15.6 FUZZY LOGIC CONTROLLER
15.6.1 Components of Fuzzy Logic Controller (FLC)
15.6.2 Fuzzy IF-THEN Rules
15.7 FLC-GA BASED STRUCTURAL OPTIMIZATION
15.8 APPLICATIONS
15.8.1 Optimum Truss
15.8.2 112 Bar Dome Space Truss
SUMMARY
PROGRAMMING ASSIGNMENT
SUGGESTED FURTHER READING
REFERENCES
Index