Combinatorial evolution of regression nodes in feedforward neural networks

✍ Scribed by Gregor P.J. Schmitz; Chris Aldrich

• Publisher: Elsevier Science
• Year: 1999
• Tongue: English
• Weight: 429 KB
• Volume: 12
• Category: Article
• ISSN: 0893-6080
• DOI: 10.1016/s0893-6080(98)00104-x

✦ Synopsis

A number of techniques exist with which neural network architectures such as multilayer perceptrons and radial basis function networks can be trained. These include backpropagation, k-means clustering and evolutionary algorithms. The latter method is particularly useful as it is able to avoid local optima in the search space and can optimise parameters for which no gradient information exists. Unfortunately, only moderately sized networks can be trained by this method, owing to the fact that evolutionary optimisation is very computationally intensive. In this paper a novel algorithm (CERN) is therefore proposed which uses a special form of combinatorial search to optimise groups of neural nodes. Oriented, ellipsoidal basis nodes optimised with CERN achieved significantly better accuracy with fewer nodes than spherical basis nodes optimised by k-means clustering. Multilayer perceptrons optimised by CERN were found to be as accurate as those trained by advanced gradient descent techniques. CERN was also found to be significantly more efficient than a conventional evolutionary algorithm that does not use a combinatorial search.