Optimization of the hidden unit function in feedforward neural networks

✍ Scribed by Osamu Fujita

• Publisher: Elsevier Science
• Year: 1992
• Tongue: English
• Weight: 790 KB
• Volume: 5
• Category: Article
• ISSN: 0893-6080
• DOI: 10.1016/s0893-6080(05)80136-4

✦ Synopsis

A novel objective fimction is proposed.for optimizing the hidden unit function in feedforward neural networks. This objective fiznction represents the performance of the hidden unit at minimizing the least squared output errors of the linear output zmit. This is derived from the decrease in the output errors due to the addition of the hidden units. The optimized ozttput state vectors of the hidden units span a proper state space, which includes the desired output vectors for the network. The optimization (maximization of the objective fimction) is equal to minimizing the angle between the desired output vector and tile projection of the hidden unit's output state vector onto the orthogonal complement of the subspace spanned by the other state vectors. Tile approximate solution can be obtained using tile gradient ascent algorithm. This optimization method is usefid in constructingfitlly connected feedforward neural networks and for minimizing tile size of layered networks.