Efficient information transfer and anti-Hebbian neural networks

✍ Scribed by Mark D. Plumbley

• Publisher: Elsevier Science
• Year: 1993
• Tongue: English
• Weight: 859 KB
• Volume: 6
• Category: Article
• ISSN: 0893-6080
• DOI: 10.1016/s0893-6080(05)80127-3

✦ Synopsis

We attempt to maximise mutual information in a neural network where the output signal is comtpted by noise in the transmission process to the next stage. We assume that the average power available for transmission is limited and that the transmission noise is uncorrelated equal-variance Gaussian noise. Using the Lagrange multiplier technique, we construct a utility fimction that should be maximised to achieve our optimum. This optimum will be achieved when the output signal components are also uncorrelated and of equal variance. For a linear network with lateral inhibitor)., connections, using an anti-Hebbian algorithm with modified self-inhibitory connections we guarantee to increase our utility fimction over time and converge to the required optimum, although not by steepest ascent. For a network with inhibitor), interneurons, a combined Hebbian/anti-Hebbian algorithm will similarly increase the utility function over time to find the optimum. This network with interneurons suggests that negative feedback may be used in a perceptual system to optimise forward transmission of information.