Bidirectional clustering of weights for neural networks with common weights

✍ Scribed by Kazumi Saito; Ryohei Nakano

• Publisher: John Wiley and Sons
• Year: 2007
• Tongue: English
• Weight: 445 KB
• Volume: 38
• Category: Article
• ISSN: 0882-1666
• DOI: 10.1002/scj.20535

✦ Synopsis

Abstract

This paper proposes a method which succinctly structures neural networks having a few thousand weights. Here structuring means weight sharing where weights in a network are divided into clusters and weights within a cluster have the same value. We newly introduce a weight sharing technique called bidirectional clustering of weights (BCW), together with second‐order optimal criteria for both cluster merging and splitting. Our experiments using two artificial data sets showed that the BCW method works well to find a succinct network structure from a neural network having about 2000 weights in both regression and classification problems. © 2007 Wiley Periodicals, Inc. Syst Comp Jpn, 38(10): 46–57, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20535