A hierarchical biologically-inspired unsupervised learning algorithm for clustering high dimensional data

Abstract

A central property of biological learning systems is the categorization of incoming data based on associations between multiple features or descriptors of a phenomenon that might be generated by distinct systems and hence do not abide by the same rules of representation. Distinguishing a rose, for example, is based on visual information (color and shape) as well as smell information. To mimic such associative processing is therefore a principal target in artificial learning systems. Unsupervised learning algorithms aim to provide categorical representations of unknown data sets based on a set of descriptors of features. For these algorithms to work efficiently, it is often necessary to trim the set of input features via a pre-processing stage. The choice of relevant features is often preformed independently from the learning algorithm. In the presence of multiple attribute features (the rose example), the selection of relevant features is far less obvious particularly in unsupervised scenarios for unknown datasets. In this thesis we develop an unsupervised learning system which can provide categorical segmentation of data sets that (i) have multiple-features and (ii) have high dimensional description in any given feature space. We present a two-level hierarchy that uses biologically inspired learning methods. The lower level is a variant of Self-Organizing Maps (SOM), that will process every attribute on its own. The higher level is a variant of an Adaptive Resonance Theory 2 (ART-2) network that will form data clusters based on the SOM output. A feedback system between the two levels biases SOM learning based on information-theoretic measures and provides attentional filtering of low-level input arriving at the SOM. This feedback acts to identify important dimensions and SOM units and to reward such dimensions and units at the next learning iteration. A dictionary of attentional filters is added to detect input vectors that are not sufficiently well represented in the algorithm and make the appropriate changes