Self-Organizing Map Overview

A self-organizing map (SOM) or self-organizing feature map (SOFM) is a type of artificial neural network that is trained using unsupervised learning to produce a low-dimensional (typically two dimensional), discretized representation of the input space of the training samples, called a map. Self-organizing maps are different than other artificial neural networks in the sense that they use a neighborhood function to preserve the topological properties of the input space.
A self-organizing map showing US Congress voting patterns visualized in Synapse. The first two boxes show clustering and distances while the remaining ones show the component planes. Red means a yes vote while blue means a no vote in the component planes (except the party component where red is Republican and blue is Democrat).

This makes SOM useful for visualizing low-dimensional views of high-dimensional data, akin to multidimensional scaling. The model was first described as an artificial neural network by the Finnish professor Teuvo Kohonen, and is sometimes called a Kohonen map.

Like most artificial neural networks, SOMs operate in two modes: training and mapping. Training builds the map using input examples. It is a competitive process, also called vector quantization. Mapping automatically classifies a new input vector.

Training

The training steps of a SOM are simple.

1. Initialise the interconnecting weights with random values.


2. Present an input pattern to the network.


3. Choose the output neuron with the highest activation (the “winner”).


4. Update the weights of neurons that are within the “neighbourhood” of the winner, using a relative learning factor.


5. Reduce the learning factor monotonically.


6. Reduce the size of the “neighbourhood” monotonically.


7. Repeat from step two until only small updates are observed.

The spread of the neighbourhood function will initially include all neurons on the grid, gradually reducing to include only the winning neuron.

To provide an example of what to expect from a SOM, I have prepared a simple example that will attempt to group twenty-five foods into regions of similarity, based on three parameters, which are protein, carbohydrate and fat.

Therefore, the challenge for this SOM is to reduce data containing three dimensions down to two, whilst retaining meaning. It does this by automatically indentifying differentiating features that will have the greatest effect.

Input data

The input data is as follows:

Output

After running this data through the SOM, the foods were placed on a 10x10 grid representing their relative similarities. A graphical representation is shown below.



How has the feature map grouped items together, whilst crushing three dimensions into two? Well, a number of zones have formed. Water, which contains no protein, carbs or fat, has been pushed to the bottom right. Directly above in the top right hand corner, sugar, which is made almost entirely of carbs, has taken hold. In the top left corner, butter reigns supreme, being almost entirely fat. Finally, the bottom left is occupied by tuna, which has the highest protein content of the foods in my sample. The remaining foods live between these extremes, with a junk food zone occupying the centre ground.

Source Code

The latest source code (C#, Java and Microsoft Excel versions) can be downloaded from the http://datamining.codeplex.com site.

References

• SOM - Wikipedia
• The source code in C# was found here
• Tom German. Self Organizing Maps
• Timo Honkela. Description of Kohonen's Self-Organizing Map
• Self-organizing maps researches
• Clustering of Self-Organizing Map
• Neural Network Based Clustering using Self Organizing Map (SOM) in Excel