Notes

A neural network that contains two layers and implements a winner take all strategy in the output layer. Rather than taking the output of individual neurons, the neuron with the highest output is considered the winner. SOM's are typically used for clustering related problems where the output neurons represent groups that the input neurons are to be classified into. SOM's may employ a competitive learning strategy.
The principal goal of an SOM is to transform an incoming signal pattern of arbitrary dimension into a one or two dimensional discrete map, and to perform this transformation adaptively in a topologically ordered fashion. We therefore set up our SOM by placing neurons at the nodes of a one or two dimensional lattice. Higher dimensional maps are also possible, but not so common. The neurons become selectively tuned to various input patterns (stimuli) or classes of input patterns during the course of the competitive learning. The locations of the neurons so tuned (i.e. the winning neurons) become ordered and a meaningful coordinate system for the input features is created on the lattice. The SOM thus forms the required topographic map of the input patterns.


Kohonen Network
The name Self-Organizing Map (SOM) signifies a class of neural-network algorithms in the unsupervised-learning category. The central property of the SOM is that it forms a nonlinear projection of a high dimensional data manifold on a regular, low-dimensional (usually 2D) grid. In the display, the clustering of the data space as well as the metric-topological relations of the data items are clearly visible. If the data items are vectors, the components of which are variables with a definite meaning such as the descriptors of statistical data, or measurements that describe a process, the SOM grid can be used as a groundwork on which each of the variables can be displayed separately using grey-level or pseudocolor coding. This kind of combined display has been found very useful for the understanding of the mutual dependencies between the variables, as well as of the structures of the data set.
The most promising fields of application of the SOM are
o Data mining at large, in particular visualization of statistical data and document collections,
o Process analysis, diagnostics, monitoring, and control,
o Biomedical applications, including diagnostic methods and data analysis in bioinformatics,
o Data analysis in commerce, industry, macroeconomics, and finance.



Algorithm:
1. Initialize input nodes, output nodes, and connection weights: Use the top N terms as the input vector and create a two-dimensional map (grid) of M output nodes (say 20-by-10 map of 200 nodes). Initialize weights Wij from N input nodes to M output nodes to small random values.

2. Present each document in order: Describe each document as an input vector of N coordinates. Set a coordinate to 1 if the document has the corresponding term and to 0 if there is no such term. Each document is presented to the system several times.

3. Compute distance to all nodes: Compute Euclidean distance dj between the input vector and each output node j:


Where Xi(t) can be 1 or 0 depending on the presence of i-th term in the document presented at time t. Here, Wij is the vector representing position of the map node j in the document vector space. From a neural net perspective, it can also be interpreted as the weight from input node i to the output node j.

4. Select winning node j* and update weights to node j* and its neighbours: Select winning node j*, which produces minimum dj. Update weights to nodes j* and its neighbours to reduce the distances between them and the input vector xi(t):



5. Label regions in map: After the network is trained through repeated presentations of all documents assign a term to each output node by choosing the one corresponding to the largest weight (winning term). Neighbouring nodes which contain the same winning terms are merged to form a group. Similarly, submit each document as input to the trained network again and assign it to a particular concept in the map. The resulting map thus represents regions of important terms/concepts with the documents assigned to them. Concept regions that are similar (conceptually) appear in the same neighbourhood.