09dw Dm Kohonennetwork

8/12/2019 09dw Dm Kohonennetwork

1/23

1

. ([email protected], 08-9275-9797)

Kohonen

Networks


2/23

2

1. Self-Organizing Maps

Kohonen Networks developed in 1982 by Tuevo Kohonen

Initially applied to image and sound analysis

Represent Self-Organizing Map (SOM)

Special class of Neural NetworksSOM

Convert high-dimensional input to low-dimensional, discrete map

Applicable to cluster analysis

Structure output nodes to clusters of nodes

Nodes in close proximity more similar, compared to those farther

apart


3/23

3

1. Self-Organizing Maps (contd)Based on Competitive Learning, where output nodes compete tobecome winning node (neuron)

Nodes become selectively tuned to input patterns during the

competitive learning process (Haykin)

Example SOM architecture shown with two inputs,AgeandIncome

Age Income

Output Layer

Input Layer

Connections with Weights


4/23

4

1. Self-Organizing Maps (contd)Every connection between two nodes has weight

Weight values initialized randomly 01

Adjusting weights key feature of learning process

Attribute values are normalized or standardized

SOMs do not have hidden layer

Data passed directly from input layer to output layer


5/23

5

1. Self-Organizing Maps (contd)SOM Process Example

Input record hasAge= 0.69 andIncome= 0.88

Attribute values forAgeandIncomeenter through respective input

nodes

Values passed to all output nodes

These values, together with connection weights, determine value of

Scoring Function for each output node

Output node with best score designated Winning Node for record


6/23

6

1. Self-Organizing Maps (contd)SOMs exhibit three characteristics

Competition

Output nodes compete with one another for best score

Euclidean Distance function commonly used

Winning node produces smallest distance between inputs and

connection weights

Cooperation

Winning node becomes center of neighborhood

Output nodes in neighborhood share excitement or reward


7/23

7

1. Self-Organizing Maps (contd)Emulates behavior of biological neurons, which are sensitive tooutput of neighbors

Nodes in output layer not directly connected

However, share common features because of neighborhood

behavior

Adaptation

Neighborhood nodes participate in adaptation (learning)

Weights adjusted to improve score function

For subsequent iterations, increases likelihood of winning records

with similar values


8/23

8

2. Kohonen Networks

Kohonen Networks are SOMs exhibiting Kohonen LearningNodes in neighborhood of winning node adjust their weights

Adjustment is linear combination of input vector and current

weight vector:

ratelearning10,

nodeoutputparticularforweights,ofsetcurrent,...,,

recordthforvaluesfield,...,,

where,)(

21

21

,,,

jmwww

nmxxx

wxww

mjjjj

nmnnn

CURRENTijniCURRENTijNEWij

w

x


9/23

9

3. Kohonen Networks Algorithm

InitializeAssign random values to weights

Initial learning rate and neighborhood size values assigned

LOOP: For each input vector x, do:Competition

For each output node j, calculate scoring function D(wj, xn)

i

niijnj xwxwD 2)(),(DistanceEuclidean

Find winning node J, that minimizes D(wj, xn)


10/23

10

3. Kohonen Networks Algorithm(contd)Cooperation

Identify output nodes j, within neighborhood of J defined by

neighborhood size R

Adaptation

Adjust weights of all neighborhood nodes j:

)( ,,, CURRENTijniCURRENTijNEWij wxww

Adjust learning rate and neighborhood size (decreasing), as needed

Nodes not attracting sufficient number of hits may be pruned

Stop when termination criteria met


11/23

11

4. Example of a Kohonen Network Study

Use simple 2 x 2 Kohonen NetworkNeighborhood Size = 0, Learning Rate = 0.5

Input data consists of four records, with attributesAgeandIncome

(values normalized)

Records with attribute values:

1 x11 = 0.8 x12 = 0.8 Older person with high income

2 x21 = 0.8 x22 = 0.1 Older person with low income

3 x31 = 0.2 x32 = 0.9 Younger person with high income

4 x41 = 0.1 x42 = 0.1 Younger person with low income


12/23

12

4. Example of a Kohonen Network Study(contd)Initial network weights (randomly assigned):

w11 = 0.9 w21 = 0.8 w12 = 0.9 w22 = 0.2

w13 = 0.1 w23 = 0.8 w14 = 0.1 w24 = 0.2

Figure shows network topology for example

Input layer = 2 nodes, and output layer = 4 nodes


13/23

13

4. Example of a Kohonen Network Study(contd)

Node 1 Node 2

Node 3 Node 4

w11

w12

w13

w14

w21

w22

w23

w24

Age Income

Input Layer

Output Layer


14/23

14

4. Example of a Kohonen Network Study(contd)First Record x1= (0.8, 0.8)

Competition Phase

Compute Euclidean Distance between input and weight vectors

92.0)8.02.0()8.01.0(),(:4

70.0)8.08.0()8.01.0(),(:3

61.0)8.02.0()8.09.0(),(:2

10.0)8.08.0()8.09.0(),(:1

22

14

22

13

22

12

22

11

xwDNode

xwDNode

xwDNode

xwDNode

The winning node is Node 1 (minimizes distance = 0.10)

Node 1 may exhibit affinity (cluster) for records of older persons

with high income


15/23

15

4. Example of a Kohonen Network Study(contd)First Record x1= (0.8, 0.8)

Cooperation Phase

Neighborhood Size R = 0

Therefore, nonexistent excitement of neighboring nodes

Only winning node receives weight adjustmentAdaptation Phase

Weights for Node 1 adjusted, where j = 1 (Node 1), n = 1 (First

record), and learning rate = 0.5:

8.0)8.08.0(5.08.0

)(5.0:

85.0)9.08.0(5.09.0

)(5.0:

,2112,21,21

,1111,11,11

CURRENTCURRENTNEW

CURRENTCURRENTNEW

wxwwIncome

wxwwAge


16/23

16

4. Example of a Kohonen Network Study(contd)Note direction of weight adjustments

Weights move toward input field values

Initial weight w11= 0.9, adjusted in direction of x11= 0.8

With learning rate = 0.5, w11moved half the distance from 0.9 to 0.8

Therefore, w11updated to 0.85

Node 1 becomes more proficient at capturing records of older, higher

income persons


17/23

17

4. Example of a Kohonen Network Study(contd)Second Record x2 = (0.8, 0.1)

Competition


71.0)1.02.0()8.01.0(),(:4

99.0)1.08.0()8.01.0(),(:3

14.0)1.02.0()8.09.0(),(:2

78.0)1.08.0()8.085.0(),(:1

22

24

22

23

22

22

22

21

xwDNode

xwDNode

xwDNode

xwDNode

Node 2 is the winning node with distance = 0.14

Node 2 weights (0.9, 0.2) most similar to input record values (0.8, 0.1)

Records of older persons and low income may cluster to Node2


18/23

18

4. Example of a Kohonen Network Study(contd)Adaptation

Weights for Node 2 adjusted, where j = 2 (Node 2), n = 2 (Second


15.0)2.01.0(5.02.0

)(5.0:

85.0)9.08.0(5.09.0

)(5.0:

,2222,22,22

,1221,12,12

CURRENTCURRENTNEW

CURRENTCURRENTNEW

wxwwIncome

wxwwAge

Again, weights move towards input field values

Initial w12= 0.9, adjusted to 0.85 (direction of x12= 0.8)

Initial w22= 0.2, adjusted to 0.15 (direction of x22= 0.1)

Node 2 develops affinity for records of older, lower income persons


19/23


20/23

20


Weights for Node 3 adjusted, where j = 3 (Node 3), n = 3 (Third



Initial w13= 0.1, adjusted to 0.15


Node 3 develops affinity for records of Younger person with high

income persons


21/23

21

4. Example of a Kohonen Network Study(contd)Second Record x4 = (0.1, 0.1)

Competition


71.0)1.02.0()8.01.0(),(:4

99.0)1.08.0()8.01.0(),(:3

14.0)1.02.0()8.09.0(),(:2

78.0)1.08.0()8.085.0(),(:1

22

24

22

23

22

22

22

21

xwDNode

xwDNode

xwDNode

xwDNode

Node 4 is the winning node with distance = 0.1

Node 4 weights (0.1, 0.2) most similar to input record values (0.1, 0.1)

Records of Younger person with low income may cluster to Node4


22/23

22


Weights for Node 3 adjusted, where j = 3 (Node 3), n = 3 (Third





Node 3 develops affinity for records of Younger person with high

income persons


23/23

23

4. Example of a Kohonen Network Study(contd)Similarly, records x3= (0.2, 0.9) and x4= (0.1, 0.1) are processedby network

Summary

Four output nodes represent distinct clusters

Simple example demonstrates network algorithm

Shows basic competition and Kohonen learning

Cluster Associated With Description

1 Node 1 Older person with high income

2 Node 2 Older person with low income

3 Node 3 Younger person with high income

4 Node 4 Younger person with low income

09dw Dm Kohonennetwork

Documents

Transcript of 09dw Dm Kohonennetwork