EarthWeb   
HomeAccount InfoLoginSearchMy ITKnowledgeFAQSitemapContact Us
     

   
  All ITKnowledge
  Source Code

  Search Tips
  Advanced Search
   
  

  

[an error occurred while processing this directive]
Previous Table of Contents Next


6.7.2. Fuzzy clustering in revealing relationships within data

The concept of fuzzy sets is of interest in preprocessing of huge learning sets and reveal essential information that the neural network should be concentrating during its training. This allows reducing a training time, achieving stable learning and producing better generalization abilities of the neural network. We discuss two examples of the neurofuzzy synergy, namely a fuzzy perceptron (Keller and Hunt, 1985) and conditional clustering - a version of the clustering algorithm that is particularly suited when emphasizing relationships between variables. In both these cases the underlying principle is that fuzzy clustering organizes the data and produces meaningful information seeds around which the network is shaped up.

6.7.2.1. Fuzzy perceptron

The domain knowledge about learning is acquired through some preprocessing data of the training before running the learning scheme. This is the case in the construction known as a fuzzy perceptron (Keller and Hunt, 1985). In its original setting a single-layer perceptron, see Chapter 2, is composed of a series of linear processing units equipped with these threshold elements. The basic perception-based scheme of learning is straightforward. Let us start with a two-category multidimensional classification problem. If the considered patterns are linearly separable then there exists a linear discriminant function f such that

where x, wRn+1

After multiplying the class 2 patterns by -1 we obtain the system of inequalities.

k = 1, 2 … N. In other words, f(xk, w) = 0 defines a hyperplane partitioning the patterns; all class 1 patterns are located at the same side of this hyperplane. Assuming that xks are linearly separable. The perception algorithm (see below) guarantees that the discriminating hyperplane (vector w) is found in a finite number of steps.

the loop is repeated until no updates of w occur

The crux of the preprocessing phase as introduced in Keller and Hunt (1985) is to carry out clustering of data and determine the prototypes of the clusters as well as the class membership of the individual patterns. The ensuing membership values are used to monitor the changes Let uik and u2k be the membership grades of the k -th pattern. Definitely, u1k + u2k = 1. The outline of the learning algorithm is the same as before. The only difference is that the updates of the weights are governed by the expression

p >1. These modifications depend very much on the belongingness of the current pattern in the class. If u1k = u2k = 0.5 then the correction term is equal to zero and no update occurs. On the other hand if u1k =1 then the updates of the weights are the same as those encountered in the original perceptron.

6.7.2.2. Conditional (context-sensitive) clustering as a preprocessing phase in neural networks

Before getting into the aspect of context sensitive clustering, it is instructive to elaborate on the standard version of the well-known clustering method known in the literature as Fuzzy C - Means (FCM). Assume that x1, x2, …, xN are n-dimensional patterns defined in Rn. The objective function is defined as a sum of squared errors

with U = [uik] being a partition matrix, , and “c” standing for the number of the clusters; p >1 is often referred to as a fuzzification parameter. ||.|| forms a distance function being computed between xk and the prototype (vi) of the i-th cluster (group), denotes a family of c × N partition matrices, namely

Formally, the optimization problem becomes expressed in the form


minU, v1, v2, …, vc Q

subject to


The iterative scheme leading either to a local minimum or a saddle point of Q is well-known and concerns a series of updates of the partition matrix that are written down as

i=1, 2, …, c, k=1, 2, …, N.

The prototypes of the clusters are obtained in the form of weighted averages of the patterns

The conditioning aspect (context sensitivity) of the clustering mechanism is introduced into the algorithm by taking into consideration the conditioning variable (context) assuming the values f1, f2, …, fN on the corresponding patterns. More specifically, fk describes a level of involvement of xk in the assumed context, . In other words, acts as a data mining filter (or a focal element) by focusing attention on some specific subsets of data. More descriptively, we solve the clustering problem

  cluster data under the condition that the output data belong to a given context (viewed as a fuzzy set.).

The idea of this form of clustering in the setting of training neural networks is illustrated in Fig. 6.20.


Figure 6.20  Context sensitive clustering in the design framework of neural networks

The way in which fk can be associated with or allocated among the computed membership values of xk, say u1k, u2k, …, uck, is not unique. Two possibilities are worth exploring:

  we admit fk to be distributed additively across the entries of the k-th column of the partition matrix meaning that

k=1, 2, … N

  we may request that the maximum of the membership values within the corresponding column equals fk,

k=1, 2, …, N.

We confine ourselves to the first way of distribution of the conditioning variable. Bearing this in mind, let us modify the requirements to be met by the partition matrices and define the family


Previous Table of Contents Next

Copyright © CRC Press LLC

HomeAccount InfoSubscribeLoginSearchMy ITKnowledgeFAQSitemapContact Us
Products |  Contact Us |  About Us |  Privacy  |  Ad Info  |  Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Read EarthWeb's privacy statement.