EarthWeb   
HomeAccount InfoLoginSearchMy ITKnowledgeFAQSitemapContact Us
     

   
  All ITKnowledge
  Source Code

  Search Tips
  Advanced Search
   
  

  

[an error occurred while processing this directive]
Previous Table of Contents Next


This form of encoding is self-evident. The strings could be quite long in the case of larger networks. For multilayer architectures, this encoding may also lead to some epistasis (meaning that changes of one portion of the string affect some other elements and result in the changes of the fitness function).

While the above encoding takes place in the connection space, there is another encoding that orients towards encoding some units (rather than connections). We refer to this form as encoding in the activation space. The scheme as discussed in Hassoun and Song (1993) is illustrated here with the use of a neural network with a single hidden layer (this method generalizes to multilayer structure; we will analyze the option later on along with additional enhancements). For the time being let us concentrate on a neural network with a single layer. The training (learning) set consists of input-output pairs x(k), target(k), k=1, 2, …, N. defined in Rn and [0, 1]m, respectively. The dimension of the hidden layer is equal to “m”. The activation levels of the neurons therein are denoted by z(1), z(2), … z(N) where z(k) results from the input set to x(k). Likewise, the responses of the output neurons are symbolized by y(k). As the network consists of a single hidden layer, the learning requires an error to backpropagate from the output layer. The number of connections makes the dimension of the learning problem high. The idea of encoding realized in the activation space works as follows. Arrange the vectors of activation of the hidden layer in the form

The population of these strings is used in the GA optimization. The fitness function computed for each Z concerns the mapping error of the network. Here the learning is easier since for a given Z there are two separate learning subproblems.

  learning input - hidden layer connections (W)
  learning hidden - output layer connections (V)

In both cases, the learning can be based on a standard delta rule. If the nonlinearities in the hidden layer are two-valued, then we end up with binary strings in the GA search space (being either in {0, 1} or {-1, 1}).

Extension to many-layered neural network is straightforward. The optimization converts into a number of independent delta-rule learning tasks with the strings concatenating the activation levels of all the neurons distributed across the hidden layers.

Let us compare the dimensions of the two GA search spaces; the one resulting from the connection coding and the other dealing with the activation levels of neurons. Again, we look into the network with a single hidden layer. The number of the connections (assuming full connectivity and encountering for the bias terms) is equal to

For the encoding of the activation levels we get

and the dimension of this string depends explicitly on the number of training patterns. To simplify the analysis, let us put n = h = m. Then the first encoding requires a string of the size n2 + n2 + n + n = 2n (n + 1) while in the second option we get n N. To retain the same dimension of the search space (that does not necessarily imply that the optimization tasks are of the same level of complexity) we require that

or

Now if the size of the training set (N) is quite high, the second encoding scheme may result in an excessively large search space. The problem can be eliminated when subscribing to two-phase learning, Fig. 8.24. Here the GA learning develops a genotype with the aid of a meaningful subset of the training set whose size N′ is far smaller than the original data set, N′ << N. The phenotype is then refined via a standard BP method utilizing all training data.


Figure 8.24  Two-level learning strategy in neural networks

Let us underline that the design of neural networks, as commonly encountered in the existing literature, is a single-phase process. All training data are directly used to construct their nonlinear neural network mapping, Fig. 8.24(i). This gives rise to many undesired phenomena: a long learning time, memorization effect (poor generalization), and a lack of robustness. The two-level learning follows the concept of a hybrid learning including GAs and gradient-based methods. The GA attempts to build a skeleton (blueprint) of a neural network and thus has to rely on the most representative patterns in the training set. The evolutionary phase is followed by the ensuing detailed learning of the phenotype (Lamarckian evolution); the latter being performed by a high resolution gradient-based optimization, Fig. 8.24(ii). The formation of a reduced set of the training data can be done in many ways including specialized clustering methods.

8.8.2. Fuzzy genetic optimization of neural networks

A parametric optimization of neural networks serves as a useful case study of the proposed methodology. For the time being let us confine ourselves to feed forward neural networks.

The connections of the neurons are mapped on the linguistic space of term defined in [-a, a] (the selection of the range itself can be done depending upon the problem at hand). The entire network is coded as a string of bits; as the number of lines is not high, this makes the strings short. The evaluation of the strings is done based upon the values of the fitness function that could be inversely proportional to the mapping error of the network.

Obviously for each string we evaluate fitness with the aid of the sampling method. The result of the GA optimization is a neural network with fuzzy connections; Fig. 8.25.


Figure 8.25  Neural network with nonnumeric connections

If interested in a semi-quantification format of the network one can stop optimization of the neural network at this point. The network has a high descriptive power. Since the connections are fuzzy sets (more precisely, fuzzy numbers) the output of the network becomes a fuzzy number. The calculations of the output follow the extension principle. The neural network in Fig. 8.25 is also a semi-numeric skeleton of a family of numeric neural network.


Previous Table of Contents Next

Copyright © CRC Press LLC

HomeAccount InfoSubscribeLoginSearchMy ITKnowledgeFAQSitemapContact Us
Products |  Contact Us |  About Us |  Privacy  |  Ad Info  |  Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Read EarthWeb's privacy statement.