EarthWeb   
HomeAccount InfoLoginSearchMy ITKnowledgeFAQSitemapContact Us
     

   
  All ITKnowledge
  Source Code

  Search Tips
  Advanced Search
   
  

  

[an error occurred while processing this directive]
Previous Table of Contents Next


More specifically, A1(x) = G(x; 0, 0.02), A2(x) = G(x; 0.25, 0.02), A3(x) = G(x; 0.50, 0.02), A4(x) = G(x; 0.75, 0.02), and A5(x) = G(x; 1.00, 0.02).

First the experimental data are associated with the corresponding elements of the unit interval so that the performance index becomes minimized - these results are given in a tabular form.

Learning of the mapping has been completed using the standard gradient-based technique The confidence factors standing in the original data set are not involved in the learning procedure and all data are treated uniformly. The vector of the parameters of the neural network, param, to be adjusted concerns the connections between the hidden and output layers as well as the parameters of the sigmoid functions (spreads and modal values). As the learning was highly sensitive to the changes of the latter, the learning rate used in the training was kept at a low level - the experiments were completed for α = 0.0005. The values of Q for several sizes of the hidden layer are visualized in Fig. 6.16.


Figure 6.16  Performance index as a function of “n”

As is clearly visible, a significant improvement occurs at n = 5; subsequently this case is discussed in detail. To visualize the character of learning Fig. 6.17 shows the changes of Q as they occur in the first 400 learning epochs.


Figure 6.17  Performance index in successive learning epochs

The form of the nonlinear mapping produced by the network is illustrated in Fig. 6.18 - this is shown in comparison with the experimental data. Subsequently, Fig. 6.19 displays the membership functions resulting from the process of context adaptation - to ease a comparison the original membership functions defined in the unit interval are included as well, Fig. 6.19(ii).


Figure 6.18  Neural network approximation of experimental data


Figure 6.19  Membership functions after nonlinear calibration

6.7. Knowledge-based learning schemes

6.7.1. Metalearning and fuzzy sets

Even though guided by detailed gradient - based formulas, the learning of neural networks can be enhanced by making use of some domain knowledge acquired via intense experimentation (learning). By running a series of a mixture of successful and unsuccessful learning sessions one can gain a qualitative knowledge on how an efficient learning scenario should look.

In particular, some essential qualitative associations can be established by linking the performance of the learning process and the parameters of the scheme being utilized. Two detailed examples illustrate this point:

1.  The highly acclaimed backpropagation (BP) scheme used in the training of multilevel neural networks is based upon the gradient of the performance index (objective index) Q. The basic update formula reads as

where wij denotes a connection (weight) between the two neurons (i and j) and α is regarded as a learning rate, α ∈ (0, 1). Similarly describes a gradient of Q expressed with respect to wij. It is obvious that higher values of α result in more profound changes (updates) of the connection. Higher values of α could also result in faster learning that, unfortunately, comes at the expense of its stability (oscillations and overshoots of the values of Q). After a few learning sessions one can easily reveal some qualitative relationships that could be conveniently encapsulated in the form of “if - then” rules (Silva and Almeida, 1990).

The pertinent rules are included in Table 6.1. We may also refer to them as metalearning rules as they capture a way in which learning has to be completed.

Table 6.1 BP-oriented learning rules
ΔQ Δα
NB
NM
NS
Z
PS
PM
PB
PB
PM
PS
Z
NM
NB
NB

These rules are fairly monotonic (yet not symmetric) and fully comply with our intuitive observations. In general, any increase in Q calls for some decrease of α; when Q decreases, the increases in à are made more conservative. The linguistic terms in the corresponding rules are defined in the space (universe) of changes of Q (antecedents) and a certain subset of [0, 1] (conclusions). Similarly, the BP learning scheme can be augmented by taking into account a so-called momentum term; the primary intent of this expansion is to avoid eventual oscillations or reduce their amplitude in the values of the minimized performance index. This makes the learning more stable yet adds one extra adjustable learning parameter in the update rule. Similarly, we can propose more detailed control rules governing the changes of learning rate with respect to learning time and learning error; a sample of the rules is included below.

The learning metarules rules can also be formulated at the level of some parameters of the networks. The essence of the following approach is to modify activation functions of the neurons in the network. Consider the sigmoid nonlinearity (that is commonly encountered in many neural architectures).

We assume that the steepness factor of the sigmoid function (γ) is modifiable. As the changes of the connections are evidently affected by this component (γ), we can easily set up the following metarules:

  if performance index then γ

Summing up, two design issues should be underlined:

  the considered approach is fully experimental. The role of fuzzy sets is to represent and properly summarize the available domain knowledge.
  while the control protocol (rules) seems to be universal, the universes of discourse should be modified according to the current application (problem). In other words, the basic linguistic terms occurring therein need to be adjusted (calibrated). The realization of this phase calls for some additional computational effort and could somewhat offset the benefits originating from the availability of the domain knowledge.


Previous Table of Contents Next

Copyright © CRC Press LLC

HomeAccount InfoSubscribeLoginSearchMy ITKnowledgeFAQSitemapContact Us
Products |  Contact Us |  About Us |  Privacy  |  Ad Info  |  Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Read EarthWeb's privacy statement.