EarthWeb   
HomeAccount InfoLoginSearchMy ITKnowledgeFAQSitemapContact Us
     

   
  All ITKnowledge
  Source Code

  Search Tips
  Advanced Search
   
  

  

[an error occurred while processing this directive]
Previous Table of Contents Next


2.3.1. Radial Basis function neural networks

This class of neural networks exhibits a diversity of processing units. The input layer plays an important role as an interface between the neural network and its environment. It is built of locally tuned receptive fields called radial basis functions (RBFs). In general, a RBF neural network (Bianchini et al., 1995; Chen et al., 1991; Hartman et al, 1990) is a feedforward architecture composed of two layers; the first one is built out of the receptive fields while the output layer is a collection of some linear processing units, Fig. 2.4. Thus the main nonlinear effect of neurocomputing resides within the nonlinear RBFs.


Figure 2.4  RBF neural network - a general topology

The detailed formulas governing the network are summarized below

  processing realized by the RBF layer

i=1, 2, …, h. K(.) is a positive radially symmetric function centered at vi and distributed according to the spread (width) defined by σi. Both x and vi are defined in Rn. The centers vi are also referred to as the prototypes of the receptive fields.

  processing performed at the output layer is linear with respect to the activation levels furnished by the RBF layer

j=1, 2, …, m. There could be a significant number of RBFs; especially we can consider Gaussian RBFs defined as

Does this type of nonlinear preprocessing delivered by the receptive fields support representation of the problem at hand? Definitely yes, if the RBF layer is properly designed. Some examples illustrate our claim and underline the importance of the receptive fields for the processing completed by the network.

Example 1. The small Exclusive - OR like data set, Fig.2.5, does call for a neural network with a single hidden layer to cope with patterns that are not linearly separable. On the other hand, the same problem can be solved by selecting four RBFs and completely giving up on the hidden layer. This is possible as the preprocessing layer has made the original data linearly separable.

On the other hand, the use of a series of RBFs distributed as visualized in Fig.2.6, may not be suitable and eventually deteriorate the learning process and result in a relatively poor performance of the overall network.


Figure 2.5  A small two-dimensional XOR-like classification problem


Figure 2.6  XOR-like problem with three RBFs whose distribution in x1 - X2 plane may lead to poor learning of the network

Example 2. The network with hidden layer(s) has only two inputs (x1, x2). The neural network with the preprocessing layer is reduced in depth but the size of the input layer is enlarged; the number of nodes there is equal to the number of the RBFs, Fig. 2.7. Each RBF is assigned to a separate cluster of data that belong to the same class.


Figure 2.7  Data set and resulting topology of the neural network and RBF neural network

Example 3. The classification situation as shown in Fig. 2.8 calls for the focal regions (RBFs) of different granularity - those of higher granularity occur at the regions where the decision (classification) boundary exhibits a highly nonlinear character.


Figure 2.8  RBFs of different granularity

2.4. Learning in neural networks

2.4.1. Neural networks as universal approximators

It is eventually the most fundamental finding about the nature of neural networks that states that they are universal approximators. It is beneficial to recall one of the formulations of this important fact (Cybenko, 1989).

Let f be a continuous sigmoidal -like function and F denote a continuous real-valued function from [0, 1]n (or any compact subset of Rn). Then for any positive constant ε there exists a collection of vectors w1, w2, …, wh, u ,and τ, such that

where

and w = [w1 w2 … wh], τ = [τ1 τ2 … τh].

The realization of the approximation of G, as portrayed in Fig. 2.9, calls for the three computational layers:

  an input layer where all input variables are summarized,
  a hidden layer of size “h” that builds up the nonlinear components of the approximation model,
  an output layer composed of a single linear neuron with the weight vector u


Figure 2.9  Approximation of F through a network with a single hidden layer; highlighted are the connections of the j-th node situated in the hidden layer

The approximation capability of the network means that any continuous F can be approximated to any required accuracy (we can make ε as small as required), so roughly speaking, the neural network can replace G, if necessary.

There are a number of other findings restating the same idea of approximation capabilities of the neural networks under slightly different assumptions or by making specific restrictions on the form of the network or nonlinear functions; the reader may refer to Hornik (1993) and Speecher(1993) for more details.

The approximation theorem is definitely an important existential result: it states that there exists a relevant neural network with these approximation abilities. It does not state, however, how to construct the network. Being important per se, the approximation theorem does not help design and train the neural network. For instance, it says nothing about the size of the hidden layer of the network (h) that is sufficient to approximate the function. This, in fact, promotes some alternative approaches (and alternative topologies of the networks) that can be used to handle the same approximation problem. Each of such implementations could be come with its own learning efficiency.1


1The situation is somewhat similar to that existing in pattern recognition. The Bayesian classifier is known to be optimal yet its design is not feasible; thus we usually confine ourselves to a whole family of suboptimal classifiers.


Previous Table of Contents Next

Copyright © CRC Press LLC

HomeAccount InfoSubscribeLoginSearchMy ITKnowledgeFAQSitemapContact Us
Products |  Contact Us |  About Us |  Privacy  |  Ad Info  |  Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Read EarthWeb's privacy statement.