Brought to you by EarthWeb
IT Library Logo

Click Here!
Click Here!

Search the site:
 
EXPERT SEARCH -----
Programming Languages
Databases
Security
Web Services
Network Services
Middleware
Components
Operating Systems
User Interfaces
Groupware & Collaboration
Content Management
Productivity Applications
Hardware
Fun & Games

EarthWeb Direct EarthWeb Direct Fatbrain Auctions Support Source Answers

EarthWeb sites
Crossnodes
Datamation
Developer.com
DICE
EarthWeb.com
EarthWeb Direct
ERP Hub
Gamelan
GoCertify.com
HTMLGoodies
Intranet Journal
IT Knowledge
IT Library
JavaGoodies
JARS
JavaScripts.com
open source IT
RoadCoders
Y2K Info

Previous Table of Contents Next


4.4. HOW MUCH V&V?

A general comment on V&V is that it is poorly understood and generally disliked. Adequate V&V is also expensive. This is not less true when applied to expert systems. Traditionally, it is one of the first activities to be cut when the project costs get tight. The need for V&V, especially for the benefits of reduced expert system maintenance, is seldom understood by management. Nevertheless, there is considerable agreement that V&V will always show a positive cost-avoidance benefit over the life of a system. Careful V&V will do so in some cases even when assessed just for the development stage.

How much V&V? is then a natural question within industrial applications. Nonfunctional features are relatively easy to define, and there are methods that can be applied straightforwardly to check them. However, conformity to the functional part of the specification is another matter. It is generally agreed that we cannot prove (guarantee) by testing that a program conforms to its specification. Therefore, it is important to define what level of checking will be sufficient for the specified purposes. However, there is no simple answer to the question "How much V&V should we do to ensure the quality of the system/subsystem/module/procedure?," as several factors should be taken into consideration.

Of particular concern from the point of view of V&V is the system complexity. The complexity of the system is defined by several characteristics which make the system harder to develop and analyze. Generally, the higher the complexity, the greater the opportunity for errors and the greater the need for V&V. However, there is another factor that needs to be considered in determining the extent of estimated V&V required: system integrity. System integrity refers to the joint capability of a system to operate for long periods without failures, to fail gracefully with reasonable warnings, to be able to recover rapidly without much difficulty, and to avoid causing expensive damage to property or harm to people or the environment. How much integrity is required of a system will be a function of several factors. Thus, a highly complex system with a low degree of required integrity should probably not need as much V&V as a highly complex system with a very high degree of required system integrity.

If we translate the system complexity and integrity requirements into quality requirements, then the above-mentioned question should be answered in connection with another one, i.e.: "How critical is the quality of the system/subsystem/ module/ procedure?"

5. RESEARCH INITIATIVES

In Europe, many of the V&V projects have taken place under the European Strategic Programme for Research and Development in Information Technology. Some of the completed projects are KADS, VALID, VITAL, and VIVA. An ongoing project, demonstrating the feasibility of extending the expert system technology to safety critical systems, is Safe-KBS.

In U.S., the majority of research on V&V involves the development of tools to support automatic code-checking. NASA and its contractors have been involved in a number of verification and validation projects.

6. FUTURE TRENDS AND SUMMARY

6.1. HYBRID INTELLIGENT SYSTEMS

Hybrid intelligent systems are systems that combine the advantages offered by expert systems, neural networks, and fuzzy systems. The need for such systems has emerged due to the fact that we are at a stage of development in which further increase in performance of existing tools requires the use of intelligent tools offered by these systems. Until now, their use was limited to stand-alone architectures; today the intelligent hybrid systems are the emerging technology that could take advantage of the best of each technology's features. This synergistic approach recognizes both similarities and differences between these systems, and suggests that for many particular applications the resultant hybrid system could reflect the best aspects of each component of the system (Vermesan and Vermesan, 1996).

One landmark awaited in the maturing of hybrid system technology is verification and validation. Clearly, there is little benefit in employing such a complex system unless it can be trusted to perform its function. Verification and validation of expert systems is an established discipline that has accumulated valuable experience over recent years. There are now many verification and validation techniques, plus considerable experience and expertise in using them. The natural question is whether these techniques can be expanded and adapted to cope with hybrid systems, or new ones are needed.

In the case of rule-based systems, a difficult process is to guarantee that the addition, deletion, or modification of rules does not leave the system in a state of chaos. However, in the case of a hybrid system, e.g., neural network expert system, the process may not be that difficult due to the fact that automated tools can help. The most important automated tool is the learning algorithm itself. It can assure that by adding, deleting, or modifying training examples, the knowledge base remains consistent with the set of training examples. Two validation procedures with extracted rules and encoded rules are presented in Vermesan and Vermesan (1995).

Furthermore, one would expect to be able to use the good techniques from V&V of rule-based systems to fuzzy rules. It might appear that a fuzzy system can be verified and validated more easily than conventional rules, because a fuzzy system uses vague terms to explain the control actions, and would therefore be easier to understand and cope with. In reality, the verification and validation problems may be more difficult. As everything in fuzzy logic is a matter a degree, the consistency model valid for expert systems does not work in the case of fuzzy rules. The concept of consistency must be refined into a notion of degree of consistency. Moreover, verifying and validating a hybrid system is not simply a problem of verifying and validating each component separately. An integrated view is needed. Therefore, building models of the hybrid system and expressing them in a formal or less formal way may be a good practice. Such models can always provide an objective measure for the executable system.


Previous Table of Contents Next

footer nav
Use of this site is subject certain Terms & Conditions.
Copyright (c) 1996-1999 EarthWeb, Inc.. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Please read our privacy policy for details.