Brought to you by EarthWeb
IT Library Logo

Click Here!
Click Here!

Search the site:
 
EXPERT SEARCH -----
Programming Languages
Databases
Security
Web Services
Network Services
Middleware
Components
Operating Systems
User Interfaces
Groupware & Collaboration
Content Management
Productivity Applications
Hardware
Fun & Games

EarthWeb Direct EarthWeb Direct Fatbrain Auctions Support Source Answers

EarthWeb sites
Crossnodes
Datamation
Developer.com
DICE
EarthWeb.com
EarthWeb Direct
ERP Hub
Gamelan
GoCertify.com
HTMLGoodies
Intranet Journal
IT Knowledge
IT Library
JavaGoodies
JARS
JavaScripts.com
open source IT
RoadCoders
Y2K Info

Previous Table of Contents Next


6.2. CONCLUSION: CURRENT STATE AND OPEN PROBLEMS

Research and development in expert system verification and validation has emerged as a distinct field only in the last decade, as expert systems have become more prevalent in real-world applications.

A lot of V&V work is found in the literature as structural verification. Extensive research has been oriented especially toward rule-based systems, i.e., detecting errors and anomalies such as redundancy, dead-end rules, subsumption, auxiliary rules, circular rules, etc. This has led to construction of tools for automatic detection of these errors and anomalies, some of which have been described as verification and validation systems. These anomaly detection techniques can be useful in highlighting areas of the knowledge base that require attention.

In real applications, many attempts have been made to formulate an underlying model. For various reasons many of these models are not explicitly defined. As a consequence, the validity of the knowledge base depends on the modeling process in ways that are difficult to check. The presence of explicit models of expertise have a higher impact on V&V, as it introduces new complexities to the V&V process compared to the other approaches and compared to conventional programs. Working from an explicit model gives a much higher degree of confidence concerning the quality of the expert systems.

It is fairly clear from the exposition of V&V methods, techniques, and systems that there is no one technique or approach that is appropriate for handling all V&V aspects of a complete expert system (another lesson from V&V of conventional software). Therefore, it would be worthwhile to inquire how these techniques can interact and communicate with each other in a useful manner. At the same time, further research is needed to compare and contrast specific alternative expert system verification and validation paradigms.

The many approaches reported by both research and industry suggest a coupling of V&V techniques to all stages of the expert system development. However, the question at what time should V&V of an expert system be considered -- before it is constructed, while it is being constructed, or after it has been constructed -- is still open. There are numerous, convincing arguments for various design and V&V methods, although V&V of expert systems is as fraught with problems and pitfalls as V&V of any software system. Fortunately, the existing knowledge and experience acquired through many years of software engineering can be utilized. In this way, repeating the same errors as in conventional software development can at least be avoided, and the attention can be focused on problems that are not common in conventional software, problems that are specific to expert systems and which add new difficulties to the V&V activities.

Another landmark awaited in the maturing of V&V technology is its active adoption by the industry. Although many good theoretical techniques and methods are published in the literature, one cannot go directly from the published examples to more complex examples. Nowadays, systems tend to become more and more complex and therefore the abstract models that have been used so far need to be reconsidered. The idealizing assumptions made for the development of an expert system must deal with the complexity of the environment in which the expert system runs. To develop operational, dependable, and reliable systems, developers need to work harder to define the scope of the application, the limitation of the domain, the requirements, and to verify and validate rigorously those aspects of the system. It is only in the context of practical applications that the various V&V methods will reveal their true worth.

Finally, this chapter has presented a view of the multifaceted work that is carried out in the area of verification and validation expert systems. Much ground has been covered, starting from theoretical foundation of V&V, continuing with V&V systems developed by research and industry, and finally approaching the industrial needs for V&V of expert systems. The perspective put forward in this chapter is that V&V is one of the primary activities that requires addressing by software developers, independent V&V teams, and certification specialists. The current research initiatives and industrial needs provide some evidence to suggest that more work is needed to make the existent V&V methods better understood and to answer the difficult questions of the open problems.

REFERENCES

Coenen, F. and Bench-Capon, T. (1993). Maintenance of Knowledge-Based Systems,
Academic Press, London.
Lee, S. and O'Keefe, R.M. (1993). Subsumption anomalies in hybrid knowledge bases,
International Journal of Expert Systems: Research and Applications, 6(3), pp. 299-318.
Liebowitz, J. (1986). Useful approach for evaluating expert systems,
Expert Systems, 3(2), pp. 86-96.
Miller, L.A. (1990). Dynamic testing of knowledge bases using the heuristic testing approach,
Expert Systems With Applications, Vol. 1, pp. 249-269.
Preece, A.D. and Shinghal, R. (1994). Foundation and application of knowledge base verification,
International Journal of Intelligent Systems, Vol. 9, pp. 683-701.
Preece, A.D., Shinghal, R., and Batarekh, A. (1992). Principles and practice in verifying
rule-based systems, The Knowledge Engineering Review, Vol. 7:2, 115-141.
Vermesan, A.I. (1996). A definition of subsumption anomalies in conceptual models of
object-oriented KBSs. In Proc. of AAAI96 Workshop on Verification and Validation of Knowledge-Based Systems and Subsystems, Portland, OR.
Vermesan, A.I. and Bench-Capon, T. (1995). Techniques for the verification and validation of
knowledge-based systems: A survey based on the symbol/knowledge level distinction, In Software, Testing, Verification & Reliability, John Wiley & Sons, pp. 233-271.
Vermesan, A.I. and Vermesan, O. (1996). An object-oriented framework for NN-ES hybrid
systems, In Taylor, J.G. (Ed.), Neural Networks and Their Applications, John Wiley & Sons, chap. 15, pp. 205-227.
Vermesan, O. and Vermesan, A.I. (1995). The use of hybrid intelligent systems in
telecommunications, In Liebowitz, J. and Prerau, D. (Eds.), Worldwide Intelligent Systems, IOS Press, chap. 10, pp. 186-226.
Wielinga, B.J., Schreiber, A.T., and Breuker, J.A. (1992). KADS: A modelling approach to
knowledge engineering, Knowledge Acquisition (Special issue: The KADS approach to knowledge engineering), March, 4(1), pp. 5-53.


Previous Table of Contents Next

footer nav
Use of this site is subject certain Terms & Conditions.
Copyright (c) 1996-1999 EarthWeb, Inc.. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Please read our privacy policy for details.