Brought to you by EarthWeb
IT Library Logo

Click Here!
Click Here!

Search the site:
 
EXPERT SEARCH -----
Programming Languages
Databases
Security
Web Services
Network Services
Middleware
Components
Operating Systems
User Interfaces
Groupware & Collaboration
Content Management
Productivity Applications
Hardware
Fun & Games

EarthWeb Direct EarthWeb Direct Fatbrain Auctions Support Source Answers

EarthWeb sites
Crossnodes
Datamation
Developer.com
DICE
EarthWeb.com
EarthWeb Direct
ERP Hub
Gamelan
GoCertify.com
HTMLGoodies
Intranet Journal
IT Knowledge
IT Library
JavaGoodies
JARS
JavaScripts.com
open source IT
RoadCoders
Y2K Info

Previous Table of Contents Next


3.2.2. Method-Specific Techniques

Method-specific techniques are generally those that are more formal in nature. Formality means that the properties of the product or process are well defined and consequently easier to express in a structured and formal way. The method specific actions thus exploit the advantage of having the results of the software development amenable to such formal checks. These methods may offer from rigorous checks of the code functionality to mathematical proof of specification refinements. They are more focused on specific program features or problems than the non-method specific actions.

Empirical Methods. The literature on empirical testing of expert systems discusses a variety of methods. In general, empirical methods involve running a prototype system on a selection of test cases, and assessing the results.

Although many of the techniques used for testing conventional software can be used, expert systems pose a number of problems for validation. The number of paths through a large expert system makes exhaustive testing impossible. One solution to this problem has been to test specific aspects of the expert system such as the most frequently used paths, most critical elements in the knowledge base, the integration of the knowledge base with other parts of a more complex system.

The classical way is inspired by Turing's test: the responses of the expert system, together with those from a human expert, are presented to an independent human expert. Evaluation of the results from empirical methods can be done in an informal, qualitative manner, or by using quantitative methods.

These methods are in general "black-box validation techniques." As the name suggests, these techniques are used to determine the correctness of the system viewed as a "black box." Given some statements of user requirements, tests are applied to the system to establish whether or not it meets its requirements. The test cases are designed so as to exercise the system in a broad range of situations. Black-box tests are difficult to perform when there are many special-case situations that require special testing.

Logical Methods. These kind of methods are called "logical" due to the logical nature of the tests. Such methods are applied to detect anomalies such as inconsistency, redundancy, subsumption, circularity, unreachable goal, or unfireable rule.

These techniques are used to establish the internal correctness of the software, and therefore they are called "white-box" techniques. Glass-box techniques complement black-box techniques, revealing anomalies such as unreachable components, missing components, and incompatible or incorrect components. These techniques are difficult to apply when the expert system is internally very complex.

3.3. EXPERT SYSTEM DEVELOPMENT METHODOLOGIES

Many of the methods and techniques mentioned in the previous sections are product oriented, in the sense that the V&V activities are oriented mainly toward the knowledge base, and to some extent to the inference engine and user interface. The process-oriented approach to V&V is connected with the more sophisticated methodologies and environments for the construction of expert systems. Central to these more principled methodologies has been one of the fundamental premises of SE, which distinguishes clearly the functional specification of a system from its design and implementation. Instead of directly encoding the knowledge gathered from the human expert, the aim is to build an equivalent of the functional specification, i.e., a knowledge level description or a conceptual model. Consequently, the focus of verification and validation has been concentrated on these models and their refinements rather than on the executable representation.

Some methodologies and environments, such as MEKAS, MAKE, and KADS, are described in Vermesan and Bench-Capon (1995) and Wielinga et al. (1992). They are concerned with developing a knowledge level model and transforming it to the symbol level. A major contribution of these approaches is that the V&V techniques are not isolated from the overall expert system development life cycle, but are part of safe and well-defined methodologies, which also provide techniques for moving to an executable representation.

The MAKE (Maintenance Assistance for Knowledge Engineers), for instance, addresses the latter stage in expert system development, which focuses on the actual construction of the KB. Its prime focus was the construction of maintainable systems, in domains based on regulations that are particularly vulnerable to change. It is argued that there is a strong relation between maintenance and V&V: maintenance involves the detection of flaws in the knowledge base, and any changes that are made must themselves be validated and verified. MAKE prescribes a development methodology that produces systems that are more capable of verification, validation, and maintenance. To be effective, these activities should not be considered only when the system has been produced, but must inform the whole development process. Central to this philosophy is that V&V should always be carried out at a higher level than the level of executable code. The primary focus of V&V must be on the models, and the transition between them, rather than simply on the executable representation.


FIGURE 11 The waterfall expert system life-cycle paradigm.


Previous Table of Contents Next

footer nav
Use of this site is subject certain Terms & Conditions.
Copyright (c) 1996-1999 EarthWeb, Inc.. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Please read our privacy policy for details.