Brought to you by EarthWeb
IT Library Logo

Click Here!
Click Here!

Search the site:
 
EXPERT SEARCH -----
Programming Languages
Databases
Security
Web Services
Network Services
Middleware
Components
Operating Systems
User Interfaces
Groupware & Collaboration
Content Management
Productivity Applications
Hardware
Fun & Games

EarthWeb Direct EarthWeb Direct Fatbrain Auctions Support Source Answers

EarthWeb sites
Crossnodes
Datamation
Developer.com
DICE
EarthWeb.com
EarthWeb Direct
ERP Hub
Gamelan
GoCertify.com
HTMLGoodies
Intranet Journal
IT Knowledge
IT Library
JavaGoodies
JARS
JavaScripts.com
open source IT
RoadCoders
Y2K Info

Previous Table of Contents Next


2.4. MODERN TIMES

To conclude this short picture of the developments of NLP techniques in the 1960s and the 1970s, we will mention here the beginnings of a field that is particularly important for setting up more comfortable, natural, and efficient interfaces between people and computer systems, i.e., the field of the "speech understanding systems." First work in this area, in the 1960s, concerned some successes in the set up of isolated-word recognition systems, using the technique of (1) comparing the incoming speech signal with the internal representation of the acoustical pattern of each word pertaining to a relatively small vocabulary, and (2), selecting the best match by using some sort of distance metrics. Systems like these could not, however, deal with the problem of connected speech signals, where the utterances contain whole phrases or sentences: these signals have, in fact, very few characteristics in common with the simple concatenation of the signals of the single words. This is due mainly to the fact that the pronunciation of the different words changes when these words are associated to form a sentence. Final syllables are often altered in the junction, and sometimes dropped; finding the boundaries between words in connected speech is, in reality, one of the most difficult aspects of the whole problem.

Confronted with this situation, the Advanced Research Projects Agency of the U.S. Department of Defense (ARPA) promoted, starting in 1971, a 5-year program in speech understanding research. Several projects were therefore financed within this framework: at Bolt, Beranek, and Newman, Inc. (BBN), SRI International, etc. At Carnegie-Mellon University, the systems developed within the ARPA framework were HEARSAY-I and DRAGON in the first phase of the program (1971-1973), HARPY, and HEARSAY-II by 1976. The HEARSAY systems are particularly well known, given that they were the first AI systems using the "blackboard" architecture. This last one is based on the idea of making use of independent knowledge sources that cooperatively solve a problem by posting hypotheses on a global data structure (blackboard) -- in the HEARSAY projects, the knowledge sources concerned acoustics, phonetics, syntax, and semantics. This architecture has been applied in several domains besides speech, and still constitutes one of the most powerful tools used by the AI community to set up complex, interconnected systems. The ARPA program ended in September 1976; the best performances (an error rate of 5% for connected speech, taking into account a limited vocabulary of 1000 words) were attained by the HARPY program, very efficient thanks to the large utilization of precompiled knowledge, but very difficult to modify.

In the 1980s, there were no real novelties to mention in our field, excepting the rise of a new wave of pessimism about the future of (industrial) NLP. This was produced by many concomitant factors, e.g., the difficulties of transforming the Schankian prototypes into industrial products (see, however, the relative success of ATRANS, a system capable of dealing correctly with bank telex in natural language), the disappointments with the expectations arising from the NL interfaces to DBs (the 1985 OVUM report on NLP spoke of an "exponential growth" of the sales of industrial products in this field for the beginning of the 1990s, a largely unrealized prevision), more in general, the morose climate generated by the "AI Winter," etc. In 1989, the Financial Times spoke about the NLP commercial systems as "not yet robust enough," characterized by a modest coverage, etc. NLP people were invited to abandon the idea of building up big, complex systems, and to concentrate instead on linguistic theory and on the theoretical and formal study of accurately stripped-down, toy problems.

Fortunately enough, a mini-revolution took place at the end of the 1980s. This event had been prepared by the considerable publicity stirred up about some successful programs for extracting information from NL texts created in an industrial environment -- see the GE's SCISOR system already mentioned in the previous subsection. SCISOR -- in some sort, an outgrowth of De Jong's FRUMP, see before -- was characterized by the presence of a linguistic analysis that (1) was, in some way, inspired by the AI techniques for text understanding promoted by the Yale school, but that (2) included a real syntactic phase, and, mainly, (3) was "shallow," i.e., it performed only a surface analysis of the texts without trying to attain a "deep understanding" of these texts. SCISOR is, therefore, one of the first examples of a trend that is now particularly popular in the NLP domain, i.e., that of building up systems that combine the use of AI techniques with that of robust but shallow methods of textual analysis. This trend -- leading to systems that, like SCISOR, do not really try to understand the "meaning" of the texts analyzed -- combines well with the resurgence of the interest for the statistical methods, triggered in this case mainly by the recent success of the modern methods for speech understanding, based only on pure stochastic techniques (see also subsection 3.3 and Section 4).

The success of this new type of NLP philosophy is also deeply indebted to some recent, ARPA-sponsored initiatives, such as the MUC (Message Understanding) and TREC (Text Retrieval) Conferences, or the Tipster Text Program, which have powerfully contributed to the popularity of above "shallow but robust" approach. We will conclude this (biased and incomplete) history of NLP saying few words about such initiatives -- please note that, even if they address mainly the problem of the extraction of information from texts, their impact on the aspect of the whole field has been considerable; moreover, these sorts of techniques can also be useful in a strict expert systems context, e.g., to help to set up large knowledge bases.

The MUC conferences (see also Cowie and Lehnert, 1996) are, in reality, competitions. Very succinctly, the contestant systems -- previously developed and tested on training data -- must read news agency-like messages and then perform an information extraction task. This task consists of automatically filling predefined, object-oriented data structures called "templates" with information extracted from the news stories; templates correspond to a fixed-format formulation of the canonical who, what, where, when, and why questions about the characteristics of a specific event in a particular domain. The templates and their slots are then scored by using an automatic scoring program with analyst-produced templates as the "keys." The Fifth Message Understanding Conference (MUC-5), in August 1993, focused, e.g., on information extraction from news stories concerning two different domains, joint ventures (JV) and microelectronics (ME). For each application, the extraction task could be performed in either English and/or Japanese, giving rise, therefore, to four combinations: English Joint Ventures, Japanese Joint Ventures, English Microelectronics, and Japanese Microelectronics. Some very general, interesting conclusions that proceed from the global, semi-automatic scoring of the results are: (1) the most frequent type of error committed by the MUC-5 systems was to miss pertinent information; (2) summary scores for error per response fill indicate a better performance in Japanese than in English for both domains (this may be explained by a linguistic structure that is less varied in the Japanese than in the English documents, and by the fact that domain concepts are expressed, in the Japanese texts, in a more standardized way); (3) summary scores indicate a slightly better performance in the microelectronics domain than in the joint ventures domain for both languages, English and Japanese; (4) a study conducted on the English microelectronics task indicates that the best machine system performs with an error rate of about 62%, a little less than twice the error rate of a highly skilled human analyst (33%). However, very good performances (34%) have been attained by the "best" system (the SHOGUN system, jointly developed by General Electric and Carnegie Mellon University) on some limited tasks, such as, e.g., extracting information in the joint ventures domains with only the core portion of the templates being scored. These results, even if they certainly show that machine performance is at least comparable with human performance, should however tone down any over-enthusiastic prediction about their appearance on the market as commercial products.


Previous Table of Contents Next

footer nav
Use of this site is subject certain Terms & Conditions.
Copyright (c) 1996-1999 EarthWeb, Inc.. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of EarthWeb is prohibited. Please read our privacy policy for details.