Method and System for Computing Semantic Logical Forms From Syntax Trees

The present invention relates to the field of natural language processing ("NLP"), and more particularly, to a method and system for generating a logical form graph from a syntax tree.

Computer systems for automatic natural language processing use a variety of subsystems, roughly corresponding to the linguistic fields of morphological, syntactic, and semantic analysis to analyze input text and achieve a level of machine understanding of natural language. Having understood the input text to some level, a computer system can, for example, suggest grammatical and stylistic changes to the input text, answer questions posed in the input text, or effectively store information represented by the input text.

Morphological analysis identifies input words and provides information for each word that a human speaker of the natural language could determine by using a dictionary. Such information might include the syntactic roles that a word can play (e.g., noun or verb) and ways that the word can be modified by adding prefixes or suffixes to generate different, related words. For example, in addition to the word "fish," the dictionary might also list a variety of words related to, and derived from, the word "fish," including "fishes," "fished," "fishing," "fisher," "fisherman," "fishable," "fishability," "fishbowl," "fisherwoman," "fishery," "fishhook," "fishnet," and "fishy."

Syntactic analysis analyzes each input sentence, using, as a starting point, the information provided by the morphological analysis of input words and the set of syntax rules that define the grammar of the language in which the input sentence was written. The following are sample syntax rules:

sentence=noun phrase+verb phrase

noun phrase=adjective+noun

verb phrase=adverb+verb Syntactic analysis attempts to find an ordered subset of syntax rules that, when applied to the words of the input sentence, combine groups of words into phrases, and then combine phrases into a complete sentence. For example, consider the input sentence: "Big dogs fiercely bite." Using the three simple rules listed above, syntactic analysis would identify the words "Big" and "dogs" as an adjective and noun, respectively, and apply the second rule to generate the noun phrase "Big dogs." Syntactic analysis would identify the words "fiercely" and "bite" as an adverb and verb, respectively, and apply the third rule to generate the verb phrase "fiercely bite." Finally, syntactic analysis would apply the first rule to form a complete sentence from the previously generated noun phrase and verb phrase. The result of syntactic analysis, often represented as an acyclic downward branching tree with nodes representing input words, punctuation symbols, phrases, and a root node representing an entire sentence, is called a parse.

Some sentences, however, can have several different parses. A classic example sentence for such multiple parses is: "Time flies lie an arrow." There are at least three possible parses corresponding to three possible meanings of this sentence. In the first parse, "time" is the subject of the sentence, "flies" is the verb, and "like an arrow" is a prepositional phrase modifying the verb "flies." However, there are at least two unexpected parses as well. In the second parse, "time" is an adjective modifying "flies," "like" is the verb, and "an arrow" is the object of the verb. This parse corresponds to the meaning that flies of a certain type, "time flies," like or are attracted to an arrow. In the third parse, "time" is an imperative verb, "flies" is the object, and "like an arrow" is a prepositional phrase modifying "time." This parse corresponds to a command to time flies as one would time an arrow, perhaps with a stopwatch.

Syntactic analysis is often accomplished by constructing one or more hierarchical trees called syntax parse trees. Each leaf node of the syntax parse tree generally represents one word or punctuation symbol of the input sentence. The application of a syntax rule generates an intermediate-level node linked from below to one, two, or occasionally more existing nodes. The existing nodes initially comprise only leaf nodes, but, as syntactic analysis applies syntax rules, the existing nodes comprise both leaf nodes as well as intermediate-level nodes. A single root node of a complete syntax parse tree represents an entire sentence.

Semantic analysis generates a logical form graph that describes the meaning of input text in a deeper way than can be described by a syntax parse tree alone. The logical form graph is a first attempt to understand the input text at a level analogous to that achieved by a human speaker of the language.

The logical form graph has nodes and links, but, unlike the syntax parse tree described above, is not hierarchically ordered. The links of the logical form graph are labeled to indicate the relationship between a pair of nodes. For example, semantic analysis may identify a certain noun in a sentence as the deep subject or deep object of a verb. The deep subject of a verb is the doer of the action and the deep object of a verb is the object of the action specified by the verb. The deep subject of an active voice verb may be the syntactic subject of the sentence, and the deep object of an active voice verb may be the syntactic object of the verb. However, the deep subject of a passive voice verb may be expressed in an agentive-by phrase, and the deep object of a passive voice verb may be the syntactic subject of the sentence. For example, consider the two sentences: (1) "Dogs bite people" and (2) "People are bitten by dogs." The first sentence has an active voice verb, and the second sentence has a passive voice verb. The syntactic subject of the first sentence is "Dogs" and the syntactic object of the verb "bite" is "people." By contrast, the syntactic subject of the second sentence is "People" and the verb phrase "are bitten" is modified by the agentive-by phrase "by dogs." For both sentences, "dogs" is the deep subject, and "people" is the deep object of the verb or verb phrase of the sentence. Although the syntax parse trees generated by syntactic analysis for sentences 1 and 2, above, will be different, the logical form graphs generated by semantic analysis will be the same, because the underlying meaning of the two sentences is the same.

Further semantic processing after generation of the logical form graph may draw on knowledge databases to relate analyzed text to real world concepts in order to achieve still deeper levels of understanding. An example knowledge base would be an on-line encyclopedia, from which more elaborate definitions and contextual information for particular words can be obtained.

In the following, the three NLP subsystems--morphological, syntactic, and semantic--are described in the context of processing the sample input text: "The person whom I met was my friend." FIG. 1 is a block diagram illustrating the flow of information between the NLP subsystems. The morphological subsystem 101 receives the input text and outputs an identification of the words and senses for each of the various parts of speech in which each word can be used. The syntactic subsystem 102 receives this information and generates a syntax parse tree by applying syntax rules. The semantic subsystem 103 receives the syntax parse tree and generates a logical form graph.

FIGS. 2-5 display the dictionary information stored on an electronic storage medium that is retrieved for the input words of the sample input text during morphological analysis. FIG. 2 displays the dictionary entries for the input words "the" 201 and "person" 202. Entry 201 comprises the key "the" 203 and a list of attribute/value pairs. The first attribute "Adj" 204 has, as its value, the symbols contained within the braces 205 and 206. These symbols comprise two further attribute/value pairs: (1) "Lemma"/"the" and (2) "Bits"/"Sing Plur Wa6 Det Art B0 Def." A lemma is the basic, uninflected form of a word. The attribute "Lemma" therefore indicates that "the" is the basic, uninflected form of the word represented by this entry in the dictionary. The attribute "Bits" comprises a set of abbreviations representing certain morphological and syntactic information about a word. This information indicates that "the" is: (1) singular; (2) plural; (3) not inflectable; (4) a determiner; (5) an article; (6) an ordinary adjective; and (7) definite. Attribute 204 indicates that the word "the" can serve as an adjective. Attribute 212 indicates that the word "the" can serve as an adverb. Attribute "Senses" 207 represents the various meanings of the word as separate definitions and examples, a portion of which are included in the list of attribute/value pairs between braces 208-209 and between braces 210-211. Additional meanings actually contained in the entry for "the" have been omitted in FIG. 2, indicated by the parenthesized expression "(more sense records)" 213.

In the first step of natural language processing, the morphological subsystem recognizes each word and punctuation symbol of the input text as a separate token and constructs an attribute/value record for each part of speech of each token using the dictionary information. Attributes are fields within the records that can have one of various values defined for the particular attribute. These attribute/value records are then passed to the syntactic subsystem for further processing, where they are used as the leaf nodes of the syntax parse tree that the syntactic subsystem constructs. All of the nodes of the syntax parse tree and the logical form graph constructed by subsequent NLP subsystems are attribute/value records.

The syntactic subsystem applies syntax rules to the leaf nodes passed to the syntactic subsystem from the morphological subsystem to construct higher-level nodes of a possible syntax parse tree that represents the sample input text. A complete syntax parse tree includes a root node, intermediate-level nodes, and leaf nodes. The root node represents the syntactic construct (e.g., declarative sentence) for the sample input text. The intermediate-level nodes represent intermediate syntactic constructs (e.g., verb, noun, or prepositional phrases). The leaf nodes represent the initial set of attribute/value records.

In some NLP systems, syntax rules are applied in a top-down manner. The syntactic subsystem of the NLP system herein described applies syntax rules to the leaf nodes in a bottom-up manner. That is, the syntactic subsystem attempts to apply syntax rules one-at-a-time to single leaf nodes to pairs of leaf nodes, and, occasionally, to larger groups of leaf nodes. If the syntactic rule requires two leaf nodes upon which to operate, and a pair of leaf nodes both contain attributes that match the requirements specified in the rule, then the rule is applied to them to create a higher-level syntactic construct. For example, the words "my friend" could represent an adjective and a noun, respectively, which can be combined into the higher-level syntactic construct of a noun phrase. A syntax rule corresponding to the grammar rule, "noun phrase=adjective+noun," would create an intennediate-level noun phrase node, and link the two leaf nodes representing "my" and "friend" to the newly created intermediate-level node. As each new intermediate-level node is created, it is linked to already-existing leaf nodes and intermediate-level nodes, and becomes part of the total set of nodes to which the syntax rules are applied. The process of applying syntax rules to the growing set of nodes continues until either a complete syntax parse tree is generated or until no more syntax rules can be applied. A complete syntax parse tree includes all of the words of the input sentence as leaf nodes and represents one possible parse of the sentence.

This bottom-up method of syntax parsing creates many intermediate-level nodes and sub-trees that may never be included in a final, complete syntax parse tree. Moreover, this method of parsing can simultaneously generate more than one complete syntax parse tree.

The syntactic subsystem can conduct an exhaustive search for all possible complete syntax parse trees by continuously applying the rules until no additional rules can be applied. The syntactic subsystem can also try various heuristic approaches to first generate the most probable nodes. After one or a few complete syntax parse trees are generated, the syntactic subsystem typically can terminate the search because the syntax parse tree most likely to be chosen as best representing the input sentence is probably one of the first generated syntax parse trees. If no complete syntax parse trees are generated after a reasonable search, then a fitted parse can be achieved by combining the most promising sub-trees together into a single tree using a root node that is generated by the application of a special aggregation rule.

FIG. 6 illustrates the initial leaf nodes created by the syntactic subsystem for the dictionary entries initially displayed in FIGS. 2-5. The leaf nodes include two special nodes, 601 and 614, that represent the beginning of the sentence and the period terminating the sentence, respectively. Each of the nodes 602-613 represent a single part of speech that an input word can represent in a sentence. These parts of speech are found as attribute/value pairs in the dictionary entries. For example, leaf nodes 602 and 603 represent the two possible parts of speech for the word "The," that are found as attributes 204 and 212 in FIG. 2.

FIG. 7-22 show the rule-by-rule construction of the final syntax parse tree by the syntactic subsystem. Each of the figures illustrates the application of a single syntax rule to generate an intermediate-level node that represents a syntactic structure. Only the rules that produce the intermediate-level nodes that comprise the final syntax tree are illustrated. The syntactic subsystem generates many intermediate-level nodes which do not end up included in the final syntax parse tree.

In FIGS. 7-14, the syntactic subsystem applies unary syntax rules that create intermediate-level nodes that represent simple verb, noun, and adjective phrases. Starting with FIG. 15, the syntactic subsystem begins to apply binary syntax rules that combine simple verb, noun, and adjective phrases into multiple-word syntactic constructs. The syntactic subsystem orders the rules by their likelihood of successful application, and then attempts to apply them one-by-one until it finds a rule that can be successfully applied to the existing nodes. For example, as shown in FIG. 15, the syntactic subsystem successfully applies a rule that creates a node representing a noun phrase from an adjective phrase and a noun phrase. The rule specifies the characteristics required of the adjective and noun phrases. In this example, the adjective phrase must be a determiner. By following the pointer from node 1501 back to node 1503, and then accessing morphological information included in node 1503, the syntactic subsystem determines that node 1501 does represent a determiner. Having located the two nodes 1501 and 1502 that meet the characteristics required by the rule, the syntactic subsystem then applies the rule to create from the two simple phrases 1501 and 1502 an intermediate-level node that represents the noun phrase "my friend." In FIG. 22, the syntactic subsystem generates the final, complete syntax parse tree representing the input sentence by applying a trinary rule that combines the special Begin1 leaf node 2201, the verb phrase "The person whom I met was my friend" 2202, and the leaf node 2203 that represents the final terminating period to form node 2204 representing the declarative sentence.

The semantic subsystem generates a logical form graph from a complete syntax parse tree. In some NLP systems, the logical form graph is constructed from the nodes of a syntax parse tree, adding to them attributes and new bi-directional links. The logical form graph is a labeled, directed graph. It is a semantic representation of an input sentence. The information obtained for each word by the morphological subsystem is still available through references to the leaf nodes of the syntax parse tree from within nodes of the logical form graph. Both the directions and labels of the links of the logical form graph represent semantic information, including the functional roles for the nodes of the logical form graph. During its analysis, the semantic subsystem adds links and nodes to represent (1) omitted, but implied, words; (2) missing or unclear arguments and adjuncts for verb phrases; and (3) the objects to which prepositional phrases refer.

FIG. 23 illustrates the complete logical form graph generated by the semantic subsystem for the example input sentence. Meaningful labels have been assigned to links 2301-2306 by the semantic subsystem as a product of the successful application of semantic rules. The six nodes 2307-2312, along with the links between them, represent the essential components of the semantic meaning of the sentence. In general, the logical form nodes roughly correspond to input words, but certain words that are unnecessary for conveying semantic meaning, such as "The" and "whom" do not appear in the logical form graph, and the input verbs "met" and "was" appear as their infinitive forms "meet" and "be." The nodes are represented in the computer system as records, and contain additional information not shown in FIG. 23. The fact that the verbs were input in singular past tense form is indicated by additional information within the logical form nodes corresponding to the meaning of the verbs, 2307 and 2310.

The differences between the syntax parse tree and the logical form graph are readily apparent from a comparison of FIG. 23 to FIG. 22. The syntax parse tree displayed in FIG. 22 includes 10 leaf nodes and 16 intermediate-level nodes linked together in a strict hierarchy, whereas the logical form graph displayed in FIG. 23 contains only 6 nodes. Unlike the syntax parse tree, the logical form graph is not hierarchically ordered, obvious from the two links having opposite directions between nodes 2307 and 2308. In addition, as noted above, the nodes no longer represent the exact form of the input words, but instead represent their meanings.

Further natural language processing steps occur after semantic analysis. They involve combining the logical form graph with additional information obtained from knowledge bases, analyzing groups of sentences, and generally attempting to assemble around each logical form graph a rich contextual environment approximating that in which humans process natural language.

Prior art methods for generating logical form graphs involve computationally complex adjustments to, and manipulations of the syntax parse tree. As a result, it becomes increasingly difficult to add new semantic rules to a NLP system. Addition of a new rule involves new procedural logic that may conflict with the procedural logic already programmed into the semantic subsystem. Furthermore, because nodes of the syntax parse tree are extended and reused as nodes for the logical form graph, prior art semantic subsystems produce large, cumbersome, and complicated data structures. The size and complexity of a logical form graph overlayed onto a syntax parse tree makes further use of the combined data structure error-prone and inefficient. Accordingly, it would be desirable to have a more easily extended and manageable semantic subsystem so that simple logical form graph data structures can be produced.

The present invention is directed to a method and system for performing semantic analysis of an input sentence within a NLP system. The semantic analysis subsystem receives a syntax parse tree generated by the morphological and syntactic subsystems. The semantic analysis subsystem applies two sets of semantic rules to make adjustments to the received syntax parse tree. The semantic analysis subsystem then applies a third set of semantic rules to create a skeletal logical form graph from the syntax parse tree. The semantic analysis subsystem finally applies two additional sets of semantic rules to the skeletal logical form graph to provide semantically meaningful labels for the links of the logical form graph, to create additional logical form graph nodes for missing nodes, and to unify redundant logical form graph nodes. The final logical form graph generated by the semantic analysis subsystem represents the complete semantic analysis of an input sentence.

FIG. 1 is a block diagram illustrating the flow of information between the subsystems of a NLP system.

FIGS. 2-5 display the dictionary information stored on an electronic storage medium that is retrieved for each word of the example input sentence: "The person whom I met was my friend."

FIG. 6 displays the leaf nodes generated by the syntactic subsystem as the first step in parsing the input sentence.

FIGS. 7-22 display the successive application of syntax rules by the syntactic subsystem to parse of the input sentence and produce a syntax parse tree.

FIG. 23 illustrates the logical form graph generated by the semantic subsystem to represent the meaning of the input sentence.

FIG. 24 shows a block diagram illustrating a preferred computer system for natural language processing.

FIG. 25 illustrates the three phases of the preferred new semantical subsystem.

FIG. 26 is a flow diagram for the new semantic subsystem (NSS).

FIG. 27 displays the first set of semantic rules.

FIG. 28A displays a detailed description of the semantic rule PrLF.sub.-- You from the first set of semantic rules.

FIG. 28B displays an example application of the semantic rule PrLF.sub.-- You from the first set of semantic rules.

FIG. 29 displays the second set of semantic rules.

FIGS. 30A-30B display a detailed description of the semantic rule TrLF.sub.-- MoveProp from the second set of semantic rules.

FIG. 30C displays an example application of the semantic rule TrLF.sub.-- MoveProp from the second set of semantic rules.

FIG. 31 displays a flow diagram for apply.sub.-- rules.

FIG. 32 displays a flow diagram for phase one of the NSS.

FIG. 33 displays the third set of semantic rules.

FIGS. 34A-34C display a detailed description of the semantic rule SynToSem1 from the third set of semantic rules.

FIG. 34D displays an example application of the semantic rule SynToSem1 from the third set of semantic rules.

FIG. 35 displays a flow diagram for phase two of the NSS.

FIGS. 36-38 display the fourth set of semantic rules.

FIG. 39A displays a detailed description of the semantic rule LF.sub.-- Dobj2 from the fourth set of semantic rules.

FIG. 39B displays an example application of the semantic rule LF.sub.-- Dobj2 from the fourth set of semantic rules.

FIG. 40 displays the fifth set of semantic rules.

FIGS. 41A-41C display a detailed description of the semantic rule PsLF.sub.-- PronAnaphora from the fifth set of semantic rules.

FIG. 41D displays an example application of the semantic rule PsLF.sub.-- PronAnaphora from the fifth set of semantic rules.

FIG. 42 displays a flow diagram for phase three of the NSS.

FIG. 43 is a block diagram of a computer system for the NSS.

FIGS. 44-59 display each successful rule application by the NSS as it processes the syntax parse tree generated for the example input sentence.

The present invention provides a new semantic method and system for generating a logical form graph from a syntax tree. In a preferred embodiment, a new semantic subsystem (NSS) performs the semantic analysis in three phases: (1) filling in and adjusting the syntax parse tree, (2) generating an initial logical form graph, and (3) generating meaningful labels for links of the logical form graph and constructing a complete logical form graph. Each phase constitutes the application of one or two sets of rules to either a set of syntax tree nodes or to a set of logical form graph nodes.

The NSS addresses the recognized deficiencies in prior art semantic subsystems described above in the background section. Each phase of the NSS is a simple and extensible rule-based method. As additional linguistic phenomena are recognized, rules to handle them can be easily included into one of the rule sets employed by the NSS. In addition, the second phase of the NSS generates an entirely separate logical form graph, rather than overlaying the logical form graph onto an existing syntax parse tree. The logical form graph data structure generated by the NSS is therefore simple and space efficient by comparison with prior art logical form graph data structures.

FIG. 24 is a block diagram illustrating a preferred computer system for a NLP system. The computer system 2401 contains a central processing unit, a memory, a storage device, and input and output devices. The NLP subsystems 2406-2409 are typically loaded into memory 2404 from a computer-readable storage device such as a disk. An application program 2405 that uses the services provided by the NLP system is also typically loaded into memory. The electronic dictionary 2411 is stored on a storage device, such as a disk 2410, and entries are read into memory for use by the morphological subsystem. In one embodiment, a user typically responds to a prompt displayed on the output device 2403 by entering one or more natural language sentences on an input device 2404. The natural language sentences are received by the application, processed, and then passed to the NLP system by way of the morphological subsystem 2406. The morphological subsystem uses information from the electronic dictionary to construct records describing each input word, and passes those records to the syntactic subsystem 2407. The syntactic subsystem parses the input words to construct a syntax parse tree and passes the syntax parse tree to the semantic subsystem 2408. The semantic subsystem generates a logical form graph from the received syntax parse tree and passes that logical form graph to other NLP subsystems 2409. The application program then can send and receive information to the natural language subsystem 2409 in order to make use of the machine understanding of the input text achieved by the NLP system, and then finally output a response to the user on an output device 2403.

FIG. 25 illustrates the three phases of the preferred new semantic subsystem. Phases 1-3 of the NSS are shown as 2502, 2504, and 2506, respectively. The states of the relevant data structures input and output from each phase of the NSS are displayed in FIG. 25 as labels 2501, 2503, 2505, and 2507. The NSS receives a syntax parse tree 2501 generated by the syntactic subsystem. The first phase of the NSS 2502 completes the syntax parse tree using semantic rules, and passes the completed syntax parse tree 2503 to the second phase of the NSS 2504. The second phase of the NSS generates an initial logical form graph 2505 and passes that initial logical form graph to the third phase of the NSS 2506. The third phase of the NSS applies semantic rules to the initial logical form graph in order to add meaningful semantic labels to the links of the logical form graph, to add new links and nodes to fill out the semantic representation of the input sentence, and occasionally to remove redundant nodes. The complete logical form graph 2507 is then passed to other NLP subsystems for use in further interpreting the input sentence represented by the logical form graph or in answering questions or preparing data based on the input sentence.

A flow diagram for the NSS is displayed in FIG. 26. The flow diagram shows successive invocation of the three phases of the NSS, 2601, 2602, and 2603. In the following, each phase of the NSS will be described in detail.

In phase one of the NSS, the NSS modifies a syntax parse tree received from the syntactic subsystem by applying two different sets of semantic rules to the nodes of the syntax parse tree. These semantic rules can alter the linkage structure of the syntax tree or cause new nodes to be added.

The NSS applies a first set of semantic rules to resolve a variety of possible omissions and deficiencies that cannot be addressed by syntactical analysis. Application of these first set of semantic rules effect preliminary adjustments to the input syntax parse tree. The linguistic phenomena addressed by the first set of semantic rules include verbs omitted after the words "to" or "not," but understood to be implicit by a human listener, missing pronouns, such as "you" or "we" in imperative sentences, expansion of coordinate structures involving the words "and" or "or," and missing objects or elided verb phrases. FIG. 27 lists a preferred first set of semantic rules applied by the NSS in phase one. For each rule, the name of the rule followed by a concise description of the linguistic phenomenon that it addresses is shown. The general format of each semantic rule is a set of conditions which are applied to a syntax parse tree node or logical form graph node and a list of actions that are applied to the syntax parse tree or logical form graph. For example, the NSS applies the conditions of each rule of the first set of semantic rules to the list of syntax records that represents the syntax parse tree and, for each rule for which all the conditions of that rule are satisfied, the NSS performs the list of actions contained in the rule, resulting in specific changes to the syntax parse tree. Of course, the actual form of each semantic rule depends on the details of the representation of the syntax parse tree and logical form graph, for which many different representations are possible. In the following figures, a semantic rule is described by a conditional expression preceded by the word "If" in bold type, followed by a list of actions preceded by the word "Then" in bold type. The "If" part of the semantic rule represents the conditions that must be applied to a syntax parse tree node or logical form graph node and found to be true in order for the rule, as a whole, to be applied to the node, and the "Then" expression represents a list of actions to be performed on the syntax parse tree or logical form graph. The displayed expression closely corresponds to the computer source code expression for the semantic rule.

FIG. 28A displays an English-language representation of the semantic rule PrLF.sub.-- You from the first set of semantic rules. As can be seen in FIG. 28A, the "If" expression concerns the values of various attributes of the syntax parse tree node to which the rule is applied, and the "Then" expression specifies the creation of a pronoun node for the lemma "you" and a noun phrase node parent for the pronoun node and the attachment of the created nodes to the syntax parse tree.

FIG. 28B shows an example of the application of the semantic rule PrLF.sub.-- You to the syntax parse tree 2801 generated by the syntactic subsystem for the sentence "Please close the door." Application of PrLF.sub.-- You results in the modified syntax parse tree 2802, with two new nodes 2803 and 2804 connected to the root node for the sentence. This semantic rule has the purpose of explicitly placing an understood "you" of an imperative sentence into the syntax parse tree.

After all semantic rules of the first set of semantic rule that can be applied to the input syntax parse tree have been applied, the NSS makes main adjustments to the preliminarily-adjusted syntax parse tree by applying to the nodes of the preliminarily-adjusted syntax parse tree a second set of semantic rules. This second set of rules include rules that serve to identify and resolve long-distance attachment phenomena, to transform verbal phrases into verbs with prepositional phrase objects, and to replace, in certain cases, the word "if" with an infinitive clause.

FIG. 29 lists a preferred second set of semantic rules applied by the NSS in phase one. For each rule, the name of the rule followed by a concise description of the linguistic phenomenon that it addresses is shown. FIGS. 30A-30B display an English-language representation of the semantic rule TrLF.sub.-- MoveProp from the second set of semantic rules. As can be seen in FIGS. 30A-30B, the "If" expression concerns the values of various attributes of the syntax parse tree node to which the rule is applied and various related syntax parse tree nodes, and the "Then" expression specifies a rather complex rearrangement of the syntax parse tree.

FIG. 30C shows an example of the application of the semantic rule TrLF.sub.-- MoveProp to the syntax parse tree 3001 generated by the syntactic subsystem for the sentence "I have no desire to see the man." Application of TrLF.sub.-- MoveProp results in the modified syntax parse tree 3002. The infinitive clause represented by node 3003 in the original syntax parse tree has been moved from its position as a child of node 3004 to being a child 3005 of the root DECL1 node 3006 of the modified syntax parse tree. This semantic rule has the purpose of moving clauses like the infinitive clause 3003 from a lower level to a higher level in the syntax tree to facilitate the subsequent transition from the syntax parse tree to a logical form graph.

In the preferred embodiment of the present invention, semantic rules are statements in a programming language that, when executed, create a new tree or graph node from one, two, or occasionally more existing tree or graph nodes and create appropriate links between the newly created node and the existing tree or graph nodes. In the preferred embodiment, the left-hand portion of a semantic rule specifies characteristics that the existing node or nodes must have in order for the rule to be applied. The right-hand portion of the semantic rule specifies the type of new node to be created, and the values for the new node's attributes. The rules described in FIG. 28 and in FIG. 30 exemplify this form.

In the preferred embodiment of the present invention, each syntax parse tree and each logical form graph is represented as a list of nodes, with the links between the nodes represented by attribute values within the nodes. Each set of rules is also represented as a list. Application of set of rules to a syntax parse tree involves selecting successive nodes from the list of nodes and attempting to apply to each selected node each rule from the list of rules representing the set of rules. A particular rule can be successfully applied to a node if that node has the characteristics specified in the left-hand portion of the rule. Occasionally, a new node may be created as a result of a successful rule application, or an existing node may be marked for deletion.

A flow diagram for the subroutine "apply.sub.-- rules" which applies a set of rules to a list of nodes representing a syntax parse tree or logical form graph is displayed in FIG. 31. The subroutine "apply.sub.-- rules" is called by the NSS to apply each set of rules during each of the three phases of the NSS. In step 3101, apply.sub.-- rules receives a list of nodes as its first argument and a list of rules as its second argument. Steps 3102 through 3107 represent an outer loop, each iteration of which attempts to apply all of the input rules from the input list of rules to successive nodes selected from the input list. Steps 3103 through 3106 represent an inner loop, each iteration of which attempts to apply a rule selected from the list of input rules to a node selected from the input list of nodes. In step 3102, apply.sub.-- rules selects the next node from the input list of nodes, starting with the first. In step 3103, apply.sub.-- rules selects the next rule from the input list of rules, starting with the first. In step 3104, apply.sub.-- rules determines whether the selected node has the characteristics specified in the left-hand part of the selected rule. If the node has the specified characteristics, then apply.sub.-- rules applies in step 3105 the selected rule to the selected of node. If apply.sub.-- rules determines in step 3106 that there are more rules to attempt to apply to the selected node, apply.sub.-- rules returns to step 3103 to select the next rule. If apply.sub.-- rules determines in step 3107 that there are more nodes to attempt to apply the rules of the input rule list, apply.sub.-- rules returns to step 3102 to select the next node.

A flow diagram for the processing done in the first phase of the NSS is displayed in FIG. 32. In step 3201, the variable "parameter1" is assigned to be the list of syntax parse tree nodes that comprise the syntax parse tree generated by the syntactic subsystem and input into the NSS. In step 3202, the variable "parameter2" is assigned to be a list of the first set of semantic rules displayed in FIG. 27. In step 3203, the NSS invokes the subroutine "apply.sub.-- rules," passing to the subroutine the variables "parameter1" and "parameter2." The subroutine "apply.sub.-- rules" applies the first set of semantic rules to the syntax parse tree to effect preliminary adjustments. In step 3204, the variable "parameter1" is assigned to be the list of syntax parse tree nodes that comprise the preliminarily-adjusted syntax parse tree. In step 3205, the variable "parameter2" is assigned to be a list of the second set of semantic rules displayed in FIG. 29. In step 3206, the NSS invokes the subroutine "apply.sub.-- rules," passing to the subroutine the variables "parameter1" and "parameter2." The subroutine "apply.sub.-- rules" applies the second set of semantic rules to the syntax parse tree to effect main adjustments.

In phase two of the NSS, the NSS applies a third set of semantic rules to the nodes of the adjusted syntax tree. Each successful rule application in phase two creates a new logical form graph node. By applying this third set of rules, the NSS creates a new logical form graph. The nodes of the logical form graph consist of only semantically meaningful attributes and a pointer back to the corresponding syntax tree node. Unlike in prior art semantic subsystems, the logical form graph nodes created by the NSS in phase two are completely separate and distinct from the syntax parse tree nodes. The NSS constructs a skeleton of the logical form graph that comprises links, stored as attributes within the nodes, that interconnect the nodes of the logical form graph.

In FIG. 33, a list of the third set of semantic rules applied by the NSS in phase two is displayed. For each rule, FIG. 33 displays the name of the rule followed by a concise description of the linguistic phenomenon that it addresses. There are only three rules in this third set of rules, and only the first rule, SynToSem1, is commonly used. The second and third rules apply only to special situations when a fitted parse was generated by the syntactic subsystem, and the adjusted syntax parse tree therefore contains a fitted parse node.

FIGS. 34A-34C display an English-language representation of the semantic rule SynToSem1 from the third set of semantic rules. As can be seen in FIGS. 34A-34C, the "If" expression concerns the values of various attributes for the syntax parse tree node to which the rule is applied and various related syntax parse tree nodes, and the "Then" expression specifies the creation of a logical form graph node and placement of the new node within the incipient logical form graph.

FIG. 34D shows an example of the application of the semantic rule SynToSem1 to the syntax parse tree 3401 generated by the syntactic subsystem for the sentence "The book was written by John." Application of SynToSem1 results in the skeletal logical form graph 3402. The skeletal logical form graph has three nodes with temporary modifiers labeling the links. Attributes have been assigned to the new nodes, based on the syntactic attributes of the syntax parse tree nodes from which they were created. There are far fewer nodes in the logical form graph than in the corresponding syntax parse tree, because the logical form graph represents the semantic meaning of the sentence. The linguistic significance of the words "the," "was," and "by" in the original sentence is or will be incorporated into the attributes and labels of the logical form graph, and the complex node hierarchies which emanated from their presence as leaf nodes in the syntax parse tree are not necessary in the logical form graph.

FIG. 35 displays a flow diagram for phase two of the NSS. In step 3501, the variable "parameter1" is assigned the list of nodes representing the adjusted syntax parse tree. In step 3502, the variable "parameter2" is assigned to be a list of the third set of semantic rules displayed in FIG. 33. In step 3503, the NSS invokes subroutine "apply.sub.-- rules" to apply the third set of semantic rules to the nodes of the adjusted syntax parse tree, thereby creating a new logical form graph corresponding to the adjusted syntax parse tree.

In phase three of the NSS, the NSS applies a fourth set of semantic rules to the skeletal logical form graph to add semantically meaningful labels to the links of the logical form graph. These new labels include "deep subject" ("Dsub"), "deep object" ("Dobj"), "deep indirect object" ("Dind"), "deep predicate nominative" ("Dnom"), "deep complement" ("Dcmp"), and "deep predicate adjective" ("Dadj"). In FIGS. 36-38, a list of the fourth set of semantic rules applied by the NSS in phase three is displayed. For each rule, FIGS. 36-38 display the name of the rule followed by a concise description of the linguistic phenomenon that it addresses.

FIG. 39A displays an English-language representation of the semantic rule LF.sub.-- Dobj2 from the fourth set of semantic rules. As can be seen in FIG. 39A, the "If" expression concerns the values of various attributes of the logical form graph node to which the rule is applied, and the "Then" expression specifies the labeling of a link in the logical form graph.

FIG. 39B shows an example of the application of the semantic rule LF.sub.-- Dobj2 to the logical form graph 3901 generated by the NSS for the sentence "The book was written by John." Application of LF.sub.-- Dobj2 to a logical form graph containing a passive clause identifies the syntactic subject as the deep object of the action. This is accomplished, in FIG. 39B, by relabeling link 3903 from a temporary modifier to the label 3904 indicating a deep object relationship.

As the final step in phase three, the NSS makes final adjustments to the logical form graph by applying a fifth set of semantic rules. This set of rules include rules that serve to unify a relative pronoun with its antecedent, find and explicitly include missing pronouns, resolve number ellipsis, provide missing deep subjects, unify redundant instances of personal pronouns, and contract coordinate structures expanded in the first sub-step of semantic analysis. These rules also deal with the problem of taking a pronoun (or "proform") and identifying the noun phrase to which it refers. In many cases, it is not possible to identify the correct noun phrase referent with the level of information that the logical form graph provides. In these cases, a list of the most likely candidates is created, and further processing is postponed until later steps of the NLP system that employ more global information. In FIG. 40, a list of the fifth set of semantic rules applied by the NSS in phase three is displayed. For each rule, FIG. 40 displays the name of the rule followed by a concise description of the linguistic phenomenon that it addresses.

FIGS. 41A-41C display an English-language representation of the semantic rule PsLF.sub.-- PronAnaphora from the fifth set of semantic rules. As can be seen in FIGS. 41A-41C, the "If" expression concerns the values of various attributes of the logical form graph node to which the rule is applied, and of related logical form graph nodes, and the "Then" expression specifies the addition of a logical form graph node representing an omitted referent of a pronoun.

FIG. 41D shows an example of the application of the semantic rule PsLF.sub.-- PronAnaphora to the logical form graph 4101 generated by the NSS for the sentence "Mary likes the man who came to dinner, and Joan likes him too." Application of PsLF.sub.-- PronAnaphora to a logical form graph containing a pronoun node with a referent in a different part of the logical form graph adds a new node to which the pronoun node is directly linked. In FIG. 41D, the new node 4103 has been added by application of PsLF.sub.-- PronAnaphora to indicate that the node "he1" refers to "man."

A flow diagram for the processing done in phase three of the NSS is displayed in FIG. 42. In step 4201, the variable "parameter1" is assigned to be the list of logical form graph nodes that comprise the logical form graph generated during phase two of the NSS. In step 4202, the variable "parameter2" is assigned to be a list of the fourth set of semantic rules displayed in FIGS. 36-38. In step 4203, the NSS invokes the subroutine "apply.sub.-- rules," passing to the subroutine the variables "parameter1" and "parameter2." The subroutine "apply.sub.-- rules" applies the fourth set of semantic rules to the logical form graph to add semantically meaningful labels to the links of the logical form graph. In step 4204, the variable "parameter1" is assigned to be the list of the logical form graph nodes that comprise the meaningfully-labeled logical form graph generated in step 4203. In step 4205, the variable "parameter2" is assigned to be a list of the fifth set of semantic rules displayed in FIG. 40. In step 4206, the NSS invokes the subroutine "apply.sub.-- rules," passing to the subroutine the variables "parameter1" and "parameter2." The subroutine "apply.sub.-- rules" applies the fifth set of semantic rules to the logical form graph to effect final adjustments.

FIG. 43 is a block diagram of a computer system for the NSS. The computer 4300 contains memory with the semantic rules 4304-4308 and rule application engine 4303. The rule application engine, under control of a central processing unit, applies the five sets of rules to the syntax parse tree 4301 to generate a corresponding logical form graph 4302. The syntax parse tree is preferably generated by the morphological and syntactic subsystems, which are not shown. The syntax tree and logical form graph can also be used to accomplish a subsequent task requiring information analogous to that which a human reader would obtain from the input sentences. For example, a grammar checker program might suggest a new phrasing for the input sentence that more accurately or concisely states what was stated in the input sentence. As another example, a computer operating system might perform computational tasks described by the input sentence. As still another example, information contained in the input sentence might be categorized and stored away for later retrieval by a database management system.

The following discussion and FIGS. 44-59 describe the complete NSS processing of the example sentence "The person whom I met was my friend." Each semantic rule that is applied by the NSS will be described, along with a representation of the results of the rule application.

No preliminary adjustment rules from the first set of semantic rules are successfully applied to the syntax parse tree input into the NSS from the syntactic subsystem during phase one. One main adjustment rule from the second set of semantic rules is applied to the input syntax parse tree. FIG. 44 displays the syntax parse tree 4400 in the form it is input. Note that it is represented in FIG. 44 slightly more simply than in FIG. 22. The NSS successfully applies the semantic rule TrLF.sub.-- LongDist1, displayed in FIG. 29 as rule 1, to the relative clause node RELCL1, 4401, of the syntax parse tree 4400 to generate the adjusted syntax parse tree 4402. The effect of applying rule TrLF.sub.-- LongDist1 is the introduction of a direct object attribute in the noun phrase node 4403 to indicate that the word "whom" is the direct object of the phrase "I met." Normally, in English, the direct object of a verb follows the verb. Because "whom" does not follow "I met" in the sentence that was parsed to produce the syntax tree 4400, the fact that "whom" is the direct object of "I met" was not identified by the application of syntactic rules.

Seven rules from the third set of semantic rules are successfully applied in phase two of the NSS. In FIG. 45, the NSS successfully applies the semantic rule SynToSem1, displayed in FIG. 33 as rule 1, to the determinate pronoun node DEPT2, 4501, of the syntax parse tree to generate the logical form graph node "my" 4502. In FIG. 46, the NSS successfully applies the semantic rule SynToSem1 to the noun phrase node NP4, 4601, of the syntax parse tree to generate the logical form graph node "friend" 4602 and the link 4603 with the temporary semantic label "Tmods" 4604. In FIG. 47, the NSS successfully applies the semantic rule SynToSem1 to the noun phrase node NP3, 4701, of the syntax parse tree to generate the logical form graph node "I" 4702. In FIG. 48, the NSS successfully applies the semantic rule SynToSem1 to the noun phrase node NP2, 4801, of the syntax parse tree to generate the logical form graph node "whom" 4802. In FIG. 49, the NSS successfully applies the semantic rule SynToSem1 to the relative clause node RELCL1, 4901, of the syntax parse tree to generate the logical form graph node "meet" 4902 and the link 4903 with the temporary semantic label "Tmods" 4904. In FIG. 50, the NSS successfully applies the semantic rule SynToSem1 to the noun phrase node NP1, 5001, of the syntax parse tree to generate the logical form graph node "person" 5002 and the link 5003 with the temporary semantic label "Tmods" 5004. In FIG. 51, the NSS successfully applies the semantic rule SynToSem1 to the declarative sentence node DECL1, 5101, of the syntax parse tree to generate the logical form graph node "be" 5102 and the link 5103 with the temporary semantic label "Tmods" 5104. Thus, with the completion of phase two of the NSS, a skeletal logical form graph has been created.

Six rules from the fourth set of semantic rules are successfully applied in phase three of the NSS. In FIG. 52, the NSS successfully applies the semantic rule LF.sub.-- Dusb1, displayed in FIG. 36 as rule 1, to the logical form graph node "be" 5201 to generate the link label "Dsub" 5202 and the link 5203 with the temporary semantic label "Tmods" 5204. In FIG. 53, the NSS successfully applies the semantic rule LF.sub.-- Dnom, displayed in FIG. 36 as rule 10, to the logical form graph node "be" 5301 to generate the link label "Dnom" 5302. In FIG. 54, the NSS successfully applies the semantic rule LF.sub.-- Props, displayed in FIG. 38 as rule 21, to the logical form graph node "person" 5401 to generate the link label "Props" 5402. In FIG. 55, the NSS successfully applies the semantic rule LF.sub.-- Dusb1, displayed in FIG. 36 as rule 1, to the logical form graph node "meet" 5501 to generate the link label "Dsub" 5502. In FIG. 56, the NSS successfully applies the semantic rule LF.sub.-- Dobj1, displayed in FIG. 36 as rule 3, to the logical form graph node "meet" 5601 to generate the link labeled "Dobj" 5603 to link the node "meet" to the node "whom" 5602. In FIG. 57, the NSS successfully applies the semantic rule LF.sub.-- Ops, displayed in FIG. 38 as rule 22, to the logical form graph node "friend" 5701 to generate the link label "PossBy" 5702.

One rule from the fifth set of semantic rules is successfully applied in phase three of the NSS. In FIG. 58, the NSS successfully applies the semantic rule PsLF.sub.-- RelPro, displayed in FIG. 40 as rule 1, to the logical form graph node "whom," displayed as 5602 in FIG. 56, to generate the link labeled "Dobj" 5801 and to remove the node "whom." In FIG. 59, the NSS successfully applies the semantic rule PsLF.sub.-- UnifyPron, displayed in FIG. 40 as rule 10, to the logical form graph to consolidate the nodes "I" and "my" into a single node. This is the last rule applied successfully by the NSS. FIG. 59 thus displays the final, complete logical form graph generated by the NSS for the input sentence "The person whom I met was my friend."

Although the present invention has been described in terms of a preferred embodiment, it is not intended that the invention be limited to this embodiment. Modifications within the spirit of the invention will be apparent to those skilled in the art. The scope of the present invention is defined by the claims that follow.