Automatic topic identification and switch for natural language search of textual document collections

[0002] This application is related in subject matter to co-pending U.S. patent application Ser. No. 09/570,788 filed May 15, 2000, by Thilo W. Goetz et al. for “Interactive Automated Response System” and assigned to the assignee of this application. The disclosure of application Ser. No. 09/570,788 is incorporated herein by reference.

[0003] 1. Field of the Invention

[0004] The present invention generally relates to a method of determining whether a user is continuing a topic or switching to another topic within a search and, more particularly, to restricting the search to the previous results if the user is continuing a topic, and to searching an entire database if the user is switching topics.

[0005] 2. Background Description

[0006] State of the art search engines have no notion of session history or dialog context. Prior art methods thus often return a great number of responses, and a user cannot easily determine if the desired answer is among them. Narrowing the search on a returned results set typically requires explicitly and manually adding to a previously defined search expression via a Boolean AND function or some other formal expression. If a user wants to initiate a new search outside a previous results set, he or she must take an explicit action. Some systems return results and place them in folders (i.e., categories), which the user can manually inspect and select such that a follow-on search is restricted to a particular folder or category. However, here too the user must manually indicate whether or not he or she is continuing a topic or switching topics. These modes of search are laborious, requiring that the user make many decisions and take many actions.

[0007] It is therefore an object of the invention to provide a method for iteratively drilling-down on a user's textual free-form natural language query using a session history to interpret successive queries in the context of previous queries on a topic or topics.

[0008] It is another object of the invention to provide a method for recognizing from a user's current free-form natural language query and a session history of previous queries by the user an implicit switch in topic.

[0009] According to the invention, by maintaining a session history of the user's free-form natural language input and by automatically determining whether there is a topic or context switch, the search process is substantially simplified and is more effective; that is, more accurate answers to a user's queries are found faster. In addition, as the present invention operates on free-form natural language input, automatically constructing the actual search expressions, the complexity of constructing successive search expressions is obviated.

[0010] If the system determines the user is, according to its session history and tests, asking successive questions within a given topic or context, the system keeps searching within a previously determined given set of previous responses on that context or topic. This effectively narrows the documents found allowing the user to quickly and accurately find just the documents of interest. If the system determines the user has implicitly changed context or topic, based on its session history and tests, it searches all the information at its disposal; i.e., all of the collections of documents.

[0011] The foregoing and other objects, aspects and advantages will be better understood from the following detailed description of a preferred embodiment of the invention with reference to the drawings, in which:

[0012] FIG. 1 is a flowchart illustrating the logic of the process implemented according to a first embodiment of the invention; and

[0013] FIG. 2 is a flowchart illustrating the logic of the process implemented according to a second embodiment of the invention.

[0014] Referring now to the drawings, and specifically to FIG. 1, user input query in the form of free-form natural language text is received at input block 102. The user query may be either typed in on a keyboard or spoken into a microphone. In the case of spoken input, the system receiving the user query would include a speech recognition system, such as IBM's ViaVoice system, to produce an electronic text document that is processed by the system. In decision block 104, a determination is made whether or not there has been prior input (i.e., is the input query provided in block 102 the first input query, or a second or subsequent input query?). If there has been prior input query, the new input query is combined with previous input query(ies) from the session history maintained by the system, as described in more detail below.

[0015] Assuming that this is the first input query, new search terms are created in function block 106 based only on the free-form natural language input. This is done using a parser to identify relevant textual words, which words are used create the search terms. Using these new search terms, one or more databases of the system are searched in function block 108, and the results of the search are returned in function block 110. The prior input is updated with the new input in function block 112, and in function block 114, the prior results are updated with the new results. The results of updated results from function block 114 are then displayed in function block 116.

[0016] If there has been a prior input query as determined in decision block 104, the new input query is combined with prior input queries stored in a session history in function block 118. Based on this combined input, combined search terms are created in function block 120. In function block 122, the prior results (PR) are searched utilizing the combined search terms from created in function block 120. The results of the search are returned in function block 124, and the new results (NR) are processed. Note that NR is always a subset of PR. Whenever NR is nonempty, the user is drilling down. However, when NR is empty, it means the search terms contributed by the new input do not apply to any of the prior results, and so a context switch has taken place. A determination is made in decision block 126 as to whether there has been a context switch by testing if NR is nonempty. If the new results are not within the categories of the previous responses, the system detects a context switch in the user's input query, and the process goes to function block 106 where new search terms are created just as if this were a first input query. However, if the new results are within the categories of the previous responses, the prior input is updated with the combined input in the session history in function block 128 before the process goes to function block 114 where the prior results are updated with the new results.

[0017] In either case, when the search results are displayed in function block 116, the user is prompted to either provide an additional input query or to indicate whether the search is completed. A determination is made in decision block 130 based on the user's input as to whether the search is to be continued. If so, the process returns to function block 102 to receive the user's next input query; otherwise, the process terminates.

[0018] FIG. 2 shows a refinement of the process of FIG. 1. In this flow diagram, the same reference numerals as used in FIG. 1 are used to designated identical or equivalent steps. Function block 108 of FIG. 1 is replaced by new function blocks 201, 203 and 205. Thus, when new search terms are created in function block 106 using a parser, the next step in function block 201 is to categorize the new input. Then in function block 203, final new search terms are created by adding terms and/or weight terms based on the assigned categories. Using the final new search terms, the system data bases are searched in function block 205.

[0019] For the case where the user input query is not the user's first query, function blocks 120 and 122 of FIG. 1 are replaced with new function blocks 207, 209, 211, and 213. After the combined input has been created in function block 118, the combined input is categorized in function block 207. Combined search terms are created from the combined input in function block 209. Then final combined search terms are created in function block 211 by adding terms and/or weight terms based on categories. Using these final combined search terms, the prior results are searched in function block 213.

[0020] The advantage of the refinements of this second embodiments in which the user's inputs are categorized and final search terms are created by adding terms and/or weight terms based on the assigned categories is to further improve the accuracy and efficiency of the process.

[0021] While the invention has been described in terms of preferred embodiments, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the appended claims.