Time-Constrained Inference Strategy for Real-Time Expert Systems

The present invention relates generally to a control process for expert systems, and, more particularly, to an expert system control process for resolving problems in time-constrained situations.

In certain applications, a significant need exists for expert systems which can operate to provide satisfactory decisions in critically time-constrained situations. This is often the case in military applications. Although the invention is certainly not limited to military applications, the following discussion will pertain to a military situation for purposes of example only. For example, in advanced avionics applications, it would be very desirable to have an intelligent system to assist pilots in making in-flight tactical decisions. In combat situations, pilots face severe workload stress in terms of the magnitude, diversity, and speed of the required decisions. Furthermore, pilots are overwhelmed by the quantity of incoming data and communications upon which they must base life-critical decisions.

Pilots are confronted with numerous dynamically changing situations, each of which requires precise activation of one of the dozens of switches, buttons, and knobs covering their controls. While each of the aircraft's hundreds of components and systems serve legitimate purposes, the technologies which created them have outpaced the technology for intelligently interfacing these systems to the pilot.

In order to support the next generation of advanced single-pilot helicopters and aircraft, the technology must be developed to provide intelligent information processing capabilities in the cockpit. These intelligent processing systems will off-load lower-level decision-making tasks from the pilot and provide tactical advisories in various situation assessment tasks. The resulting cockpit environments will be characterized by enhanced, efficient performance and reduced pilot workload.

Emerging techniques in knowledge-based expert systems are providing the solution to this dilemma by developing methods for representing and applying knowledge stored in a computer to solve problems that are generally thought to require significant human expertise (e.g. see Davis, R., "Expert Systems: Where Are We? And Where Do We Go From Here?", A.I. Magazine, Spring 1982, which is hereby incorporated by reference). These knowledge-based expert systems will complement the pilot's performance by off-loading lower-level tasks so that the pilot may concentrate on higher-level tactical and strategic objectives.

Although expert system technology has been evolving since the mid 1960's, the development of expert systems for time-constrained environments is still in its infancy. Referring to a military example, the real-time cockpit requirements for knowledge-based decision aids introduce several areas of expert system technology requiring further research and development. State-of-the-art expert systems have not yet confronted real-time constraints either in military situations or elsewhere. Typically, in the past, real-time performance and expert systems have not been considered in the same system.

An "expert system" is an intelligent computer program that uses knowledge and inference procedures to solve problems that are difficult enough to require significant human expertise. An expert system consists of:

(1) A knowledge base of domain facts and heuristics associated with the problem. The facts constitute a body of information that is widely shared, publicly available, and generally agreed upon by experts in a field. The "heuristics" are mostly private, little-discussed rules and strategies of good judgment, plausible reasoning, and good guessing that characterize expert-level decision-making and drastically limit search in large problem spaces. This knowledge is used by the system in reasoning about the problem.

(2) A control structure for symbolically processing and utilizing the information stored in the knowledge base to solve the problem. This control structure is also commonly referred to as the inference engine.

(3) A global data base to serve as a working memory to keep track of the problem status, input data, and relevant facts and history of the solution progression.

(4) Explanation systems to allow the user to challenge and examine the reasoning process underlying the system's answers. This includes a user friendly interface to facilitate user interaction with the system.

(5) Knowledge acquisition system to facilitate the addition of new knowledge into the system. Knowledge acquisition is an ongoing process, thus the knowledge must evolve over time through several iterations of trial and error. This interactive transfer of expertise from a human expert to the expert system is required in order to achieve an operationally acceptable level of performance.

The fundamental structure of an expert system is shown in FIG. 1. The major architectural distinction between a knowledge-based expert system and a conventional computer program is that an expert system maintains a clear, distinct separation between the general knowledge about the problem (knowledge base 10) and the reasoning mechanisms for symbolically applying this knowledge (control structure 12). Instead of being programmed to follow step-by-step procedures, an expert system employs a few general problem-solving procedures for symbolically applying knowledge and heuristics to find solutions to general classes of problems. As shown in FIG. 1, input data 16 is coupled to the control structure 12 and the knowledge base 10 through the global data base 14. A user interfaces with the expert system through the natural language interface 18.

Over the past decade, several Artificial Intelligence (A.I.) research groups have built highly specialized knowledge-based expert systems for a variety of domains (e.g. medicine, chemistry, geology, computer configurations, electronic fault diagnosis, etc.). The number of organizations developing expert systems has grown sharply over the past five years. Furthermore, research has rapidly spread and intensified in academia, spurred by this influx of industry and government funds. Researchers have recognized the potential applicability of expert systems for a wide variety of applications. However, this research still has not addressed the real-time constraints prevalent in military applications.

Previous university and industrial research activities addressing real-time constraints have focused on developing faster symbolic processing machines and multi-processing architectures (e.g. see Halstead, R. H., "Multilisp: A Language for Concurrent Symbolic Computation", MIT Labortory for Computer Science, August 1984, which is hereby incorporated by reference). Present-day expert systems claim real-time performance simply by virtue of their application domain being small enough not to require sophisticated processing. However, using conventional expert system inferencing techniques on these enhanced architectures will require that an upper bound be placed on the size of the knowledge base to ensure real-time processing. This upper limit on the size of a knowledge base is unacceptable for most military applications since the knowledge base is expected to grow and expand over time. The complexity of military scenarios and other complex situations mandates that a new generation of expert system inferencing techniques be developed to efficiently process these knowledge bases in real-time.

The present inventors have recognized that faster machines and limited domains do not satisfy the technology requirements for real-time expert systems. Accordingly, they have undertaken research to address the problem-solving paradigms that enable time-constrained decisions-making in complex situations.

Traditional expert system control strategies may fail to provide responses in time-stressed scenarios. Most expert system control strategies employ a depth-first search through a knowledge base in order to reach a decision. However, in a time-constrained environment such as that found in many military situations, this depth-first traversal is not guaranteed to achieve a reliable solution, or more importantly any solution at all.

An expert system control strategy contains the problem solving mechanisms for symbolically processing and utilizing the information stored in the knowledge base to solve problems. Two reasoning mechanisms are commonly used in expert systems, either alone or in combination, to process the knowledge base. These mechanisms are:

(1) forward-chained (data-driven) inferencing; and

(2) backward-chained (goal-driven) inferencing.

The specific implementation of a reasoning mechanism depends on the nature of the problem and the representation of the knowledge being manipulated.

When data or basic ideas serve as a starting point, then forward-chaining is the natural direction for the problem solution. In these data-driven inferencing systems, the system reasons forward from the facts to a solution by repeatedly evaluating the antecedents of all rules in the knowledge base until a steady-state condition is reached. However, these forward-chained systems may never terminate if the initial evidence does not lead to a conclusion. Thus, accordingly, most expert systems in practice tend to be goal-driven.

When a goal state or hypothesis serves as an initial focus point, then the inverses of inference rules are applied to find a path back to the initial state. In this goal-directed inferencing scheme, the system works backward from a hypothetical solution (goal) to find evidence supporting that goal. Backward-chaining is applicable to most diagnosis and planning problems.

This backward-chained inferencing strategy explores each potential hypothesis in order of likelihood. The system searches the knowledge base for a rule(s) whose consequent matches the desired conclusion. The system then attempts to recursively verify each resulting rule in turn until the antecedent part of the rule matches the initial problem description stored in the global data base. Each hypothesis is explored until all hypotheses have been exhausted. The system then ranks the hypotheses by the degree to which they are supported by the evidence and presents the resulting conclusion to the user.

The limitation, however, of this depth-first backward-chained inferencing strategy is that the evaluation of the knowledge base is independent of the amount of processing time available in the given scenario. In time-stressed scenarios, the system might inappropriately plunge depth-first into an insignificant path of reasoning and not achieve a solution within the bounded response time. Traditional expert system inferencing strategies contain no focusing mechanism to identify the most influential, cost-effective knowledge to evaluate in the knowledge base. Thus, these traditional inferencing strategies are inappropriate for time-constrained decision-making domains.

Accordingly, it is an object of the present invention to provide an expert system control process which allows for improved operation in a time-constrained environment.

A further object of the present invention is to provide an expert system control process which is particularly useful in time-constrained situations for resolving problems in a manner similar to that which would be used by a human expert faced with these time constraints.

To accomplish these and other objects, the inventors have developed an expert system inferencing strategy to provide reliable responses in time-constrained situations. This research and development has focused on the problem-solving paradigms which lead to reliable time-constrained decisions. In particular, they have constructed an expert system control strategy resembling the way an expert in the field reasons about time-constrained situations. This inferencing strategy considers the most influential pieces of data first, and less significant data as time permits. The significance of a piece of knowledge is determined by the potential change in the final decision resulting from evaluation of that node. Thus, the rule which appears most likely to alter the final conclusion is tested first and less significant data is examined as time permits. The level of belief in the final conclusion is proportional to the amount of processing time for the given scenario. Thus, this inferencing mechanism adjusts the decision-making process based upon the immediacy of the required solution. In short, the optimality of the solution is appropriately dependent upon the processing time available in the given scenario.

To accomplish this "best first" approach, merit values (e.g. see Slagle, J. R. and Halpern, E., "An Intelligent Computer Strategy for Computer Consultation", NRL Memorandum Report 4789, April 1982, which is hereby incorporated by reference) are assigned to each of the propositions (nodes) of the inferencing network to guide the search. These merit values are based on the degree of influence which the particular node has on the top node of its particular inferencing network and the processing cost of evaluating the particular node. A "best first" algorithm is then applied based on the merit values so that the best data can be evaluated first to give a decision based on the best data within the time limitations imposed. The certainty of the expert system response increases in proportion to the amount of processing time available.

FIG. 1 shows the basic structure of an expert system employing the control process of the present invention;

FIG. 2 shows an inference network used in the control process of the present invention;

FIG. 3 shows an individual network node of the inference network of FIG. 2;

FIG. 4 shows a prototype expert system for weapon-to-threat assignment as an example of utilizing the control process of the present invention; and

FIGS. 5a and 5c show the arrangement of the rules of the knowledge base shown in Appendix 4 for the prototype expert system of FIG. 4.

Basically, the present inventors have developed a time-constrained inference network to provide reliable expert system responses for applications where significant real-time constraints exist. Particularly, a deep-level inference network to represent domain knowledge, similar to the partitioned semantic network in the PROSPECTOR system (e.g. see R. O. Duda et al, "Development of the PROSPECTOR Consultant System for Mineral Exploration", Final Report for SRI Projects 5821 and 6415, Artificial Intelligence Center, SRI International which is hereby incorporated by reference), has been implemented to support time-constrained decision-making and additionally provide a mechanism for the propagation of dynamically changing data. A military application will be given as an example, although the invention is not limited to this, and can be practiced in any situation where a time-constrained decision must be made using an expert system.

Referring to FIG. 1, the knowledge base 10 for this inference network is represented as a set of propositions. Each proposition has an associated numeric value representing its likelihood (i.e. the relative significance of processing that node). Initially, a default value is assigned to each proposition in setting up the knowledge base to represent an estimate of the likelihood. As will be explained later, the propostion's value will subsequently be calculated during processing of given problems to establish a value other than the estimated default value. A proposition may have antecedent propositions from which its value may inferred, and may itself be an antecedent of consequent propositions.

Each proposition is represented as a node in the inference network as shown in FIG. 2. The top node G represents the final conclusion of the expert system. The other nodes in the system below the top node G represent individual propostions related to the top node and possess associated values. Using backward-chained inferencing, the top node represents a basic proposition which the user wishes to have the expert system's likelihood estimate on. The nodes below this represent other propositions which the knowledge base programmer establishes as affecting the proposition of the top node. Generally, a knowledge base will have several top nodes since, obviously, it will be desired to analyze a number of different propositions. In fact, depending on the information sought by the user, any node in the system can become a top node if the user designates that particular proposition as the information he wants evaluated. The nodes which affect this designated proposition then become the lower nodes in the inference network. The leaf nodes in the network are referred to as terminal nodes whose values are retrieved from sensor data and user input.

Links (or "edges") in the inference network connect the proposition nodes together. These links (inference rules) define the implications for organizing the network into antecedent-consequent relationships, thus allowing for the propagation of information.

Assignment in the network is the process by which the value associated with a node is determined from the values associated with its antecedents. The inference links specify how the probability of a consequent proposition is assigned based upon the probabilities of its antecedent propositions. The arguments to the assignment function correspond to the antecedents of that node. Propagation involves a sequence of assignments initiated at any node and continues through the consequents of that node until the top proposition is reached.

A real-time inferencing strategy has been developed by this invention to focus the solution path on the most influential node in the network with the least associated processing cost. This inferencing strategy utilizes a best-first search algorithm to select the most "meritorious" proposition at any level in the inference network to process. A control mechanism utilizing these merit values has been developed to ensure that the most relevant facts are considered in a cost-effective prioritized manner and thus, guarantees a reliable response within the given time constraint.

Each node in the inference network of FIG. 2 is represented in the format illustrated in FIG. 3. As seen there, each node contains the following:

NODE LABEL: The node label specifies the unique name for the node. It is formed by concatenating the datum function with the particular context.

DATUM FUNCTION: The datum function is a function whose domain is a set of contexts and whose codomain is any set of numbers.

CONTEXT: The context is a set of variables upon which the value of a node depends. The context specifies the domain for the datum function. Propositions with no particular context are global contexts or terminal nodes and are independent of antecedent propositions.

CONDOMAIN OF DATUM FUNCTION: The codomain of a datum function is any specified set of numbers. For a probablistic datum fucntion, this numeric value can fall between zero and one or can fall between negative one and positive one similar to probabilistic certainty functions in the EMYCIN system (e.g. see B. G. Buchanan et al, Rule-Based Expert Systems, Addison-Wesley, 1984 which is hereby incorporated by reference). A descriptive datum function represents a physical attribute of an object and may assume any value.

DEFAULT: Each node in the inference network has an associated default value that the assignment function returns when no antecedent values are supplied. The default value is the a priori value of a node and msut belong to the codomain of the datum function. As noted earlier, this will only serve as the node's value until the node is expanded (i.e. processed or evaluated) in light of the particular circumstances the user has provided to give the node's value. This default value may be context sensitive and may be derived by a bounded set of production rules.

ASSIGNMENT FUNCTION: The assignment function specifies how the value associated with a node is to be computed based upon the values associated with its antecedent's. The arguments to the assignment function correspond directly to the values of the antecedents of the node. The assignment functions consist of the Bayesian logical connectives (AND, OR and NOT) and any expert-defined linearly differentiable function.

OR Assignment Function

OR (x,y, . . . , z)=1-[(1-x) (1-y) . . . (1-z)]

The OR assignment function assigns the product of the complement probabilities of the antecedents to the complement of the consequent.

AND Assignment Function

AND (x,y, . . . , z)=(x) (y) . . . (z)

The AND assignment function assigns the product of the probabilities of the antecedents to the consequent.

NOT Assignment Function

NOT (x)=(1-x)

The NOT assignment function assigns the complement of the probability of the antecedent to the consequent.

EXPERT-DEFINED Assignment Function

The EXPERT-DEFINED assignment function allows the expert to define a differentiable mathematical function for combining the values of antecedents to compute the value of a consequent. EXPERT (x,y, . . . z)=f(x,y, . . . , z) must be defined by the expert such that f'(x,y, . . . , z) exists.

CONFIDENCE RATING: The inference rules associated with the assignment function may themselves contain uncertainty. A numeric procedure has been developed to indicate the confidence rating of the expert in each particular assignment function. The confidence rating is a number between 0 and 1 associated with each assignment function to indicate the degree of belief in the consequent when the antecedents are known to be true (i.e. one--see Cohen, P. R., and Grinberg, M. R., "A Theory of Heuristic Reasoning about Uncertainty", A. I. Magazine, Summer 1983, which is hereby incorporated by reference). These confidence ratings are combined and propagated through each consequent assignment function. This use of a confidence rating is similar to the use of a certainty factors in the MYCIN system (e.g. see B. G. Buchanan et al, Rule-Based Expert Systems, Addison-Wesley 1984). A generalization of Bayes' theorem is utilized to derive the confidence rating of nodes in the inference network. Whereas Bayes' theorem requires masses of statistical data, the generalization of this theorem utilizes subjective expert judgment in its place. Thus the domain expert must initially specify the confidence rating of each node. This relaxation of Bayes' theorem requires that the domain expert be consistent and accurate in assignment of confidence ratings throughout the network.

NODE VALUE: The value of a node is a number within the codomain of the datum function and is obtained by applying the assignment function to the node values of its antecedent propositions. The resulting value is combined with the confidence rating of the node to yield the derived degree of belief in the consequent. The combining function for incorporating the confidence rating is specified initially by the domain expert.

TIME-MERIT VALUE: The time-merit value of a node represents the cost-effective influence value that a node has on the top proposition. A high merit value implies that a proposition exerts considerable influence on the top proposition with little associated processing cost. A low merit value indicates that expansion of that proposition will have little effect on the top proposition or that expansion of that node can be accomplished only at a high processing cost. Processing cost is generally defined in terms of the total computer processing time that is required for determining the value of the particular node. This depends both on what antecedent nodes must be processed first to establish the value of the present node, and the particular assignment function of the present node. On the other hand, processing cost is not necessarily limited to this definition, and could be defined to reflect consumption of other resources of the computer system such as memory, input-output channel bandwidth, peripheral device allocation etc. Obviously, the term "processing cost" depends upon the architecture of the particular computer system. For example, in a parallel processing multicomputer system, real computation time may be less than total processor time. Thus, 100 msec. of total processor time could be allocated between ten processors so that the real computation time would only be about 10 msec.

Conversely, in a time-shared uni-processor system 100 msec of processor time might require over 1 sec. of real-time due to the need for sharing processor time with other users. Thus, the term "processing cost" can be a reflection of real-time or actual processor time or some combination of both, depending upon the relative importance of these to the user.

Calculation of merit will now be discussed. As noted previously, the merit value of a node represents the cost-effective influence value that a node exerts on the top proposition in the inference network. Consider the inference network shown in FIG. 2 with top proposition G and antecedents G.sub.i. Each subscript indicates an additional level in the inference network. The value stored at G.sub.i is P.sub.i. Each value of P is determined by applying the assignment function of a node to the values of its antecedent proposition.

The time-merit of an untried proposition is defined as: ##EQU1## where dP represents the expected change in the node value P of the top proposition G, and d(C.sub.ij . . . st) represents the associated processing cost to expand the untried proposition G.sub.ij . . . st. The subscript notation "ij . . . st" indicates a multi-level index which identifies the node. For example, P.sub.ij is the jth antecedent of P.sub.i. The absolute value is taken since both positive and negative changes in the top proposition are of equal impact. Thus, the time-merit value of a proposition G is equal to the absolute value of the expected ratio of the potential change in the node value of the top proposition to the cost of expanding the node G (e.g. the time required to evaluate that node). A more useful and meaningful representation of a time-merit value is derived by applying the chain rule to the time-merit equation: ##EQU2##

The last factor in the chain rule expansion is the only one involving the cost of expanding an untried node, and is termed the "self-merit" of a node. The self-merit indicates the expected change in a proposition's node value P.sub.ij. . . st with respect to the cost of processing the proposition, C.sub.ij . . . st. The self-merit value of a node as indicated by this last factor is assigned by the domain expert. The specific scale and units of a self-merit value are not important, as long as they are consistently relative to one another throughout the network. Large self-merit values should be assigned to nodes whose required processing time is short and whose value is likely to change a great deal.

A brief discussion will now be given regarding how the domain expert could go about determining an appropriate self-merit value. Basically, it should be recognized that self-merit is a factored probability parameter, since the numerator term in its derivation is a probability. The denominator term is a cost-constant (or at least a constant cost-estimate). The self-merit value is the expected change in a node's value resulting from evaluation of that node, divided by a factor representing node evaluation cost. The expected change is viewed with respect to the default value for the node.

In the absence of any knowledge about the probability distribution of node values, a default value of 0, representing complete uncertainty, should be chosen. In a boolean logic system nodes would have only two possible values, representing true or false. In a bayesian system, the equivalent values would be 1 or -1, with no in-between values. The bayesian system allows for in-between values as a way of representing uncertainty, but the underlying concept is essentially binary.

Now let's say our expert knows or feels from his experience that a node's value is equally likely to be either -1 or 1 at any given point in time lacking any other knowledge about a specific situation. At the time of determining the self-merit value, we know that node computation time (cost) is, say, 1 unit. Therefore, the only thing remaining to be determined is the expected change in value of the node resulting from the evaluation. Since he doesn't know which value the node will have before computation, he chooses a default value of 0. In this case, the expected change would be 1, as shown from the following calculation: ##EQU3##

The above statement says that the expected change is 1 because there is a 50% probability of a value change (from the default value) of 0 to -1; and there is a 50% probability of a value change from 0 to 1. In either case, the absolute value of the change is 1.

But this is a special case. Consider another example:

The expert knows (or feels) from experience that a particular node is likely to have the value 1 just about all the time--say, 99% of the time. The other 1% of the time it will be -1. He assigns a default value of 1.

From the above discussion, the utility of the self-merit term should be more apparent. When the expert has knowledge about the likely evaluation of a node, the self merit value allows expression of that knowledge. In particular, when the expert sees a probability distribution for node values that is strongly asymmetric across the possible range, the system can take advantage of this knowledge to reduce computation time. The asymmetric probability distribution is equivalent to reduced uncertainty and in turn, reduced expected change in node value from the default. Now, like human experts, our expert system can "tend to ignore" certain rules which "usually" yield the same value.

Each of the remaining factors in the chain rule expansion of the merit function is a link-merit, which indicates the change in the node value of a consequent proposition due to the change in the node value of an antecedent proposition. The link merit corresponds to the degree of influence exerted by an antecedent on its direct consequent. The link merit is calculated by differentiating the assignment function associated with the link connecting the antecedent/consequent pair with respect to the given antecedent. Examples of different types of link merits are as follows: AND Link Merit: A consequent whose truth value is dependent upon verification of all its antecedents is the logical AND of those antecedents. Assuming all antecedents are independent, then a probablistic analysis describes the AND function as:

Thus, the probability assigned to a consequent, H, given the current probabilities (values) for each of its antecedents A.sub.i (i=l, . . . , n) is the product of the antecedent probabilities. Differentiating the consequent probability with respect to a single antecedent and substituting, the AND Link Merit is derived as: ##EQU4## The AND link-merit states that the power of an antecedent to change its hypothesis probability is inversely proportional to that antecedent's probability. Thus the antecedent of lowest probability has the highest link-merit value among all antecedents of an AND Proposition. Intuitively, the antecedent with the lowest probability is the one primarily responsible for holding down the consequent probability of an AND link.

OR Link Merit: The OR function is logically true when any of its antecedents are true. Assuming independence of antecedents, then the probablistic OR function is written as:

The consequent probability is the complement of the product of the complements of all the antecedent probabilities. Differentiating with respect to an individual antecedent and substituting, the OR link-merit function is derived as: ##EQU5## The OR Link-merit states that the power of an antecedent to change its consequent probability is inversely proportional to the complement of that antecedent's probability. Thus, the antecedent of highest probability has the greatest link-merit of all the antecedents of an OR node.

NOT Link Merit: The logical NOT is defined as:

A subscript is not required since a NOT link has only one antecedent. The link merit of the NOT function is derived as: ##EQU6##

In summary, the time-merit value of a node in the inference network is the product of all of the link merits on all of the nodes directed from the node to the top proposition, multiplied by the self-merit value of that node. The merit values for the nodes are used in conjunction with a "best-first" algorithm in a manner which will be discussed below to operate within given time-constraints. Regarding the calculation of merit values in general, it should be realized that the initial calculations for merit value assignments are based on the assigned default values given to each node. Thus, referring to FIG. 3, an initial merit for each node is provided based on the assigned default values. However, during processing values of antecedent nodes will change as actual calculated values replace default values. Thus, the respective merit values for the nodes will also change since the merit of any given node will depend on the values of its antecedent nodes.

The control strategy for searching through the inference network utilizing a best-first algorithm based on time-merit values to select the most cost-effective influential proposition at any level in the network to evaluate will now be discussed. The inference strategy continuously updates the nodes values for the top propositions. When the available processing time is about to expire, the default values for all non-evaluated propositions are applied in all active assignment functions and the resulting node values are propagated through the inference network to yield the final result in a bounded time.

This inferencing strategy employs a two step algorithm to select the next proposition for evaluation:

1. The system identifies the untried proposition with the largest merit value on the tree (i.e. the inference network) and calculates the merit values for all of its children (i.e. expands the identified node to all of its antecedent nodes).

2. At each level, the best time-merit value is backed up to the top proposition. At the top level, the untried position with the highest merit value is identified and evaluated.

The overhead computation time for evaluation time-merit values is proportional to tree depth rather than tree size, since only the time-merits of the newly expanded node are computed and backed up. This best-first strategy is analogous to moving up a tree of winners. A cutoff merit value may be used to limit or increase the total number of nodes investigated. Only those nodes with a time-merit value above the cutoff will be investigated. This cutoff merit value does not alter the order in which nodes are investigated, only the total number of nodes investigated. If no propositions remain with a merit value above the cutoff value, then traversal of the inference network is complete.

This inference strategy ensures that the most significant data is processed first in a prioritized manner. At any instant, the current node value of the top proposition represents the most reliable estimate of the actual conclusion that can be made within the given time constraint. The level of confidence in the estimated node value of the top proposition increases in proportion to the amount of processing time utilized.

An example of the present invention will now be given by describing a prototype expert system for tactical weapon-to-threat assignment (e.g. see Callero, M., Jamison, L., and Waterman, D. A., "TATR": An Expert Aid for Tactical Air Targeting", RAND Corporation, January 1982; and Slagle, J. R., Cantone, R., and Halpern, E., "BATTLE": An Expert Systems for Fire Support Command and Control", NRL Memorandum Report 4847, July 1982, which are hereby incorporated by reference). This avionics application is a well-bounded weapon-to-threat assignment domain which clearly illustrates the requirements for time-constrained expert system decision making. The real-time aspect of this weapon-to-threat assignment problem requires that weapon deployment responses be made to the pilot within an acceptable response time for each threat scenario.

This prototype expert system provides reliable time-constrained tactical weapon deployment advisories to pilots by assimilating:

mission profile data;

target/threat sensor data;

weapon availability data;

aircraft monitoring data;

environmental data; and

weapon-to-threat assignment knowledge

A model knowledge base for this weapon-to-threat assignment domain has been constructed to provide realistic responses. The knowledge base contains the domain specific facts, rules, and heuristics associated with weapon-to-threat assignment. The focus of this example is centered on development of a real-time control strategy rather than on a sophisticated knowledge base for weapon-to-threat assignment. Thus, a representative subset of weapons, targets, threats, and assignment strategies has been selected. These weapons, targets, and threats selected are typical of more complex scenarios. Appendix 1 represents weapon stores for the prototype system while Appendix 2 represents the potential targets and their threats to be encountered. Appendix 3 shows the weapon-to-threat effectiveness matrix used for the prototype, thereby illustrating which weapons are effective for given targets. Appendix 4 lists the rules of the knowledge base. FIGS. 5a to 5c represent a graphic illustration of Appendix 4.

The weapon-to-threat assignment knowledge for this model knowledge base was collected from various books and references, such as Janes' Weapon Systems. Future expert level performance can arise from collecting an extensive set of weapon and threat knowledge, rather than a large collection of domain independent methods. The development process for this prototype expert system for weapon-to-threat assignment is composed of seven phases:

(1) Requirements Specifications;

(2) Design of time-Constrained Inference Strategy;

(3) Knowledge Engineering;

(4) Implementation of Real-Time Expert System;

(5) Refinements and Demonstration;

(6) Implementation of Traditional Control Strategy; and

(7) Comparison and Evaluation of Time-Constrained versus

Traditional Expert System Inference Strategy

Weapon-to-threat assignment is a critical function for pilots requiring many complex time-constrained decisions involving evaluation of the battlefield scenario to determine the importance of enemy targets and threats, selection of appropriate targets and threats to attack, and assignment of suitable weapons and tactics to attack those targets and threats. The weapon-to-threat assignment decision results from time-constrained human judgements integrating the many factors regarding enemy target/threat capabilities, weapon system effectiveness, and the battlefield scenario. The dynamic stressful cockpit environment requires that critical real-time choices be made among the various alternatives.

This prototype expert system, illustrated in FIG. 4, receives input information from five primary sources. This information includes:

(1) Weapon Stores data;

(2) Pre-processed target and threat sensor data;

(3) Mission profile and tactics data;

(4) Environmental data for the battlefield scenario; and

(5) Aircraft monitoring data.

The output of this expert system is a recommendation to the pilot regarding the optimal weapon deployment response for the existing threat scenario. The response time of the system is bounded by a maximum acceptable value dependent upon the lethality of the threat scenario. The ultimate weapon deployment decision resides with the pilot.

The knowledge base 10 for this prototype expert system contains the domain specific facts, rules, and heuristics associated with weapon-to-threat assignment. In particular, the knowledge base 10 contains specific knowledge regarding weapon characteristics and capabilities (Appendix 1), target and threat characteristics (Appendix 2), and weapon assignment rules and heuristics (Appendices 3 and 4). The target/threat sensor information, as well as the heuristic procedures for weapon-to-threat assignment are incomplete and inexact. This prototype system is designed to accommodate and propagate uncertainty in sensor data and inference rules. Associated with each recommendation to the pilot is a confidence rating to express the "level of belief" associated with the recommendation based upon the certainty of the input data and the heuristics utilized in the time-constrained inference strategy.

The time-constrained inference network described earlier was implemented for this weapon-to-threat assignment domain. The control strategy for the inference network utilizes a best-first traversal of the knowledge base based upon the merit values of each node to determine the most influential and cost-effective node to process. In particular, it is especially important to realize that the "best first" algorithm is utilized in conjunction with the merit value calculations discussed earlier since it is these merits values which establish what is, in fact, "best".

The knowledge representation for this prototype system for weapon-to-threat assignment is a combination of:

(1) Frames: to represent declarative weapon, target/threat characteristics and capabilities. Each frame is a modular, decomposable "chunk" of knowledge organized into a fixed format. The slots within the frame are internal variables which are filled in by tokens to represent an instantiation of that frame; and

(2) Inference Network: to store procedural knowledge and heuristics for weapon-to-threat assignment. The inference network consists of a collection of condition-action pairs (inference rules) representing weapon-to-target assignment knowledge. For example, following the format discussed earlier with regard to FIG. 3, the node in the inference network for computing the operational verification of deploying an AGM-78 is represented as:

The inference network control strategy was implemented to provide time-constrained responses for this weapon-to-threat assignment expert system. Appendix 5 contains a listing of the LISP implementation of this control structure which was run on a Harris H800 computer with a LISP 5.0 Interpreter. This inferencing strategy utilizes a best-first pruning algorithm, based upon time/influence constraints reflected in the calculated merit values, to generate and select the best weapon allocation plan. A weighted value is assigned to each node in the inference network, corresponding to that node's ability to alter the final conclusion. A time/impact value is assigned to each link of the connecting nodes to represent the processing cost associated with investigating that node. In time-constrained situations, the next node selected for investigation will correspond to the node with the greatest potential influence on the final conclusion and the least associated processing cost. Thus, this control mechanism insures that the solution strategy focuses on the most influential events in a cost-effective prioritized manner.

Initially, the user is prompted to initialize the global data base 14 by entering each weapon aboard the aircraft. As noted earlier, the possible weapons are listed in Appendix 1. The user is then prompted to simulate the operations of a target/threat recognition system by entering the potential target classifications and associated certainties. Appendix 2 contains a listing and description of each potential target in the knowledge base 10 of the prototype system. The user then enters the range of the given target/threat and the aircraft altitude to complete initialization of the global data base 12. A print mode may be enabled to allow the user to view the justification for the decision.

For the time-constrained inference strategy in this example, the user is prompted to enter the number of nodes to be evaluated. This time constraint corresponds to the number of nodes in the network that will be expanded for each proposition by the inference strategy. Generally, this number of nodes value will be replaced by a time-value which corresponds to the maximum response time in the given situation. However, as noted earlier, this "time" can either reflect real-time or processor time, depending on what is most important to the user.

In order to evaluate the performance of this time-constrained expert system inference strategy, the inventors have also implemented a traditional expert system control strategy for the same weapon-to-threat assignment domain used for the prototype example. Appendix 6 contains a listing of the LISP implementation of this traditional control strategy also arranged for the Harris H800 computer. This control strategy utilizes an exhaustive backward-chained. depth-first traversal of the knowledge base, similar to the EMYCIN system.

The knowledge base for this traditional expert system is a set of production rules consisting of antecedent-consequent pairs, representing weapon-to-threat assignment knowledge. Additionally, each production rule contains an associated certainty factor to indicate the domain expert's level of confidence in the consequent of a particular rule, given the certainty of its antecedents. The traditional backward-chained control structure for this traditional expert system constructs a set of hypotheses, with each hypothesis representing the potential assignment of one weapon from the weapon stores aboard the aircraft. The control strategy then attempts to verify all the antecedents of each hypothesis, in turn, until all hypotheses have been evaluated.

The traditional system searches the knowledge base for all the rules whose consequent matches the desired hypothesis. The control strategy recursively attempts to verify each antecedent of the resulting rules in turn until the antecedent part of a rule matches the initial problem description stored in the global data base. When all hypotheses have been exhausted, the system ranks the hypotheses by the degree to which they are supported by the evidence and the highest ranking hypothesis is presented to the user.

A comparison between the time-constrained inference strategy and traditional expert system inference strategy was made to evaluate the applicability of this real-time inference strategy for time-constrained avionic domains. Evaluation of the performance of the system indicates that this best-first strategy for selecting the most influential node in the network is far superior to traditional depth-first expert system search strategies, since traversal is not constrained to a particular order. It more closely resembles the way a human expert solves a time-constrained problem, by considering the most influential cost-effective factors first. Alternatives which don't appear promising are disregarded. Refinements to this inference strategy will make it generally applicable to a wide variety of military problems.

A logical question may be raised regarding the inference strategy of the present invention due to the degree of jumping around in the inference network resulting from the best-first strategy. A traditional depth-first strategy remains within a single sub-tree for the length of time and will never return to that subtree again. On the other hand, the best-first strategy of the present invention jumps all around the inference network in a seemingly bewildering fashion. However, since this system is designed to operate in real-time and the input data is received from sensors rather than from questioning the user, this jumping around is transparent to the user. More generally, this merit strategy may even be utilized within a depth first traversal of an inference network to order the antecedents of a node before it is expanded by the depth-first traversal.

The following is a sample scenario to demonstrate the reasoning process of the expert system. Appendix 7 contains a detailed listing of the reasoning process utilized in the traditional expert system control strategy. Appendix 8 contains a detailed listing of the reasoning process for time-constrained inference strategy when the number of nodes to expand was entered as five. The "trace" function of the LISP language was used to obtain a listing of each rule being tested in turn. The scenario includes:

WEAPON STORES: AGM-88

WASP

ERAM

MK-84H

TARGETS: TANK FORMATION (certainty=0.8)

SAM SITE SA-4 (certainty=0.2)

TARGET RANGE: 6000 meters

A/C ALTITUDE: 550 feet

The human "expert" answer to this scenario would assign the ERAM to the given target location since it is deduced with high probability that the target is a tank formation. The AGM-88 is excluded since it is only effective against radiating targets (SAM, radar sites, communications, etc.) and thus ineffective against a non-radiating tank. The MK-84H is effective against primarily hard structures such as bridges and buildings and thus inappropriate. Both the ERAM and the WASP are highly effective against multiple armoured targets, however the target range is close to the maximum range limit of the WASP and thus potentially uncertain. On the other hand, the maximum range of the ERAM is significantly greater than the existing range to the target and thus more certain. Additionally, the ERAM is slightly more effective at low-level delivery than is the WASP.

Indeed, both the traditional and time-constrained expert system control strategies indicate the ERAM as being the first assignment choice and the WASP as being the second choice.

However, the number of antecedent-consequent clauses being verified to achieve this solution was 80 in the traditional expert system strategy and only 34 in the time-constrained strategy. It is apparent that this time-constrained inference strategy provides a feasible solution to the time-constrained expert system reasoning problem. More complex and detailed knowledge bases will require that even more antecedent-consequent relationships will be evaluated. Therefore, it can be seen that in time-constrained situations, the knowledge base can not be effectively traversed by traditional backward-chained depth-first traversal.

Although the present invention has been described with regard to a particular example, it is to be understood that the invention is not limited to this example. For example, although the present invention is especially useful in military applications such as that described in the example, it is not limited to military applications. On the contrary, the present invention can be used in any application where time constraints are an important consideration, for example, as a scheduling algorithm for multiprocessing expert systems. Similarly, although the LISP implementation given in Appendix 5 is well suited to the requirements of the invention, other program languages can be used. Further, even though the invention is particularly useful for time-constraint situations, with the "processing cost" being a reflection of time, it is to be understood that the "time" can be real-time or processor time or some combination of the two, depending upon what actual constraints face the user. As discussed earlier, military users will often face real-time constraints. On the other hand, in time-sharing environments, a user might find processor time to be more of a concern for cost or processor time allocation reasons. Or, as shown in the prototype, a simple node number limitation could be used as a "time-constraint" even though the actual real-time or processor time for this would not be known beforehand. Further, as mentioned previously, processing cost could actually be defined with regard to consumed resources (e.g. memory required etc.) of the computer system other than time, if these were of particular importance to the user. This would have to be reflected in the merit value calculation by assigning relative merits to each node based upon the effect which expanding the node during processing would have on the top node and the amount of the resource in question which would be consumed by thus expanding the node.

Accordingly, although a preferred embodiment of the invention has been shown and described, it will be apparent to those skilled in the art that changes may be made without departing from the principles and spirit of the invention, the scope of which is defined by the following claims. ##SPC1##