Adaptive Mechanisms for Execution of Sequential Decisions

1. Field of the Invention

The present invention generally relates to the field of artificial intelligence and, more particularly, to adaptive mechanisms for increasing the speed of execution of sequential decisions. As used herein, artificial intelligence means that part of computer science concerned with designing intelligent computer systems. Such systems are finding increasing application in problem-solving, logical reasoning, language understanding, programming, learning, expert systems, robotics and vision.

2. Description of the Prior Art

Among the most important aspects to problem-solving methods in artificial intelligence are (1) representation and (2) search. The present invention is concerned with the efficiency of search in the decision making process. Search strategies in current practice typically have the following characteristics:

1. they use fixed control;

2. they are insensitive to experience; and

3. they do not exploit the usage pattern of the search program.

Search strategies that use fixed control are inherently limited in achieving good run-time performance. A fixed search stategy is illustrated here by way of an example of pathfinding in a graph. Consider a directed graph (digraph for short) such as the one shown in FIG. 1 of the drawings. A program may be written in PROLOG to search for paths between specified pairs of nodes in the digraph. PROLOG is a computer program language which derives its name from PROgramming in LOGic and is a major artificial intelligence implementation language. See, for example, D. H. D. Warren, L. M. Pereira and F. Pereira, "PROLOG--The Language and Its Implementation Compared with LISP", Proceedings of the Symposium on Artificial Intelligence and Programming Languages (1977), published by the ACM. An example of such a program is shown below.

adj(a,b).

adj(a,c).

adj(a,d).

adj(a,e).

adj(b,c).

adj(b,d).

adj(c,d).

adj(e,f).

conn(X,Y)<-ne(X,Y)&adj(X,Y).

conn(X,Y)<-ne(X,Y)&adj(X,Z)&conn(Z,Y).

conn(X,Y)<-fail.

Suppose a top-down, left-to-right search strategy (i.e., depth-first search) with backtracking is used as provided by a Prolog system. Suppose the search program is queried whether a is connected to f by typing "<-conn(a,f).". Then, execution of the search will involve the traversal of the search tree shown in FIG. 2(a) before a path is found from a to f. The sequence of nodes visited in the search process is shown by a dashed line in the forward direction and by a dotted line in the reverse direction. Each query of the form "<-conn(a,f)." represents an instance of the set of problem inputs that may be presented to the search program. For an instance I of the problem, let the search tree traversed be denoted as T(I). The main characteristic to observe with regard to a fixed search strategy is the following. If the search program were given a repetitive sequence of problem instances of the form

I,I,I, . . . ,I (sequence of n I's), then the search trees traversed in the repetitive solution of the problem instances will be

T(I),T(I),T(I), . . . ,T(I) (sequence of n T(I)'s).

With respect to I, if the search program were inefficient to begin with, then no matter how many times I is solved, the search program will be exactly as inefficient as it were before it solved the first instance of I. The search program using fixed control does not learn to improve its efficiency to take advantage of the fact that, in its experience, it has solved certain problem instances more frequently than certain other problem instances. We believe that for realistic problems, not all input instances are equally likely to occur. Assuming the set of problem instances encountered exhibit a nonuniform distribution, this can be exploited by search programs to optimize their average run-time performance.

A significant disadvantage of a fixed search strategy, such as the depth-first search provided by PROLOG, is that a programmer has to be aware of the fixed order in which the search will be conducted for various inputs. If the programmer is concerned with efficiency, then he is responsible for writing the program in such a way that the fixed search strategy will find solutions efficiently for the set of problem instances he wishes to solve. For instance, suppose the pathfinding program is predominately used for finding a path from a to f and rather infrequently used for finding paths between other pairs of nodes. Then, a good strategy for improving performance is to write the program such that the search involved in finding a path from a to f is made as efficient as possible even at the expense of increasing the search effort for finding paths between some other pairs of nodes. Knowing that PROLOG uses a fixed depth-first search strategy, the programmer could achieve the desired objective by simply rewriting the pathfinding program as follows:

adj(a,e).

adj(a,c).

adj(a,d).

adj(a,b).

adj(b,c).

adj(b,d).

adj(c,d).

adj(e,f).

conn(X,Y)<-ne(X,Y)&adj(X,Y).

conn(X,Y)<-ne(X,Y)&adj(X,Z)&conn(Z,Y).

conn(X,Y)<-fail.

The search tree traversed in finding a path from a to f is as shown in FIG. 2(b), and the search effort involved is efficient. The average performance can thus be improved.

In general, the task of organizing the program for efficient execution by a fixed search strategy is complicated for two reasons. First, for moderate to large-sized search programs, understanding the structure of search trees traversed for different problem instances soon becomes difficult. Second, and more important, the programmer typically would have no a priori knowledge of the workload; i.e., the relative frequencies with which different problem instances will be submitted to the program over the program's lifetime. The programmer will typically have knowledge only of what constitutes valid problem inputs. These drawbacks are intrinsic in the nature of any fixed search strategy. The subject invention is directed to approaches which overcome the limitations of a fixed search strategy.

In the prior art, U.S. Pat. No. 4,368,509 to Li discusses mechanisms for adaptive self-optimization. Li's patent is based on an algorithm that discovers dependencies among variables. His algorithm generates a sequence of tests that adaptively discovers the dependencies. Li's technique allows the algorithm to produce a set of test inputs on which observations are made. Then, according to a particular mathematical strategy, a new set of inputs is generated. However, for the same set of inputs submitted at different instants in time, the system is controlled in an identical manner resulting in exactly identical performance.

As will become clear from the description which follows, Li's patent differs from our invention in a very fundamental manner. The key characteristics of our invention are adaptive learning and capability for self-improvement of efficiency of systems embodied with our invention. This is achieved by keeping a prior history of search performance. If a sequence of identical problem inputs were submitted to a system embodied with our invention, the efficiency with which the problem instance is solved can vary from one instance of the problem to the next in a way that tends to minimize the search effort involved in solving the problem. This is achieved by the adaptive learning mechanisms of the present invention. In contrast, Li's patent does not possess the unique characteristic of the invention, namely, adaptively learning to improve efficiency in a systematic manner so that frequently encountered classes of problem instances are solved with improved efficiency as the system is used on a repeated basis.

U.S. Pat. No. 4,472,770 to Li is more general than the above-referenced patent to Li; however, the comments given above apply to this patent as well.

U.S. Pat. No. 4,210,962 to Marsh et al provides a demonstration that dynamic programming problems can be solved rapidly enough for use in real-time applications. Marsh et al describe a special purpose parallel computer to speed up the solution of dynamic programming problems. The parameters of the dynamic programming problem are supplied as inputs to the dynamic programming processor. The main issue addressed by this patent is fast computation of the optimal solution to problems. There is no learning and adaptation involved. The efficiency with which an instance of a dynamic programming problem is solved does not depend on history; i.e., how often the problem instance was solved before by the processor disclosed in the patent. Thus, there is a fundamental difference between the Marsh et al patent and our invention which always tends to bias the system toward the efficient solution of frequenty encountered problem instances.

U.S. Pat. No. 4,499,596 to Casey et al discloses the idea of adapting a system in a manner that is influenced by the sequence of inputs presented to the system. Casey et al describe the logic of an algorithm that has been used for efficient classification of characters in the context of optical character recognition systems. The process of classification is then used to generate compact coded vector representation of scanned characters. This can result in reduced storage and transmission requirements for facsimile application. The objectives of the Casey et al patent do not include optimization of classification time or storage requirements. Casey et al's scheme adapts in the sense that it is able to receive an arbitrary sequence of text characters, one at a time. Each character that is encountered for the first time is automatically assigned a class index. Any subsequent encounter of the same character will result in retrieval of the class index of the character for output. Thus, adaptation in the context of the Casey et al patent means ability to classify character strings that are seen by the system for the first time. It should be noted that the purpose of adaptation is not optimization of any specified objective function, such as classification time. The method of adaptation described in the Casey et al patent is based on appropriately extending a decision tree network representation whenever a new character is encountered. It is of a heuristic nature. The decision tree representation generated by the algorithm is one of many possible decision trees that may be used to classify characters. There is no formal theoretical basis provided for choosing among the many possible representations. There is no concept of learning in the Casey et al patent. Given identical character sequences at different time instants, the classification algorithm always executes identical steps and requires the same amount of time and storage for execution. The efficiency of the algorithm for solving a problem instance does not depend on whether the problem instance was solved previously. Thus, there is fundamental difference between the Casey et al patent and our invention.

It is therefore an object of the present invention to provide adaptive mechanisms for the faster execution of sequential decisions.

It is a further object of the subject invention to provide an adaptive-learning mechanism for improving the performance of applications that require sequential-decision making.

It is another object of the invention to provide adaptive mechanisms for increasing the speed of execution of sequential decisions using a parallel computer.

It is yet another object of the instant invention to provide an adaptive mechanism which enables multiple processors to learn, adapt, and cooperate in an effective way to speed up the execution of such applications.

According to the invention, an adaptive mechanism is provided which reorders a chain of decisions based on observations made during their execution in order to produce an ordering that achieves faster execution. Eventually, the average performance over a period of time converges to the optimum performance. The algorithm is used to maintain a near-optimal sequence for trying the set of alternatives. It is assumed that the costs (for success and failure) and probabilities for the different alternatives are not known when the program is written, so that the programmer cannot determine the optimal ordering from a prior knowledge. When the search program is presented a sequence of problem instances to solve, it follows the initial ordering of the search tree. As it solves these problem instances, it also monitors the amount of search effort (CPU time or number of instructions or some other dominant measure) it spends in trying the different alternatives. The information it gathers from repeated observations of the execution of the search program is used to produce automatically a near-optimal ordering of the alternatives.

Typical of the applications where this mechanism can be effectively used are certain kinds of expert systems and fault diagnosis systems. An expert system often requires that a list of rules be scanned. As soon as an applicable rule is found, the rule is fired and the scanning of the list of rules is repeated. Associated with each rule are a cost (e.g., number of instructions or execution time) of deciding whether the rule is applicable and a probability with which the rule will fire. Certain fault diagnosis and isolation systems require the execution of a decision chain for selecting one out of N possible outcomes that result in follow-up actions such as module replacement or detailed testing of specific modules. The diagnosis system applies a sequence of test procedures to a faulty machine until one procedure terminates with a positive finding. Each test takes a certain amount of time and is likely to turn up a positive result with a certain probability.

The adaptive mechanism according to this invention is presented in the context of optimization of expert-system applications. The mechanism is used to maintain a near-optimal sequence for scanning rule-lists in expert systems. Assume, for example, that a program contains a sequential decision chain with either mutually exclusive or independent outcomes, and that each decision in the chain has associated with it some fixed cost and probability. It is well known that the optimal ordering of the decisions orders them by descending values of the ratios of their probabilities to their costs. See, for example, W. E. Smith, "Various Optimizers for Single-Stage Production", Naval Research Logistics Quarterly, 3(1956), pp. 59-66, and H. A. Simon and J. B. Kadane, "Optimal Problem-Solving Search: All-or- None Solutions", Artificial intelligence, 6(3), 1975, pp. 235-247.

In its most basic form, the adaptive mechanisms according to the invention are applied to a single processor system. In a sequel, the invention is described in the context of two processors which cooperate and maintain a near optimal sequence for scanning rule-lists in expert systems. These basic mechanisms can be extended to parallel computers with more than two processors.

The foregoing and other objects, aspects and advantages of the invention will be better understood from the following detailed description of preferred embodiments of the invention with reference to the drawings, in which:

FIG. 1 is a graphical representation of a directed graph or digraph;

FIGS. 2(a) and 2(b) are graphical representations of search trees;

FIG. 3 is a flow chart of the adaptive mechanism as implemented in a single processor;

FIG. 4 is a graph showing the results of applying the adaptive mechanism of the invention implemented in a single processor to a decision chain with seven decisions;

FIGS. 5.1 and 5.2, taken together, are a flow chart of adaptive mechanism as implemented on plural processors; and

FIG. 6 is a graph showing the results of applying the adaptive mechanism of the invention implemented on two processors to a decision chain with thirteen decisions.

Before describing the first embodiment of the invention as it would be implemented on a single processor, it is necessary to first define certain variables. Let N be the total number of rules in a list scanned by an expert system. Let the observed frequencies of firing of different rules be stored in a 1-dimensional array FREQ, with FREQ(i), i=1, . . . ,N, containing the total number of times Rule i has fired. When the expert system is used for the very first time, the FREQ values are set to zero. Let COST be a 1-dimensional array such that COST(i), i=1, . . . ,N, contains the accumulated cost of testing if Rule i is applicable. Let ATTEMPTS be a 1-dimensional array such that ATTEMPTS(i), i=1, . . . ,N, contains the number of times a rule has been tested for firing. Then the average cumulative cost for Rule i is COST(i)/ATTEMPTS(i). Let RATIO be a 1-dimensional array such that RATIO(i), i=1, . . . ,N, contains the value ##EQU1## This is an estimate of the ratio of probability of firing to cost per test for each rule.

Referring now to the drawings, and more particularly to FIG. 3, the adaptive mechanism according to the invention executes the following procedure. The first test made in decision block 10 is to determine if the expert system is being used for the very first time. If so, as indicated in function block 12, the counters used to count the variables are initialized to zero and the SCANLIST is set to the initial sequence of the rule list. A list of rules is maintained in sequence, and the rules are ordered by nonincreasing RATIO values. This list is called the SCANLIST. If, on the other hand, the expert system is not being used for the very first time, the initialization of function block 12 is skipped, and control goes directly to function block 14 where the scan cycle begins. Whenever a rule has to be chosen for firing, the rules are selected in order from SCANLIST, so that Rule i is tested before Rule j if RATIO(i) >RATIO(j). When a rule, say Rule k, is tested for firing, the following actions are taken. ATTEMPTS(k) is incremented by 1, and COST(k) is increased by the cost of the testing. When a test for Rule k is successful, indicating that the rule will fire, FREQ(k) is increased by 1, and RATIO(k) is updated. This is indicated in function block 16. Then a test is made in decision block 18 to determine if the updated values indicate a significant change that warrants adaptation of SCANLIST. If the test is positive, Rule k is reordered in SCANLIST in function block 20 to maintain the ordering to be nonincreasing by RATIO values; otherwise, function block 20 is skipped and control is passed to decision block 22 which determines if the execution of the expert system is to be terminated. If not, the process loops back to function block 14. However, if the execution of the expert system is to be terminated, then control passes to function block 24 where the values of SCANLIST, COST, FREQ, ATTEMPTS, and RATIO are saved for future use by the expert system. The appendix contains a program listing in the PL/1 programming language which simulates the procedures illustrated in the FIG. 3 flow chart.

The expected cost of firing a rule with the adaptive mechanism according to the invention typically converges to a near-optimal cost after a number of rules have been fired. FIG. 4 shows the adaptive mechanism applied to a decision chain with seven decisions. The higher graph shows the convergence when the chain has been deliberately placed in the worst possible initial ordering. The lower graph shows the convergence for an initial ordering chosen at random. The optimal cost for these data is 62.6, and both examples achieve near-optimal performance within 500 decisions, with the random initial ordering actually converging within 200 decisions. The observed improvement in average cost of testing for these two examples varied between three and four to one. The improvement and convergence rates observed are problem dependent and, therefore, do not necessarily reflect which might be observed in practice.

There are several variations of the basic idea that achieve nearly the same performance improvement at lower overhead costs for the mechanism. Since the majority of the performance improvement occurs in the first few hundred decisions, the reordering of the rules can be done with varying frequency, occurring for almost every decision at first, and eventually being done rather rarely. It is possible to change the order of only the rules that fire successfully, or to reorder all rules including those that did not fire, when a reordering occurs. Because the cost of execution of a test may not vary significantly each time the test is performed, it may not be necessary to maintain both COST and ATTEMPTS. Instead, each rule may be associated with a fixed constant cost per test, so that it is sufficient to have COST(i) for each Rule i, with the initial value of COST(i) determined by counting instructions in the test or from the first execution of the test. In this case, the computation of RATIO(i) reduces to computing the value FREQ(i)/COST(i). Another simplification is possible in considering rule reordering. When a ratio changes sufficiently for its corresponding rule to be moved elsewhere in the decision chain, instead of moving that rule to its designated position, it is simpler to swap the rule with the neighboring rule in the direction that the rule should move. Convergence may be slower with this scheme, but the lower overhead may yield better overall performance.

Consider next the application of the invention to a multi-processor system as represented by M processors. Again, it is necessary to define variables. Let ADAPT.sub.-- INTERVAL be a variable containing the frequency with which the M processors will cooperate in sharing their knowledge. The value of this variable is initially set to unity when the multiprocessor algorithm is used for the first time. Let COUNT be a variable for counting the number of rules that have fired. When the multiprocessor algorithm is first used, COUNT is initialized to zero.

As shown in the flow chart of FIG. 5, the multiprocessor system executes the following procedure. As in the single processor implementation, the first decision that must be made, by processor P1, is whether this is the first time the expert system is being used, as indicated in decision block 26. If it is, the list of rules is partitioned into M sublists L1, L1, . . ., LM of equal (or nearly equal) size in function block 28. On the other hand, if the expert system is not being used for the first time, then function block 28 is skipped and control is passed to function block 30 where the scan cycle begins. Processor Pi (i=1,2, . . . ,M) keeps the list of rules in Li, ordering them by nonincreasing RATIO values. Whenever a rule has to be chosen for firing, Pi considers rules in the order in which they appear in Li. Processor Pi will consider Rule k for firing before Rule 1 if RATIO(k) >RATIO(1), assuming both Rules k and 1 are in Li. When a rule, say Rule k, has been tested by Pi for firing, the following actions are taken. ATTEMPTS(k) is incremented by 1, and COST(k) is increased by the cost of the testing. When one of the processors, say Pj, finds a rule, say Rule k, that can fire, then the following additional actions are taken by Pj. COUNT is incremented by 1. FREQ(k) is incremented by 1 and RATIO(k) is updated. Then Rule k is reordered in list Lj to maintain the ordering to be nonincreasing RATIO values. As soon as Rule k fires, Pj also notifies all the other processors through an interrupt mechanism to stop further processing of their respective lists.

The M processors cooperate as follows in sharing the knowledge they have gained from observations. A test is made in decision block 32 by processor Pj, that just fired a rule, to determine if it is time for adaptation. If COUNT equals ADAPT.sub.-- INTERVAL, control is transferred by Pj to P1 to carry out the adaptation process. Processor P1 resets COUNT to 0 and takes the following actions in function block 34. It combines the rules contained in L1, L2, . . ., LM and orders the combined list of rules in nonincreasing order of their RATIO values. This list is called the MASTER.sub.-- LIST. P1 reassigns rules to L1, L2, . . ., LM as follows. A rule is placed in position k of list Li if it occurs in position {(k-1)M+i} of MASTER.sub.-- LIST. If, on the other hand, it is not time for adaptation, control passes to function block 36 where the processor Pj, that just fired a rule in block 30, signals the start of scanning of respective rule lists again, and control loops back to function block 30.

After function block 34, a test is made in decision block 38 by processor P1 to determine if the execution of the expert system is to be terminated. If not, control passes to function block 40 were processor P1 signals the resumption of scanning of the rule lists, and control loops back to function block 30. If, on the other hand, execution is to be terminated, control passes to function block 42 where the contents of the counters are saved along with the current ordering of the lists for future use.

The appendix contains a program listing in the PL/I programming language which simulates a special case of the above descirbed multiprocessor procedures, namely, for implementation on a two-processor system. With a two-processor implementation, the expected cost of firing a rule with the multiprocessor procedure typically converges to a value somewhat greater than half the optimal cost achievable with a single processor. The observed speedup in simulation study of the multiprocessor procedure varied between 1.43 and 1.85. FIG. 6 shows the multiprocessor procedure applied to a decision chain with thirteen decisions. The graph shows the convergence for an initial ordering chosen at random. The observed improvement in the cost of testing in this example was 2.3. Convergence occured within 1000 decisions at a cost of 33.7. For the data used in this example, the optimal cost of testing with one processor is 48.2. The observed speedup is thus 1.43. Performance improvement relative to initial ordering, convergence rates and speedup factors observed are problem dependent and, therefore, do not necessarily reflect which might be observed in practice.

There are several variations of the basic multiprocessor mechanism that may achieve nearly the same performance improvement at lower overhead costs for the mechanism. Again, since the majority of the performance improvement occurs in the first few hundred decisions, the multiprocessors may be made to cooperate very frequently at first, and eventually very rarely. This could be accomplished by executing the multiprocessor algorithm initially with ADAPT.sub.-- INTERVAL set to unity and then resetting ADAPT.sub.-- INTERVAL to a larger value. Further, instead of partitioning the decision chain into M sublists that are considered exclusively by the M processors each time the rule list is scanned in parallel, a common list consisting of all the decisions in the chain can be maintained in nonincreasing order of RATIO values. Rules can be tested according to the sequence in the common list. Whenever a processor becomes available, it can choose the next rule in the list which has not yet been tested in the current scan cycle. As soon as a rule has been tested successfully for firing by one processor, it can stop the current scan cycle and also interrupt the other processors to stop further processing. This scheme may have comparable convergence and overall performance as the basic multiprocessor scheme.

While the invention has been described in terms of two specific implementations, one on a single processor and one on a multiprocessor system, it will be understood by those skilled in the art that the invention can be practiced in various alternatives and with various modifications within the scope and spirit of the invention as defined in the appended claims. ##SPC1##