Software Discovery System

This invention relates to systems for discovering information in collections of data and, more particularly, to discovery systems for obtaining specifications of computer programs.

An important problem in software engineering is the maintenance of "legacy" programs, that is, programs which were originally written many years ago but are still being used, and consequently must be modified when bugs are discovered or new features are added. The legacy program problem is becoming ever more pressing. Software, unlike the hardware it runs on, does not wear out, and consequently can be used for as long as there are computers which can execute it. Further, in many modern devices such as telephone switches or data base systems, the software is the most expensive component of the system, and consequently will be completely rewritten only if there is no other practical alternative.

One aspect of the legacy program problem is understanding the program. While understanding software is never easy, understanding a legacy program becomes more difficult over time. The program's original developers have typically left or are doing other things, and whatever documentation originally existed generally does not adequately reflect the many modifications of the software. Indeed, understanding legacy programs is so difficult that it is estimated that more than half of the time spent maintaining a large legacy program is in fact spent learning to understand it.

Software engineers have developed a number of techniques for making programs more understandable. One common method is to construct a "specification" of that system--an abstract description of the system presented in different language, typically a language that is more concise and/or declarative than the language in which the system is implemented. Because no specifications exist for many software systems, techniques for automatic "specification recovery" have been frequently proposed in the software engineering community as an aid in maintaining software systems.

Techniques are known for recovering a specification by analyzing the source code of a software system. Examples of these techniques are described in T. J. Biggerstaff "Design Recovery for Maintenance and Reuse", IEEE Computer, July 1989; P. T. Breuer and K. Lano "Creating Specifications from Code: Reverse Engineering Techniques", Journal of Software Maintenance: Research and Practice, Vol. 3, pp. 145-162, 1991; and C. Rich and L. Wills "Recognizing a Program's Design: A Graph-Parsing Approach", IEEE Software, January 1990. One difficulty with these techniques is that they are limited to source code; in many legacy systems, the source code has been lost or bugs have been fixed by "patching" the executable code, so that there is no longer an exact correspondence between the source code and the executable code. Another difficulty is that source code is complex, and consequently, its analysis is not easy.

What is needed in dealing with legacy programs is a way of getting a specification for the program which simply avoids source code analysis and instead concentrates on what the legacy program actually does.

The invention gets a specification for a program by simply learning the specification from the results of a set of executions of the program. In more detail, the user must instrument the software system being analyzed so that a trace of all program actions (at some appropriate level of abstraction) is recorded whenever the program is executed. Next, the instrumented program is run on a set of representative test problems. This produces a set of traces describing the program's actions on these test problems.

Then, a learning method that produces hypotheses in a first-order abstract declarative language, for instance Prolog, is used to construct from these execution traces one or more general descriptions of the behavior of the program. A declarative bias may be used to restrict the learner to output only those general descriptions that are considered a priori to be plausible. Finally, one or more specifications of the program are derived from these general descriptions of the programs behavior.

FIG. 1 shows an overview of the invention.

FIG. 2 illustrates the space of Prolog programs that must be searched to find a specification.

FIG. 3 describes the method used to find a Prolog program consistent with a grammar and a set of examples.

FIG. 4 describes the method used to find all Prolog programs consistent with a grammar and a set of examples, and to express this set of programs in a compact.

FIG. 5 illustrates the space of programs searched by these methods for another example problem.

FIG. 6 shows an example relation and view.

FIG. 7 shows an example trace.

FIG. 8 shows an example generalized description and an example specification.

FIG. 9 shows an example declarative bias.

FIG. 10 shows another example relation and view.

The reference numbers in the figures have three or more digits; the rightmost two digits are reference numbers within a figure; the remaining digits are a figure number. Thus, the element with the reference number 103 first appears in FIG. 1.

The following Detailed Description will first provide some background in the area of the use of inductive learning to produce programs and will then explain how inductive learning techniques are used in the present invention to produce specifications of existing programs.

5.1 Using Inductive Learning to Generate Programs

Experiments have been made in the area of machine learning in reversing the normal process of programming. Normally, the programmer has some idea of the desired results, writes a program to attain the results, tests the program to see whether it attains them, and then repeatedly modifies and tests the program until it does in fact attain them. The idea in the machine learning experiments was to see whether it was possible to give a machine learning system the desired results and have it "learn" a correct program in a first-order abstract, declarative language such as Prolog. Some prior-art techniques for learning Prolog programs are described in J. R. Quinlan "Learning Logical Definitions from Relations", Machine Learning, Vol. 5, No. 3, 1990 and S. Muggleton and C. Feng, "Efficient Induction of Logic Programs", Inductive Logic Programming, Academic Press, 1992.

Machine learning techniques have also been applied to the software engineering tasks of "data reification" and finding "loop invariants", as described in I. Bratko and M. Grobelnik "Inductive learning applied to program construction and verification", Proceedings of the Third International Workshop on Inductive Logic Programming, Bled, Slovenia 1993. Data reification is one of the steps performed in converting an abstract specification of a program to executable code. Loop invariants are logical statements that are true every time a certain point in a program is reached; they are often used in proving the correctness of programs.

A problem with the above learning techniques is that with program results of any complexity whatever, the number of hypotheses the learning system must deal with is enormous. The problem has been addressed by means of a declarative bias which specifies a set of constraints which the hypotheses must satisfy and thereby reduces the number of hypotheses. The declarative bias is provided by the user of the learning system and in effect makes the user's knowledge concerning certain properties of the results available to the learning system. The declarative bias is specified in a declarative language such as a grammar or a set of templates. Techniques for learning Prolog programs with a declarative bias are described in J.-U. Kietz and S. Wrobel "Controlling the Complexity of Learning in Logic Through Syntactic and Task-Oriented Models" Inductive Logic Programming, Academic Press, 1992; and W. W. Cohen "Compiling Knowledge into an Explicit Bias" Proceedings of the Ninth International Conference on Machine Learning, Aberdeen, Scotland, 1992.

In the preferred embodiment of the present invention, the foregoing techniques for obtaining a program from a set of results are applied to the problem of obtaining a specification of a pre-existing program. The method of application is this:

The preexisting program is executed on sets of problem data to obtain traces of the executions; and

the learning system is applied to the traces to produce a specification as described above.

The specification is of course a high-level specification in a first-order declarative language such as Prolog. Great improvements in efficiency can be attained by further providing a declarative bias which reflects the understanding of the system's user about certain aspects of the results produced by the preexisting program.

FIG. 1 provides an overview of a preferred embodiment of software discovery system 101. A software system 103, running on a computers for which a specification is to be obtained is instrumented using instrumentation 105 to produce instrumented software system 107, which, when executed on example problems 109, produces behavior traces 111. In the preferred embodiment, the instrumentation is produced as follows:

print statements are added to the source code of the subroutines that form the interface to the data base system so that all database accesses are traced;

print statements are added to the source code of the routines that generate the view tuples, so that generation of the view tuples is traced; and

an AWK program which is run on the output. of the print statements to convert the output to a form suitable for the machine learning system.

In the preferred embodiment, it is thus the source code which is instrumented. In other embodiments, the instrumentation may be done by running software system 103 in an execution environment such as those provided by debuggers like the well-known gdb debugger or modifying the object code for software system 103.

The behavior traces 111 and user-provided constraints on specifications 113 are provided to learning system 115, which analyzes behavior traces 111 using constraints 113 to produce a generalized description 119 of the behavior of software system 113. System 101 then produces specification 121 from generalized description 101.

In the preferred embodiment, the technique of FIG. 1 is used to produce specifications in Prolog for a set of database views. The Prolog language is described in "The Art of Prolog: Advances Programming Techniques" by Leon Sterling and Ehud Shapiro, MIT Press 1986, and "Programming in Prolog" by W. F. Clocksin and C. S. Mellish, Springer-Verlag 1981. Prolog is well known to those skilled in the arts of artificial intelligence and software engineering; however, other specification languages, for instance relational algebra, could also be used without departing from the spirit and scope of the invention.

The database system for which specifications are to be recovered is a subsystem of a telephone switching system. A large part of this system is a database, which for the purposes of this example can be regarded as a conventional relational database. In such a database, relationships between items of data are expressed by means of tables. Each table is made up of columns and rows (also called tuples. The columns represent classes of values and each row is a related set of values belonging to the classes. There are two kinds of tables in the system: relations, which express the underlying relationships between the data items in the data base, and views, which are tables made by operations which select rows and columns from one or more of the relations.

The database of the example is optimized for retrieval, and hence information is often stored internally in a redundant and unintuitive format. Thus, the end user accesses the database through one or more of the views. The views are essentially "virtual relations"--they can be accessed and updated in much the same manner as actual relations, but are designed to present a simpler interface to the underlying data. For reasons of efficiency each view is implemented as a C-language subroutine that supports the necessary view operations (for instance, inserting, updating, and deleting tuples.) These C subroutines can be quite complex. For instance, in updating a view, it is often necessary to check that certain integrity constraints are maintained. Integrity constraints may affect many database relations and checking them is often non-trivial.

The preferred embodiment automatically constructs Prolog specifications for these views. The specifications describe how the views are derived from the underlying relations, but need not describe all details of the implementation, such as the integrity constraints enforced by the implementation or the implementation's error handling procedures.

As a concrete example of a specification recovery task, consider the relation r 601 of FIG. 6. The relation has two rows 607 and five columns 611. One way in which a user of system 103 can access the information in relation 601 is by view v.sub.1 603, which is a projection of columns 1,3,5 and 4 of relation r. A Prolog specification of this view is shown at 605 of FIG. 6. Specification 605 states that v.sub.1 is produced from r by taking columns A,B,D,and C of r and reordering them as columns A,B,C, and D of v.sub.1.

In the preferred embodiment, the code that implements view v.sub.1 is first instrumented by the user so that when each tuple t of the view is constructed, a trace is made of the tuple t of v.sub.1 and the tuple of r used to construct that tuple t. The trace is shown in FIG. 7, where trace 701(1) contains the first tuple of view v.sub.1 703 and the tuple of r 705 used to construct tuple 703 and trace 701(2) contains the corresponding information for the second tuple of view v.sub.1.

The Prolog specification 121 for view v.sub.1 is then learned from the trace. The technique used to learn Prolog specification 121 works by finding a program P.sub.s such that P.sub.s .LAMBDA.DB.perp-right.x.sub.i for each program trace x.sub.i, where DB is the switch database. P.sub.s must also be a program which agrees with the constraints of constraint specification 113. Program P.sub.s is shown at 803 in FIG. 8.

Program P.sub.s 803 is arrived at by means of program P.sub.t 801, which corresponds to generalized description 119 of FIG. 1. Program P.sub.t 801 in FIG. 8 is generated by learning system 115 from trace 701 of the present example. From program P.sub.t 801, it can be determined that the prolog program P.sub.s 803 is a specification of v.sub.1 which is consistent with traces 701. P.sub.s is produced from P.sub.t by a simple syntactic transformation in which the head of P.sub.s is simply the first argument of the head of P.sub.t, and the body of P.sub.s is the body of P.sub.t.

Prolog learning technique for finding P.sub.t 801 will now be described in detail. The technique is able to efficiently finds a program P.sub.t 801 which generalizes traces 701 and to ensure that this program P.sub.t can be easily converted to a valid specification P.sub.s 803. The learning system is further constrained to output only hypothesis programs that satisfy constraints 113. In the preferred embodiment, the user specifies constraints 113 by providing an "augmented antecedent description grammar". Augmented antecedent description grammars are an extension of antecedent description grammars, which are described in W. W. Cohen "Compiling Knowledge into an Explicit Bias" Proceedings of the Ninth International Conference on Machine Learning, Aberdeen, Scotland, 1992. An augmented antecedent description grammar is a series of rules of the form

Each rule specifies that the symbol s can be rewritten as the series of symbols t.sub.1, . . . , t.sub.k, provided that conditions c.sub.1, . . . , c.sub.n are true. If any of the t.sub.i 's or c.sub.j 's contain variables (which are in this discussion indicated by words beginning with uppercase letters) then s must be rewritten as t'.sub.1, . . . , t'.sub.k where t'.sub.i is the result of replacing the variables in t.sub.i with values that also make each c.sub.j true. For instance, the rule

would allow replacing the symbol "s(2)" with the sequence "t(add1(2)),u(2)".

Any symbol that cannot be rewritten using any of the rules in a grammar is called a "terminal symbol" of the grammar. One particular symbol in the grammar is designated as the "start symbol" of the grammar. A "sentence" of a grammar is a sequence of the terminal symbols that can be derived from the start symbol by performing rewriting steps in accordance with the rules of the grammar. An augmented antecedent description grammar has a start symbol of the form "body(A)" where "A" is some possible head of a Prolog clause. Any specification of the program being learned must be a Prolog program consisting of a single clause

such that the sequence of symbols B.sub.1, . . . , B.sub.k is a sentence of the grammar, and consequently, learning system 115 can ignore any possible specification for which the sequence of symbols is not a sentence of the grammar.

As a concrete example, consider the augmented antecedent description grammar 901 shown in FIG. 9. The first rule says that the start symbol "body(View,[Rel])" can be always be rewritten as the symbol "projection(View,Rel)". The second rule says that "projection(View,Rel)" can be rewritten as the sequence of symbols "Rel,give.sub.-- values(ViewCols,RelCols)" wherever the conditions "declared.sub.-- view(View,ViewCols)" and "declared.sub.-- relation(Rel,RelCols)" are true. For instance, if the user has recorded as true the facts

declared.sub.-- view(v.sub.1 (R,S,T,U),[R,S,T,U]).

declared.sub.-- relation(r(V,W,X,Y,Z),[V,W,X,Y,Z]).

(representing the declarations that v.sub.1 is the view for which a specification is to be derived, that r is a relation in the switch database, and specifying that the lists of arguments for the predicates representing v.sub.1 and r should be [R,S,T,U] and [V,W,X,Y,Z] respectively) then the symbol "body(A)" can be rewritten as

r(V,W,X,Y,Z),give.sub.-- values([R,S,T,U],(V,W,X,Y,Z])

If the following additional grammar rules are present

give.sub.-- values([X.vertline.[ ]], Ys).fwdarw.give.sub.-- value(X, Ys) where true.

give.sub.-- values([X.vertline.Xs], Ys).fwdarw.give.sub.-- value(X, Ys),give.sub.-- values(Xs, Ys) where Xs.noteq.[ ].

give.sub.-- value(X, Ys).fwdarw.X=Y where member(Y, Ys).

then "body(A)" can be rewritten to any of the following sentences: ##EQU1##

Thus the learner could hypothesize any of these Prolog clauses: ##EQU2##

Hence the grammar rules above define the set of all views that construct some projection of the columns in the relation r. In an appropriate context, all of these hypotheses might be considered plausible a priori. Also, all of these clauses can be converted easily to a specification of v.sub.1. All that needs to be done is to replace the head of the clause with its first argument: for example the clause

trace(v.sub.1 (R,S,T,U),[r(V,W,X,Y,Z)]).rarw.r(V,W,X,Y,Z),R=V,S=W,T=X,U=Y.

can be converted to

v.sub.1 (R,S,T,U).rarw.r(V,W,X,Y,Z),R=V,S=W,T=X,U=Y.

The constraints specified by the augmented antecedent description grammar of the preferred embodiment include two that reflect special properties of the database system to which the system is applied. The first of these constraints is that any hypothesis must access tuples in the data base in an order which is consistent with the trace of each view tuple. The second is that any hypothesis must be composed of generative clauses, that is, when v(X.sub.1, . . . , X.sub.n) is the head of a clause in a view specification, each X.sub.i always appears somewhere in the body of the clause. In other words, each hypothesis must use only relations known from the examples to be relevant. For instance, the specifications derived from the clauses generated by the ADG of the example above contain only the variables R,S,T,U in their heads, and all of these variables must also occur in every clause body as well. Thus the specifications produced by the above ADG are generative.

To efficiently search through this space of possible hypotheses for one that is consistent with the examples the following technique is used. Consider the graph of all sequences of symbols that can be derived by the grammar in which an edge connects sequences s.sub.1 and s.sub.2 if and only if s.sub.2 can be derived from s.sub.1 by applying a single grammar rule to the leftmost non-terminal symbol in s.sub.1. This graph is traversed in a depth-first manner, except that whenever any sequence s.sub.2 is reached containing a terminal symbol that is not contained in the parent sequence s.sub.1, a test is made to see whether the clause

is consistent with the examples, where t.sub.1, . . . , t.sub.k are the terminal symbols in s.sub.2. If this test fails, then the portion of the graph below this point is not searched. (Eliminating portions of a search space in this way is called "pruning" the search space.) If this test succeeds and s.sub.2 is a sentence, then s.sub.2 is returned as a consistent hypothesis. A description of this search algorithm is given in FIG. 3.

The graph and the search procedure is illustrated in FIG. 2, which shows the portions of graph 203 making up an initial portion of search space 201 imposed by the grammar above. For the examples given above, the parts of the graph below the boxed nodes in the figure would not be searched by the search strategy. The portion of the graph below the node labeled a will be pruned because the condition S=V is not true for the examples. Likewise the portions of the graph below the nodes labeled b, c and d will be pruned because the conditions R=W, R=Z, and S=Z do not hold.

One disadvantage of this search procedure is that it returns only a single specification. For the purpose of better understanding the software, it may be helpful to know if there is more than one plausible specification that is consistent with the traces. At the user's discretion, the discovery system of the invention will employ an alternative search procedure that finds all specifications consistent with both the examples and the user's grammar. As another motivation for considering this problem, it is sometimes the case that (for reasons of efficiency data is stored redundantly in a database. FIG. 10 shows an example of such a case. In relation "n" 1003, there is an integrity constraint that requires the "login" field of relation "n" to be identical to the "lname" field: Thus, the specifications 1005 and 1007 of FIG. 10 are both equally valid specifications of view v.sub.2 1001.

To find sets of consistent specifications, another depth-first search of the same graph is used. A description of this search algorithm 401 is given in FIG. 4. This search procedure uses the same rules for pruning away parts of the graph; however, the search does not stop when the first consistent hypothesis is found. Instead, at each nonleaf node in the graph, all child nodes are searched, and the union of the sets of consistent hypotheses found at any child node are returned.

To keep the set of hypotheses from growing too large, a special compact representation is used for unions on hypotheses. Notice that at the leaf nodes in the graph, the only sets that must be represented are singleton sets. At the next-highest level of the graph, the only sets that must be unioned together are sets of sentences that were derived from a string .alpha.N.beta. by rewriting the first nonterminal symbol N to a string of terminal symbols. That is, the only unions that must be represented are of the form

where the .tau..sub.i 's are sequences of terminal symbols. Such sets can also be represented by an expression of the form

More generally, one can show by mathematical induction that at any level i steps away from the leaves, the only unions that must be represented are of the form

where .alpha., .beta., and all of the .gamma..sub.i 's can be represented by an expression containing ".times." symbols, as shown above. Thus one can always represent sets compactly with an expression of the form

where .alpha., .beta., and the .gamma..sub.i 's are themselves expressions for sets, written in this compact notation.

An a simple example of the operation of this procedure, suppose that one has the grammar

body(a).fwdarw.B,C

B.fwdarw.b.sub.1.

B.fwdarw.b.sub.1.

B.fwdarw.b.sub.3.

C.fwdarw.c.sub.1.

C.fwdarw.c.sub.2.

and the examples are consistent with all of the following clauses:

a.rarw.b.sub.1,c.sub.1.

a.rarw.b.sub.3,c.sub.1.

a.rarw.b.sub.1,c.sub.2.

a.rarw.b.sub.3,c.sub.2.

The search graph for the above example appears as graph 501 in FIG. 5 and the sets RET returned by algorithm 401 at each node in graph 501 would be as follows:

Node 1: b.sub.1,c.sub.1

Node 2: b.sub.1,c.sub.2

Node 3: b.sub.1,(c.sub.1 .times.c.sub.2)

Node 4: .0.

Node 5: b.sub.3,c.sub.1

Node 6: b.sub.3,c.sub.2

Node 7: b.sub.3,(c.sub.1 .times.c.sub.2)

Node 8: (b.sub.1 .times.b.sub.3),(c.sub.1 .times.c.sub.2)

Node 9: (b.sub.1 .times.b.sub.3),(c.sub.1 .times.c.sub.2)

The final output to the user would be the set of clauses listed above as being consistent with the data, expressed in this compact form:

Another shortcoming of the implementation described above is that it cannot deal with situations where data is not simply copies from a relation to a view, but rather undergoes a conversion step. For example, in the specification below, view v.sub.6 is a copy of q with every value of "0" in the third column replaced by the empty string.

v.sub.6 (VProblemCode, VSeverity, VHelpBufId).rarw.

RPCode=VProblemCode,

q(RPCode, RLevel, RTextBufld),

VSeverity=RLevel,

zero2nullstr(RTextBufId, VHelpBufId).

zero2nullstr(0,").

zero2nullstr(Id,Id).rarw.Id.noteq.0.

Learning views with conversion functions like zero2nullstr is very difficult since a wide range of conversions are performed. We addressed this problem by extending the implementation to generate view specifications that contain determinations. A determination between two variables X and Y is denoted YX, and indicates that the value of Y can be functionally derived from the value of X. Using determinations, v.sub.6 could be specified

v.sub.6 (VProbleraCode, VSeverity, VHelpBufH).rarw.

RPCode=VProblemCode,

q(RPCode,RLevel,RTextBufId),

VSeverity=RLevel,

VHelpBufIdRTextBufId.

Specifications with determinations are even more abstract than the Prolog specifications generated by the preferred embodiment; in particular, they cannot be used to find all tuples in a view. However, they are still useful for program understanding.

The learning system of the preferred embodiment requires almost no changes to incorporate determinations. The only difference is that in the pruning phase, the test to see if a clause

A: -t.sub.1, . . . , t.sub.k

(where the t.sub.i 's are terminal symbols) is a bit more complex. A terminal symbol "XY" is considered consistent *unless* there are two examples in which (X, Y) are bound to (x1,y1) and (x2, y2) respectively where y1=y2 but x1 and x2 are different. Put another way, the symbol "XY" is consistent whenever it is consistent to believe that X can be functionally derived from Y given the data.

5.2 Results of Application of the Techniques

As earlier indicated, the foregoing techniques have been used to obtain specifications of views in a database belonging to a telephone switching system. In that application, specifications were extracted for a number of views, including views that contain many (up to 92) columns, views that depend on relations that contain many (up to 90) columns, and views that depend on more than one relation (for example, views that are a projection of the join of two relations, rather than the projection of a single relation.) The invention has also been applied to views that have large implementations, consisting of up to 2900 lines of C source code.

An important feature of the foregoing techniques is the following: while system 101 cannot return a specification 121 for every program it analyzes, the specifications 121 which it does return will be correct. For that reason, system 101 can be usefully employed to make a first analysis of a set of programs, while other tools can be applied to those programs for which system 101 does not return specifications.

The discovery system of the invention can be implemented on a digital computer. Such a digital computer is programmed to accept input in the form of an instrumented software system, a set of example problems for the software system, an augmented antecedent description grammar, and then to carry out the method described above. Creation of such a program is well within the capability of one skilled in the art.

5.3 Conclusion

The foregoing Detailed Description has disclosed to those skilled in the arts of machine learning and software discovery how machine learning techniques may be applied to traces generated by executions of a pre-existing program to obtain a specification of the pre-existing program. While the techniques disclosed herein are the best presently known to the inventor, other learning techniques could also be used without departing from the spirit and scope of the invention. Other search methods could be employed to search the space defined by a declarative bias: for instance, the search method for antecedent description grammars described in W. W. Cohen "Compiling Knowledge into an Explicit Bias" Proceedings of the Ninth International Conference on Machine Learning, Aberdeen, Scotland, 1992. Also, learning techniques employing different "learning protocols" could be employed. Moreover, learning techniques are known that do not require a declarative bias, but instead constrain the hypotheses of the learner by negative examples which are generated automatically from positive examples, together with information about the completeness of the set of positive examples, and such techniques could be employed in the invention. One such techniques is described in J. R. Quinlan "Learning Logical Definitions from Relations", Machine Learning, Vol. 5, No. 3, 1990. Further, learning techniques are known that enrich the set of examples provided by posing "queries" about the system being learned; one such technique is described in "Algorithmic Program Debugging" by Ehud Shapiro, MIT Press, 1982. The dashed portions of FIG. 1 indicate how such queries could be incorporated into the invention. Employing either of these sorts of learning methods would be consistent with the spirit and scope of the invention. It will also be clear to those skilled in the art that specification languages other than Prolog could be employed; that other classes of specification recovery problems could be considered, for example recovering specifications of communications protocols, could also be considered, again without departing from the spirit and scope of the invention; and that other learning techniques could be employed to carry our the method of the invention.

That being the case, the Detailed Description is to be understood as being in all respects illustrative and exemplary, but not restrictive, and the scope of the invention is not to be defined by the Detailed Description, but rather by the claims as interpreted with the full breadth permitted by the law.