Quick difference and update for tree structure data

[0002] 1. Field of the Invention

[0003] The present invention relates to an efficient mechanism for the differentiation and update of data structured in tree format. The invention has particular application to version management of tree structured data but the tree differentiation process according to the invention can differentiate any two trees, regardless of whether they are successive versions or not.

[0004] 2. Background Description

[0005] The information used by computer programs can be represented in multiple formats. There is, however, a growing trend towards moving much of these data into a standardized tree structured for-mat. The XML (eXtensible Markup Language) and DOM (Document Object Model) standard proposals are two leading efforts in this trend. The prospect of having a preferred data representation format raises the question of the adequacy of the existing mechanisms of data management.

[0006] Consider the case of information (source code, textual data, etc.) that varies over time. Version management is then a critical issue. At the core of the version management issue lies the following problem: given two successive states of the information, it is necessary to be able to describe the informational difference between these two versions, and to bring older versions up to date with newer ones.

[0007] In the current approach, version management is achieved by managing external representations of the data, rather than the data itself. The information is first converted into some conventional representation, typically a sequence of text lines or bytes, which is then processed. The internal structure of the data is irrelevant in this model; rather, all information is treated as a more or less unstructured sequence of tokens.

[0008] While sufficient for many computational purposes, this model has two major drawbacks. The first one arises from the intrinsic ambiguity of the conversion to an external format: the same data can have multiple external representations in a given format. As a result irrelevant difference reports (“false negatives”) are often generated.

[0009] Second, and most important, the way differences are reported (e.g., in terms of byte or line mismatches) bears no relation with the intrinsic structure of the data, and requires an additional “interpretative” step to infer the actual informational difference.

[0010] It would seem then that there are big advantages in doing version management directly on the internal representation of the data. When tree structured data is considered, however, there is one major obstacle: the high computational cost of the tree differentiation algorithms. As an indication of this, consider the cost of the optimal tree differencing algorithm: for labeled ordered trees, the cost is no less than

[0011] Nodes(tree 1)×leaves(tree 1)×Nodes(tree 2)×leaves(tree 2), that is, the cost increases at least quadratically when the size of the tree increases. For unordered labeled trees the situation is still worse. Optimal differentiation has exponential cost in the worst case,

[0012] Nodes(tree 1)×Nodes(tree 2)+[leaves(tree 1)]!×3**[leaves(tree 1)1×Nodes(tree 2)×. . .

[0013] The consequence is that tree differencing algorithms become impractical for most realistic problems as soon as the size of the trees involved starts to grow, unless large amounts of time and computing power are available.

[0014] It is therefore an object of the present invention to provide an efficient mechanism for the differentiation and update of data structured in tree format.

[0015] According to the invention, there is provided an efficient differencing and update mechanisms that operates directly on the native representation of the data. No irrelevant differences are introduced by the process. The difference reports generated are stated in terms of the basic operations that can be performed on a data tree, thus requiring no additional interpretation, and making it amenable to direct human inspection and understanding. This mechanism can become a central part of any version management system.

[0016] The foregoing and other objects, aspects and advantages will be better understood from the following detailed description of a preferred embodiment of the invention with reference to the drawings, in which:

[0017] FIG. 1 is a simplified representation of the process of editing tree structured data by means of elementary tree editing operations;

[0018] FIG. 2 is a flow diagram representing the logic of the fast tree differencing process according to the invention; and

[0019] FIG. 3 is a flow diagram representing the logic of the tree update process according to the invention.

[0020] Referring now to the drawings, and more particularly to FIG. 1, there is shown by way of example a graphical representation of editing a tree structure of the information database using elementary tree editing operations, like “change”, “delete”, or “insert”. More particularly, the tree structure on the left, denoted T1, is modified by three editing operations to produce the tree on the right, denoted T2. In FIG. 1, a “delete” operation is shown removing leaf C from T1, a “change” operation is shown modifying the information (label) of node D in T1 to become node Z in T2, and an “insert” operation is shown inserting the new node H between node E and the leaf F. Herein, a sequence of editing operations that transforms a tree T1 to a tree T2 will be called an “edit path” between T1 and T2.

[0021] The processes illustrated in FIG. 1 are editing functions well understood in the data processing arts for editing tree structured data. The problem in maintaining such a tree structured database is how to efficiently differentiate and update successive versions of the data.

[0022] The invention improves the difference and update process by working directly on the native tree representation of the data, and expressing the differences between the trees in terms of basic “tree edit” operations (like “delete”, “change”, or “insert”). In turn, this requires addressing the problem of the high computational cost of tree differentiation.

[0023] To solve this problem, a two step tree differencing process is introduced. The first step is a fast sub-tree matching procedure that uses a digital finger print (also called message digest or digest value herein) to detect identical sub-trees. See, for example, R. Rivest, “RFC 1321 The MD5 Message-Digest Algorthim”, MIT Laboratory for Computer Science and RSA Data Security, Inc.,April 1992, and H. Maruyama, K. Tamura and R. Uramoto, “Digest Values in DOM (DOMHASH) Proposal”, June 1998. This step reduces the size of the input trees, and with it the computational cost of the next step. The second step finds the difference between the simplified trees by computing the shortest edit path between them; that is, the least expensive sequence of tree editing operations required to transform one tree into another. A standard tree editing distance algorithm is used for this purpose. See, for example, K. Zhang and D. Shasha, “Simple Fast Algorithm for the Editing Distance Between Trees and Related Problems”, SIAM J. Comput., Vol. 18, No. 6, pp. 1245-1262, December 1989. The difference is represented as a sequence of tree editing operations which the update process used to transform an old version of the data into the new one.

[0024] FIG. 2 shows the flow diagram for the fast tree differencing process according to the invention. The inputs to the process are two rooted tree structures, T1, T2. The output of the process is a sequence of tree edit operations E which edits T1 into T2.

[0025] Each node of the two input trees is assigned a digest value that identifies the sub-tree rooted at the node. The digest value of a node is defined in such a way as to uniquely identify sub-trees within an extremely high confidence level (see the DOMHASH proposal, supra). This is represented in FIG. 2 by function block 201. In function block 202, all the nodes in tree T1 are added to a fast look up table where the digest values are used as keys.

[0026] Next, a fast matching process in blocks 203 to 205 finds unique matches of digest values between nodes in trees T1 and T2. The digest value of each node in tree T2 is looked up in the table to find if there is a unique match with a node in T1, as shown in function block 204 and decision block 205. If a unique match is found, the sub-trees rooted at the matching nodes are removed from the original trees and replaced by two identical leaf nodes in function block 206. Information about the replaced branches of the tree is stored in a “reduction map” in function block 207 for later use in the second phase of the algorithm. The process then loops back to decision block 203 in order to complete the processing of all the nodes in tree T2.

[0027] When all the nodes in tree T2 have been processed, the reduction process is complete. This phase produces two simplified sub-trees T1′ and T2′. If the input trees T1 and T2 are unordered trees, an additional simplification is carried out by ordering the simplified trees T1′ and T2′ according to a given canonical ordering. This involves imposing a predefined ordering among the children of every node. It permits treating unordered trees as ordered trees and thereby apply more efficient difference algorithms. This is shown in decision block 208 and function block 209.

[0028] A tree editing distance algorithm (like the one referenced above by Zhou and Shasha) finds the differences between the two trees (the edit path between them) in function block 210. Tree distance algorithms require the use of a “cost function” that provides the cost of each possible editing operation. The cost function used by the algorithm in function block 210 is dynamically generated based on the reduction map information stored in the previous phase of the process. In this way, the editing cost of individual operations in the simplified tree is kept consistent with that of the original tree. The difference between the trees is output as a sequence of tree edit operations E′, which may include operations like “change”, “delete”, “insert”, or others. Observe that E′ edits T1′ and T2′. The reduction map information is used in function block 211 to obtain the sequence E, which edits T1 and T2. Additionally, the same information from function block 206 can be used to restore trees T1′ and T2′ to their original states T1 and T2 in function block 212.

[0029] FIG. 3 is a flow diagram of the update process according to the invention. The input to the process is the source tree structure T1 and a sequence of edit operations E. The output of the process is the target tree T2.

[0030] The sequence of editing operations E is used to modify tree T1. The update process consists of two separate steps. In the first step, the sequence of edit operations is scanned to compile a list of all the nodes of the target that are not present in T1. This is shown in FIG. 3 in blocks 301 and 302, where a new operation is selected from E if possible, and in decision block 303, where the operation is checked to determine if a new node has to be added to tree T1. In that case, the required new node is added to T1 in function block 304. Then the process loops back to decision block 301 unit the complete sequence of operations E has been inspected.

[0031] The result of step one is in general a modified tree T1′. In step two, each operation from E is applied to the extended tree T1′; e.g., nodes may be deleted, have their information changed or be reattached to a different place in the tree. Hence, in blocks 305 and 306, the sequence of operations E is traversed once more, and each operation is applied to the extended tree T1′ in function block 307. Once all operations have been processed, the resulting tree T2 is output.

[0032] While the invention has been described in terms of a single preferred embodiment, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the appended claims.