Method of Removing Unneeded Data From DB2 Logs and Other Data Sets Having Displacement-Dependent Data

The invention relates to a method for removing unneeded data in DB2 log data sets. With the benefit of this disclosure, those of ordinary skill will appreciate that the method has a broader applicability: it can likewise be used to remove unneeded data in other data sets that use a relative byte address (RBA) identifier for records or other data items, and from which the unneeded data would thus normally be unremovable.

FIG. 1 illustrates the context of the invention using a greatly simplified view of a hypothetical DB2 log record:

In many cases, a great deal of information in the data portion 115 of the DB2 log record 105 is unneeded for data recovery, yet takes up considerable storage space in the storage medium 100. Because the data portion 115 of a record 105 is identified only by the relative byte address of the record 105, this unneeded data cannot be removed from the log record without compromising the integrity of the RBAs of data that follows the removed data.

In an electronic, magnetic, optical, or similar data structure such as a DB2 log data set that conforms to the IBM MVS data set format requirements (referred to simply as a "data set") and that contains displacement-dependent data, e.g., data that is identified by relative byte address, data that is determined to be unneeded or otherwise removable is overwritten with a compressible binary pattern such as a sequence of binary zeros. The resulting substitute data set can be compressed to occupy less storage space and require less time for data recovery.

FIG. 1 is a block-diagram view of a hypothetical DB2 log data set.

FIG. 2 is a flow chart of a method in accordance with the invention.

FIG. 3 is an expanded-view flow chart of specific method steps in the method shown in FIG. 2.

FIG. 4 is a block-diagram view of a substitute data set resulting from processing the DB2 log data set using the method illustrated in FIGS. 2 and 3.

One illustrative embodiment of the invention is described below as it might be implemented using a computer program to control a general-purpose programmable computer. In the interest of clarity, not all features of an actual implementation are described in this specification. It will of course be appreciated that in the development of any such actual implementation (as in any software development project), numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system- and business-related constraints, which will vary from one implementation to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking of software engineering for those of ordinary skill having the benefit of this disclosure.

4.1 Overview

FIG. 2 shows a high-level overview of an improved method, executed by a computer program, for removing selected data from a DB2 log data set or other data set in which relative byte address is used to identify data items:

The method illustrated in FIG. 2 can be performed on an arbitrary DB2 log data set. It can, for example, be invoked just as DB2 has completed an archive-log process to archive an active log data set, so that a compressed substitute data set is stored in place of the original archived data set.

4.2 Identifying and Removing Removable Data

FIG. 3 shows, as part of the step 205 of overwriting unneeded data, an illustrative method of identifying records 105 containing data 115 that can be removed from the original data set 102. Again, for illustrative purposes the record 105 is assumed to be a DB2 log record, in a DB2 log data set 102 that has just been archived by DB2.

A list of some record types and subtypes that normally can be removed safely is set out in Table 1 below; other types and subtypes may also be removable in particular applications:

4.3 Decompressing Compressed Data Sets

In the special case of DB2 data sets, it will be apparent to those of ordinary skill having the benefit of this disclosure that if DB2 issues a DASD read-type request specifying a data record 105 at a particular relative byte address, the substitute data set 402 must be decompressed if it was compressed prior to be stored. In one embodiment, the entire data set can be decompressed and then recompressed when the data request from DB2 has been fulfilled. Alternatively, a table of relative byte addresses can be maintained. That table may show, for a given record 105/405, both the record's RBA in the original data set 102 and its RBA in the substitute data set 402. A data request from DB2 can then be fulfilled by retrieving and decompressing only the needed data record 405 and not the entire compressed substitute data set 402.

4.4 Some Advantages of the Invention

Those of ordinary skill having the benefit of this disclosure will recognize that compression of a DB2 log data set provides provides significant performance benefits. Less storage space is required to maintain the log data set. Moreover, recovery of data from a log data set is considerably speeded up because unneeded data need not be read to obtain the desired data; this benefit is especially significant when log data sets are archived to magnetic tape (which must be fast-forwarded or rewound to obtain desired data) instead of disk storage or other direct-access storage.

It will be appreciated by those of ordinary skill having the benefit of this disclosure that numerous variations from the foregoing illustration will be possible without departing from the inventive concept described herein. Accordingly, it is the claims set forth below, and not merely the foregoing illustration, which are intended to define the exclusive rights claimed in this application program.