Method for database assisted file system restore

[0002] 1. Field of Invention

[0003] The present invention relates generally to the field of computer information archiving and backup recovery. More specifically, the present invention is related to the control of a filesystem recovery process by a database management system. The method of this invention has particular application in restoring a filesystem and files to any point in time in the event of a filesystem crash.

[0004] 2. Discussion of Prior Art

[0005] Data archiving and restoration have always been important to critical computer applications and data. The prior art in this area describes many different approaches to improving speed, reliability, efficiency, and storage requirements of archival solutions. Independently, database management systems (DBMS) have undergone their own development. Aside from the various models such as relational, hierarchical and object-oriented, database management systems have evolved to manage visual, binary and even distributed data. The application of DBMS principles to filesystem archival and recovery, however, has not been adequately addressed as indicated in the following discussion of related prior art.

[0006] The patent to Elliott et al. (4,945,474) provides for a database transaction logging method which allows operational recovery after an I/O error. The disclosed restoration is limited to a database rather than restoring the underlying filesystem.

[0007] The patent to Combs et al. (5,577,220) describes a mechanism for taking a snapshot of a CPU state and restoring to that state if needed. The system described can only be restored to the snapshotted state rather than any previous state.

[0008] The patent to Hanes (5,754,848) provides for a disaster recovery system which first restores a traditional filesystem (i.e 8.3 filenames) which has libraries to allow subsequent recovery using long filenames. The described method relates to translating between filenaming conventions rather than maintaining a recoverable history of filesystem operations.

[0009] The patent to Masada (5,754,782) provides for a document backup and restoration system in a GroupWare environment. However, no discussion of combining both a database restoration and a filesystem recovery is provided.

[0010] The patent to Whiting et al. (5,778,395) provides for a space reducing data archiving system which stores only file changes after an initial filesystem snapshot is stored, but fails to discuss synchronizing a particular version with an accompanying database state.

[0011] The patent to Luick (5,793,944) describes a hardware method of data storing and restoring upon the occurrence of a variety of errors and is limited to teaching a method of capturing various hardware states rather than capturing filesystem directory structure states.

[0012] The patent to Bailey et al. (5,794,252) provides for a database transaction logging system to assist with database duplication. The logging system discussed, however, fails to consider the maintenance of filesystem directory transactions.

[0013] The patent to Morris (5,813,017) provides for a distributed data backup system which reduces transmission bandwidth and storage requirements by archiving files as delta (i.e changed) files across a network. However, no teaching is provided which synchronizes a retrieved delta version with a desired database restoration state.

[0014] The IBM Technical Disclosure Bulletin entitled “Table Relocation Alternative For Restore” describes an interactive method of restoring backed-up data to a filesystem having a different structure than that of the filesystem from which it was originally archived; although, no provision for restoring the filesystem is discussed.

[0015] The IBM Technical Disclosure Bulletin entitled “Method to Ensure the Integrity of File Operations in a Database Transaction” teaches a COMMIT and ROLLBACK algorithm to ensure database transactions do not leave a filesystem in an inconsistent state. However, relating the consistent database to a previous directory structure is not taught.

[0016] The IBM Technical Disclosure Bulletin entitled “Changed Data Only Backup and Recovery” details a multi-version backup system which archives only changes to files which have previously been archived but fails to discuss how to restore files to an underlying filesystem directory structure which might also have changed.

[0017] Whatever the precise merits, features and advantages of the above cited references, none of them achieve or fulfill, individually or in combination, the purposes of the present invention. They fail to provide for a database managed table which logs filesystem directory operations and without such a log, the prior art fails to teach a method of restoring a filesystem directory structure to a previous known state. Furthermore, without a point-in-time directory recovery method, the above mentioned prior art fails to provide for a method of synchronizing a database restore to a filesystem directory structure restore and file restore.

[0018] A database management system (DBMS) is used to maintain a record of a filesystem's directory structure (registry database). The creation, removal or update of a directory results in a record, pertaining to that event, being stored into a database table of similar events. In addition, files are also linked to a DBMS (user database) and archived according to its management rules (i.e. DBMS references and/or controls the linked files). In the event of a filesystem crash, the DBMSs are used to restore the filesystem through the use of the directory structure database table which allows reconstruction of the filesystem to any previous filesystem state. Furthermore, files can then be recovered to match that state and thereby reconcile external files linked to a database to any previous state.

[0019] While described as two separate databases, implementing the registry database and the user database as a single database is also considered within the scope of the present invention. The registry database and the user database are logical entities and mapping them to the same physical database is functionally equivalent to mapping them to two different physical databases. One difference worth mentioning, however, is that the registry database is created by the system and is not normally visible to users, while the user database is created by the user. Throughout this specification, DBMS is used when generally referring to these databases regardless of their actual physical mapping.

[0020] FIG. 1 illustrates the table structure of the present invention for storing directory operations information.

[0021] FIG. 2 illustrates a flowchart for the algorithm of the present invention which restores an entire filesystem's directory structure.

[0022] FIGS. 3a-3gcollectively illustrate the progression of a filesystem's recovery during reconstruction performed according to the present invention.

[0023] FIG. 4 illustrates a flowchart for an algorithm of the present invention which restores a single subdirectory structure.

[0024] FIG. 5 illustrates a flowchart for an algorithm of the present invention which synchronizes a file recovery operation with a filesystem directory restoration.

[0025] While this invention is illustrated and described in a preferred embodiment, the device may be produced in many different configurations, forms and materials. There is depicted in the drawings, and will herein be described in detail, a preferred embodiment of the invention, with the understanding that the present disclosure is to be considered as a exemplification of the principles of the invention and the associated functional specifications of the materials for its construction and is not intended to limit the invention to the embodiment illustrated. Those skilled in the art will envision many other possible variations within the scope of the present invention.

[0026] Relational database management systems (RDBMSs) have proven to be a popular and powerful model for managing large amounts of diverse data. Recently, object-oriented capabilities have been added to traditional RDBMS systems in an effort to extend the data being managed. One area that has received a lot of attention is the integration between an RDBMS and the filesystem. An example of such efforts is the extension of IBM's® DB2® UDB® product to include DataLinks®. This technology enables an organization to continue storing data in the filesystem to take advantage of filesystem capabilities, while at the same time coordinating the management of these files and their contents with associated data in an RDBMS. These benefits are especially pronounced with particularly large files of unstructured or semi-structured data such as documents, images, video clips, and engineering drawings.

[0027] An RDBMS has become indispensable for managing traditional business data with integrity, security, and reliability but historically has not been as effective with the unstructured data identified above. These types of data (e.g. video, images, etc.) remain outside the DBMS in a filesystem, but are often related in some way to traditional data stored in the DBMS. A very simple example is that of a product's picture (a digital image file) which is related to an inventory database maintained by a manufacturer which continually needs updating with regard to pricing, inventory, part number, or design specifications.

[0028] There are many reasons to leave the unstructured data in a filesystem rather than bring it into the formal DBMS environment. In particular, RDBMSs do not know how to deliver isochronous data effectively, as in streamed video or audio. RDBMSs with their inherent overhead (e.g. parser, optimizer, etc.) have difficulty matching the performance of the filesystem in accessing files. Also, by keeping files in the filesystem, they can be located close to an application thereby optimizing performance by minimizing network traffic. Furthermore, existing applications that directly access operating systems files can continue to be used without modification.

[0029] Filesystems, however, do not provide the same comprehensive data management facilities as a DBMS, such as transaction control, enforcement of integrity constraints, and backup and recovery strategies. DataLinks® and other similar technology enables DBMSs, particularly DB2® UDB®, to manage data stored in external operating system files as if the data were stored directly in the database.

[0030] One particularly important feature of DataLinks® is that it automatically generates a backup of any external file as soon as it is linked to the database. Further, through its DataLinks® file manager, different versions of a file are tracked and backed-up to support point-in-time recovery of that file. In the event of a database crash, other database error, or data roll-back, the DBMS and DataLinks® file manager accomplishes a database recovery by using a transaction log and linked file information to synchronize files containing referenced data to the correct point of database consistency.

[0031] The present invention expands the above capabilities to handle the much more catastrophic event of a filesystem crash. In the event of a filesystem crash, the underlying operating system files and directory structures are lost and must be recovered before any attempt is made at restoring a database. Traditionally, filesystems were archived using either full or incremental strategies. Typically, each night, an entire filesystem was archived to tape or other storage media. Alternatively, each night, only those files which had been modified since the last full backup were archived to a tape. In the event of a filesystem crash, the archived tapes were then used to restore the filesystem to the state of the last backup.

[0032] In general, there are two major shortcomings to this strategy. First, any changes made to the filesystem subsequent to the time of the backup are lost; and second, restoring the filesystem to the state of an arbitrary point in time is not possible. In particular to filesystems under DBMS control, additional weaknesses become evident. The traditional backup and restore method severely limits which database backup and restore operations can be performed. In particular, only those database operations consistent with the underlying filesystem directory structure can be accomplished.

[0033] In order to support the capability of recovering from a filesystem crash, a database table is created to maintain information about the directory structure of the filesystem. The described embodiment assumes a hierarchical directory structure but other equivalent directory structure's are also capable of being tracked.

[0034] In the following discussion, the term “directory” is used to refer to both filesystem directories and subdirectories. If some differentiation is needed, terms like “parent” or “child” are used to further distinguish “directory”.

[0035] In the present invention, when a directory is created, the database table receives an entry describing the new directory and when a directory is deleted from a filesystem, the corresponding entry in the table is marked “deleted” and time stamped. When a filesystem crashes, the database table is used to rebuild the filesystem directory structure. After the directory structure is rebuilt appropriate files are then restored.

[0036] The various known methods of detecting filesystem directory-modifying operations are not part of the present invention. Instruction trapping, library replacement and modification, and other functionally-equivalent, industry-standard methods to detect when a directory-modifying command is performed, are all contemplated by the present invention.

[0037] FIG. 1 illustrates an exemplary database table which tracks filesystem operations. Each entry occupies a row of the table and details an element in a filesystem directory structure. Note that the operation and recording of it is assumed to be atomic. The columns identify the information needed by a recovery process to restore a directory structure to any previous point in time.

[0038] FSID 102 is a filesystem identifier that uniquely identifies a filesystem which can be locally present or distributedly attached.

[0039] INODE 104 is an identifier internal to a particular filesystem which keeps track of the location of a directory. When a filesystem is first created under operating system control, one of the areas allocated to management information rather than data storage is an INODE list. The INODE list contains an array of INODEs which themselves contain important information about an associated file. File ownership, file access permissions and disk address are some examples of information in an INODE. Furthermore, an INODE is referred to by its INODE number—where in the INODE list it occurs. The INODE number of the root directory of a filesystem is assumed to be 0 in the current invention. However, referring to root directories using numbers other than 0 are also contemplated to be within the scope of the present invention. In each filesystem, the operating system maintains a directory information table which relates directory entries (e.g. files, directories, symbolic links) with INODE numbers.

[0040] Parent INODE 106 is the INODE number of each entry's parent directory.

[0041] Directory Name 108 is the character string for the directory represented by a particular entry.

[0042] Status 110 is a flag to keep track of whether a directory is currently active or inactive (removed).

[0043] Time Stamp 112 represents the time of the directory's creation. This value is represented as either a time or a log sequence number; either representation identifies this entry's temporal relation to other database events.

[0044] Rm_time 114 is similar to Time Stamp 112 but represents when a directory was deleted. An active directory has this field blank.

[0045] Information 116 is a field for storing metadata about a directory entry. Possible metadata information includes directory security attributes, mount information, owner, and access control list. The particular data in FIG. 1 illustrates a filesystem having the following active directories: /dlfs1/Usr/Datalinks, /dlfs2/Image/a_m, and /dlfs2/Image/n_z. Where the FSID for /dlfs1 is 12 and the FSID for /dlfs2 is 14. Directories /dlfs1/User and /dlfs2/Image/a_z have previously been deleted and are inactive.

[0046] As for the table management structures, the preferred primary key of the table illustrated in FIG. 1 is (INODE, FSID). This key selection allows for the creation of an index which allows quick scanning during any of the rebuild operations described later. Also, a self-referencing foreign key constraint is defined on (FSID, Parent INODE). This constraint ensures that the parent node of a child directory must exist before the child directory and its INODE value can be inserted as an entry in the table.

[0047] Given the database table structure identified in FIG. 1, the following algorithm is used to rebuild an entire filesystem hierarchy in the event of a filesystem or disk crash:

[0048] Step 1. Locate all the root-level directories by searching for all entries in the table with INODE value equal to 0 (zero). For each such located directory re-create them in the filesystem and perform steps 2 through 5. An index scan of the table's primary key is one efficient method of locating these entries.

[0049] Step 2. For each table entry located in step 1, first extract the (INODE, FSID) pair, second set the tuple (c_inode, c_fsid) equal to the extracted tuple, and third use (c_inode, c_fsid) to perform steps 3-5.

[0050] Step 3. Identify all children directories of a parent directory by finding all records that satisfy the following test:

[0051] (Parent INODE==c_inode) and

[0052] (FSID==c_fsid)

[0053] Extract the Directory Name from each identified child directory entry and create the extracted directories under the current parent directory.

[0054] Step 4. For each child directory identified in step 3, first extract its (INODE, FSID) tuple, second set the tuple (c_inode, c_fsid) equal to the extracted tuple, and third return to step 3. This step effectively sets the child directory located in the previous step 3 to the new parent directory for which children directories of the new parent directory will be searched upon returning to step 3.

[0055] Step 5. Recursively apply steps 3 and 4 until no more matching children directories are found while performing step 3.

[0056] The algorithm identified above is illustrated in the flowchart of FIG. 2: When an entire filesystem restore is started 200, the first step 202 locates a root level directory entry by searching the database table for entries having INODE values equal to 0. If no root level directory is located, then the filesystem restore is complete at step 204. When, however, a root level directory entry is located then a directory is re-created, step 206, in the filesystem for that entry. The next step, 208, extracts the FSID and INODE values from the directory entry located in step 202 and defines a current directory by setting the values c_inode and c_fsid equal to the extracted INODE and the extracted FSID values, respectively.

[0057] In step 210, the database table is searched for a child directory of the current directory by locating entries which have Parent INODE values equal to c_inode and FSID values equal to c_fsid. When a child directory entry is found, step 220 extracts the Directory Name value and then recreates the extracted directory under the current directory. At step 222, the FSID and INODE values are extracted from the directory entry located in step 220. The present current directory values are stored in step 224 for later use and a new current directory is defined in step 226 by setting c_inode and c_fsid equal to the INODE and FSID values extracted in step 222.

[0058] With these new current directory values, step 228 returns to step 210 to continue. If an entry corresponding to a child directory of the current directory is not located in step 210, then step 212 determines whether the current directory is a root level directory. If the current directory is a root level directory, then step 214 returns to step 202 which locates the next root level directory entry to process. If the current directory is not a root level directory, then step 216 changes the current directory by restoring c_fsid and c_inode to the values previously stored in step 224. With these new current directory values, step 218 returns to step 210 to continue.

[0059] Performing the above algorithm reconstructs an entire file system top down from the table by first finding the root-level directories for each filesystem and then recursively finding its children directories and its children's children directories, etc. as illustrated in the sequence of FIGS. 3athrough 3gwhich detail the progression of the directory structure rebuilding according to the table entries in FIG. 1.

[0060] Rather than rebuilding an entire filesystem, the database table also has sufficient information to rebuild a single filesystem or a particular subdirectory of the filesystem given an (INODE, FSID) pair using the following algorithm.

[0061] Step 1. Preferably using an index scan on the table's primary key, find the table entry with a given (INODE, FSID) pair.

[0062] Step 2. Extract the Directory Name and its Parent INODE value from the entry matched in the previous step.

[0063] Step 3. Using the Parent INODE extracted in step 2 along with the given FSID, first identify the parent directory's entry in the table and then extract its Directory Name and INODE.

[0064] Step 4. Recursively apply the above process until the parent INODE value is NULL.

[0065] Step 5. Build the full path name for the desired subdirectory by concatenating all the extracted directory names in last-found, first-out order.

[0066] As previously indicated, the above algorithm which is illustrated by the flowchart of FIG. 4, allows a specific subdirectory to be rebuilt using the table of filesystem directory entries discussed earlier. Given the INODE and FSID of the desired subdirectory, a matching table entry is located in step 402 and the Directory Name and Parent INODE values are extracted in step 406 from the located entry. If no matching entry is found in step 402, then the process stops in step 404.

[0067] The extracted Parent INODE value is tested in step 408. If it is not NULL, then the table is scanned in step 412 to locate the entry with an FSID value equal to the FSID value given in step 402 and an INODE value equal to the extracted Parent INODE value. Step 414 continues by extracting the directory name of the entry located in step 412 and also extracting its Parent INODE value. Step 414 completes by returning control to step 408 with the newly extracted Parent INODE value.

[0068] In step 408, if the extracted Parent INODE value is NULL (i.e. a root level directory) then upwardly traversing the subdirectory path is finished and control transfers to step 410. Step 410 rebuilds the desired subdirectory in the filesystem by re-creating the subdirectory's path according to all the extracted Directory Names in a last-found, first-out order.

[0069] Once the path from the root level to a subdirectory is reconstructed, the reconstruction of all descendants of the subdirectory is possible in step 416. By using steps 2 through 5 of The Complete Rebuild Algorithm on the (INODE, FSID) pair for a given subdirectory, a complete rebuilding of the descendants of the given subdirectory structure is accomplished.

[0070] When a filesystem crashes (disk fails), the directory structure of the file system first needs to be rebuilt. The complete rebuild algorithm is used, without any constraints, to restore a filesystem completely under DBMS control or alternatively, a more efficient method of reconstruction is also possible. Specifically, a filesystem backup is restored first from a traditional archival media to return the filesystem to the state at the time the backup was performed. Then the database table of directory entries is selectively applied to bring the filesystem to a specified state.

[0071] As an example, if a user specifies the application of all entries with a time stamp (or log sequence number) occurring after the backup was performed, then the filesystem directory structure is reconstructed back to the state right before the disk crash. Other selections of time stamp values for controlling the application of table entries allow reconstruction of the directory structure to any previous state; possible examples include a particular time of day or the last known time of a consistent or known-good database state.

[0072] Once the filesystem is rebuilt, files which occupy the filesystem must also be restored. A reconciliation agent within the DBMS is needed to accomplish this task. As discussed earlier, when a file is under DBMS control, an archive copy of the file and its changes are saved and each archived version is associated with a unique identifier and stored in a database table. In a filesystem recovery process, after the directory structure of the filesystem is reconstructed, the DBMS then performs a reconciliation process to synchronize a database state to the reconstructed filesystem directory state. The following algorithm describes a method for restoring DBMS-managed external files to a user specified database state once the directory structure of underlying filesystem is reconstructed to that same state.

[0073] Step 0. Restore a filesystem directory structure to a desired state using any of the above mentioned algorithms.

[0074] Step 1. A database agent receives and sets an identifier for the desired database state. Either a time stamp or log sequence number is sufficient to identify a database state.

[0075] Step 2. The database agent scans the database catalog tables to locate tables which have columns referencing external files under DBMS control.

[0076] Step 3. For each table found in step 2, the agent scans the table and extracts location information (URL) for each referenced external file.

[0077] Step 4. The agent forwards each URL and the database state identifier to a restoration process at a node identified by the URL.

[0078] Step 5. The restoration process uses the URL and database state information to search and select a file from an archived file table. Each archived file is associated with a database state identifier. If more than one file entry exists for a URL, the archived version which matches the specified database state is selected.

[0079] Step 6. The restoration process restores the file selected in step 5 to the filesystem from the archival device.

[0080] When all the tables have been processed, all files in the reconstructed directory structure that are linked to the DBMS are restored to match the requested database state. As an optimization to the above algorithm, the state of the archive table (in step 5) is first compared with the one in the filesystem, if it exists. If the states are the same then step 6 is unnecessary and is skipped.

[0081] The flowchart in FIG. 5 illustrates the above detailed algorithm for synchronizing a database file restore with a filesystem directory structure restore. First, a filesystem directory structure is restored, in step 500, to a desired state using any of the previously discussed methods. A database agent then receives this desired state information and in step 502 sets an identifier, such as a time stamp or log sequence number, which corresponds to the desired state. In step 504, the managed database catalog tables are searched to locate database tables that have columns referencing external files. If no such table is located in step 504, then all managed files have been restored and the file synchronization completes at step 506. If, however, a table is located in step 504, it is then scanned, entry by entry to extract (in step 508) file archive location information for each referenced external file.

[0082] When the scan, in step 508, for an entry in a given table fails to locate any more referenced external files in that table, control returns to step 504 in order to locate additional tables which manage external files.

[0083] Each location information extracted in step 508 is passed, along with the database state identifier, to a restoration process in step 510. The restoration process uses the location information to locate (step 512) and retrieve a referenced file from an archival media (step 516). If more than one version of the file exists, as determined by step 514, then the restoration process, in step 518, selects the version that matches the desired database state. In step 520, if the filesystem does not have the correct version of the file, the selected file version is restored from the archival media to the filesystem which was, itself, previously restored in step 500 and then returns to step 508 in order to locate the next referenced file in the current table being scanned.

[0084] As indicated, when all external file references from all managed tables have been processed and restored, step 506 is reached and the synchronization is complete.

[0085] A system and method has been shown in the above embodiments for the effective implementation of rebuilding a filesystem directory structure under database control in the event of a filesystem crash. While various preferred embodiments have been shown and described, it will be understood that there is no intent to limit the invention by such disclosure, but rather, it is intended to cover all modifications and alternate constructions falling within the spirit and scope of the invention as defined in the appended claims. For example, the present invention should not be limited by computer operating system, database management system, database management model, directory structure, DBMS-file linking technology, the type of user interface, computer hardware platform, e-commerce platform, network operating system, archiving software, or archiving hardware. In addition, the present invention can be implemented locally on a single PC, connected workstations (i.e. networked-LAN), across extended networks such as the Internet or using portable equipment such as laptop computers or wireless equipment (RF, microwaves, infrared, photonic, etc.)