Method and apparatus for processing transactional file system operations to enable point in time consistent file data recreation

[0001] 1. Field of the Invention

[0002] Embodiments of the present invention generally relate to a data protection system and, more importantly, to a method and apparatus for processing transactional file systems operations to provide point in time consistent file data to support file data recovery and replication from transactional file system.

[0003] 2. Description of the Related Art

[0004] In a typical computing environment, one or more computing devices use a variety of resources to process and store file data. For example, employees within a large organization use computers, data storage devices, networking components and/or the like for performing the day-to-day activities. The employees may store the file data in the data storage devices in the form of data volumes for retrieval at a later date. Occasionally, the file data in the data volumes becomes corrupted or erroneous and a previous storage state of the file data is needed to avoid a disruption in the day-to-day activities of the organization.

[0005] Many organizations use a variety of data protection systems to preserve storage states of the data volumes in case of a disaster. For example, a snapshot of the data volume at a particular point-in-time may be captured through a snapshot service in order to facilitate recovery of lost or corrupted data (e.g., files, emails and/or the like). In other words, the snapshot is a representation of a storage state of the data volume at the particular point-in-time. Furthermore, a continuous data protection system is configured to recreate any version of a file existing at any point-in-time using the captured snapshot and a log or journal of input/output activity for that file.

[0006] The data volumes may be organized into files and directories (i.e., a file system, such as File Allocation Table (FAT), New Technology File System (NTFS)). For example, the data volumes may be organized in accordance with a transactional file system, such as Transactional NTFS. Generally, a transactional file system volume is fault-tolerant and consistent. A transaction can either be finished completely (e.g., a committed transaction) or reverted completely (e.g., a rolled back transaction), but not necessarily both at any given point in time. This means that if there is a crash or power failure, after recovery, the storage state of the data volume will be consistent. A significant amount of computing overhead, however, is required to maintain data consistency.

[0007] Current data protection systems are unable to restore or replicate file data associated with a transaction that was in progress during the snapshot of the data volume. For example, mounting a snapshot causes any in progress (i.e., incomplete) transactions to be rolled back. As such, the snapshot does not include file data that is point in time consistent. Hence, the use of transactions prevents the data protection software from recreating point in time consistent file data from the snapshot. In addition, the file data may only be replicated or recovered to a data volume that supports transactional file systems. Transactional file system operations cannot be played on a non-transactional file system.

[0008] Therefore, there is a need in the art for a method and apparatus for processing transactional file system operations to enable point in time consistent file data recreation and recovery from transactional file systems.

[0009] Embodiments of the present invention generally comprise a method and apparatus for processing transactional file system operations to enable point in time consistent file data recreation and recovery from transactional file systems. In one embodiment, a method for processing transactional file system operations to provide point in time consistent file data comprises processing input/output activity associated with file data that is used in a computing environment to identify at least one transaction and segregating the input/output activity based upon the at least one transaction to enable point in time consistent data recreation for the file data.

[0016] FIG. 1 is a block diagram of a system 100 for processing transactional file system operations to enable point in time consistent file recreation according to one or more embodiments of the present invention. The system 100 includes a computing environment 102, a server 104 and a storage 106 in which each is coupled to each other through a network 108.

[0017] The computing environment 102 provides various computer resources (e.g., storage, network, file system, backup, restore and/or the like) to one or more client computers. Furthermore, the one or more client computers in the computing environment 102 produce various forms of information (e.g., customer data, application data, financial data and/or the like) that is stored as file data 110 (e.g., a volume that comprises a number of files in accordance with a file system). The file data 110 may be backed up by the server 104 to the storage 106 through the network 108 according to one or more embodiments.

[0018] The server 104 is a type of computing device (e.g., a laptop, a desktop, a Personal Desk Assistant (PDA), a tablet, a mobile phone and the like) that comprises a central processing unit (CPU) 112, various support circuits 114 and a memory 116. The CPU 112 may comprise one or more commercially available microprocessors or microcontrollers that facilitate data processing and storage. Various support circuits 114 facilitate operation of the CPU 112 and may include clock circuits, buses, power supplies, input/output circuits and/or the like. The memory 116 includes a read only memory, random access memory, disk drive storage, optical storage, removable storage, and the like. The memory 116 further includes various software packages, such as a recorder 118, data protection software 120 and a playback module 122.

[0019] The storage 106 generally includes various components (i.e., hardware and software) that are configured to manage storage resources. The storage 106 may include one or more data storage devices (e.g., hard disk drives, optical drives, magnetic tape drives, a storage array (e.g., a RAID configuration) and/or the like), one or more storage controllers and software that provides an interface (e.g., Integrated Drive Electronics/AT Attachment (IDE/ATA), Small Computer Systems Interface (SCSI) and/or the like) to the one or more data storage devices for the server 104. The storage 106 includes input/output activity information 124 and a backup image 126.

[0020] The network 108 comprises a communication system that connects computers by wire, cable, fiber optic, and/or wireless links facilitated by various types of well-known network elements, such as hubs, switches, routers, and the like. The network 108 may employ various well-known protocols to communicate information amongst the network resources. For example, the network 108 may be part of the Internet or intranet using various communications infrastructure such as Ethernet, WiFi, WiMax, General Packet Radio Service (GPRS), and the like. Furthermore, the network 106 may form a portion of a Storage Network Area (SAN) using various communications infrastructure such as Ethernet, Fibre Channel, InfiniBand, and the like.

[0021] Generally, the input/output activity information 124 includes one or more storage and/or file system operations associated with the file data 110 and stored in one or more journals or streams. In one embodiment, the input/output activity information 124 includes one or more transactional and non-transactional file system operations. As explained further below, the input/output activity information 124 may segregated into one or more transactional journals and/or one or more non-transactional journals.

[0022] The backup image 126 may be organized in a proprietary format to represent a storage state of a volume (i.e., the file data 110) according to one or more embodiments. The backup image 126 may be generated by the data protection software 120 (e.g., SYMANTEC NetBackup products, SYMANTEC BackupExec products, snapshot software and/or the like). In one embodiment, the backup image 126 (e.g., a full backup image or an incremental backup image) includes the file data 110. In another embodiment, the backup image 126 (e.g., a snapshot image) includes references or pointers to addresses (e.g., locations of files) within the file data 110. In one embodiment of the present invention, the backup image 126 is a point in time consistent copy of the file data 110 at a particular point in time (i.e., a recovery or backup point).

[0023] The storage 106 facilitates permanent storage (i.e., a backup) of a storage state of the file data 110 at one or more points in time. In one embodiment, the file data 110 at the particular point in time is stored as the backup image 126. In addition, subsequent input/output activity (e.g., file system operations) is recorded in the input/output (I/O) activity information 124. In one embodiment, the input/output activity information 124 may be played back to recover the file data 110 at a later point in time. In another embodiment, the input/output activity information 124 may be used to update the backup image 126 to represent the file data 110 at the later point in time. Accordingly, the updated backup image 126 includes a copy of the file data 110 that is consistent with the later point in time.

[0024] The recorder 118 includes software code that is configured to record input/output activity associated with the file data 110. In one embodiment, one or more transactions are applied to the file data 110 during the input/output activity. Furthermore, the recorder 118 segregates the input/output activity based upon one or more transactions. For example, the input/output activity is transmitted to the recorder 118 as a single journal (e.g., a chain of file system operations) and then, split into one or more transactional journals and/or a non-transactional journal where each transaction is recorded in a journal of the one or more transactional journals. In one embodiment, the segregated input/output activity is stored as the input/output activity information 124.

[0025] In another embodiment, the input/output activity is stored in the input/output activity information 124 as the single journal. At a later date (e.g., during system recovery), the recorder 118 segregates the single journal into one or more journals based on the one or more transactions to enable point in time consistent file recreation. For example, the single journal is split into a number of journals in which one or more journals are associated with one or more incomplete transactions at a particular point in time (e.g., rolled back transactions when a snapshot image is mounted). As another example, the single journal is split into one or more pre-snapshot transactional journals, one or more post-snapshot transactional journals and one or more non-transactional journals.

[0026] In one embodiment, the recorder 118 stores one or more file system operations that are prior to and/or subsequent to a backup (e.g., a snapshot, a full backup and/or the like) of the file data 110 in the input/output information 124. The one or more file system operations may be associated with one or more transactions and/or one or more non-transactions that are applied to the file data 110 before and/or after the generation of the backup image 126. Furthermore, the recorder 118 segregates the one or more file system operations based upon the one or more transactions.

[0027] In one embodiment, the recorder 118 stores one or more transactional file system operations that are applied before the generation of the backup image 126 (e.g., a snapshot) in a pre-snapshot transactional journal. In another embodiment, the recorder 118 splits the pre-snapshot transactional journal operations that are applied before the generations into a journal for incomplete transactions (e.g., one or more transaction in progress during the generation of the snapshot) and a journal for completed transactions. As explained further below, the journal for incomplete transactions may be used to update the snapshot to be point in time consistent. Additionally, the recorder 118 stores one or more file system operations (transactional and non-transactional) after the generation of the backup image 126 in a post-snapshot journal.

[0028] In another embodiment, the recorder 118 stores the one or more file system operations in a stream or a journal as a chain of events (i.e., in order of application on the file data 110). Alternately, the recorder 118 segregates the one or more file system operations into multiple journals during recordation. In either embodiment, the one or more file system operations are split into one or more journals for the one or more transactions. Accordingly, the input/output information 124 includes a journal for each of the one or more transactions and a journal for all of the one or more non-transactions.

[0029] For example, the recorder 118 segregates the input/output activity information 124 based on transaction ids of the one or more file system operations. For each transaction id, the recorder 118 creates a journal and stores each and every file system operation associated with the transaction id in the journal. As the recorder 118 processes a particular file system operation for recordation, the recorder 118 selects a journal that corresponds with a transaction id of the particular file system operation according to one embodiment. In one embodiment, the transaction id of non-transactions is NULL or 0. As such, file system operations having such a transaction id are stored in the non-transactional journal.

[0030] The recorder 118 segregates the input/output information 124 to enhance replication and/or recovery of the file data according to one embodiment. A copy of the segregated input/output information 124 permits the playback module 122 to use non-transactional file system functions to replay transactional file system operations. For example, the data protection software 120 may use non-transactional versions of “read”, “write”, “open” and “close” functions to perform corresponding transactional versions. The removal of “commit” and “roll back” file system operations permits data recovery or replication between transactional and non-transactional file systems. Therefore, the data protection software 120 may restore or replicate a data volume in a transactional file system to a data volume a non-transactional file system and vice versa.

[0031] In one embodiment, the recorder 118 grooms the one or more post-snapshot journals. For example, the one or more post-snapshot journals may include file system operations for completed transactions (e.g., the one or more in-progress transactions). Naturally, the completed transactions may either be committed or rolled back. A post-snapshot journal associated with a rolled back transaction may be deleted because the file system operations were never fully applied to the data volume. Hence, the one or more post-snapshot journal is groomed of the rolled back transaction. In another embodiment, the recorder 118 grooms the one or more pre-snapshot transactional journals of any rolled back transactions.

[0032] In yet another embodiment, the recorder 118 grooms one or more transactional journals and a non-transactional journal. As mentioned herein, the input/output information 124 may be segregated into a transactional journal for each transaction of the one or more transactions and a journal for all non-transactions based on transaction ids of the file system operations. For example, the recorder 118 may identify a rollback file system operation in a particular transactional journal and remove the transactional journal from the input/output activity information 124. Alternatively, upon processing a rollback file system operation for recordation, the recorder 118 deletes a transactional journal having a matching transaction id as the rollback file system operation.

[0033] As another example, the recorder 118 may identify a commit operation in a particular transactional journal and associate the particular transactional journal with the non-transactional journal. For example, the recorder 118 associates the particular transactional journal with the non-transactional journal using related file names, transaction IDs and/or the like. Since the transaction associated with the particular transactional journal is committed, the one or more file system operations may be replayed by the playback module 122 using corresponding non-transactional functions. Alternatively, upon processing a commit file system operation for recordation, the recorder 118 associates a transactional journal having a matching transaction id as the commit file system operation with the non-transactional journal.

[0034] As yet another example, the recorder 118 may determine that one or more portions of a particular journal (e.g., a transaction journal and/or a non-transaction journal) are to be groomed (i.e., removed) after a period of time. For instance, the particular journal may be a transactional journal that is no longer needed for restoration and/or replication because a point in time consistent backup image that post-dates the transactional journal is available. Furthermore, the transactional journal may be associated with a non-transactional journal when the transaction is committed. As such, the recorder 118 deletes the associated transactional journal while grooming the non-transactional journal.

[0035] The playback module 112 includes software for facilitating point in time consistent recovery and/or replication of the file data 110 in the computer environment 102. In one embodiment, the playback module 122 is configured to use the backup image 126 and the input/output activity information 124 to create (i.e., recreate) files that are consistent with a recovery point for the file data 110 (e.g., a data volume in the computing environment 102). In operation, the playback module 122 mounts the backup image 126 to access a storage state of the file data 110 (e.g., a data volume). In one embodiment, the playback module 122 updates the backup image 126 using a segregated copy of input/output information 124.

[0036] For example, the playback module 122 uses one or more pre-snapshot transactional journals to update a mounted snapshot image with one or more transactions that were in progress during the snapshot of the data volume. Mounting the snapshot image causes a rollback of the one or more in progress transactions. The one or more pre-snapshot transactional journals indicate one or more file system operations associated with the one or more in progress transactions and prior to a timestamp of the snapshot. Such file system operations are applied to the mounted snapshot image to update the data volume. As a result, the storage state of the data volume is consistent with the timestamp of the snapshot.

[0037] Subsequently, the playback module 122 may use one or more post-snapshot journals to modify the mounted snapshot image to be point in time consistent with a recovery point. The one or more post-snapshot journals may include one or more file system operations for one or more transactions and/or one or more non-transactions that were performed after a timestamp of the snapshot. Such file system operations may be applied to the mounted snapshot image until the recovery point. As a result, the mounted snapshot image includes a point in time consistent data volume.

[0038] FIG. 2 is a method 200 for processing transactional file system operations to enable point in time consistent file recreation according to one or more embodiments of the present invention. The method 200 starts at step 202 and proceeds to step 204.

[0039] At step 204, input/output activity is recorded. At step 206, one or more transactions are identified. At step 208, the input/output activity (e.g., the input/output activity information 124 of FIG. 1) is segregated based upon the one or more transactions. At step 210, a determination is made as to whether the one or more transactions are completed. If it is determined that a transaction of the one or more is incomplete, then the method 200 returns to step 204. If it is determined that the one or more transactions are completed, then the method 200 proceeds to step 212. At step 212, the segregated input/output activity is groomed. After step 212, the method 200 returns to step 204.

[0040] FIG. 3 is a method 300 for segregating the input/output activity according to one or more embodiments of the present invention. The method 300 starts at step 302 and proceeds to step 304.

[0041] At step 304, a file system operation is processed. At step 306, a determination is made as to whether the file system operation is transactional. If it is determined that the file system operation is transactional, then the method 300 proceeds to step 308. At step 308, the file system operation is added to a journal with a matching transaction id. If it is determined that the file system operation is non-transactional, then the method 300 proceeds to step 318. At step 318, the file system operation is added to a non-transactional journal.

[0042] At step 310, a determination is made as to whether the file system operation is a roll back operation. If it is determined that the file system operation is a roll back operation, then the method 300 proceeds to step 312. If it is determined that the file system operation is not a roll back operation, then the method 300 proceeds to step 314. At step 312, the journal with the matching transaction id is deleted. Accordingly, the transactional journals are groomed and the journal with the matching transaction id is removed.

[0043] At step 314, a determination is made as to whether the file system operation is a commit operation. If it is determined that the file system operation is not a commit operation, then the method 300 proceeds to step 318. If it is determined that the file system operation is a commit operation, then the method 300 proceeds to step 316. At step 316, the journal with the matching transaction id is associated with a non-transactional journal. Subsequently, the method 300 proceeds to step 320. At step 320, a determination is made as to whether there is a next file system operation. If it is determined that there is a next file system operation, then the method 300 returns to 304. If it is determined that there is no next file system operation, then the method 300 proceeds to 322. At step 322, the method 300 ends.

[0044] FIG. 4 illustrates segregated input/output activity information according to one or more embodiments of the present invention. In one embodiment, the input/output activity information is stored in a journal 400 as a chain of file system operations on a time scale. The journal 400 is segregated (e.g., by the recorder 118 of FIG. 1) into a non-transactional journal 402, a transactional journal 404 and a transactional journal 406.

[0045] Accordingly, each non-transactional file system operation is segregated from the journal 400 and stored in the non-transactional journal 402. For example, file system operations “Open 1.TXT” and “WRITE ABC to 1.TXT” are determined to be not associated with a transaction (e.g., transaction id=0). Hence, file system operations “Open 1.TXT” and “WRITE ABC to 1.TXT” are stored in the non-transactional journal 202 to segregate from file system operations associated with a transaction “TXF 1” or a transaction “TXF 2.”

[0046] Furthermore, each transactional file system associated with the transaction “TXF 1” or the transaction “TXF 2” is segregated from the journal 400 and stored in the transactional journal 406 or the transactional journal 404, respectively. For example, the file system operation “Open 4.TXT” is determined to be associated with the transaction “TXF 2”. The transactional journal 404 is associated with a transaction id that matches the transaction “TXF 2”. As such, the file system operation “Open 4.TXT” is stored in the transactional journal 404. As another example, the file system operation “Open 3.TXT” is determined to be associated with the transaction “TXF 1”. The transactional journal 406 is associated with a transaction id that matches the transaction “TXF 1”. Therefore, the file system operation “Open 3.TXT” is stored in the transactional journal 406.

[0047] Additionally, the transactional journal 404 and the transactional journal 404 may be groomed to optimize data recovery. For instance, a rollback file system operation for the second transaction is identified in the journal 400 and/or the transactional journal 404. As such, the transactional journal 404 may be deleted from the input/output activity information. Moreover, a commit file system operation for the transaction “TXF 1” is identified in the journal 400 and/or the transactional journal 406. Hence, the transactional journal 406 may be associated with the non-transactional journal 402 because the transaction “TXF 1” is completed and the file system operations may become visible to any process.

[0048] Optionally, a snapshot may be generated at any point in time within the journal 400. As an example, the snapshot may be generated while the second transaction is currently in progress (i.e., not completed). As such, the transactional journal 404 may be further segregated into a pre-snapshot transactional journal and a post-snapshot transactional journal. Furthermore, because the second transaction is rolled back, the pre-snapshot transactional journal and the post-snapshot transactional journal may be deleted.

[0049] FIG. 5 is a method 500 for providing a point in time consistent snapshot to enable recovery of file data according to one or more embodiments of the present invention. The method 500 starts at step 502 and proceeds to step 504.

[0050] At step 504, a recovery point is established. At step 506, a snapshot generated prior to the recovery point is mounted (e.g., a snapshot having a timestamp before the recovery point). At step 508, a determination is made as to whether there are one or more incomplete transactions (e.g., transactions in progress at the timestamp associated with the generation of the snapshot). If it is determined that there are one or more incomplete transactions, then the method 500 proceeds to step 510. If it is determined that there are no incomplete transactions, then the method 500 proceeds to step 514.

[0051] At step 510, one or more pre-snapshot transactional journals are accessed. At step 512, one or more file system operations are replayed. In one embodiment, the one or more file system operations are applied to the snapshot. Hence, the snapshot is update to be consistent with the point in time associated with the generation of the snapshot. At step 514, a post-snapshot journal is accessed. Alternatively, one or more post-snapshot transactional journals and a post-snapshot non-transactional journal are accessed as explained above. At step 516, one or more file system operations are replayed until a timestamp associated with the recovery point. At step 518, file data is recovered from the updated snapshot that is point in time consistent with the recovery point. At step 520, the method 500 ends.

[0052] While the foregoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.