Tracking in-progress writes through use of multi-column bitmaps

[0001] The present patent application is a continuation of U.S. patent application Ser. No. 10/326,432, filed on Dec. 19, 2002, now U.S. Pat. No. 6,907,507 entitled “TRACKING IN-PROGRESS WRITES THROUGH USE OF MULTI-COLUMN BITMAPS” and is incorporated by reference herein in its entirety and for all purposes.

[0002] Many businesses rely on large-scale data processing systems for storing and processing their data. Insurance companies, banks, brokerage firms, etc., rely heavily on data processing systems. Often the viability of a business depends on the reliability of access to data contained within its data processing system. As such, businesses seek reliable ways to consistently protect their data processing systems and the data contained therein from natural disasters, acts of terrorism, or computer hardware and/or software failures.

[0003] Businesses must be prepared to eliminate or minimize data loss and recover quickly with useable data after an unpredicted event such as a hardware or software failure in the data processing system. When an unexpected event occurs that causes loss of data, businesses often recreate the data using backup copies of their data made on magnetic storage tape. Restoring data from a backup tape is typically a time-consuming process that often results in a substantial loss of business opportunity. Business opportunity is lost because the data processing system cannot process new data transactions while data is being recreated from backup copies. Further, restoration from tape usually involves loss of data. For most businesses, this kind of data loss and down time is unacceptable. Mirrored data volume and virtual point-in-time (PIT) backup technologies offer better solutions for full and accurate restoration of business critical data. Mirrored data volume and virtual PIT technologies not only minimize or eliminate data loss, but also enable rapid recovery of a data processing system when compared to conventional bulk data transfer from sequential media.

[0004] FIG. 1 illustrates an exemplary data processing system 10 that employs mirrored data volume and virtual PIT backup technology. Data processing system 10 includes a host node 12 coupled to data storage systems 14–18. Data storage systems 14–18 include data memories 24–28, respectively. As will be more fully described below, data memory 24 stores a primary data volume, data memory 26 stores a virtual PIT backup copy of the primary data volume, and data memory 28 stores a mirrored copy of the data volume. Data processing system 10 shown in FIG. 1 and its description herein should not be considered prior art to the invention disclosed and/or claimed.

[0005] Host node 12 may take form in a computer system (e.g., a server computer system) that receives and processes requests to read or write data to the primary data volume stored within memory 24. In response to receiving these requests, host node 12 generates read or write-data transactions for reading or writing data to the primary data volume within data memory 24. It is noted that host node 12 is capable of accessing each of the memories 24–28 or memory internal to host node 12 for reading or writing data to the data volumes stored therein.

[0006] Each of the data memories 24–28 may include several memory devices such as arrays of magnetic or optical discs. The primary data volume maybe distributed across several memory devices of memory 24. Likewise, the virtual PIT copy of the primary data volume is distributed across several memory devices of data memory 26, and the mirrored copy of the primary data volume is distributed across several memory devices of data memory 28. Each of the data memories 24–28 may include n_max physical memory blocks into which data can be stored. More particularly, data memory 24 includes n_max memory blocks allocated by host node 12 for storing data of the primary data volume, data memory 26 includes n_max memory blocks allocated to store the virtual PIT backup copy of the primary data volume, and data memory 28 includes n_max memory blocks allocated to store the mirrored copy of the primary data volume. Corresponding memory blocks in data memories 24–28 are equal in size. Thus, memory block 1 of data memory 24 is equal in size to memory block 1 of data memories 26 and 28.

[0007] Host node 12 creates the virtual PIT backup copy of the primary data volume according to the methods described in U.S. patent application No. 10/143,059 entitled “Method and Apparatus for Creating a Virtual Data Copy,” and U.S. patent application Ser. No. 10/254,753 now U.S. Pat. No. 6,912,631, filed Sep. 25, 2002 and entitled “Method and Apparatus for Restoring a Corrupted Data Volume,” each of which is incorporated herein by reference in entirety. These references describe how host node 12 reads and writes data to the virtual PIT copies. Additionally, the above referenced applications describe how the virtual PIT backup copy can be converted to a real PIT backup copy using a background copying process executed by host node 12.

[0008] Host node 12 creates the virtual PIT backup copy of the primary data volume when host node receives or internally generates a command to create a PIT copy of the primary data volume. Initially (i.e., before any virtual PIT copy is created in data memory 26) data memory 26 contains no data. When host node 12 creates the first virtual PIT backup copy in memory 26, host node 12 creates a pair of valid/modified (VM) maps, such as maps 30 and 32 shown in FIG. 2. FIG. 2 also shows a third map 34 which will be more fully described below. VM maps 30 and 32 correspond to the primary data volume and the virtual PIT copy thereof, respectively, stored within data memories 24 and 26, respectively. Hence, VM maps 30 and 32 will be referred to as primary VM map 30 and PIT backup copy VM map 32.

[0009] Each of the VM maps 30 and 32 include n_max entries, each entry having two bits. Each entry of primary VM map 30 corresponds to a respective block of data memory 24, while each entry of the virtual PIT backup copy VM map 32 corresponds to a respective block of data memory 26. The first and second bits in each entry of VM maps 30 and 32 are designated V_n and M_n, respectively. V_n of each entry, depending on its state (i.e., logical 0 or logical 1), indicates whether block n of the associated memory contains valid data. V_n of the primary VM map 30 indicates whether a corresponding memory block n in memory 24 stores valid data. For example, when set to logical 1, V₂ of VM map 30 indicates that block 2 of data memory 24 contains valid data of the primary data volume, and when set to logical 0, V₂ of the VM map 30 indicates that block 2 of data memory 24 contains no valid data of the primary data volume. V_n of the PIT backup copy VM map 32 indicates whether memory block n in memory 26 stores a valid point-in-time copy of data in block n of memory 24. For example, V₂ of VM map 32, when set to logical 1, indicates that block 2 of memory 26 contains a valid copy of data that existed in block 2 of memory 24 at the time the PIT backup copy was first created or at the time the PIT backup copy was last refreshed. U.S. application Ser. No. 10/326,427, now U.S. Pat. No. 6,880,053, entitled “Instant Refresh of A Data Volume” describes one method of refreshing a PIT backup copy and is incorporated herein by reference in entirety. V₂ of VM map 32, when set to logical 0, indicates that block 2 of memory 26 does not contain a valid copy of data. The V_n bit of PIT backup copy VM map 32 is used to determine when data in block n of memory 24 is to be copied to block n of memory 26. More particularly, when host node 12 generates a write-data transaction for writing data to block n of memory 24, host node 12 checks the status of V_n in PIT backup copy VM map 32. If V_n is set to logical 0, then the PIT backup copy in memory 26 lacks a valid copy of data of block n in memory 24. Before data can be written to block n of memory 24 in accordance with the write-data transaction, data in block n of memory 24 must first be copied to block n of memory 26. If, on the other hand, V_n is set to logical 1, data can be written to block n of memory 24 in accordance with the write-data transaction without first copying data to block n of memory 26.

[0010] M_n in each entry of VM map 30 and PIT copy VM map 32, depending upon its state, indicates whether data within a respective block n of the corresponding memory is modified (or new) since the time the PIT backup copy was first created or last refreshed. For example, when set to logical 1, M₃ of the primary VM map 30, indicates that block 3 of data memory 24 contains data which has been modified since the time the PIT backup copy was first created or last refreshed. When set to logical 0, M₃ of the primary VM map of 30 indicates that data has not been modified or written to Block 3 of data memory 24 since the time the PIT backup copy was first created or refreshed.

[0011] When VM maps 30 and 32 are first created by host node 12, each entry of PIT VM map 32 is set to logical 0 thus indicating that data memory 26 contains no valid or modified data. For purposes of explanation, it is presumed that each block of data memory 24 contains valid data of the primary volume. Accordingly, VM of each entry within primary VM map 30 is initially set to logical 1. Lastly, M_n of each entry in primary VM map 30 is initially set to logical 0.

[0012] As noted above, data memory 28 stores a mirrored copy of the primary data volume. A mirrored copy is considered a real time copy of the primary data volume. Each time host node 12 writes data to the primary data volume stored in memory 24 via a write-data transaction, host node also generates a transaction to write the same data to the mirrored copy stored within memory 28. Thus, the mirrored copy, as its name implies, closely tracks the changes to the primary data volume. The mirrored copy within memory 28 provides an immediate backup data source in the event that data storage system 14 fails as a result of hardware and/or software failure. If data storage system 14 suddenly becomes unusable or inaccessible, host node 12 can service client computer system read or write requests using the mirrored volume within data memory 28.

[0013] Host node 12 is also subject to hardware and/or software failure. In other words, host node 12 may crash. When host node 12 recovers from a crash, host node 12 expects contents of the data memory 24 to be consistent with the contents of the data memory 28. Unfortunately, host node 12 may crash after a write-data transaction is completed to data memory 24, but before the mirrored copy can be updated with the same data of the write-data transaction. If this is to happen, the contents of the primary data copy stored in memory 24 will be inconsistent with the mirrored copy in memory 28 when host node 12 recovers from its crash.

[0014] Host node 12 uses the dirty region (DR) map 34 shown in FIG. 2 to safeguard against inconsistencies between memories 24 and 28 after crash recovery. DR map 34 includes n_max entries corresponding to the n_max memory blocks in data memories 24 and 28. Each entry includes one bit which, when set to logical 0 or logical 1, indicates whether respective memory blocks in data memories 24 and 28 are the subject of an in-progress write-data transaction. For example, host node 12 sets DR₂ to logical 1 before host node 12 writes data to memory block 2 in data memories 24 and 28. Setting or clearing a bit in DR map 34 requires a write (i.e., an I/O operation) to memory that stores DR map 34. After the write transaction is completed to memory block 2 of data memories 24 and 28, host node 12 switches the state of DR₂ back to 0 using another write operation.

[0015] If host node 12 crashes, the contents of DR map 34 is preserved. After recovering from the crash, host node 12 copies the contents of memory blocks in data memory 24 to respective memory blocks in data memory 28, or vice versa, for each respective DR_n bit set to logical 1 in DR map 34. After these memory blocks are copied, host node 12 is ensured that the primary data volume of data memory 24 is consistent with the mirrored copy of data memory 28.

[0016] Host node 12 is burdened by having to generate two separate write operations to update VM map 30 and DR map 34 with each write-data transaction to the primary data volume. More particularly, with each write-data transaction, host node 12 must generate a first write operation to update VM map 30 and a second write operation to update DR map 34. Additionally, a substantial amount of time is needed to complete separate write operations to VM map 30 and DR map 34. To illustrate, FIG. 3 is a flow chart illustrating operational aspects of modifying or writing data of the primary volume within data memory 24 in accordance with a write-data transaction. More particularly, when host node 12 receives a request to write data to the primary volume from one of the computer systems, host node 12 generates a write-data transaction to write data to block n of memory 24. In step 42, host node 12 accesses VM map 32 to determine whether V_n is set to logical 1. If V_n of the VM map 32 is set to logical 0, then host computer 12, as shown in step 44, copies the contents of block n of memory 24 to the corresponding block n in memory 26. Thereafter, in step 46 host computer 12 sets V_n of the VM map 32 to logical 1. Before host computer system 12 writes or modifies data of block n of memory 24 that stores a portion of the primary volume and block n of memory 28 that stores a portion of the mirrored copy, host computer system 12 must set M_n and DR_n of primary VM map 30 and DR map 34, respectively, to logical 1. It is noted that M_n and DR_n are in separate tables, and thus have separate addresses within memory. Accordingly, host computer system 12 must generate separate writes or 10 functions to set M_n and DR_n to logical 1. The time it takes, for example, for the read/write head (not shown) to travel across a magnetic disc to reach the appropriate sector that stores M_n of VM map 30, followed by the time it takes for the magnetic read/write to travel across the magnetic disc to reach another sector that stores DR_n of the DR map 34, may be substantial. Once M_n and DR_n are set to logical 1 in steps 50 and 52, host node 12 writes data to block n in memory 24 as shown in step 54 and writes data to block n of memory 28.

[0017] Disclosed is a method and apparatus for tracking in-progress writes to a data volume and a copy thereof using a multi-column bit map. The method can be implemented in a computer system and, in one embodiment, includes creating a data volume in a first memory, and creating a copy of the data volume in a second memory. In response to the computer system receiving a request to write first data to the data volume, the computer system switches the state of first and second bits of a map entry in a memory device, wherein the state of the first and second bits are switched using a single write access to the memory device.

[0025] The use of the same reference symbols in different drawings indicates similar or identical items.

[0026] The present invention relates to an apparatus and method for tracking in-progress write I/O's through the use of multi-column bit maps. FIG. 4 illustrates (in block diagram form) relative components of a data processing system 60 employing one embodiment of the present invention. Data processing system 60 is similar to data processing system 10 of FIG. 1. Data processing system 60 includes a host node 62 coupled to data storage systems 64–68. It is noted that the term “coupled devices” should not be limited to devices coupled directly together by a communication link. Two devices may be coupled together even though communication occurs through an intervening device.

[0027] Data storage systems 64–68 include memories 74–78, respectively. Memory 74 stores one or more primary data volumes. For purposes of explanation, memory 74 stores one primary data volume, it being understood that the present invention will not be limited thereto. Memory 76 stores a point-in-time (PIT) backup copy of the primary data volume whether real or virtual, while memory 78 stores a mirrored copy of the primary data volume.

[0028] The primary data volume (and the PIT backup and mirrored copies thereof) is a collection of files that store data. While it is said that files store data, in reality data is stored in physical blocks of memory 74 which are allocated to the files by host node 62. Host node 62 may take form in a computer system (e.g., a server computer system) that includes a data storage management system 70 in the form of software instructions executing on one or more data processors (not shown). Data storage management system 70 may include a file system (not shown) and a system (not shown) for managing the distribution of volume data across several memory devices. Volume Manager™ provided by Veritas Corporation of Mountain View, Calif., is an exemplary system for managing the distribution of volume data across several memory devices. Host node 62 generates read or write-data transactions in response to host node 62 receiving requests from client computer systems to read or write data to the primary data volume.

[0029] As noted, memories 74–78 store the primary data volume, the PIT backup copy of the primary data volume, and the mirrored copy of the primary data volume, respectively. Data memories 74–78 may take form in one or more dynamic or static random access memories, one or more arrays of magnetic or optical data storage disks, or combinations thereof. Data memories 74–78 should not be limited to the foregoing hardware components; rather, data memories 74–78 may take form in any hardware, software, or combination of hardware and software in which data may be accessed and persistently stored. Data memories 74–78 may take form in a complex construction of several hardware components operating under the direction of software. The memories may take form in mirrored hardware. It is further noted that the present invention may find use with many types of redundancy/reliability systems. For example, the present invention may be used with Redundant Array of Independent Disks (RAID) systems. Moreover, the present invention should not be limited to use in connection with the host node of a data storage network. The present invention may find use in a storage switch or in any of many distinct appliances that can be used with a data storage system.

[0030] Each of the data memories 74–78 includes n_max physical blocks of memory into which data can be stored. It is noted that any or all of the memories 74–78 may have more than n_max memory blocks. However, the first n_max blocks in memories 74–78 are allocated by host node 62 for storing the primary data volume, the PIT backup copy of the primary data volume, and the mirrored copy of the primary data volume, respectively. Corresponding memory blocks in memories 74–78 are equal in size. Thus, memory block 1 of memory 74 is equal in size to memory blocks 1 in memories 76 and 78. Each of the memory blocks within memory 74 may be equal in size to each other. Alternatively, the memory blocks in memory 74 may vary in size.

[0031] Host node 62 can access each of the blocks in memories 74–78 in response to generating read or write-data transactions. The read or write-data transactions are generated by host node 62 in response to host node 62 receiving read or write requests from client computer systems coupled thereto. The primary data volume stored in memory 74 is the “working” volume of data processing system 60. Thus, when client computer systems request a read or write access to data, the access is directed to the primary data volume. The PIT backup copy of the primary data volume within memory 76 acts as a safeguard against data corruption of the primary data volume due to a software or operator error. The mirrored copy within memory 78 provides a alternative source to data in the event that data storage system 64 is rendered inaccessible to host node 62 due to hardware and/or software failure.

[0032] Host node creates a virtual PIT backup copy of the primary data volume by creating primary VM/DR map 80 and a PIT VM map 82 shown within FIG. 5. These maps may be stored within one of the memories 74–78, in a memory within host node 62 or in a memory (not shown) attached to host node 62. The contents of each of the maps 80 and 82 is accessible by host node 62 using a read or write operation. Maps 80 and 82, like maps 30 and 32 described above, include n_max entries. Each entry of primary VM/DR map 80 corresponds to a respective block of data memories 74 and 78, while each entry of the VM map 82 corresponds to a respective block of data memory 76. VM map 82 is substantially similar to map 32, while VM/DR map 80 is substantially similar to a combination of VM map 30 and DR map 34.

[0033] Each entry in primary VM/DR map 80 consists of three bits designated V_n, M_n, and DR_n. V_n in the primary VM/DR map, depending on its state, indicates whether a corresponding block in memory 74 contains valid primary volume data. For example, when set to logical 1, V₂ of the primary VM/DR map 80 indicates that block 2 of memory 74 contains valid primary volume data, and when set to logical 0, V₂ of primary VM/DR map 80 indicates that block 2 of data memory 74 contains no valid primary volume data. V₂ of VM map 82, when set to logical 1, indicates that block 2 of memory 76 contains a valid copy of data that existed in block 2 of memory 74 at the time the PIT backup copy was first created or at the time the PIT backup copy was last refreshed. V₂ of VM map 82, when set to logical 0, indicates that block 2 of memory 76 does not contain a valid copy of data.

[0034] M_n in each entry of primary VM/DR map 80, depending upon its state, indicates whether data has been written to or modified in the corresponding block n of memory 74 since the last time the PIT backup copy of the primary data volume was created or refreshed. For example, when set to logical 1, M₃ of primary VM/DR map 80 indicates that data has been written to block 3 of memory 74 since the last time PIT backup copy was refreshed. Likewise, when set to logical 0, M₃ of the primary VM/DR map 80 indicates that data has not been written or modified in block 3 of the memory 74 since the last time a PIT backup copy was refreshed.

[0035] DR_n in each entry, depending on its state, indicates whether the corresponding block n of memories 74 and 78 are the subject of an in-progress write-data transaction. For example, when set to logical 1, DR₅ of VM/DR map 80 indicates that memory block 5 of memories 74 and 78 are the subject of an in-progress write-data transaction. When set to logical 0, DR₅ of primary VM/DR map 80 indicates that memory block 5 of memories 74 and 78 are not the subject of an in progress write transaction.

[0036] Multiple bits in entries of primary VM/DR map 80 can be set by host node 62 using a single write operation to memory that stores VM/DR map 80. In one embodiment, multiple bits of an entry can be updated with a single write operation since the bits are stored adjacent to each other in the memory that stores VM/DR map 80. More particularly, host node can write a multi-bit binary value to an entry of VM/DR map 80 using a single write operation thereby updating or changing one or more bits of the entry. For example, suppose V₃, M₃, and DR₃ of entry 3 of VM/DR map 80 are initially set to logical 1, logical 0, and logical 0, respectively. In other words, entry 3 of VM/DR map 80 stores binary value “100” where the most significant bit (i.e., the left most bit) corresponds to V₃ and the least significant bit (i.e., the right most bit) corresponds to DR₃. Suppose further that host node 62 generates a write-data transaction for writing data to block 3 of memory 74. Before or after host node 62 writes data to block 3, host node 62 can write the binary value “111” to entry 3 of VM/DR map 80 thereby changing the state of M₃ and DR₃ from logical 0 to logical 1 (V₃ is maintained at logical 1 even though V₃ may be overwritten) using a single write operation. In contrast host node 12, described in the background section above, requires separate operations to update M₃ of VM map 30 and DR₃ of DR map 34 where one operation changes the state of M₃ of VM map 30 while the other operation changes the state of DR₃ of DR map 34. It is noted that host node 62 in this example could write the binary value “x11” to entry 3 of VM/DR map 80 instead of the binary value “111.” In this alternative embodiment, V₃ could be masked from the write operation so that it is not overwritten by the binary value “x.” Indeed, any one of the V_n, M_n, or DR_n bits can be masked when a multi-bit binary value is written to VM/DR map.

[0037] In yet another alternative embodiment, multiple multi-bit entries in VM/DR map 80 can be modified by host node 62 via a single write operation. For example, suppose host node 62 generates a write-data transaction for writing data to block 4 of memory 74 after host node writes data to block 3 of memory 78. Further suppose that entries 3 and 4 of VM/DR map 80 store binary values “111” and “100,” respectively, at the time host node 62 generates the write-data transaction for writing data to block 4 or memory 74. DR₃ is set to logical 1 thus indicating that block 3 in memories 74 and 76 are the subject of a write data transactions when, in fact, they are not. VM/DR map 80 can be updated, before or after data is modified in block 4 of memory 78 in accordance with the new write-data transaction, by writing binary values “110” and “111” to entries 3 and 4, respectively, via a single write operation since entries 3 and 4 rather than two write operations where the first operation writes binary value “110” to entry 3 and the second operation writes binary value “111” to entry 4. Entries 3 and 4 can be updated with a single write operation since entries 3 and 4 of map 80 are adjacent to each other in the memory that stores VM/DR map 80. It is noted that V₃, V₄, and M₃ are not updated in the sense that the value of these bits do not change after the single write operation to VM/DR map 80. After the single write operation, DR₃ is properly updated to logical 0 thus indicating that block 3 in memories 74 and 78 are no longer subject to a write-data transaction, and M₄ is updated to logical 1 thus indicating that data of block 4 in memory 74 has been modified.

[0038] Operational aspects of tracking in-progress writes is best understood with reference to the flow chart of FIG. 6 which shows operational aspects of one embodiment of the present invention. Tracking in-progress writes begins after host node 62 receives a request to write data to the primary data volume. Host node 62 identifies the block n of memory 74 that is the target of the received write request. Host node 62 generates a write-data transaction for writing data to block n in step 90. Host node 62, in step 92, accesses VM map 82 to determine the state of V_n. If V_n is set to logical 0, then host node 62 copies the data of block n in memory 74 to block n of memory 76 and subsequently sets V_n of VM map 82 to logical 1 with a write operation to the memory that stores map 82. If V_n of VM map 82 is determined to be set to logical 1 in step 92 or after V_n is set to logical 1 in step 96, the process proceeds to step 100 were host node sets M_n and DR_n bits of VM/DR map 80 using a single write operation to memory that stores VM/DR map 80. Thereafter in steps 102 and 104, block n of memories 64 and 68 are updated with the write data of the request in step 90. It is noted that steps 102 and 104 can be reversed in order or performed in parallel. Eventually, when block n of memories 64 and 68 have been updated with the new data, host node 62, in asynchronous fashion, accesses entry n of VM/DR map 80 and sets the DR_n bit to logical 0, thus indicating that the data write transactions to block n of memories 74 and 78 have completed and block n of memory 74 stores valid and modified data.

[0039] Because only one write operation is needed to update the M_n and DR_n bits of primary VM/DR map 80, the amount of processing overhead on host node 62 is reduced when compared to host node 12 in FIG. 1. Moreover, because write operation of step 100 shown in FIG. 6 requires a single access to a physical address within memory that stores the primary VM/DR map 80, the amount of time needed to perform M_n and DR_n updating is substantially reduced.