US 8,548,944 B2Grant
De-duplication based backup of file systems
Issue Date:2013-10-01
•21 Claims
•11 Drawing Sheets
Abstract
File system backups are performed by copying information describing changes in the file system since a previous point in time. To restore data, a virtual restored file system (VRFS) structure is created corresponding to a snapshot of data copied from the file system that is stored in the backup file system. A client can read the data stored in the backup file system pointed at by the VRFS structure as well as write to the backup file system. Multiple clients can share blocks of data stored on the backup file system via the same VRFS structure or via multiple VRFS structures. If a client writes to a data block pointed at by a VRFS, a copy of the data block is made to which the client makes changes so that the data in the original data blocks is saved for other clients.
Metadata
Assignee
- Delphix Corp.
Inventor
- Jedidiah Yueh
Application Information
Application Number:US 13/183,131
Filing Date:2011-07-14
Priority Date:2010-07-15
Art Unit:2168
Classifications
IPC:
G06F7/00G06F17/00
Patent Drawings (11 sheets)
Description
CROSS REFERENCES TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional Patent Application No. 61/364,652 filed Jul. 15, 2010, which is incorporated by reference in its entirety.
BACKGROUND
[0002] This invention relates generally to backup and restore of file systems, and in particular to storage efficient backup.
[0003] File systems provide permanent storage for data that is critical to an organization. Therefore, file systems form an important part of an organization's information technology. File systems are used for persistent storage of data by applications used in an organization, for example, email servers, document management systems, source code management systems and the like. Organizations make copies of the data stored in the file systems to avoid loss of data in case the original data in the file system is destroyed or corrupted. The process of making copies of the data is called a backup. Conventional systems that perform backups of data store the data in storage devices with large storage capacity, for example, magnetic tapes. These devices typically have slow access time. Conventional systems restore data from the backup storage in case of loss of data stored in the original file system. The process of restoring data of a large file system can take significant amount of time resulting in loss of business during the time the data is unavailable.
SUMMARY
[0004] Embodiments perform backups of a file system by copying changed blocks of data from the file system to a storage system. In case of loss of data or if the source file system is unavailable, a virtual restored file system (VRFS) structure is created using blocks of data stored in the storage system. The VRFS structure points at data blocks copied at various points in time. Since only changed blocks of data are stored in the storage system, the amount of data copied during backup is small. Also restoration of data is efficient because creation of the VRFS structure takes significantly less time than conventional restoring of file systems from a backup storage device. Furthermore, a client can directly read or write in the storage system while the source file system is down, instead of waiting for data to be restored on another file system.
[0005] In an embodiment, the backup system receives data blocks for a plurality of point-in-time copies of a source file system. The backup system stores the data blocks in a storage system. A data block can be associated with multiple point-in-time copies, for example, if a data block did not change since a previous point-in-time copy was obtained, the data block may be associated with the previous point-in-time copy as well as with a subsequent point-in-time copy. The backup system receives a request to restore information from the source file system for a target system. The backup system generates a virtual restored file system by linking a set of files to stored data blocks of the storage system and mounting the set of files on the target system.
[0006] In an embodiment, the backup file system performs backups of multiple source file systems. In an embodiment, a client can update the data stored on the backup file system pointed at by the VRFS structure. Multiple clients can share blocks of data stored on the backup file system via the same VRFS structure or via multiple VRFS structures. If a client writes to a data block pointed at by a VRFS, the changed data block is copied. This allows the original data blocks to be used by other clients.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] FIG. 1( a) (prior art) shows a conventional approach towards performing backups of file systems using a backup storage.
[0008] FIG. 1( b) (prior art) shows a conventional approach towards making data stored in a backup storage available for use in case of loss of data of the original file system.
[0009] FIG. 2( a) is a diagram illustrating how backups are performed using a de-duplication based backup system, in accordance with an embodiment of the invention.
[0010] FIG. 2( b) is a diagram illustrating how data backed up using a de-duplication based backup system is made available for use by a client in case of loss of data, in accordance with an embodiment of the invention.
[0011] FIG. 3 is a diagram illustrating how blocks of data copied at different points in time are organized in a backup file system to allow provisioning of a VRFS, in accordance with an embodiment of the invention.
[0012] FIG. 4 illustrates how data blocks stored in a backup file system can be shared by file structures created for different VRFS structures, in accordance with an embodiment of the invention.
[0013] FIG. 5 illustrates a system architecture for a de-duplication based backup system, in accordance with an embodiment of the invention.
[0014] FIG. 6 is a flowchart of a process for receiving data from a production server for performing backups using a de-duplication based backup system, in accordance with an embodiment of the invention.
[0015] FIG. 7 is a flowchart of a process for provisioning a VRFS structure corresponding to a given point-in-time, in accordance with an embodiment of the invention.
[0016] FIG. 8 illustrates an embodiment of a computing machine that can read instructions from a machine-readable medium and execute the instructions in a processor or controller.
[0017] The figures depict various embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.
DETAILED DESCRIPTION
[0018] FIG. 1( a) shows a conventional approach towards performing backups of file systems storing data, for example, data used by production servers 115. Multiple file systems 110 can be backed up using a single backup system 100. Clients 140 access the data stored in file systems 110 by interacting 145 with the production server 115. The client 140 that accesses the file system 115 can be executing on a computer system separate from the production server 115 or it can be a program executing within the production server 115. The data from the file systems 110 is periodically copied 160 to the backup system 100. The backup system 100 stores the backup data it receives on a backup storage device 130, for example, a magnetic tape storage device.
[0019] FIG. 1 and the other figures use like reference numerals to identify like elements. A letter after a reference numeral, such as “110A,” indicates that the text refers specifically to the element having that particular reference numeral. A reference numeral in the text without a following letter, such as “110,” refers to any or all of the elements in the figures bearing that reference numeral (e.g. “110” in the text refers to reference numerals “110A” and/or “110B” in the figures).
[0020] FIG. 1( b) illustrates the conventional approach towards making data available from a backup system 100 using a restore operation. If there is a system failure 125 of the production server 115(a) or the file system 110(a), the client 140 is unable to use the data of the file system 110(a). The latest backup of the data from file system 110(a) is typically retrieved 165 by the backup system 100 from the backup storage 130 and restored 170 onto a file system 110(c). The client 140 can use 175 the data available on the file system 110(c), which represents a recent copy of the data from the file system 110(a). Typically, the performance of backup storage devices 130 is slower compared to storage devices used for file systems 110 of production system. As a result, the overall time taken to restore the data from the backup storage device 130 to the file system 110(c) can be significant, resulting in loss of potential business during the restore operation.
[0021] FIG. 2 illustrates a de-duplication based backup system 200 that eliminates the slow restore operation 170 for making data available in case of file system failures. As shown in FIG. 2( a), the de-duplication based backup system 200 receives from the file systems 110, blocks of data that have changed since a previous point in time. A block of data can store any information in the file system including data, executable program code, or instructions. In one embodiment, the changed blocks of data are received periodically. In these embodiments, each set of data blocks received by the de-duplication based backup system 200 corresponds to data blocks that changed since the last time the blocks were extracted from the file system 110. The operation of receiving the blocks of data from the production server is also referred to as the changed-block copy operation. In an embodiment, the changed blocks of data can be sent to the de-duplication based backup system 200 by an agent running on the production server 110. That agent can either identify changed files and portions of files (data blocks), by either operating at the file system level or at a volume level. The data blocks may or may not be fixed in size.
[0022] Alternatively, the de-duplication based backup system 200 sends a request to execute application programming interface (API) calls using libraries on the production server 115 to retrieve the changed blocks. In this configuration there is no need to have a backup agent on the production server 115 since the process is initiated and controlled by the de-duplication based backup system 200. A de-duplication based backup system 200 that does not require a backup agent on the production server is called an agent-less backup system. The de-duplication based backup system 200 stores the changed blocks of data in a backup file system 210. Typically, the amount of data corresponding to the changed blocks of the file system 110 is significantly less than the entire data of the file system 110. As a result the amount of data stored in the backup file system 210 can be significantly less than the amount corresponding to a full backup of the file system as shown in FIG. 1 . That amount can be further reduced via compression or other data reduction techniques, such as using hashes as pointers to already stored identical blocks. Accordingly, the backup file system 210 can utilize a storage device that is significantly faster than a typical backup storage device 130. In an embodiment, the backup file system can be stored on a storage device that is at least as fast as the storage device used by the file system 110. For example, the backup file system 210 can be stored on a hard drive or a solid state drive (SSD). Storing the backup data on a fast storage device enables the possibility of allowing applications running on clients 140 to directly access the data stored in the de-duplication based backup system 200 in case the original file system 110 isn't available without significantly degrading the performance of those applications.
[0023] FIG. 2( b) illustrates how a de-duplication based backup system 200 makes data available for a client 140 to use. If there is a failure 125 of the file system 110(a) or the production server 115(a) the client 140 can directly access the data stored in the de-duplication based backup system 200. The de-duplication based backup system 200 presents a set of data blocks that correspond to the latest data of the file system 110(a). In an embodiment, the client can request the de-duplication based backup system 200 to provide access to data corresponding to a particular point in time that is different from the latest point in time. The data blocks corresponding to a point in time may be presented to a server and operating system as either a network mounted file system (e.g. over NFS), or as a disk volume (e.g. over FC or iSCSI).
[0024] The set of data blocks representing the file system 110 at a particular point in time may comprise data blocks copied by the de-duplication based backup system 200 from the file system 110 at different points in time depending on when the data block was changed. The set of data blocks corresponding to the data of the file system 110(a) can be made available to the client 140 via a file-sharing system 220 that allows the client 140 to access the backup file system 210. The operation of making the set of data blocks available is called provisioning of a virtual restored file system (VRFS) corresponding to the data of the failed file system 110(a). For example, the file-sharing system 220 may provide access to the backup file system 210 via network file system (NFS) mount operation. A portion of the backup file system 210 may be NFS-mounted to the client 140, thereby allowing client 140 to access files stored on the mounted portion of the backup file system 210. The client 140 can read as well as write to the mounted portion of the backup file system 210. All or part of the file system at a specific point in time may be mounted.
[0025] The process illustrated by FIG. 2( b) eliminates the expensive data restore operation 170 illustrated in FIG. 1( b) that requires copy 170 of a large amount of data from a backup storage device. The backup file system 210 is used to store the backup copies and also can be accessed directly by the client 140 for read/write purposes. A virtual restored file system (VRFS) structure is created that allows the client 140 to access a snapshot of the data stored on the backup file system 210 that is copied from the file system 110. The time required to provision a VRFS corresponding to the failed file system 110(a) is significantly less than the time required for a restore operation 170. In an embodiment, all of the steps to mount the file system or volume can be fully automated, by having a toolkit on the target server (the toolkit comprising software modules), which has the privileges to mount the file system or volume. In an embodiment, when the production server 115(a) or file system 110(a) that previously failed starts working again, the changes made to the backup file system 210 by the client 140 are transferred to the production server 115(a) and applied to the file system 110(a). In an embodiment, applying the changes comprises copying the data blocks that were changed in the de-duplication based backup system 200 over to the file system 110a and replacing the corresponding data blocks of the file system 110a with the received data blocks. Subsequently, the client 140 can revert back to using the file system 110(a) of the production server 115(a).
[0026] FIG. 3 is a diagram illustrating how blocks of data copied at different points in time are organized in a backup file system to allow provisioning of a VRFS. The file system 110 shown in FIG. 4 comprises data blocks F1, F2, F3, and F4. Assume time T0 is the first point-in-time at which the backup file system 210 is used to backup data from file system 110. At time T0 all blocks from the file system 110 are copied 315(a) to the backup file system and stored as data blocks F11, F12, F13, and F14 respectively. Assume that between time T0 and T1, blocks F2 and F4 are changed. At time T1, only data blocks F2 and F4 are copied 315(a) from file system 110 to the backup file system 210 and stored as blocks F22 and F24 respectively. Similarly, assuming data blocks F3 and F4 change between time T1 and T2, at time T2, these blocks are copied to the backup file system 210 and stored as blocks F33 and F34 respectively.
[0027] If the file system 110 fails after time T2, a snapshot of data from the file system 110 corresponding to the point in time T2 can be made available to a client. The data structure used for provisioning a VRFS corresponding to point T2 is shown as a set 320 of pointers to the data blocks. Each pointer Pi points at the latest copy of data block i modified before time T2. For example, pointer P1 points at data block F11 copied at time T0, pointer P2 points as block F22 copied at time T1, and pointer P3, P4 point at blocks F33, F34 copied at time T2. If the client 140 writes to a data block pointed at by the VRFS 320, a copy of the corresponding data block is made. The client 140 subsequently reads/writes to the copy of the data block. This allows the original copy of the block stored in the backup file system 210 to be saved for use by other clients, for example, to provision other VRFS structures.
[0028] FIG. 4 illustrates how multiple snapshots of the file system 110 can be supported at the same time by the backup file system 210 if necessary. The VRFS 320(a) corresponding to the file system snapshot for time T2 points to the copies of data blocks obtained on or before time T2. For example, pointer V1 points to block F11 copied at time T0, pointer V2 points to block F22 copied at time T1, and pointers V3, V4 that point at blocks F33, F34 respectively. On the other hand, VRFS 320(a) corresponding to the file system snapshot for time T1 points to the latest copy of a data block copied on or before time T1. Accordingly, pointer U1 points to block F11 copied at time T0, pointer U2 points to block F22 copied at time T1, pointer U3 points to block F13 copied at time T0 and pointer U4 points at block F24 copied at time T1. Data blocks of the backup file system 210 can be shared by multiple VRFS structures corresponding to different points in time. For example, block F22 is shared by the VRFS corresponding to time T1 as well as VRFS corresponding to time T2. Furthermore, a VRFS can be shared by multiple clients 140. In one embodiment, these different points in time can all be mounted by a single target server or by multiple target servers. In one embodiment, multiple points in time can be mounted on a single server under multiple mount points or under a single mount point, with directories representing multiple points in time.
[0029] The changes made to the backup file system 210 using a VRFS 320 can be merged back to the source file system 110. However if there are multiple copies of a data block that have been updated by clients 140, the updates to the data block may conflict with each other. Merging these copies of the data block to the original file system 110 may require further analysis of the copies of the data block to determine if the same portion of the data block is updated differently in the two copies. Conflicts may have to be resolved to determine which update from a set of conflicting updates is applied to the data while merging changes from the backup file system 210 to the original file system 110.
[0030] In some embodiments, a filter driver can monitor changes to data blocks and store the history of the changes using a suitable format. In some embodiments, variable sized data segments are copied from the file system 110 to the backup file system 210. A variable sized data segments can be smaller than a complete data block if the amount of data that changed since the last copy operation is small. As a result, a copy of variable sized segments can take less time and can be performed more frequently. For example, an operation that copies all the changed blocks since a previous changed-block copy operation can be performed once a day but the backup of the history of changes can be performed every few minutes. A restore of a file system for a given time Tm can be performed by taking data blocks corresponding to the most recent block copy taken at time Tn before the time Tm. The changes corresponding to history of changes copied after Tn and before Tm can be applied to the data copied by the changed-block copy operation to obtain a snapshot of data close to a time point before time Tm. In some embodiments, combinations of variable sized data segments and fixed sized blocks are retrieved from the file system 110 and stored on the backup file system in either compressed form or without compression. In some embodiments, changed blocks representing several different points in time are stored local to the original file system, for instance if the original file system is not connected to the network, and then sent in a batch job while maintaining the information that allows recreation of the data corresponding to multiple points in time.
System Architecture
[0031] FIG. 5 shows a high level block diagram illustrating a system environment suitable for performing backups of file systems using a de-duplication based backup system 200 and making the data stored in the backup system available to clients. The system environment comprises one or more production server 115, a de-duplication based backup system 200, an administration system 540, and one or more clients 140. Systems shown in FIG. 3 can communicate with each other if necessary via a network.
[0032] A production server 115 is a computer system that performs operations on data stored in a file system 110. For example, the production server 115 may comprise an email server, or a database server, an enterprise resource planning (ERP) application, a mobile device, or any other application that stores data. In one embodiment, a production server 115 refers to any computer system that stores data on a file system 110. The production server 115 comprises a file system 110, a backup agent 345, and a file system library 385. In alternative configurations, different and/or additional modules can be included in a production server 115.
[0033] In some embodiments, the file system 110 may be a local storage of the production server 115 or a network attached storage. The file system library 385 provides APIs useful for extracting information from the file system 110. The backup system 200 calls the APIs of the file system library 385 to copy data from file system 110 to the backup system 200. In one embodiment, the backup system 200 calls the file system library 385 APIs to determine the number of blocks of the file system that changed since a given point in time to determine whether to obtain the changed blocks. In another embodiment, the backup system 200 periodically copies at a fixed interval, blocks of file system 110 that changed since the last time a similar operation was performed. In one embodiment, the file system library 385 mounts the file system 110 of the production server 115 on the de-duplication based backup system 200 using a file sharing system similar to the file-sharing system 220. Mounting file system 110 on the backup system 200 allows transfer of data stored on the file system 110 to the backup system 200 using file system APIs.
[0034] In some embodiments the de-duplication based backup system 200 comprises one or more de-duplication based backup systems 200 that share a mapping of all the blocks to prevent duplicates from being stored across all the de-duplication based backup systems 200. For example, use of multiple de-duplication based backup systems 200 provides fault-tolerance or high-availability of the systems in case of failures. In some embodiments, the de-duplication based backup systems 200 add structured redundancy via mirroring, RAID, or other mechanism to protect the data stored in the de-duplication based backup systems 200.
[0035] In some embodiments, the production server 115 includes a backup agent 545 that coordinates transfer of data blocks from the file system 110 to the de-duplication based backup system 200. For example, the backup agent 545 can periodically monitor data blocks of the file system 110 for changes to the data blocks to identify data blocks to be copied to the backup system 200. In one embodiment, the backup agent 545 writes changed blocks of file system 110 to the backup system 200 at a fixed time interval. In another embodiment, the backup agent 545 writes changed blocks of file system 110 to the backup system 200 if the number of changed blocks exceeds a threshold number or the size of the changed data exceeds a threshold value. In another embodiment all change blocks are sent as they are stored and then stored in the de-duplication backup system as a log of change blocks that can be applied to create different points in time.
[0036] Some embodiments may not include all the modules shown in FIG. 5 , for example, the backup agent 345. The copy of data from the file system 110 that does not include the backup agent 545 is initiated by the block copy manager 510. These embodiments are called agent-less de-duplication based backup systems. These embodiments are beneficial since they require less maintenance for production server 115 by eliminating the installation and maintenance of the backup agent 545. In some embodiments the agent-less system requires less load on the production server 115 by eliminating the backup agent 545 that can consume system resources. On the other hand, a system that uses a backup agent 545 may require fewer communications between the de-duplication based backup system 200 and the production system 115 since the block copy manager 510 is not required to request backup data from the production server 115 each time. In a system where read/write operations on the file system 110 are less frequent, the backup agent 545 can significantly reduce the number of messages exchanged between the production server 115 and the de-duplication based backup system 200.
[0037] In one embodiment, the file system library 585 includes a filter driver that monitors all update operations of the file system 110. The filter driver monitors all the read/write operations performed on the file system 110. The filter driver includes the logic to track the updates made to the file systems and determine when to copy the updates to the backup system 200. The filter driver can copy the changed blocks of the file system 110 to the backup system 200 either on a periodic basis or when the amount of changed data or number of changed blocks exceeds a threshold.
[0038] The data stored in the backup file system 210 can be exposed to a client 140, for example, by using a file-sharing system 220. For example, the data stored in the backup file system 210 can be mounted on a computer system running the client 140. The client 140 can be an application or a program running on a computer system. The de-duplication based backup system 200 retrieves information available in the file system 110 of the production server 115 and stores it in the backup file system 210. The information retrieved from the file system 110 comprises data that changed since a previous retrieval for a subset of the file system 110 or for the entire disk storing the file system 110. In some embodiments, the file system 110 can be mounted as an iSCSI volume that represents a network disk (instead of a network file system). If the information corresponding to the entire disk is retrieved from the file system 110, the retrieved information can be used to boot or start up a new computer system. In one embodiment, the information retrieved from the file system 110 can be a portion of the file system 110 that is associated with an application, for example, an email server. In some embodiments, APIs provided by an application can be used to monitor and process changes to the file system made by the application, for example, history of changes made by the application can be obtained and stored. Systems and methods for retrieving data associated with database systems to a database storage systems are described in the U.S. application Ser. No. 12/603,541 filed on Oct. 21, 2009, which is incorporated by reference in its entirety.
[0039] In some embodiments, the information backed up from the file system 110 can include a set of one or more files. The de-duplication based backup system 200 retrieves changed blocks of data from the set of files and copies them onto the backup file system 210. In some embodiments, the production server 115 can have applications that have file handles open for files stored in the file system 110 when the block copy manager 510 or the backup agent 545 needs to copy data from the file system 110. An open file handle for a file indicates that the application can be in the process of changing the data in the file. Depending on the state of the application, the data of the file may not be in a consistent state. In these embodiments a previous state of the data blocks of the file system 110 that represents a consistent state of the file system is used to perform the backup.
[0040] The de-duplication based backup system 200 includes an auto-discovery manager 520, a VRFS manager 575, a block copy manager 510, a file sharing manager 570, a storage allocation manager 565, and a backup file system 210. In alternative configurations, different and/or additional modules can be included in the de-duplication based backup system 200. The block copy manager 510 interacts with the production server 115 by sending a request to the file system library 585 to retrieve information representing a point-in-time copy of the file system 110 or a portion of the file system 110. The block copy manager 510 stores the retrieved information in the backup file system 210. The VRFS manager 575 builds the data structures to create a virtual restored file system 420 based on blocks of data stored in the backup file system 210. The storage allocation manager 565 allocates storage for the information received to store the information in the backup file system 210. The storage allocation manager 565 keeps track of the various versions of each block of data that may be obtained from the production server 115. The storage allocation manager 565 also copies blocks of data if necessary, for example, when a client writes to a data block. If a block of data is copied for read-only purposes, the storage allocation manager 565 allocates only sufficient storage to keep a pointer of reference to the exiting block of data. However, if an attempt to write to the copied block of data is made, the storage allocation manager 565 allocates sufficient storage to make a copy of the block of data to avoid updating the original block of data. In some embodiments, the data blocks retrieved by the de-duplication based backup system 200 may be compressed before they are stored in the backup file system 210.
[0041] The file sharing manager 570 allows files stored in the de-duplication based backup system 200 to be shared across computers that may be connected with the de-duplication based backup system 200 over the network. The file sharing manager 570 uses the file-sharing system 220 for sharing files with clients 140. An example of a system for sharing files is a network file system (NFS). A system for sharing files may utilize fiber channel Storage area networks (FC-SAN) or network attached storage (NAS) or combinations and variations thereof. The system for sharing files may be based on small computer system interface (SCSI) protocol, internet small computer system interface (iSCSI) protocol, fiber channel protocols or other similar and related protocols. In some embodiments, the de-duplication based backup system 200 may utilize a logical volume manager. In an embodiment, files are organized in a format emulating a given file system disk layout, such as the file system of WINDOWS operating system called NTFS or the UNIX file system (UFS).
[0042] The auto-discovery manager 520 allows the de-duplication based backup system 200 to find file systems that need backups for a giver computer system or a set of computer systems. For example, the de-duplication based backup system 200 may be provided with information identifying a computer system, for example, the network address or a machine. The de-duplication based backup system 200 can automatically discover various file systems used by the computer system that may need backup. In some embodiments, the de-duplication based backup system 200 is provided information identifying applications that need backup on the computer system. The de-duplication based backup system 200 can identify all the relevant the files that are used by the identified applications that need backup. For example, the de-duplication based backup system 200 may be requested to perform backups of an email server on a given machine. The auto-discovery manager 520 can identify the files used by the email server to store email related data as well as email server configuration information that needs to be backed up. In an embodiment, the auto-discovery manager 520 can take information identifying a set of computer systems and discover the file systems associated with all the computer systems that need backup. In some embodiments, the auto-discovery manager 520 is provided access to a network to perform discovery of various machines on the network including desktop computers, laptop computers, servers etc. and the file systems on the discovered computers for which backup needs to be performed. In an embodiment, the information discovered by the auto-discovery manager 520 is reviewed by a system administrator to determine the set of file systems that need to be backed up to the backup file system 210.
[0043] The administration system 540 comprises a user interface 595 that allows a system administrator to perform administrative operations for example, specifying production systems that need to be backed up or client systems 140 that need access to the backup file system 210 in case of data loss. In some embodiments, the user interface 595 allows a system administrator to configure parameters for the auto-discovery manager 520 to perform discovery of file systems 110 for backup. The user interface 595 also allows a system administrator to specify the frequency at which data is copied from the production server 115 for performing backup.
[0044] FIG. 6 shows a flowchart of the process illustrating how a block copy manager 510 receives changed blocks from the production server 115 and saves them. The block copy manager 510 sends 610 a request for blocks changed since a previous changed-block copy operation to the production server 115. In some embodiments, the request may include information identifying the files for which change information is retrieved or information identifying a portion of the file system 110 for which changed blocks need to be retrieved. In some embodiments, the request comprises a remote call to execute instructions (or code) in the file system library 585. The file system library 585 executes instructions to identify the changed data blocks and sends the requested data to the de-duplication based backup system 200. The block copy manager 510 receives 620 the information comprising the changed blocks. The block copy manager 510 requests the storage allocation manager 565 to save 630 the blocks of data in a storage efficient manner. The block copy manager 510 waits for the next iteration of the changed-block copy operation and repeats the above steps after the waiting period. In one embodiment special processing can be applied either before, during, or after the backup process or before, during, or after the mounting process to make a point-in-time copy available. Special processes can include but are not be limited to encryption, identification and elimination of confidential data, elimination of unnecessary or temporary data (such as empty blocks), or other processes.
[0045] In some embodiments, the backup agent 545 of the production server 115 determines whether to initiate the changed-block copy operation. For example, the backup agent 545 can monitor the changes made to a portion of the file system 110, for example, a set of files. If the amount of changes as measured by the number of blocks changed or an amount of information indicating history of changes exceeds a threshold value, the backup agent may determine that a changed-block copy operation needs to be initiated. In another embodiment, even though the amount of changes may not exceed a threshold value, the backup agent 545 may initiate a changed-block copy operation if the time since a previous changed-block copy operation exceeds a threshold value. The backup agent can send the information comprising changed blocks of data or history of changes to the block copy manager 510. The block copy manager 510 sends a request to the storage allocation manager 565 to save the information. The storage allocation manager 565 saves the information in the backup file system 210.
[0046] FIG. 7 illustrates how a virtual restored file system structure can be created for a client 140 based on the backup information saved in the backup file system 210. The VRFS manager 575 receives 700 a request to create a VRFS for a given time point Tn, for example, in case there is a failure of a file system 110 that was backed up. The VRFS manager 575 identifies 710 the recent most block copy for a time point Tj that is before time Tn. In some embodiments, if information describing history of changes that occurred between the time periods Tj and Tn is available, the information is identified 720. Based on the block copy for time point Tj and the history of changes, a read write file structure is created 730 by making storage efficient copy of data blocks. For example, the file structure may include pointers to blocks of data stored in backup file system 210. In some embodiments, data changes based on history of changes are applied to the blocks of data to obtain a copy of the data block corresponding to a more recent point in time compared to Tj. A handle to the file structure created is sent 740 to the client 140. The client can perform read or write operations using the VRFS file structure.
Alternative Embodiments
[0047] In some embodiments, the de-duplication based backup system 200 can be used as a production system in case the original production server 115 fails. The de-duplication based backup system 200 provides a consolidated storage system for several applications or file systems from several computer systems. All applications that can be executed on the original file systems 110 can be executed using the backup file system 210. If backup file system 210 is used as a consolidated file system, use of a high-performance storage device (for example, solid-state drive (SSD)) to store the backup file system 210 improves the performance of all the clients 140 using the data stored in the backup file system 210. This allows the enterprise to improve performance of several clients 140 by investing in a single consolidated system instead of a large number of individual systems. For example, replacing the hard drives on several machines by SSD can be lot more expensive than replacing the drive of a single consolidated system. In some embodiments, the backup file system 210 can be used as a high-availability system that can be used by clients, for example, when the production server 115 is not available, even if there is no loss of data stored in file system 110.
[0048] In some embodiments added storage caching layers can improve the performance of the backup file system, especially if multiple blocks are shared across different systems accessing different points in time or different file systems.
[0049] The entire backup file system 210 can be copied to a conventional backup storage for performing a backup of the backup file system 210 (for example, as shown in FIG. 1 ). As a result, a conventional backup storage operation can also be performed for a set of file systems 110. The backup of the backup file system 210 using conventional backup techniques can be much simpler process than performing individual backups of all the file systems 110 of the various production servers 115 of an enterprise. This simplicity is obtained due to the fact that a single system is being backed up and also because the amount of data being backed up is smaller since it represents only the changed blocks of data from the original file systems which can be further reduced in size by using data compression. In some embodiments the backup file system can be replicated in whole or in part to another storage device.
[0050] In some embodiments, instead of creating a VRFS structure for a client to use, the data represented by the VRFS can be exported to a different file system and the client allowed to access the exported data stored in the other file system. Exporting the VRFS data can be beneficial, for example, if the applications using the data require large amount resources that are not available in the de-duplication based backup system 200.
Computing Machine Architecture
[0051] FIG. 8 is a block diagram illustrating components of an example machine able to read instructions from a machine-readable medium and execute them in a processor (or controller). Specifically, FIG. 8 shows a diagrammatic representation of a machine in the example form of a computer system 800 within which instructions 824 (e.g., software) for causing the machine to perform any one or more of the methodologies discussed herein may be executed. In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.
[0052] The machine may be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a smartphone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions 824 (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute instructions 824 to perform any one or more of the methodologies discussed herein.
[0053] The example computer system 800 includes a processor 802 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), one or more application specific integrated circuits (ASICs), one or more radio-frequency integrated circuits (RFICs), or any combination of these), a main memory 804, and a static memory 806, which are configured to communicate with each other via a bus 808. The computer system 800 may further include graphics display unit 810 (e.g., a plasma display panel (PDP), a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)). The computer system 800 may also include alphanumeric input device 812 (e.g., a keyboard), a cursor control device 814 (e.g., a mouse, a trackball, a joystick, a motion sensor, or other pointing instrument), a storage unit 816, a signal generation device 818 (e.g., a speaker), and a network interface device 820, which also are configured to communicate via the bus 808.
[0054] The storage unit 816 includes a machine-readable medium 822 on which is stored instructions 824 (e.g., software) embodying any one or more of the methodologies or functions described herein. The instructions 824 (e.g., software) may also reside, completely or at least partially, within the main memory 804 or within the processor 802 (e.g., within a processor's cache memory) during execution thereof by the computer system 800, the main memory 804 and the processor 802 also constituting machine-readable media. The instructions 824 (e.g., software) may be transmitted or received over a network 826 via the network interface device 820.
[0055] While machine-readable medium 822 is shown in an example embodiment to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store instructions (e.g., instructions 824). The term “machine-readable medium” shall also be taken to include any medium that is capable of storing instructions (e.g., instructions 824) for execution by the machine and that cause the machine to perform any one or more of the methodologies disclosed herein. The term “machine-readable medium” includes, but not be limited to, data repositories in the form of solid-state memories, optical media, and magnetic media.
Additional Configuration Considerations
[0056] The foregoing description of the embodiments of the invention has been presented for the purpose of illustration; it is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Persons skilled in the relevant art can appreciate that many modifications and variations are possible in light of the above disclosure.
[0057] Some portions of this description describe the embodiments of the invention in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combinations thereof.
[0058] Any of the steps, operations, or processes described herein may be performed or implemented with one or more hardware or software modules, alone or in combination with other devices. In one embodiment, a software module is implemented with a computer program product comprising a computer-readable medium containing computer program code, which can be executed by a computer processor for performing any or all of the steps, operations, or processes described.
[0059] Embodiments of the invention may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, and/or it may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a tangible computer readable storage medium or any type of media suitable for storing electronic instructions, and coupled to a computer system bus. Furthermore, any computing systems referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.
[0060] Embodiments of the invention may also relate to a computer data signal embodied in a carrier wave, where the computer data signal includes any embodiment of a computer program product or other data combination described herein. The computer data signal is a product that is presented in a tangible medium or carrier wave and modulated or otherwise encoded in the carrier wave, which is tangible, and transmitted according to any suitable transmission method.
[0061] Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the invention be limited not by this detailed description, but rather by any claims that issue on an application based hereon.
Claims
Post-issuance certificates present
The text shown below may not reflect later post-issuance changes.
What is claimed is:
1. A method for performing backup of file systems, the method comprising:
receiving data blocks for a plurality of point-in-time copies of a source file system, each point-in-time copy of the source file-system obtained by extracting data blocks from the source file-system that changed since a previous point-in-time copy was extracted, the source file system comprising at least a source file;
storing the data blocks on a storage system, the stored data blocks comprising one or more versions of a data block, each version corresponding to a point-in-time copy;
receiving a request to restore information obtained from the source file system for a target system; and
responsive to receiving the request to restore, creating a virtual restored file system comprising a set of files including a restored file corresponding to the source file, the creating comprising:
linking the restored file to a plurality of the data blocks stored on the storage system, the plurality of data blocks comprising at least a first data block associated with a first point in time copy and a second data block associated with a second point in time copy, and
mounting the set of files to the target system to allow the target system to access the set of files, the mounted set of files comprising the virtual restored file system.
2. The method of claim 1, further comprising:
sending a request to receive the data blocks for a point-in-time copy of the source file system.
3. The method of claim 1, wherein the access to the set of files allows read access and write access to information stored in the plurality of stored data blocks.
4. The method of claim 1, wherein the plurality of point-in-time copies of the source file system are received based on a predetermined schedule associated with the source file system.
5. The method of claim 1, further comprising:
receiving a request to read data from the virtual restored file system;
identifying a stored data block from the plurality of stored data blocks; and
sending information from the stored data block.
6. The method of claim 1, further comprising:
receiving a request to write data to the restored file of the virtual restored file system;
identifying a stored data block from the plurality of data blocks linked to the restored file;
creating a copy of the stored data block and linking the restored file to the copy of the stored data block; and
updating the copy of the stored data block based on the request to write.
7. The method of claim 1, further comprising:
receiving a request to merge updates to the virtual restored file system hack with the source file system; and
sending stored data blocks of the plurality of stored data blocks that have been updated to the source file system.
8. The method of claim 1, wherein the request to restore information is a first request to restore information, the target system is a first target system, the virtual restored file system is a first virtual restored file system, the set of files is a first set of files, the restored file is a first restored file, and the plurality of stored data blocks is a first plurality of stored data blocks, the method further comprising:
receiving a second request to restore information obtained from the source file system, the second request identifying a second target system; and
responsive to receiving the second request to restore, creating a second virtual restored file system comprising a set of files linked to data blocks of the storage system including a second restored file corresponding to the source file, the creating comprising:
linking the second restored file to a second plurality of stored data blocks stored on the storage system, the second plurality of data blocks distinct from the first plurality of data blocks hut sharing at least one data block with the first plurality of data blocks, and
mounting the second set of files to the second target system to allow the second target system to access the second set of files, the mounted second set of files comprising the second virtual restored file system.
9. The method of claim 1, wherein the storage system is a first storage system, the method comprising:
receiving a request to export the virtual restored file system to a second storage system; and
sending the plurality of stored data blocks to the second storage system.
10. The method of claim 1, wherein the request to restore is associated with a restore time point, wherein the plurality of stored data blocks comprise data blocks received as part of point-in-time copies of the source file system extracted before the restore time.
11. A method for performing backups of file systems, the method comprising:
receiving requests to perform backups of plurality of source file systems, each request identifying a source file system, the source file system comprising at least a source file;
for each request received, performing backup of an associated source file system, comprising:
receiving data blocks for a plurality of point-in-time copies of the associated source file system each point-in-time copy of the associated source file-system obtained by extracting data blocks from the associated source file-system that changed since a previous point-in-time copy was extracted, and
storing on a storage system, the data blocks, the stored data blocks comprising one or more versions of a data block, each version corresponding to a point-in-time copy; and
receiving a request to restore information obtained from a particular source file system, the request to restore identifying a target system; and
responsive to receiving the request to restore the particular source file system, creating a virtual restored file system comprising a set of files including a restored file corresponding to the source file, the creating comprising:
linking the restored file to a plurality of stored data blocks stored on the storage system, the plurality of data blocks comprising at least first data block associated with a first point in time copy and a second data block associated with a second point in time copy, and
mounting the set of files to the target system to allow the target system to access the set of files, the mounted set of files comprising the virtual restored file system.
12. The method of claim 11, further comprising:
sending a request to receive the data blocks for a point-in-time copy of the particular source file system.
13. The method of claim 11, further comprising:
receiving a request to write data to the virtual restored file system;
identifying a stored data block from the plurality of stored data blocks; and
updating the stored data block based on the request to write.
14. The method of claim 11, wherein the request to restore information is a first request to restore information, the target system is a first target system, the set of files is a first set of files, the virtual restored file system is a first virtual restored file system, and the plurality of stored data blocks is a first plurality of stored data blocks, the method further comprising:
receiving a second request to restore information obtained from the particular source file system, the request identifying a second target system; and
responsive to receiving the second request to restore, creating a second virtual restored file system comprising a set of files linked to data blocks of the storage system including a second restored file corresponding to the source file, the creating comprising:
linking the second restored file to a second plurality of stored data blocks stored on the storage system, the second plurality of data blocks distinct from the first plurality of data blocks but sharing at least one data block with the first plurality of data blocks, and
mounting the second set of files to the second target system to allow the second target system to access the second set of files, the mounted second set of files comprising the second virtual restored file system.
15. The method of claim 14, wherein one or more of the stored data blocks associated with the first set of files are also associated with the second set of files.
16. The method of claim 14, further comprising:
receiving a request to write data to the restored file of the first virtual restored file system;
identifying a stored data block from the first plurality of stored data blocks linked to the restored file;
responsive to determining that the stored data block is associated with the second virtual restored file system, creating a copy of the stored data block and linking the restored file to the copy of the stored data block; and
updating the copy of the stored data block based on the request to write.
17. The method of claim 11, wherein the request to restore is associated with a restore time point, wherein the plurality of stored data blocks comprise data blocks received as part of point-in-time copies extracted before the restore time.
18. A computer program product having a computer-readable storage medium storing computer-executable code for performing backup of file systems, the code comprising:
a block copy manager configured to:
receive data blocks for a plurality of point-in-time copies of a source file system, each point-in-time copy of the source file-system obtained by extracting data blocks from the source file-system that changed since a previous point-in-time copy was extracted, the source file system comprising at least a source file;
a storage allocation manager module configured to:
store on a storage system, the data, blocks;
a virtual restored file system manager configured to:
receive a request to restore information from the source file system for a target system; and
responsive to receiving the request to restore, create a virtual restored file system comprising a set of files including a restored file corresponding to the source file, the create comprising:
linking the restored file to a plurality of the data blocks stored on the storage system, the plurality of data blocks comprising at least a first data block associated with a first point in time copy and a second data block associated with a second point time copy, and
mounting the set of files to the target system to allow the target system to access the set of files, the mounted set of files comprising the virtual restored file system.
19. The computer program product of claim 18, wherein the virtual restored file system manager is further configured to:
receive a request to write data to the restored file of the virtual restored file system;
identify a stored data block from the plurality of data blocks linked to the restored file;
create a copy of the stored data block and link the restored file to the copy of the stored data block; and
update the copy of the stored data block based on the request to write.
20. The computer program product of claim 18, wherein the request to restore information is a first request to restore information, the target system is a first target system, the virtual restored file system is a first virtual restored file system, the set of files is a first set of files, the restored file is a first restored file, and the plurality of stored data blocks is a first plurality of stored data blocks, wherein the virtual restored file system manager is further configured to:
receive a second request to restore information obtained from the source file system, the second request identifying a second target system; and
responsive to receiving the second request to restore, create a second virtual restored file system comprising a set of files linked to data blocks of the storage system including a second restored file corresponding to the source file, the creating comprising:
linking the second restored file to a second plurality of stored data blocks stored on the storage system, the second plurality of data blocks distinct from the first plurality of data blocks but sharing at least one data block with the first plurality of data blocks, and
mounting the second set of files to the second target system to allow the second target system to access the second set of files, the mounted second set of files comprising the second virtual restored file system.
21. The computer program product of claim 18, wherein the request to restore is associated with a restore time point, wherein the plurality of stored data blocks comprise data blocks received as part of point-in-time copies extracted before the restore time.
Patent Citations (37)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US4853843(A) | 1989-08-01 | Ecklund | Applicant |
| US5634053(A) | 1997-05-01 | Noble et al. | Applicant |
| US5680608(A) | 1997-10-01 | Chang et al. | Applicant |
| US5680618(A) | 1997-10-01 | Freund | Applicant |
| US6523036(B1) | 2003-02-01 | Hickman | Applicant |
| US6883083(B1) | 2005-04-01 | Kemkar | Applicant |
| US6920457(B2) | 2005-07-01 | Pressmar | Applicant |
| US7197491(B1) | 2007-03-01 | Chou et al. | Applicant |
| US7269607(B2) | 2007-09-01 | Cotner et al. | Applicant |
| US7653794(B2) | 2010-01-01 | Michael et al. | Applicant |
| US7779051(B2) | 2010-08-01 | Friedlander et al. | Applicant |
| US7895228(B2) | 2011-02-01 | Cragun et al. | Applicant |
| US8255915(B1) | 2012-08-01 | Blanding et al. | Applicant |
| US2002/0143764(A1) | 2002-10-01 | Martin et al. | Applicant |
| US2003/0204597(A1) | 2003-10-01 | Arakawa et al. | Applicant |
| US2004/0054648(A1) | 2004-03-01 | Mogi et al. | Applicant |
| US2005/0114701(A1) | 2005-05-01 | Atkins et al. | Applicant |
| US2006/0242381(A1) | 2006-10-01 | Shatskih et al. | Applicant |
| US2007/0219959(A1) | 2007-09-01 | Kanemasa | Applicant |
| US2007/0260628(A1) | 2007-11-01 | Fuchs et al. | Applicant |
| US2008/0034268(A1) | 2008-02-01 | Dodd et al. | Applicant |
| US2008/0154989(A1) | 2008-06-01 | Arman | Applicant |
| US2009/0019246(A1) | 2009-01-01 | Murase | Applicant |
| US2009/0080398(A1) | 2009-03-01 | Mahany et al. | Applicant |
| US2009/0132611(A1) | 2009-05-01 | Brown et al. | Applicant |
| US2009/0132616(A1) | 2009-05-01 | Winter et al. | Applicant |
| US2009/0144224(A1) | 2009-06-01 | Phan et al. | Applicant |
| US2009/0177697(A1) | 2009-07-01 | Gao et al. | Applicant |
| US2009/0292734(A1) | 2009-11-01 | Miloushev et al. | Applicant |
| US2010/0070476(A1) | 2010-03-01 | O'Keefe et al. | Applicant |
| US2010/0131959(A1) | 2010-05-01 | Spiers et al. | Applicant |
| US2010/0174684(A1) | 2010-07-01 | Schwaab et al. | Applicant |
| US2011/0004586(A1) | 2011-01-01 | Cherryholmes et al. | Applicant |
| US2011/0004676(A1) | 2011-01-01 | Kawato | Applicant |
| US2011/0093435(A1) | 2011-04-01 | Zha et al. | Applicant |
| US2011/0093436(A1) | 2011-04-01 | Zha et al. | Applicant |
| US2011/0161973(A1) | 2011-06-01 | Klots et al. | Applicant |
Non-Patent Literature (20)
- “FlexVol™ abd FlexClone™ Software,” Datasheet, Network Appliance, Inc., 2004, 2 Pages.Applicant
- “FlexClone” Datasheet, Network Appliance, Inc., 2008, 2 Pages.Applicant
- Hansen, J., et al., “Lithium: Virtual Machine Storage for the Cloud,” In SoCC'10—Proceedings of the 1st ACM Symposium on Cloud Computing, Jun. 10-11, 2010, 21 Pages, [online] [retrieved on Nov. 16, 2011]. Retrieved from the Internet <URL: http://vmwareJe/files/pdf/partners/academiC/ vmware-academic-socc055- hansen-wp.pdf >.Applicant
- Meeks, J., “An Oracle Technical White Paper-Oracle Data Guard with Oracle Database 11g Release 2,” Oracle, Sep. 2009, 20 Pages.Applicant
- “NetApp SnapMirror,” Datasheet, Network Appliance, Inc., 2008, 2 Pages.Applicant
- “NetApp Snapshot Technology,” Datasheet, Network Appliance, Inc., 2004, 1 Page.Applicant
- “NetApp SnapManager for Oracle,” Datasheet, Network Appliance, Inc., 2008, 2 Pages.Applicant
- Prasad, STS., et al., “Virtual Database Technology, XML, and the Evolution of the Web”, IEEE Computer Society Technical Committee on Data Engineering, 1998, pp. 1-5.Applicant
- Rajaraman, A., et al., “Virtual Database Technology: Transforming the Internet into a Database”, IEEE Internet Computing, Jul./Aug. 1998, pp. 55-58.Applicant
- Sadagopan, S., “Introduction to WebSphere Federation Server”, IBM Data Management Solutions, 2005, pp. 1-45.Applicant
- “Virtual Databases”, Jan. 7, 2008, 4 Pages, [online] [Retrieved on Sep. 19, 2011] Retrieved from the internet <URL:http://www.db2dean.com/PreviouslVirtualDB.html>.Applicant
- Wilson, A.J., et al., “Multiple Virtual Databases to Support Multiple VOS in R-GMA”, CCLRC-Rutherford Appleton Laboratory, UK, 2006, 3 Pages.Applicant
- PCT International Search Report and Written Opinion, PCT/US2011/044209, Dec. 6, 2011, 7 Pages.Applicant
- PCT International Search Report and Written Opinion, PCT/US2010/052963, Dec. 10, 2010, 16 Pages.Applicant
- PCT International Search Report and Written Opinion, PCT/US2010/052960, Dec. 10, 2010, 17 Pages.Applicant
- PCT International Search Report and Written Opinion, PCT/US2010/060536, Feb. 28, 2011, 12 Pages.Applicant
- Office Action for U.S. Appl. No. 13/316,263, Jan. 14, 2013, 7 Pages.Applicant
- Office Action for U.S. Appl. No. 12/647,337, Nov. 26, 2012, 14 Pages.Applicant
- Supplementary European Search Report for European Patent Application No. EP 10825453, Jun. 28, 2013, 8 Pages.Applicant
- Supplementary European Search Report for European Patent Application No. EP 10825452, Jun. 28, 2013, 8 Pages.Applicant