Method, system, and program for remote copy in an open systems environment

[0002] This application is related to the following co-pending and commonly-assigned patent applications, all of which are filed on the same date herewith, and all of which are incorporated herein by reference in their entirety:

[0003] “Method, System, And Program For Selecting One of Multiple Paths to Communicate With a Device”, to David A. Burton, Robert L. Morton, and Erez Webman, U.S. patent application Ser. No. 09/591,024, currently pending ,and “Method, System, And Program For Determining A Number of Write Operations to Execute” to David A. Burton, Robert L. Morton, and Erez Webman, U.S. patent application Ser. No. 09/591,023, currently pending.

[0004] 1. Field of the Invention

[0005] The present invention relates to a system, method, and program for shadowing data at a remote site and, in particular, shadowing data in an open systems environment.

[0006] 2. Description of the Related Art

[0007] Data storage systems often include a data protection component that involves creating a back-up copy of data to use in the event of a system failure. One such data protection system involves the use of a computer at a remote site, often referred to as the secondary system, shadowing data storage at a primary site having a primary computer and storage. If there is a system failure at the primary site, then the data can be recovered from the secondary site. One such disaster recovery system that provides data shadowing is the International Business Machines Corporation (IBM) Peer-to-Peer Remote Copy (PPRC) system. In a PPRC system, a primary storage controller provides access to a primary mass storage systems, such as one or more Direct Access Storage Devices (DASDs), comprised of interconnected hard disk drives. The storage controller manages the flow of data from host systems to DASDs. A secondary controller provides a shadow copy of designated volumes in the primary DASD in a secondary mass storage system. A high speed dedicated line, such as Enterprise System Connection (ESCON) channels **, may be used to transfer data from the primary controller to the secondary controller. In PPRC systems, a relationship is established between volumes in the primary DASD and the secondary DASD by copying the designated volumes from the primary DASD to the secondary DASD through the primary and secondary controllers. After the establishment phase, any updates to the protected volumes in the primary DASD are copied over to the secondary controller and DASD to synchronize the data therebetween.

[0008] **Enterprise Storage Server and ESCON are registered trademarks and Fibre Channel Raid Storage Controller is a trademark of IBM; Windows and Windows NT are registered trademarks of Microsoft Corporation.

[0009] Current data shadowing systems require that both the primary and secondary control unit include special purpose data shadowing firmware or software to implement the data shadowing relationship between a primary and secondary controllers, such as the IBM PPRC and XRC software and firmware. In such systems, typically, the controllers involved in shadowing customer data are from the same vendor.

[0010] Many computer networks operate in an open system environment and transfer data using the Small Computer System Interface (SCSI) or Fibre Channel architectures. In an open system environment, systems may be from different vendors and would not communicate using special, vendor specific protocols, such as PPRC or XRC. Thus, there is a need to provide data shadowing in an open systems environment where the systems do not include special purpose firmware or software to implement such data shadowing.

[0011] To overcome the limitations in the prior art described above, preferred embodiments disclose a method, system, and program for providing a shadow copy of data storage areas in a primary site to data storage areas in a secondary site. A definition is made of storage areas in the primary site having data to be shadowed and corresponding storage areas in the secondary site to shadow data at the storage areas in the primary site. A shadow pair comprises one primary storage area and one secondary storage area that shadows data for the primary storage area in the pair. A standard data transfer interface protocol command, such as a SCSI or Fibre Channel command, is used to configure status storage areas in the secondary site to provide status information and data for each primary and secondary storage area. A write command in the standard data transfer interface protocol is then used to write status information to the status storage areas indicating status of the shadowed data at the secondary storage areas in the pairs.

[0012] In further embodiments, an update from a host to one storage area in the primary site in one shadow pair to update is received. The write command in the standard data transfer interface protocol is then used to write the update to the storage area for the pair to update.

[0013] Preferred embodiments provide a methodology for allowing a remote controller in an open systems environment to function as a secondary site to shadow data at a primary site. Preferred embodiments allow the primary controller to use a standard data transfer interface protocol, such as SCSI or Fibre Channel Protocol (FCP), to place status information and data at the secondary site on the state of data being shadowed. In the event of a failure at the primary site, another system can use the status information to determine the consistency of the data at the secondary site in shadowing the data at the primary site. With the preferred embodiments, the secondary controller does not include special purpose data shadowing because the primary controller uses standard data transfer protocols to maintain data shadowing status at the secondary site.

[0014] Referring now to the drawings in which like reference numbers represent corresponding parts throughout:

[0015] FIG. 1 is a block diagram illustrating a computing environment in which preferred embodiments are implemented; and

[0016] FIGS. 2-5 illustrate logic implemented in a primary controller and remote copy host to implement the data shadowing in accordance with preferred embodiments of the present invention.

[0017] In the following description, reference is made to the accompanying drawings which form a part hereof and which illustrate several embodiments of the present invention. It is understood that other embodiments may be utilized and structural and operational changes may be made without departing from the scope of the present invention.

[0018] FIG. 1 illustrates a computing environment in which preferred embodiments are implemented. Hosts 4a, bmay comprise any computing device known in the art, including servers through which other client computers can access storage or clients. The hosts 4a, beach include at least one adaptor, such as a Fibre Channel or Small Computer System Interface (SCSI) adaptor card or any other network adaptor card known in the art. The host adaptors allow the hosts 4a, bto communicate with storage controllers 6a, bvia switch 8. The switch 8 may comprise the IBM Fibre Channel Storage Hub or Switch, the IBM SAN Fibre Channel Switch, or any other switching device known in the art. A primary site includes a primary controller 6aand primary storage 10a. Designated volumes in the primary storage 10 are transferred to a secondary site comprising a secondary controller 6band secondary storage 10b. The storage devices 10a, bmay comprise a DASD or any other non-volatile storage device and system known in the art.

[0019] Preferred embodiments are implemented in an open systems environment where the primary and secondary controllers 6a, band hosts 4a, bmay comprise different types of systems from different vendors that communicate using a common protocol such as SCSI or Fibre Channel Protocol (FCP). Thus, the hosts 4a, band controllers 6a, bmay include any operating system known in the art, including the Microsoft Corporation Windows and NT operating systems or any other specialized operating system.** In such open systems environment, the primary controller 6acan use commonly used write commands, such as SCSI write commands, to copy the primary volumes to the secondary volumes in the secondary storage 10b. In such open system embodiments, the secondary controller 6bdoes not need special purpose software to coordinate the shadowing activities with the primary controller 6bas the primary controller 6aaccomplishes the shadowing by using standard interface protocol write commands. In such open systems, the primary and secondary controllers 6a, bmay comprise any controller device known in the art and the primary and secondary controllers 6a, bmay be of different models and model types, and even of different classes of storage controllers.

[0020] **Enterprise Storage Server and ESCON are registered trademarks and Fibre Channel Raid Storage Controller is a trademark of IBM; Windows and Windows NT are registered trademarks of Microsoft Corporation.

[0021] To ensure data consistency, both the primary 6asecondary 6bcontrollers maintain a state LUN 12a, bindicating their view of the state of shadowing data between the primary and secondary 6a, bcontrollers. In the event of a failure of the primary controller 6a, the hosts 4a, bcan query the state LUN 12bat the secondary controller 6bto determine the consistency of the data being shadowed between the primary and secondary controllers 6a, b. In preferred embodiments, state information is maintained for each LUN pair being shadowed between the primary 10aand secondary 10bDASDs, to allow data consistency to be determined on a LUN by LUN basis. The state LUNs 12a, bare created during the establishment phase. Below are four possible states that may be indicated in the state LUNs 12a, b:

[0022] Simplex: This is the state before the establishment of a remote copy relationship between LUNs and after the remote copy session has terminated. Simplex means that the LUN is not currently involved in data shadowing. Ending a remote copy relationship between the primary and secondary LUNs returns the two devices to the simplex state.

[0023] Duplex Pending: The state during the establishment phase where data is initially being copied from the primary LUN to the secondary LUN, but has not yet completed. At this point, data is not consistent between the primary and secondary LUNs, and the remote copy at the particular secondary LUN cannot be used. The establishment phase ends when all the data in the primary LUN has initially been copied over.

[0024] Full Duplex: The primary and secondary LUN pair are consistent except for current writes to the primary LUN received at the primary controller 6a.

[0025] Suspend Duplex: Changes to the primary LUN have not been communicated to the secondary LUN. The suspend state occurs when only one of the devices in a dual copy or remote copy LUN pair is being updated because of either a permanent error condition or an authorized user command. All writes to the remaining functional device are logged. This allows for automatic resynchronization of both LUNs when the LUN pair is reset to the duplex state. If suspend duplex is the state during the establishment phase where data is copied for the first time between the primary and secondary LUNs in the DASDs 10a, b, then the data is not consistent as the initial copy has not completed. However, if suspend duplex is the state after the initial establishment copy has completed, then the data at the secondary LUN is consistent, but not up-to-date, as recent writes were not communicated to the secondary LUN.

[0026] In preferred embodiments one or more hosts, e.g., host 4afor purposes of this discussion, would include a remote copy program 14 (shown in FIG. 1) that allows the host 4ato participate in remote copy operations as a remote copy host. The remote copy host 4awould execute the remote copy program 14 to query the secondary state LUN 12bin the event of a failure to determine the extent to which shadowed data at the secondary LUNs may be used to recover the failed primary LUNs. In preferred embodiments, the secondary state LUN 12bis configured in a redundant array of independent disks (RAID) to ensure that the state information is not lost in the event of a single drive failure in the secondary DASD 10b.

[0027] FIG. 2 illustrates logic implemented in software and/or firmware of the primary controller 6ato establish a relationship between primary LUNs in the primary DASD 10aand second LUNs in the secondary DASD 10bto provide a remote copy or data shadowing for the primary LUNs involved in the relationship. As discussed, before the secondary site can start shadowing updates to the primary site, a relationship of primary LUNs in the primary DASD 10aand secondary LUNS in the secondary DASD 10bmust be established. Control begins at block 100 with the primary controller 6ainitiating the establishment phase between LUN pairs, LUNs to be shadowed in the primary DASD 10aand the corresponding secondary LUN in the pair that will shadow the data. At block 102, the primary controller 6aconfigures a state LUN 12bin the secondary DASD 10bto include a sector for each LUN in the secondary DASD 10b. In preferred embodiments, the primary controller 6auses the vendor specific configuration commands for the secondary controller 6bto configure the state LUN 12bin the secondary DASD 10b. In preferred embodiments, the primary controller 6awould send these vendor-specific configuration commands through the Fibre Channel device driver. The primary controller 6awould further configure a primary state LUN 12athat also maintains a copy of the status.

[0028] The primary controller 6athen communicates (at block 104) the location information of the secondary state LUN 12bin the secondary DASD 10bto the remote copy host 4a. The remote copy host 4awould use such information to access the remote copy status at the secondary site in the event of a failure of the primary DASD 10a. The primary controller 6awill issue (at block 106) write commands to write “simplex” status to the LUN sectors in the primary and secondary state LUNs 12a, bcorresponding to primary and secondary LUNs, respectively, not participating in remote copy LUN pairs, i.e., not involved in data shadowing operations. The primary controller 6awill further issue (at block 110) write commands to write “duplex pending” status to the sectors in the primary and secondary state LUNs 12a, bcorresponding to primary and secondary LUNs, respectively, participating in remote copy LUN pairs.

[0029] For each remote copy LUN pair of a primary LUN i being shadowed by a secondary LUN j, the primary controller 6awill issue (at block 112) write commands to write the customer data from the primary LUN i to the secondary LUN j. The primary controller 6amay utilize the algorithm described in the co-pending application “Method, System, and Program for Determining a Number of Write Operations to Execute,” having attorney docket No. TUC9-2000-0015US1, which application is incorporated herein by reference in its entirety, to manage writes to copy the data from the primary LUN i to secondary LUN j. Upon receiving (at block 130) acknowledgment from the secondary controller 6bthat the write operations to LUN j completed at the secondary DASD 10b, the primary controller 6awill write (at block 132) “full duplex status” to the LUN i sector in the primary state LUN 12aand to the LUN j sector in the secondary state LUN 12b. At this point, the establishment phase has completed and all the data from the LUNs in the primary DASD 10ainvolved in remote copy has successfully been copied over to the LUNs in the secondary DASD 10bshadowing the data in the primary LUNs.

[0030] FIG. 3 illustrates logic implemented in firmware and/or software in the primary controller 6ato shadow updates to LUNs in the primary DASD 10athat are part of a remote copy LUN pair, i.e., being shadowed on the secondary DASD 10b. Control begins at block 150 with the primary controller 6areceiving an update to LUN i in the primary DASD 10athat is part of a remote copy LUN pair, being shadowed by LUN j in the secondary DASD 10b. The LUNs i and j may be the same LUN number or different LUN numbers. The primary controller 6aapplies (at block 152) the updates to LUN i in the primary DASD 10aand returns acknowledgment (at block 154) to the host initiating the write that the update is completed. The primary controller 6athen issues (at block 156) one or more writes, such as SCSI write commands, to copy the updates to LUN i to LUN j in the secondary DASD 10b.

[0031] At block 170, the primary controller 6areceives status on the write request to secondary LUN j of the updates to the primary LUN i. If (at block 172) the status indicates that the updates were not successfully written to the secondary LUN j or the write otherwise failed for any reason, including problems with the connection, then the primary controller 6aissues (at block 174) an asynchronous attention status, such as a SCSI Unit Attention, that remote copy is suspended for LUN i. This informs; the host 4a, bthat initiated the write operation and all other hosts that they cannot obtain current data for LUN i from the secondary DASD 10b. This status is only issued once. The primary controller then issues (at block 178) a write command to write “suspend duplex” status to the LUN i and LUN j sectors in the primary 12aand secondary 12bstate LUNs, respectively. At this point, the hosts 4a, bcan still write updates to LUN i. However, the asynchronous attention informs the hosts 4a, bthat they cannot rely on the secondary DASD 10bto provide current data. Otherwise, if the updates were successfully written to secondary LUN j, then the logic ends.

[0032] If, at block 200, the primary controller 6areceives status that the write of the “suspend duplex” status to the LUN j sector in the secondary state LUN 12bfailed, then the primary controller 6apresents (at block 202) asynchronous status to the remote copy host 4band any other host attempting to access LUN i of a failure to update status to the LUN j sector in the secondary state LUN 12b. This informs the hosts 4a, bthat they cannot rely on the status information for LUN i maintained in the secondary state LUN 12b. This asynchronous status may be implemented in a read command the remote copy host 4apresents to the primary controller 6a. The primary controller 6amay queue this read request and leave it outstanding until it needs to present status to the remote copy host 4aon a failed attempt to write “suspend duplex” status to the secondary site. The asynchronous status indicating the failure to update LUN j sector at the secondary site is only presented once to each host. Thus, the host 4a, breceiving the asynchronous status continues to believe the secondary state LUN 12bdoes not reflect the correct status for LUN j until it receives a subsequent status indicating otherwise.

[0033] FIG. 4 illustrates logic implemented in the remote copy program 14 to involve the remote copy host 4bin the remote copy operations and assist the primary controller 14awith remote copy. If the primary controller 6acould not write the “suspend duplex” status to the LUN j sector in the secondary state LUN 12b, then the remote copy host 4amay try to apply the “suspend duplex” status as it may have alternative communication paths to the secondary DASD 10bnot available to the primary controller 10a. At block 250, in response to receiving the asynchronous status from the primary controller 4a(at block 202, FIG. 3), the remote copy host 4awould issue a write to write the “suspend duplex” state to the LUN j sector in the secondary state LUN 12b. At block 270, the remote copy host 4areceives status on the write “suspend duplex” to the secondary state LUN 12b. If the status (at block 272) indicates that the write succeeded, then the remote copy host 4ainforms (at block 274) the primary controller 4aof the successful write of the status; otherwise, the primary controller 4ais notified (at block 276) that the write of the “suspend duplex” status failed.

[0034] FIG. 5 illustrates logic implemented as software and/or firmware in the primary controller 6a. Control begins at block 300 with the primary controller 6areceiving a response from the remote copy host 4bconcerning the write of the “suspend duplex” status to the LUN j sector in the secondary state LUN 12b. If the status was successfully updated (at block 302), then the primary controller 4aqueues (at block 304) the updates to LUN j. The primary controller 6awould return success to the host 4a, binitiating the update. From the no branch of block 302 or 304, control transfers to block 308 where the primary controller 6awould periodically attempt to establish a connection with LUN j in the secondary DASD 10b.

[0035] As discussed, there are two possible asynchronous statuses. One indicates that the secondary DASD 10bdoes not include current data for LUN i and the second indicates that the status for LUN i in the secondary state LUN 12bis not accurate. At block 330, the primary controller 6areceives notification that the connection with LUN j at the secondary site is reestablished. The primary controller 6athen writes (at block 332) any queued writes for LUN j to LUN j and then writes (at block 334) “full duplex” status to the LUN i and LUN j sectors in the primary 12aand secondary 12bstate LUNs, respectively. The primary controller 12afurther provides (at block 336) asynchronous status one time to any host providing updates to LUN j of the new status of LUN j, i.e., that it is available and the status for LUN j in the secondary state LUN 12bis current.

[0036] With the preferred embodiments, in the event of a failure in the primary DASD 10a, the remote copy host can read the secondary state LUN 12bto determine the status of the secondary LUNs shadowing data. If the status is full duplex, then the remote copy host 4aknows that the data at the secondary LUNs in the remote copy LUN pairs is consistent for all writes that have completed. In preferred embodiments, hosts log updates in a journal until they receives status that the update was successfully applied. In the event of a failure, for those writes that have not completed, the hosts 4a, bcan recover the unapplied writes from the host journal log, and then apply those updates to the secondary LUNs involved in remote copy to derive the current data at the time of the failure. If multiple hosts had outstanding writes to the primary LUNs in the remote copy relationship, then the uncompleted updates would have to be gathered from journal logs from all those hosts having uncompleted updates to the primary LUNs.

[0037] If there is a failure, and the status is “duplex pending”, then the data at the secondary LUN is not consistent because the establishment phase of the initial copy has not completed to that particular LUN. “Suspend duplex” means that updates have not been communicated to the secondary LUN due to a failure. In case of a failure while the secondary LUN is in “suspend duplex”, the data can be recovered from the journal logs of hosts that have outstanding updates to the primary LUNs involved in the remote copy operation.

[0038] Preferred embodiments provide a system for allowing controllers in an open systems environment to function as secondary controllers in a remote copy relationship to shadow data. In the preferred embodiments, the secondary controllers do not need any special purpose firmware or software to participate in the data shadowing relationship. Instead, the primary controller writes status information to the secondary DASD using a standard data transfer interface protocol, such as SCSI or Fibre Channel commands, that are acceptable by all systems in an open system environment. The status information is written to the secondary DASD to allow a remote copy host or agent to determine the status of remote copy operations at the time of the failure of the primary DASD.

[0039] The following describes some alternative embodiments for accomplishing the present invention.

[0040] The preferred embodiments may be implemented as a method, apparatus or program using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. The programs defining the functions of the preferred embodiment can be delivered to a computer via a variety of information bearing media, which include, but are not limited to, computer-readable devices, programmable logic, memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, SRAMs, etc.) carriers, or media, such as a magnetic storage media, “floppy disk,” CD-ROM, a file server providing access to the programs via a network transmission line, wireless transmission media, signals propagating through space, radio waves, infrared signals, etc. Of course, those skilled in the art will recognize that many modifications may be made to this configuration without departing from the scope of the present invention. Such signal-bearing media, when carrying computer-readable instructions that direct the functions of the present invention, represent alternative embodiments of the present invention.

[0041] In preferred embodiments, one primary controller 6ashadowed data on a single secondary controller 6b. However, in alternative arrangements, one primary controller may shadow data at multiple secondary controllers or multiple primary controllers may shadow data at one or more secondary controllers.

[0042] In preferred embodiments, data storage areas were delineated as LUNs. In alternative embodiments, storage areas may be divided according to different means.

[0043] The preferred logic of FIGS. 2-5 describe specific operations occurring in a particular order. In alternative embodiments, certain of the logic operations may be performed in a different order, modified or removed and still implement preferred embodiments of the present invention. Morever, steps may be added to the above described logic and still conform to the preferred embodiments. Further, operations described herein may occur sequentially or certain operations may be processed in parallel.

[0044] In preferred embodiments, the primary and secondary controllers communicated over a fibre channel interface using SCSI or Fibre Channel commands. In alternative embodiments, different command protocols may be utilized. For instance, the ESCON protocol may be used for the channel communications and the count-key-data.(CKD) protocol may be used for the input/output (I/O) commands.

[0045] Preferred embodiments were described with respect to a storage system in which data from a primary site is shadowed at a secondary site as part of a data backup and recovery system. However, the preferred method, system, and program for shadowing data may apply to non-data recovery systems, or whenever a mirror copy of data is needed.

[0046] In summary, preferred embodiments disclose a method, system, and program for providing a shadow copy of data storage areas in a primary site to data storage areas in a secondary site. A definition is made of storage areas in the primary site having data to be shadowed and corresponding storage areas in the secondary site to shadow data at the storage areas in the primary site. A shadow pair comprises one primary storage area and one secondary storage area that shadows data for the primary storage area in the pair. A standard data transfer interface protocol command, such as a SCSI or Fibre Channel command, is used to configure status storage areas in the secondary site to provide status information and data for each primary and secondary storage area. A write command in the standard data transfer interface protocol is then used to write status information to the status storage areas indicating status of the shadowed data at the secondary storage areas in the pairs.

[0047] The foregoing description of the preferred embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto. The above specification, examples and data provide a complete description of the manufacture and use of the composition of the invention. Since many embodiments of the invention can be made without departing from the spirit and scope of the invention, the invention resides in the claims hereinafter appended.