System and method for simulating disaster situations on peer to peer remote copy machines

[0002] This invention relates to testing of the operability of data storage backup systems and, more particularly, to a method of simulating disasters in peer to peer remote copy (PPRC) data storage systems.

[0003] A typical digital computer system includes a host processor for storing data and executing instructions, and one or more disk subsystems for data storage. Since stored data may be corrupted or destroyed, backup systems have been used to create multiple copies of the stored data, usually on separate disk subsystems so that in the event of a disaster, the data can be recovered from one or more of the copies. Such provision of copies is referred to as either “shadowing” or “mirroring” of the data. When mirroring is carried on by one disk subsystem reflecting data to a remotely positioned disk subsystem, those subsystems are referred to as a “peer to peer remote copy” (PPRC) pair.

[0004] Assume a PPRC data storage system comprising a host processor, a primary storage subsystem and a secondary storage subsystem. During normal operation, the host processor issues write commands to the primary storage subsystem and data is mirrored to the secondary storage subsystem by operation of the primary storage subsystem. If the primary storage subsystem suffers a malfunction, the host can then connect directly to the secondary storage subsystem and continue operation, or a remote secondary host can be used to continue operation. The success of such a system is contingent upon an accurate mirroring of data between the primary and secondary storage subsystems.

[0005] It is desirable to conduct a controlled test to assure that the PPRC system is properly mirroring data between the storage subsystems, and that the integrity of the data is faithfully maintained. Conventionally, such tests have been performed by manually simulating disasters that interfere with the operation of the PPRC system. This form of testing is time consuming and difficult because the disaster conditions must be set up through manual intervention.

[0006] It is an object of this invention to provide an improved method of testing the operation of a PPRC data storage system in disaster situations and verifying the integrity of the mirrored data without manual intervention.

[0007] A PPRC data storage system includes at least a host processor, a primary data storage subsystem and a secondary storage subsystem. Initially, the host processor communicates with the primary storage subsystem while the primary and secondary storage subsystems are remotely coupled as a PPRC pair to enable a mirroring of data therebetween.

[0008] To test a disaster scenario, the host processor configures and transmits a special command to the primary storage subsystem. Upon receipt of the command, the primary storage subsystem simulates a disaster to affect all or some of the PPRC disk volumes. The host processor senses the malfunction of the primary storage subsystem by detecting an aberration in its communications with the primary storage subsystem, and responds by establishing communications directly with the secondary storage subsystem. The secondary storage subsystem then terminates the PPRC remote copy operation with the failed primary storage subsystem. Next, the host verifies the integrity of the mirrored data by comparing data from the secondary storage subsystem to data from the primary storage subsystem. This process may be automated to cycle through a set of different types of simulated disasters.

[0009] As compared to the prior art, this method allows for relatively quick and easy verification of the operation of a PPRC data storage system and its ability to recover in the case of a disaster without requiring manual intervention.

[0010] FIG. 1 is a block diagram of a PPRC data storage system particularly adapted to carrying out the invention.

[0011] FIG. 2 is a logical flow diagram illustrating the method of the invention.

[0012] Referring to FIG. 1, a PPRC data storage system 10 generally includes a host processor 20, a primary storage subsystem 40 and a secondary storage subsystem 60. The host processor 20 may communicate directly with primary storage subsystem 40 via primary communication link 90, or with secondary storage subsystem 60 via secondary communication link 92. Primary storage subsystem 40 and secondary storage subsystem 60 are coupled via a remote copy link 94 and are configured to provide a mirroring of data therebetween. That is, secondary data 61 will be a mirror image of primary data 41.

[0013] All communication between host processor 20, primary storage subsystem 40 and secondary storage subsystem 60 involves a dispatch of a command from the device desiring an operation and a subsequent acknowledgment from the device that is the target of the command. Primary storage subsystem 40 responds with primary acknowledge 42. Secondary storage subsystem 60 responds with secondary acknowledge 62. The preferred embodiment of this invention includes write, read, PPRC configuration, setup, and disaster commands as described below.

[0014] A write command allows the initiator of the command to write data to another device. During normal, non-disaster, operation, (1) host processor 20 issues write command 21, accompanied by host data 29, to primary storage subsystem 40, and (2) primary storage subsystem 40 issues write command 43, accompanied by primary data 41, to secondary storage subsystem 60 to execute a remote copy operation. Host processor 20 does not write to secondary storage subsystem 60 during normal operation. Host processor 20 may issue write command 21, accompanied by host data 29, to secondary storage subsystem 60, only if secondary storage subsystem 60 is not configured to operate as part of a PPRC pair.

[0015] A read command allows the initiator of the command to read data from another device. Host processor 20 can issue read command 22 to primary storage subsystem 40 to read primary data 41, or to secondary storage subsystem 60 to read secondary data 61. During normal, non-disaster, operation, host processor 20 reads only from primary storage subsystem 40. Host processor 20 may read from secondary storage subsystem 60 only if secondary storage subsystem 60 is not configured to operate as part of a PPRC pair.

[0016] A PPRC configuration command 23 is used to configure primary storage subsystem 40 and secondary storage subsystem 60 as a PPRC pair. The command is issued by host processor 20 to primary storage subsystem 40. Primary storage subsystem 40 relays the command to secondary storage subsystem 60.

[0017] Setup command 25 is issued by host processor 20 to secondary storage subsystem 60 when host processor 20 wishes to establish full communications with secondary storage subsystem 60 and terminate the PPRC pair.

[0018] Disaster command 26 is issued by host processor 20 to primary storage subsystem 40 to initiate a disaster simulation 45 in primary storage subsystem 40.

[0019] During normal, non-disaster operation, host processor 20 executes jobs 30, which includes issuing write commands 21 and read commands 22 to primary storage subsystem 40. Primary storage subsystem 40 issues write commands 43 to secondary storage subsystem 60 as necessary to support the remote copy operation.

[0020] In the case of a disaster causing a malfunction of primary storage subsystem 40, host processor 20 senses an aberration in its communications with primary storage subsystem 40, via malfunction detector 28. In turn, host processor 20 issues setup command 25 to secondary storage subsystem 60. This establishes full communication, enabling read and write commands, between host processor 20 and secondary storage subsystem 60 and causes termination of the PPRC pair.

[0021] Host processor 20 includes the capability to format and issue a disaster command 26, based on a selection from a table of disaster types 27. Upon dispatch to and receipt of disaster command 26, primary storage subsystem 40 executes a disaster simulation 45. For example, disaster simulations can include a malfunction of the primary communication link 90, a time-out where primary storage subsystem 40 fails to send primary acknowledge 42 within a predetermined period of time after receiving a command, or a total failure of primary storage subsystem 40.

[0022] FIG. 2. illustrates the preferred method of the invention. Beginning with step 110, all simulated disasters are cleared from the PPRC data storage system 10. Next, in step 115, normal communications are established between host processor 20 and primary storage subsystem 40. In step 120, host processor 20 issues PPRC configuration command 23 to configure primary storage subsystem 40 and secondary storage subsystem 60 as a PPRC pair.

[0023] In step 125, jobs 30 are started, or re-started and executed. Host processor 20 issues write commands 21 and read commands 22 to primary storage subsystem 40. Concurrently, primary storage subsystem 40 issues write commands 43 to secondary storage subsystem 60 to effectuate the remote copy operation.

[0024] In step 130, host processor 20 selects a disaster type from table of disaster types 27, and in step 135, a disaster command 26 is formatted to indicate the selected disaster type and is then issued to primary storage subsystem 40. In step 140, primary storage subsystem 40 executes a disaster simulation 45 based on disaster command 26. In step 145, host processor 20 continues processing until it senses the malfunction.

[0025] After detection of the malfunction (step 145), the method proceeds to step 150 where host processor 20 issues, to secondary storage subsystem 60, a setup command 25. This establishes full communications with secondary storage subsystem 60 via secondary communication link 92 and causes termination of the PPRC status between primary storage subsystem 40 and secondary storage subsystem 60.

[0026] To verify the integrity of the data mirrored between primary data storage subsystem 40 and secondary storage subsystem 60, in step 155, secondary data 61 is compared to primary data 41. This comparison can be performed by allowing host processor 20 to continue operation using secondary data 61 and verifying proper execution of jobs 30, or in the alternative, reading secondary data 61 and primary data 41 and comparing the two sets of data. In either case, the comparison is evaluated in step 160.

[0027] In step 160, if the determination is that the data does not match, the method proceeds to step 165 where a FAIL condition is indicated.

[0028] In step 160, if the determination is that the data does match, the method proceeds to step 170 where a PASS condition is indicated and then proceeds back to step 110. From this point, the method steps will be executed as previously described, but in step 130, a different disaster type can be selected from the table of disaster types 27.

[0029] It should be understood that the foregoing description is only illustrative of the invention. Various alternatives and modifications can be devised by those skilled in the art without departing from the invention. For example, the simulation could be controlled, or the disaster type could be selected, through a user-friendly interface. Further, while the procedures required to execute the invention hereof are indicated as already loaded into the memory of the storage subsystems, they may be configured on a storage media, such as data memory 96 in FIG. 1, for subsequent loading into the controlling subsystem processors. Accordingly, the present invention is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.