Method for validating operation of a fibre link

[0001] The present invention generally relates to disk arrays utilized in Fibre Channel Storage Area Networks (SANs), or the like, and particularly to a method for validating operation of a fibre channel loop or link within such disk arrays.

[0002] Fibre channel disk arrays employed by Storage Area Networks (SANs), and the like, typically comprise multiple drive-side fibre channel loops each having a plurality of devices (e.g., up to 127 devices per loop). In typical implementations, each fibre channel loop includes two disk array controllers and multiple drive enclosures, each having a plurality of drives. Each drive enclosure may further include an Enclosure Service Module (ESM) within, which monitors environmental conditions within the enclosure and attempts to ensure that individual drives within the enclosure do not cause the entire loop to become non-functional.

[0003] Normally, a fibre channel loop must be initialized when a new device enters the loop or when an error condition is detected on the loop. When the fibre channel loop (also referred to as “link”) goes down to reinitialize for some reason, it normally completes initialization within 15–30 milliseconds. If the link fails to complete reinitialization within 3–5 seconds, the ESMs bypass their “input” external ports and perform diagnostics to determine if all the drives within the enclosure are functioning properly. This diagnostic process is accomplished by bypassing all the drives and then issuing a Loop Initialization Primitive (LIP), which is the command needed to begin initialization. However, the ESMs do not bypass their “output” external ports. Thus, when the ESMs are performing bypassing and LIPing operations, other devices on the loop see the LIP completions, and can falsely determine that the entire link is up when it is not.

[0004] When the disk array controller detects that the link is up (i.e., a “link up”), it attempts to resume processing of commands to the devices on the loop. However, if the link is not truly “up”, these commands will fail to complete. When these commands fail to complete, the disk array controller typically seeks to restore operation of the fibre link by again attempting to initialize the link again via a LIP command. However, this approach is normally not successful since it has a tendency to put the link into a perpetual LIP condition from which it may not recover without manual intervention from an operator.

[0005] Consequently, it would be desirable to provide a method for validating operation of a fibre link of such disk arrays (i.e., for validating if the link is truly up) so that the link is not put into a perpetual LIP condition when initialized.

[0006] Accordingly, the present invention is directed to a method for validating operation of a fibre link when the fibre link is initialized. In exemplary embodiments, the method includes the steps of entering a trial link up state upon receiving a command to initialize the fibre link so that normal commands to other devices within the fibre channel loop are not resumed, and thereafter entering a final link up state and resuming normal commands to other devices within the fibre channel loop. In exemplary embodiments, the method may be implemented by devices within a system such as a disk array system of a storage area network (SAN), or the like.

[0007] It is to be understood that both the forgoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed. The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate an embodiment of the invention and together with the general description, serve to explain the principles of the invention.

[0013] Reference will now be made in detail to the presently preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings.

[0014] Referring generally now to FIGS. 1 through 3, an exemplary fibre channel loop in accordance with the present invention is described. Fibre channel loop or fibre link 100 includes a disk array controller (DAC) 102 and multiple drive enclosures (three drive enclosures 104–108 are shown), each having a plurality of fibre channel drives (ten drives 110–114 are shown). Each drive enclosure 104–108 may further include an Enclosure Service Module (ESM) 116, which monitors environmental conditions within the enclosure 104–108 via the enclosure control bus 118 and attempts to ensure that individual drives 110–114 within the enclosure 104–108 do not cause the entire loop 100 to become non-functional.

[0015] FIG. 2 provides an expanded views of the first drive enclosure 104 during normal operation (i.e., when the fibre link 100 is up). As shown in FIG. 2, no ports within fibre link 100 (e.g., input and output ports 120–122) are bypassed. Consequently, self-directed frames or commands issued by disk array controller 102 will succeed.

[0016] FIG. 3 depicts the fibre link 100 shown in FIGS. 1 and 2, wherein the fibre link 100 is down because the first drive enclosure 104 is bypassing its input port 120 but not bypassing its output port 122. When disk array controller 102 detects that the link us up (i.e., a “link up”), it attempts to resume processing of commands to the devices 104–116 on the loop. However, if the link is not truly “up”, these commands will fail to complete. In the past, when commands issued by the disk array controller failed to complete, the disk array controller typically sought to restore operation of the fibre link by again attempting to initialize the link again via a LIP command. This attempt to initialize the fibre link tended to put the link into a perpetual LIP condition, where many devices on the loop are initiating LIP commands and these LIP commands fail to complete properly due to Enclosure Service Modules (ESMs) leaving and rejoining the main loop to perform diagnostics.

[0017] The present invention provides a method for validating operation of a fibre link when the fibre link is initialized. In accordance with this method, two link states: “trial link up” and “final link up” are introduced. When a device 102–116 of fibre channel loop 100 receives a LIP completion (LILP) frame (e.g., disk array controller 102), the device (e.g., disk array controller 102) first enters the trial link up state. Thereafter, the device (e.g., disk array controller 102) enters a final link up state. Preferably, normal commands to other devices (e.g., drive enclosures 106 and 108) within fibre channel loop 100 are not resumed until the device enters the final link up state. In order to enter the final link up state, the device sends a “no operation” (NOP) frame to itself via fibre channel loop 100. If the device receives its own NOP, it can safely assume that the fibre link 100 is truly up, and can enter the final link up state. Normal commands to the other devices on the loop 100 may then be resumed.

[0018] The device may also employ self-directed commands for diagnostics to ensure that the link is truly up and available for commands to be sent. While in the trial link up state, the device can send, in addition to the NOP, a command such as a read command, a write command, or the like to itself, allowing the device to determine that the loop is not only operational (i.e., the link is “up”) but is also stable.

[0019] Referring now to FIG. 4, an exemplary method for validating operation of a fibre link, such as the fibre link illustrated in FIGS. 1 through 3 is described. In the exemplary embodiment shown, the method 200 is initiated, at step 202, when a command to initialize the fibre link is received by a device within the fibre channel loop. The device then enters a trial link up state, at step 204, so that normal commands to other devices within the fibre channel loop are not resumed. In exemplary embodiments, the command to initialize the fibre link comprises a Loop Initialization Primitive (LIP) including a Loop Initialization Primitive completion frame (LILP). While in the trial link up state, the device may send and receive one or more self-directed commands via the fibre link for verifying that the link is stable, at step 206. In exemplary embodiments, these self-directed commands may include read commands, a write commands, or the like. To enter the final link up state, the device sends a no operation frame (NOP) to itself, while in the trial linkup state, at step 208. When the device receives the no operation frame (NOP), the device enters the final linkup state, at step 210, and resumes normal commands to other devices within the fibre channel loop.

[0020] The present invention allows the fibre link 100 (FIGS. 1, 2 and 3) to become operable or “come up” without excessive re-initializations (e.g., via LIP commands). Thus, employing the present invention it is typically unnecessary for an operator to manually power-cycle all devices on the fibre channel loop 100 in order to complete re-initialization of the fibre link. Preferably, the present invention may employed by any topology where elements of the whole group can be segmented, which provokes other elements not within that segment to assume that the whole group is functioning properly, and take actions based upon that assumption. For example, in embodiments of the invention illustrated herein, fibre channel loop 100 comprises a loop of a disk array system 130 employed by a storage area network 140, or the like.

[0021] In exemplary embodiments, the methods disclosed may be implemented as sets of instructions (e.g., software or firmware) readable by a device within the fibre channel loop 100 (FIGS. 1, 2 and 3). Further, it is understood that the specific order or hierarchy of steps in the methods disclosed are examples of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the method can be rearranged while remaining within the scope of the present invention. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.

[0022] It is believed that the present invention and many of its attendant advantages will be understood by the forgoing description. It is also believed that it will be apparent that various changes may be made in the form, construction and arrangement of the components thereof without departing from the scope and spirit of the invention or without sacrificing all of its material advantages. The form herein before described being merely an explanatory embodiment thereof. It is the intention of the following claims to encompass and include such changes.