Hybrid logical block virtualization system for a storage area network

[0001] 1. Field of the Invention

[0002] The present invention generally relates to shared storage and more particularly to a hybrid virtualization engine which provides a system that has the advantages of both in-band and out-of-band virtualization engines without the disadvantages of either.

[0003] 2. Description of the Related Art

[0004] Until recently, shared storage has only been implemented well within homogeneous aggregations of computer systems, IBM System 390 (IBM Corporation, Armonk, N.Y., USA) is one example where different computers have been able to share storage, this sharing is enabled by the existence of a single architecture that fits together the sharing mechanisms and storage management mechanisms.

[0005] With the advent of open systems and especially since the advent of storage area networks (SAN), no single system vendor has been able to define such an architecture or storage management system under which all logical block storage can be deployed and managed. Instead, a set of different vendors have produced different products according to different standards and application program interfaces (APIs). The result of this is that while a large range of hardware products can be physically attached on a SAN fabric, they generally do not interoperate well and they also generally present different interfaces to storage administrators and storage administration tools, making the management of the set of connected logical block devices extremely difficult.

[0006] A class of SAN products termed ‘virtualization engines’ have started to appear. The purpose of this class of product is to significantly simplify the management of logical block storage in the following ways. A virtualization engine places a mapping function in the path in which read and write data requests are made by computer systems. The purpose of the mapping function is to determine which physical storage locations hold the data requested by the computer system, to intercept write data requests and perform special actions when the physical storage locations are sources for advanced functions such as point-in-time or remote copies. To be effective, this mapping function should add only small latency to the fulfillment of the data request, and scale well. Since a SAN attaches many computer systems (possibly from several vendors) with many storage devices (also possibly from several vendors), the mapping function must provide a consistent picture to all computer systems of which physical storage locations are associated with which logical blocks that are being accessed by those computer systems. To date, two virtualization engine architectures have emerged, ‘in-band’ and ‘out-of-band’. Out-of-band virtualization engines have two separate functions which coordinate together to provide virtualization.

[0007] The first function is the mapping function, which maps host requests to physical locations on a storage device. There are generally many instances of the mapping function within the SAN. These instances are typically located in a computer system either in a device driver or host bus adapter, or in a SAN switch such as a fibrechannel switch. The mapping function is meant to operate with very low overhead, for example in a fibrechannel switch is meant to operate at the maximum rated speed of the switch.

[0008] The second function is the management function, which handles the interface to the system administrator or system administration tool. One responsibility of the management function is to respond to configuration requests by updating the master copy of the configuration and also coordinating the altering of the tables in the various instances of the mapping function to reflect new configuration. Another responsibility is to be responsive to signals from the mapping function instances indicating that data access requests require special handling, and to coordinate the actions for carrying out the special handling. There is generally only one management function in the system albeit with a backup instance of it to provide fault tolerance for the function.

[0009] In-band virtualization engines combine the two functions such that read and write data requests are serviced at the same place that configuration requests are serviced. In addition, they often have more advanced functions such as fast-write caching and more robust and performance oriented implementations of remote copy and point-in-time copy.

[0010] There is provided, according to one aspect of the invention, a hybrid virtualization engine which provides a system that has the advantages of both the in-band and out-of-band virtualization engine without the disadvantages of either.

[0011] The invention provides a system for managing logical blocks of storage that includes an out-of-band mapping unit that processes data from a host. The mapping unit has a first table and a second table. An in-band virtualization engine is operatively connected to the out-of-band mapping unit. The first table maps locations on a memory device and the second table maps to the in-band virtualization engine. The in-band virtualization engine performs storage management functions and directly maps to the locations on the memory device. The out-of-band mapping unit is dynamically altered by the in-band virtualization engine to determine which of the first table and the second table is chosen.

[0012] The out-of-band mapping unit includes a flag (e.g., indicator) for choosing between the first table and the second table. The in-band virtualization engine performs advanced functions including fast-write caching, remote copy, and point-in-time copy. The in-band virtualization engine includes a management unit and a mapping unit that coordinates updates of the first table and the second table of the out-of-band virtualization engines. The in-band virtualization engine processes configuration requests from external administrators and also performs background request processing on the memory device.

[0013] The invention provides a method for managing logical blocks of storage which combines both the in-band and out-of-band virtualization engine approaches. The invention processes data from a host using the out-of-band mapping unit. The invention dynamically alters the out-of-band mapping unit using the in-band virtualization engine so as to determine which of the first table and the second table is chosen. The invention performs the storage management functions by using a management unit in the in-band virtualization engine. The direct mapping is performed using a mapping unit in the out-of-band virtualization engine. The invention updates the first table and the second table of the out-of-band virtualization engine using the in-band virtualization engine. The invention processes configuration requests from external administrators using the in-band virtualization engine. The invention performs background request processing on the memory device using the in-band virtualization engine.

[0014] Thus, the invention provides the advantages of both the in-band and out-of-band virtualization approaches. Potentially, most of the data requests go through the relatively inexpensive mapping unit of the in-band virtualization engine directly to storage, however for that set of data for which advanced function is required or desired, the data requests could go quickly (rather than through a high latency exception route as in the out-of-band virtualization approach) to the in-band virtualization engine where the advanced function will be carried out in a performance optimized way.

[0015] In addition, with the invention, a user can provision this system flexibly to achieve cost goals. For a low cost virtualization system or where the user already has high performance back-end storage (e.g., with fast-write cache), relatively inexpensive out-of-band virtualization engine mapping units could be deployed in most cases whereas relatively few of the in-band virtualization engines might be deployed to handle the management function and the small amount of data requests resulting from advanced functions such as point-in-time copy or remote copy. On the other hand, a user could choose to deploy a significant set of in-band virtualization engines where those engines will then provide high performance fast-write cache for most of the data.

[0022] Two virtualization engine architectures (‘in-band’ and ‘out-of-band’) are first discussed herein. The out-of-band virtualization engine in FIG. 1, for example, has two separate functions which coordinate together to provide virtualization. The first function is a mapping unit (function) 122, which maps host 120 requests to physical locations on a storage device. There are generally many instances of the mapping unit 122 within the SAN. These instances are typically located in a computer system either in a device driver or host bus adapter, or in a SAN switch such as a fibrechannel switch. The mapping unit is meant to operate with very low overhead, for example, a fibrechannel switch is meant to operate at the maximum rated speed of the switch. The second unit is the management unit (function) 124, which handles the interface to the system administrator or system administration tool.

[0023] One responsibility of the management unit 124 is to respond to configuration requests by updating the master copy of the configuration and also coordinating the altering of the tables in the various instances of the mapping unit to reflect a new configuration. Another responsibility is to be responsive to signals from the mapping unit 122 instances indicating that data access requests require special handling, and to coordinate the actions for carrying out the special handling. There is generally only one management unit 124 in the system albeit with a backup instance of it to provide fault tolerance for the function.

[0024] In FIG. 2, the in-band virtualization engines combine these two functions in a single SAN unit 130, such that read and write data requests are serviced at the same place that configuration requests are serviced. In addition, in-band virtualization engines often process more advanced (e.g., storage management) functions such as fast-write caching that allows write data to be written into the virtualization engine without writing the data all the way to slower storage devices, but maintains the guarantees normally provided by the slower storage devices. Thus, data will be preserved after a hardware error or power failure. More robust and performance oriented implementations include remote copy, point-in-time copy, and transparent data migration. Remote copy allows write data to be mirrored at a remote location so that it will be preserved and accessible after a disaster at the primary location. Point-in-time copy allows an image of a logical unit at a particular point in time to be logically copied to another logical unit, without physically having to perform any copying of the data from one logical unit to another. Transparent data migration is a facility for moving data from one storage device to another. without disturbing host access of that data while it is being moved. These advanced functions require significant amounts of memory, particularly in the case of a fast-write cache that stores the write data, and distributed protocols between virtualization engines, which requires a great deal of complex software and high processing power in the processors within the virtualization engines.

[0025] Some problems normally associated with out-of-band virtualization engines are that the advanced functions such as point-in-time copy, remote copy, and transparent data migration, call for significant high latency coordination between the mapping unit 122 and the management unit 124, at the expense of increased latency for those operations. Because the mapping functions are ‘thin’, they do not offer such features as fast-write cache services that allow hosts to write data without having to wait for data-copying or moving operation from one slow storage device to another to complete. This means that the write latency induced by advanced functions such as point-in-time copy cannot be hidden behind the fast-write cache.

[0026] Some problems normally associated with in-band virtualization engines are that users may have some or many of the advanced functions available for some of their data in their current storage controllers and they may not want to purchase the same advanced function again for all their data. There is more costly hardware associated with an in-band virtualization engine of a given capacity, therefore the total system cost may be higher.

[0027] One aspect of the invention provides a hybrid virtualization engine which delivers a system that has the advantages of both the in-band and the out-of-band virtualization engines without the disadvantages of either. FIG. 3 shows such a hybrid virtualization engine. In this system, there is, as with an out-of-band virtualization engine, multiple instances of an external mapping unit (function) 140. There is also a management unit (function) 152, an internal mapping unit (function) 154, and an advanced function agent unit 156 within an in-band virtualization engine(s) 150. However, the relationship between the in-band virtualization engines 150 and the external mapping units 140 is much different than either the in-band (FIG. 2) or an out-of-band (FIG. 1) virtualization engines discussed above.

[0028] The management unit 152, receives configuration (e.g., to perform storage management functions such as point-in-time copy). In response, the in-band engine 150 updates, in a consistent manner, the internal mapping units 154 of all the in-band virtualization engines and the maps 142, 144 contained within the external mapping units.

[0029] Within the external mapping units 140, two devices (e.g., tables) are maintained for each block of data that might be accessed by the host. The first table 144 maps that block of data to the physical location of storage 126. The second table 142 maps the block of data to the in-band virtualization engine 150. For each block of data, a flag (e.g., indicator) is maintained in the external mapping unit 140 to indicate whether the first table 144 should be used to map the request directly to a physical location, or whether the second table 142 should be used to map the request to a virtual location for the in-band virtualization engine. The in-band virtualization engine 150 controls the flag to determine which table 142, 144 will be used to process the data request coming from the host 120. Therefore, the invention allows the in-band virtualization engine 150, acting in response to a configuration instruction, to dynamically control how incoming data requested from the host 120 will be processed by the mapping units 140 by changing the flag in the mapping units 140. The in-band virtualization engine 150 can dynamically direct some or all of the data requests to the storage 126 or some or all of the data requests to the in-band virtualization engine 150.

[0030] When a data request is received at the external mapping unit 140 from a host 120, that flag will cause the request to be routed either to the back-end storage 126, or to the in-band virtualization engine 150 which will then perform the storage management function required for that request and in turn will make one or more requests (using mapping unit 154) of the back-end storage 126 to perform the request along with the storage management function.

[0031] FIG. 4 is a flowchart which shows the processing of the invention. In item 400, the invention dynamically alters the out-of-band mapping unit 140. Next, in item 402, the invention processes data from the host 120 using the out-of-band mapping unit 140. The invention then proceeds either to item 404, whereby the invention maps to locations on the memory device 126 using the first table 144; or the invention proceeds to item 406 to map to the in-band virtualization engine 150 using the second table 142. After the invention completes item 406, the invention will perform storage management functions in item 408 using the advanced function (e.g., storage management) agent 156 in the in-band virtualization engine 150. In item 410, the invention directly maps to the memory locations 126 using the mapping unit 154.

[0032] FIG. 5 is a flowchart which shows the interaction of the in-band virtualization engine 150 and the out-of-band mapping unit 140 in response to a configuration change. With the invention, the configuration change dynamically switches a set of I/O requests for specific virtual disk blocks from going straight from the out-of-band mapping function 140 to back-end storage devices 126, to going from the out-of-band mapping function 140 through the in-band virtualization engine 150. In item 500, configuration requests are received by the management function 152 of the in-band virtualization engine 150. In item 502, the management function 152 prepares to process requests through the in-band virtualization engine 150 by updating the mapping tables in the in-band mapping function 154. In item 504, the management function 152 instructs out-of-band mapping function 140 to quiesce I/O requests for the affected virtual disk blocks. In item 506, the management function 152 updates the first table 144 and the second table 142. In item 508, the management function 152 then instructs out-of-band mapping function 140 to use table 142 in mapping requests for the affected virtual disk blocks. In item 510, the management function 152 finally instructs out-of-band mapping function 140 to resume I/O request processing for the affected virtual disk Optionally, in item 512, the management function 152 generates background requests to read data from one back-end storage device 126 and to write data to another back-end storage device 126, and transmits those requests to back-end storage devices 126 via out-of-band mapping function 140.

[0033] Dynamically changing of streams of I/O requests from going through the in-band virtualization engine 150 back to going straight from the out-of-band mapping function 140 to back-end storage devices 126 is accomplished using a similar sequence as described by FIG. 5.

[0034] Thus, the advanced function can be dynamically enabled or disabled for sets of blocks. For example, establishing a point-in-time copy will cause the advanced function to be enabled until a background copy is completed (by action of the in-band virtualization engine 150 making background requests of the back-end storage 126), at which point the advanced function would be disabled. The in-band virtualization engine 150 will consistently update its own mapping tables 154 and the mapping tables 142, 144 within the external mapping units 140 for each change. On the other hand, the advanced function could be set relatively statically for certain blocks of data, for example, if a user wanted to enable fast-write cache in the in-band virtualization engine for some of the data.

[0035] Thus, the invention provides the advantages of both the in-band and out-of-band virtualization approaches. Potentially most of the data requests go through the relatively inexpensive mapping unit 140 directly to storage 126. However, for that set of data for which advanced function was required or desired, the data requests could go quickly (rather than through a high latency exception route as in the out-of-band virtualization approach) to the in-band virtualization engine 150 where the advanced function will be carried out in a performance optimized way.

[0036] In addition, with the invention, a user can provision this system flexibly to achieve cost goals. For a low cost virtualization system or where the user already has high performance back-end storage (e.g., with fast-write cache), relatively inexpensive mapping units 140 could be deployed in most cases whereas relatively few of the in-band virtualization engines 150 might be deployed to handle the management function and the small amount of data requests resulting from advanced functions such as point-in-time copy or remote copy. On the other hand, a user could choose to deploy a significant set of in-band virtualization engines 150 where those engines will then provide high performance fast-write cache for most of the data. The invention provides a common pool of storage from which all data storage space needs for all computer systems can be met. This contrasts with previous generations in which open (e.g., Unix or Windows) computer systems had access to only their own data storage devices (termed direct attach storage), therefore, free storage space under one computer system could be used by that computer system and no other.

[0037] The invention provides a single place and method for storage administrator to perform many of their tasks, for example, making block storage available to computer systems, or establishing remote copies or point-in time copies of sets of data.

[0038] While the invention has been described in terms of preferred embodiments, those skilled in the art will recognize that the invention can be practiced with modification within the spirit and scope of the appended claims.