RAID protected external secondary memory

[0001] The present application is related to commonly assigned and co-pending U.S. patent application Ser. No. 10/100,150 entitled “METHOD AND APPARATUS FOR USING A SOLID STATE DISK DEVICE AS A STORAGE CONTROLLER CACHE”, filed on Mar. 18, 2002, and hereby incorporated by reference.

[0002] 1. Technical Field

[0003] The present invention relates to data storage and, in particular, to data cache in a storage controller. Still more particularly, the present invention provides a method and apparatus for using a redundant array of solid state disks as level 2 cache in a storage controller.

[0004] 2. Description of the Related Art

[0005] A storage controller is an embedded system logically connected between a host computer system and a pool of storage. FIG. 1 illustrates an example of a typical storage controller system. Input/output (I/O) host 102 sends read and write data access requests to storage module 110. The storage module includes storage controller 120 and disk drives 130. Storage controller 120 performs read and write operations to satisfy the data access requests of the I/O host.

[0006] Storage controller 120 includes I/O cache 122. This I/O cache, also referred to as storage controller level 1 cache, is located in the storage controller memory. Data blocks that are read from disk drives 0–N may be stored in the I/O cache so that frequently accessed data may be read from the faster memory device rather than the slower disk drives. Furthermore, I/O cache 122 may also serve as intermediate storage for data blocks that are written to the disk drives. Subsequent reads of these data blocks may be found in the cache, thus reducing access time.

[0007] Still further, each one of level 1 and level 2 cache memories is a single point of failure. Thus, a cache may appear as a weakness in system reliability. If the cache memory fails or communication error prevents access to the cache memory, data can either be lost or inaccessible. Data unavailability is perceived as a failure.

[0008] More particularly, redundant array of independent disks (RAID) systems, may stripe data blocks and store each stripe on a different physical disk drive. For example, in the storage controller system shown in FIG. 1, a data block written to storage module 110 may be striped into N+1 stripes, each stripe being stored on a respective one of drives 0–N 130. With the greater number of reads and writes to physical drives in RAID systems, the importance of I/O cache is increased. While a RAID system can improve the performance and reliability of the hard disks themselves, RAID does not affect the performance or dependability of the cache.

[0009] Therefore, it would be advantageous to provide an improved multi-level cache for storage controller systems.

[0010] The present invention provides a redundant array of solid state disk drives among the storage devices controlled by a storage controller. The solid state disk drives may serve as a level 2 cache using standard multi-level cache management algorithms. The solid state disks may share a drive channel with other storage devices or may have a dedicated channel. The present invention may also provide multiple solid state disk devices to avoid single points of failure. With two solid state disks, the storage processor could maintain the cache data in both devices. If one device fails, the other could be used to maintain data services. With two or more devices, other Redundant Array of Independent Disks organizations may be used to improve data-to-metadata ratio while maintaining fault tolerance. Using these Redundant Array of Independent Disks techniques, the plurality of solid state disks may then be organized as a single level 2 cache volume that serves as a second level cache for a storage controller.

[0017] The description of the preferred embodiment of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiment was chosen and described in order to best explain the principles of the invention the practical application to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.

[0018] With reference now to the figures and in particular with reference to FIG. 2, a storage controller system is illustrated in accordance with a preferred embodiment of the present invention. Input/output (I/O) host 202 sends read and write data access requests to storage module 210. The storage module includes storage controller 220 and disk drives 230. Storage controller 220 performs read and write operations to satisfy the data access requests of the I/O host. Disk drives 230 may be connected to the storage controller via channels 0–N.

[0019] The depicted example illustrated in FIG. 2 and above-described examples are not meant to imply architectural limitations. For example, drives 230 may be hard disk drives. However, other storage devices, such as tape drives, optical disk drives, and the like, may be used in addition to or in place of the hardware shown in FIG. 2.

[0020] Storage controller 220 includes I/O cache 222, which serves as the storage controller level 1 (L1) cache. I/O cache 222 may be a random access memory (RAM). A typical example of a storage controller system may allocate 1 GB of memory for storage controller level 1 cache; however, more or less memory may be allocated for cache depending on the implementation.

[0021] Advances in memory technologies have led to the emergence of solid state disks. Solid state disk devices are essentially a non-volatile random access memory connected to an I/O channel. Due to the I/O channel protocol, memory access is not as fast for solid state disks as it is for the memory on the storage controller. However, the underlying random access memory generally has much improved I/O latency, I/O rate and sustained bandwidth as compared to hard disk drives.

[0022] In accordance with a preferred embodiment of the present invention, drives 230 also include solid state disk drive 232, which serves as the storage controller level 2 cache. With the improved performance characteristics of solid state disks over hard disk drives, solid state disk drive 232 may be used as a second level cache by a storage processor using standard multi-level cache management algorithms.

[0023] To make use of external, high-performance devices such as solid-state disks as secondary I/O cache, the present invention uses Redundant Array of Independent Disks (RAID) techniques to improve the performance and reliability of the level 2 cache. For example, storage module 210 may include multiple solid state disk devices 232 to avoid single points of failure. With two solid state disk devices, storage controller 220 could maintain the same data in both devices. If one solid state disk failed, the other could be used to maintain data services. This configuration is referred to as RAID 1 (mirroring). With two or more solid state disk devices, other RAID organizations, such as striped parity (RAID 5), may also be used to improve the data to metadata ratio while maintaining fault tolerance.

[0024] With reference now to FIGS. 3A and 3B, block diagrams are shown illustrating example storage controller architectures in accordance with a preferred embodiment of the present invention. Particularly, FIG. 3A illustrates a single-memory storage controller architecture. Storage controller 300 employs a peripheral component interconnect (PCI) local bus architecture. Although the depicted example employs a PCI bus, other bus architectures such as Industry Standard Architecture (ISA) may be used. Microprocessor 302, with internal level 1 cache, and memory pool 308 are connected to PCI local bus 310 through memory controller 304. Microprocessor level 2 cache 306 is also connected to memory controller 304. PCI bridge 310 also may include an integrated memory controller and cache memory for processor 302.

[0025] In the depicted example, ethernet adapter 314, PCI to ISA bridge 312, drive channel adapters 316–318, and host channel adapter 320 are connected to PCI bus 310 by direct component connection. PCI to ISA Bridge 312 provides a connection through ISA bus 330 for basic input output system (BIOS) 332 and serial port 324.

[0026] Processor 302 is used to coordinate and provide control of various components within storage controller 300 in FIG. 3A. Instructions for the storage controller may be located on storage devices, such as BIOS 322, and may be loaded into memory pool 308 for execution by processor 302.

[0027] Memory pool 308 is a single memory pool that is logically partitioned into two regions. A first region serves as processor memory. This portion of memory is used by processor 302, for example, as “scratch pad” memory to perform the operations of the storage controller. The second region of memory pool 308 serves as I/O buffer memory or level 1 storage controller cache.

[0028] Drive channel adapters 316–318 provide drive channels for storage devices, such as hard disk drives. A storage controller may have, for example, four drive channels. Each drive channel may support multiple drives per channel. The number of drives is limited by I/O hardware and communication protocol.

[0029] In accordance with a preferred embodiment of the present invention, solid state disks 342, 344 are connected to at least one of the drive channel adapters, such as drive channel adapters 316, 318 in FIG. 3A. Solid state disks 342, 344 form a storage controller level 2 cache volume that is used to supplement the level 1 cache stored in memory pool 308. Each solid state disk may store, for example, 8 GB of data. Therefore, read request performance may be greatly improved due to an increased probability of the data residing either in the storage controller level 1 cache or the high-speed solid state disk.

[0030] Performance and reliability may also be improved by using RAID storage configurations. For example, data may be striped (RAID 0) across solid state disks 1–N. As another example, data stored on one solid state disk, such as disk 342, may be mirrored (RAID 1) on another solid state disk, such as disk 344. A combination of RAID 0 and RAID 1, referred to as RAID 1,0 or RAID 10, can also be used to stripe and mirror cache data stored on solid state disks 1–N. Further, one solid state disk may be used to store parity data (RAID 3) for the other solid state disks. Still further, cache data and parity data may be striped across three or more drives for performance and fault tolerance (RAID 5). Other RAID techniques may also be used to manage the second level cache volume stored on solid state disks 342, 344.

[0031] Turning now to FIG. 3B, a dual-memory storage controller architecture is shown in accordance with a preferred embodiment of the present invention. Storage controller 350 employs a peripheral component interconnect (PCI) local bus architecture. Although the depicted example employs a PCI bus, other bus architectures such as Industry Standard Architecture (ISA) may be used. Microprocessor 352, with internal level 1 cache, and memory pool 358 are connected to PCI local bus 360 through memory controller 354. Microprocessor level 2 cache 356 is also connected to memory controller 354. PCI bridge 360 also may include an integrated memory controller and cache memory for processor 352.

[0032] In the depicted example, Ethernet adapter 364, PCI to ISA bridge 362, drive channel adapters 366–368, and host channel adapter 370 are connected to PCI bus 360 by direct component connection. PCI to ISA Bridge 362 provides a connection through ISA bus 380 for basic input output system (BIOS) 382 and serial port 384.

[0033] Processor 352 is used to coordinate and provide control of various components within storage controller 350 in FIG. 3B. Instructions for the storage controller may be located on storage devices, such as BIOS 382, and may be loaded into memory pool 358 for execution by processor 352.

[0034] Processor 352 uses memory pool 358, for example, as a “scratch pad” memory to perform the operations of the storage controller. Memory pool 374 is connected to PCI bus 360 by memory controller 372. Memory pool 374 serves as I/O buffer memory or level 1 storage controller cache.

[0035] Drive channel adapters 366–368 provide drive channels for storage devices, such as hard disk drives. In accordance with a preferred embodiment of the present invention, solid state disks 392, 394 are connected to drive channel adapters, such as drive channel adapters 366, 368 in FIG. 3B. Solid state disks 392, 394 form a cache volume that serves as storage controller level 2 cache to supplement the level 1 cache stored in memory pool 374. Each solid state disk may store, for example, 8 GB of data. Therefore, read request performance may be greatly improved due to an increased probability of the data residing either in the storage controller level 1 cache or the high-speed solid state disk. Performance and reliability may also be improved by using RAID storage configurations as discussed above with respect to FIG. 3A.

[0036] Those of ordinary skill in the art will appreciate that the hardware in FIGS. 3A and 3B may vary depending on the implementation and the depicted examples in FIGS. 3A and 3B and above-described examples are not meant to imply architectural limitations. For example, the examples shown in FIGS. 3A and 3B illustrate bus architectures; however, the present invention may be implemented using other architectures, such as a switched architecture. For example, the present invention may be implemented using a Fibre Channel architecture.

[0037] With reference to FIGS. 4A–4D, example RAID configurations using solid state disks to implement a storage controller cache are depicted in accordance with a preferred embodiment of the present invention. More particularly, FIG. 4A illustrates an example RAID 0 configuration. Cache data stream 402 is received by RAID engine 404. The RAID engine manages the storage of the cache data stream onto solid state disks 406. Rather than storing the cache data on a single solid state disk, RAID engine stripes the data across the plurality of solid state disks. While this configuration may result in a slight increase in performance due to parallel reads to and/or writes to the disks, RAID 0 does not enhance reliability. Thus, the increased cost for the plurality of solid state disks will likely outweigh the performance benefit.

[0038] With reference now to FIG. 4B, an example RAID 1 configuration is shown in accordance with a preferred embodiment of the present invention. Cache data stream 412 is received by RAID engine 414. The RAID engine manages the storage of the cache data stream onto solid state disks 416. The RAID engine may store the cache data, for example, on a single solid state disk and mirror or copy that data onto a second solid state disk, as shown in FIG. 4B. If one of the solid state disks fails, the other disk can be used to maintain the second level storage controller cache. A combination of RAID 0 and RAID 1, referred to as RAID 1,0 or RAID 10, can also be used to stripe and mirror cache data stored on solid state disks 416.

[0039] Turning now to FIG. 4C, an example RAID 3 configuration is depicted. Cache data stream 422 is received by RAID engine 424, which manages a cache volume stored on solid state disks 426. The RAID engine stripes the cache data across a plurality of the solid state disks and generates parity data from the striped cache data. The RAID engine stores the parity data on a separate, dedicated solid state disk drive. If a solid state disk fails, the striped data can be reconstructed using the remaining stripes and the parity data. Thus, RAID 3 provides the performance enhancement of RAID 0 and fault tolerance with the dedicated parity disk.

[0040] FIG. 4D illustrates an example RAID 5 configuration in accordance with a preferred embodiment of the present invention. Cache data stream 432 is received by RAID engine 434. The RAID engine generates parity data from the cache data and stripes the cache data and the parity data across solid state disks 436. The parity data is interspersed with the striped cache data, as shown by example in FIG. 4D. If a solid state disk fails, the striped data can be reconstructed using the remaining stripes and the parity data. Thus, RAID 5 provides performance enhancement by reading from and writing to all of the solid state disks. Fault tolerance is provided by the parity information.

[0041] Other RAID configurations may be used within the scope of the present invention. The examples shown in FIGS. 4A–4D are not meant to imply architectural limitations for the present invention. For example, more or fewer solid state disks may be used depending upon the implementation.

[0042] With reference now to FIG. 5, a flowchart illustrating the operation of a storage controller is shown in accordance with a preferred embodiment of the present invention. The storage controller uses RAID techniques to create a cache volume. The storage controller then may store the cache data across the solid state disks using these RAID techniques according to the following procedure.

[0043] The process begins and receives a data access request (step 502). A determination is made as to whether the data access request is a read request or a write request (step 504). If the data access request is a read request, a determination is made as to whether the data is in level 1 cache (step 506). If the data is in level 1 cache, the process fetches the data from level 1 cache (step 508) and the process ends. If the data is not in level 1 cache in step 506, a determination is made as to whether the data is in the level 2 cache stored in the solid state disks (step 510). If the data is stored in the storage controller level 2 cache, the process fetches the data from level 2 cache volume (step 512) and the process ends. However, if the data is not stored in level 2 cache in step 510, the process reads the data from the storage device (step 514) and ends.

[0044] Returning to step 504, if the data access request is a write request, a determination is made as to whether the data is cached (step 516). If the data is cached, a determination is made as to whether the data is cached in level 1 cache or level 2 cache (step 518). If the data is cached in level 1 cache, the process overwrites the data in level 1 cache in memory (step 520) and ends. If the data is cached in level 2 cache volume in step 518, the process overwrites the data in the level 2 cache volume (step 522) and the process ends.

[0045] If the data is not cached in step 516, the process allocates space in level 1 cache for the written data (step 524). A determination is made as to whether level 1 cache needs to be flushed to make space to cache the written data (step 526). If a flush of level 1 cache is not necessary, the process ends. If, however, a flush of level 1 cache is necessary in step 526, the process writes data from level 1 cache to the level 2 cache volume (step 528) and a determination is made as to whether a flush is necessary to make space to write data in level 2 cache volume (step 530). If a flush is not necessary, the process ends. However, if a flush is necessary in step 530, the process flushes data from level 2 cache volume (step 532) and ends.

[0046] Thus, the present invention provides a second level of storage controller cache using a solid state disk device. Read request performance is improved due to an increased probability of data residing in either the storage controller level 1 cache in memory or the solid state disk device. The present invention may also provide multiple solid state disk devices to avoid single points of failure. With two solid state disks, the storage processor could maintain the cache data in both devices. If one device fails, the other could be used to maintain data services. With two or more devices, other RAID organizations may be used to improve data-to-metadata ratio while maintaining fault tolerance. Using these RAID techniques, the plurality of solid state disks may then be organized as a single level 2 cache volume that serves as a second level cache for a storage controller.

[0047] Retrieving the data from this second level cache is a less expensive operation than reading the data from the hard disks, especially if the data is striped across several disks as in a RAID storage system. Allowing the volumes to be configured as second level cacheable allows users to tune system performance for specific applications. Solid state disk devices are available in standard hard disk drive form factors. Using these devices as customer replaceable units in hard drive modules allows users to upgrade and expand simply by populating additional units in the system.