Method, system, and program for demoting data from cache based on least recently accessed and least frequently accessed data

[0003] 1. Field of the Invention

[0004] The present invention relates to a method, system, and program for managing data in cache and selecting cache entries for demotion to make room for subsequently accessed data.

[0005] 2. Description of the Related Art

[0006] In a memory cache, pages or tracks of data are copied from a storage device, such as a hard disk drive or other non-volatile storage device typically comprised of a magnetic medium, and placed into a volatile, electronic memory area referred to as cache. When tracks or data are accessed from the storage device they are loaded into cache and returned to the application requesting the data. Because the accessed data remains in cache, a subsequent request for the data can be returned from cache, which is substantially faster than retrieving the data from the storage device. Returning data from cache, referred to as a cache hit, improves performance and system throughput because a cache memory by definition provides faster access to data. A cache may also be used to provide faster access to a main volatile memory, such as a random access memory (RAM). For instance, many processors include an “on-board” cache that caches data from RAM for the processor to use and subsequently access from the faster cache memory. In both cases, disk caching and memory caching, the cache provides a high speed memory from which data may be returned faster than the storage device or main memory where the data is maintained.

[0007] After the cache utilization reaches a certain upper limit, the cache manager will demote data from cache to make room for subsequently accessed tracks. Areas of cache marked as demoted may then be overwritten by new data, thus making room for data more recently accessed from storage. In the prior art, a least recently used (LRU) algorithm is used as follows. When a track is added to a cache, a pointer to the track in cache is placed at a top of the LRU linked list. If a track already in cache is again accessed, then the pointer to that track in cache is placed at the top of the LRU list. When the cache manager determines that data must be demoted or removed from cache to make room for subsequent data accesses, the cache manager will demote tracks whose pointers are at the bottom of the list, representing those tracks that were accessed the longest time ago relative to other tracks in cache.

[0008] This prior art LRU approach works sufficiently well with sequentially accessed data, that is data that is accessed contiguously. However, such LRU algorithms are not optimal for randomly accessed data, that is tracks of storage accessed out of sequence. In such case, a demoted track in cache, even though it is the least recently accessed, may have been accessed more frequently than other tracks and is, thus, more likely to be subsequently accessed than those tracks in cache less frequently accessed. Thus, the LRU approach does not consider the frequency of access to cache entries when selecting entries for demotion.

[0009] Thus, there is a need in the art to improve cache hits, i.e., the read requests returned from cache to increase performance and data throughput.

[0010] To overcome the limitations in the prior art described above, preferred embodiments disclose a method, system, and program for caching data. Data from a device, such as a volatile memory device or non-volatile storage device, is maintained in entries in a cache. For each entry in cache, a variable indicates both a time when the cache entry was last accessed and a frequency of accesses to the cache entry.

[0011] In further embodiments, a determination is made when a number of entries in cache has reached a threshold. In response to reaching the threshold, a determination is made of a plurality of cache entries having a relatively low value for the variable with respect to the value of the variable for other cache entries. A relatively low variable value indicates that the cache entry is one of a least recently accessed entry and/or least frequently accessed entry. Those determined cache entries having the relatively low value for the variable are demoted from cache.

[0012] In still further embodiments, the variable for a cache entry is increased whenever the cache entry is accessed.

[0013] Preferred embodiments provide an algorithm and data structures for maintaining for each cache entry a ranking or variable indicating a time when the entry was last accessed and the number of accesses relative to other cache entries. This variable is used in determining which entries to demote from cache to make room for subsequently accessed data. Preferred embodiments would tend to demote among the entries that were least recently accessed those entries that are less frequently accessed. This methodology improves the cache hit ratio because an entry that is more frequently accessed relative to another entry is more likely to be accessed again in the future. Thus, the preferred methodology would select for demotion those entries less likely to be accessed in the future with respect to other entries. This is an improvement over prior art techniques that do not take into account the frequency of access when demoting cache entries.

[0014] Referring now to the drawings in which like reference numbers represent corresponding parts throughout:

[0015] FIG. 1 illustrates a cache architecture in accordance with preferred embodiments of the present invention; and

[0016] FIGS. 2 and 3 illustrate cache management logic in accordance with preferred embodiments of the present invention.

[0017] In the following description, reference is made to the accompanying drawings which form a part hereof and which illustrate several embodiments of the present invention. It is understood that other embodiments may be utilized and structural and operational changes may be made without departing from the scope of the present invention.

[0018] FIG. 1 illustrates a cache architecture in which preferred embodiments are implemented. A cache 2 includes cache entries 4a, b, c, dinto which tracks or pages of data from the storage 6 may be placed. When a track or page of data is staged into a cache entry 4a, b, c, d, a new entry 8ais added to the top of the LRU linked list 10. For each cache entry 4a, b, c, d, the LRU linked list 10 includes one entry 8a, b, c, d. Each entry 8a, b, c, din the LRU linked list 10 includes a LRU rank value 12a, b, c, dand pointer 14a, b, c, dto one cache entry 4a, b, c, d, which may comprise a page if the storage is a memory device or a track if the storage 6 is a disk drive or other non-volatile magnetic storage medium. Further, when an input/output request 5 is serviced from cache, i.e., a cache hit, then the entry 8a, b, c, din the LRU linked list 6 including the pointer to the accessed cache entry 4a, b, c, dis moved to the top of the LRU linked list 8. Although the cache 2 is only shown as having a few cache entries 4a, b, c, dand corresponding entries 8a, b, c, din the LRU linked list 10, in practice there are likely thousands of cache entries and entries in the LRU linked list 10. A bus 16 provides a data path between the cache 2 and storage 6. The cache 2 further maintains a time counter 18 that is used in calculating the LRU rank 12a, b, c, d.

[0019] The LRU rank 12a, b, c, dprovides a value for each entry 4a, b, c, din cache 2 that indicates both how frequently the entry is accessed and the time of last access. The purpose of this weighting is to allow the process demoting entries 4a, b, c, dfrom cache 2 to take into account the frequency with which an entry was accessed and remove those less frequently accessed entries. Demoting relatively less frequently accessed entries increases the likelihood of a cache hit because data that has a history of being accessed more frequently is more likely to be accessed in the future over less frequently accessed entries.

[0020] FIG. 2 illustrates logic implemented in a cache controller or control logic (not shown), referred to herein as the cache 2, to manage the entries in cache 2. Control begins at block 100 with the initiation of cache management activities by initializing the cache 2 and setting the timer counter to one. At block 102, the cache 2 receives an I/O request 5 for a track in storage 6, or in cases where the cache is caching data from a memory device, a page from memory. Still further, the cache 2 could receive a request for a page of data. If the requested track is in cache 2 as a cache entry 4a, b, c, d(at block 104), then the cache 2 services (at block 106) the I/O request 5 directly from cache 2. For a read request, the cache 2 would return the requested data to the entity requesting the data; for a write request the cache 2 would update the write data to the track location in cache. The cache 2 would then move (at block 108) the entry 8a, b, c, din the LRU linked list 10 corresponding to the cache entry 4a, b, c, dsubject to the I/O request 5 to the top of the LRU linked list 10. The cache then adds (at block 110) the modulo of the time counter 18 divided by 512 to the LRU rank 12a, b, c, dof the LRU entry 8a, b, c, dmoved to the top of the LRU linked list 10. In alternative embodiments, the time counter 18 may be divided by other values. The time counter 18 is then incremented by one (at block 112).

[0021] If the requested track is not in cache, then the cache 2 stages (at block 114) the requested track from storage to a cache entry 4a, b, c, dand adds (at block 116) the LRU entry 8a, b, c, dfor the new cache entry 4a, b, c, dto the top of LRU list 10. The LRU rank for the added cache entry is then set to zero. The I/O request 5 is then serviced (at block 118) from cache 2. Control then transfers to block 110 to set the LRU rank 12a, b, c, dfor the new entry.

[0022] The above LRU rank is weighted for previous accesses. Because the time counter 18 is incremented for every I/O access, the value added to the LRU rank for subsequently accessed cache entries increases substantially in a short period of time. Thus, LRU entries recently accessed or added to cache will have a substantially higher LRU rank than the LRU rank of entries that have not been accessed recently, even those entries accessed numerous times. However, to the extent entries have not been accessed for the same amount of time, the entry that was accessed more frequently will have a higher LRU rank because its LRU rank will be weighted with previous accesses. Thus, to the extent entries were last accessed at about the same time, which in terms of cache operations is within fractions of a second of each other, the more frequently accessed entry will have a greater weighting. As discussed, a cache entry more frequently accessed in the past is likelier to be accessed in the future over less frequently accessed entries that were last accessed at about the same time.

[0023] FIG. 3 illustrates logic implemented in the cache 2 to demote entries in cache. At block 150 the cache 2 utilization, i.e., number of cached entries, reaches an upper threshold. In response, the cache determines (at block 152) from the last 1024 entries in the LRU linked list 8, thirty-two entries that have the lowest LRU rank 12a, b, c, d. The cache 2 then demotes (at block 152) those determined thirty-two entries from cache 2. Once a track is demoted, if it is dirty data, i.e., updates to locations in storage 6, then the data will be destaged to storage 6. Once dirty data is destaged or if the data is not an update to the track in storage 6, then the cache entry 4a, b, c, dis marked as available to receive new tracks staged in from the storage 6.

[0024] As discussed, those entries having the lowest LRU rank marked for demotion are both the least recently accessed and among those entries least recently accessed recently, are less frequently accessed. Thus, the algorithm of FIGS. 2 and 3 ensures that more frequently accessed cache entries 4a, b, c, dremain in cache longer. Because a more frequently accessed entry is more likely to be subsequently accessed over less frequently accessed entries last accessed at about the same time, the preferred embodiment cache demotion scheme will improve the cache hit ratio as those entries more frequently accessed are less likely to be demoted from cache 2. In fact, in performance tests, it has been found that the throughput and performance of systems employing the cache management scheme of the preferred embodiments has improved 5-10% over systems that employ the prior art LRU demotion method.

[0025] The following describes some alternative embodiments for accomplishing the present invention.

[0026] The preferred embodiments may be implemented as a method, apparatus or program using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. The programs defining the functions of the preferred embodiment can be delivered to a computer via a variety of information bearing media, which include, but are not limited to, computer-readable devices, programmable logic, memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, SRAMs, etc.) carriers, or media, such as a magnetic storage media, “floppy disk,” CD-ROM, a file server providing access to the programs via a network transmission line, wireless transmission media, signals propagating through space, radio waves, infrared signals, etc. Of course, those skilled in the art will recognize that many modifications may be made to this configuration without departing from the scope of the present invention. Such signal-bearing media, when carrying computer-readable instructions that direct the functions of the present invention, represent alternative embodiments of the present invention.

[0027] As discussed, the cache management scheme of the preferred embodiments may be employed in a disk cache that provides high speed storage of data accessed from a storage device, such as a hard disk drive, tape, optical disk or any other magnetic storage medium or in a memory cache that caches data from a slower memory device, such as the main memory.

[0028] The preferred embodiment cache may comprise a set associative cache, direct mapped cache, fully associative cache or any and other cache designs known in the art. Still further, the cache may comprise a battery backed up cache, also referred to as a non-volatile storage unit.

[0029] The preferred logic of FIGS. 2 and 3 describes specific operations occurring in a particular order. In alternative embodiments, certain of the logic operations may be performed in a different order, modified or removed and still implement preferred embodiments of the present invention. Morever, steps may be added to the above described logic and still conform to the preferred embodiments. Further, operations described herein may occur sequentially or certain operations may be processed in parallel.

[0030] Preferred embodiments described a specific technique for weighting a variable with the frequency a track or page of data in cache has been accessed. In alternative embodiments, different algorithms may be used to factor in both the time of the most recent access and the frequency of accesses to an entry in cache.

[0031] In preferred embodiments, a LRU linked list was maintained for cache entries. In alternative embodiments, different queue structures, other than LRU linked lists, may be used in managing the data in cache.

[0032] In summary, preferred embodiments disclose a method, system, and program for caching data. Data from a device, such as a volatile memory device or non-volatile storage device, is maintained in entries in a cache. For each entry in cache, a variable indicates both a time when the cache entry was last accessed and a frequency of accesses to the cache entry.

[0033] The foregoing description of the preferred embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto. The above specification, examples and data provide a complete description of the manufacture and use of the composition of the invention. Since many embodiments of the invention can be made without departing from the spirit and scope of the invention, the invention resides in the claims hereinafter appended.