Adaptive Domain Partitioning of Cache Memory Space

This invention relates to data storage management and control. More particularly, the invention relates to a memory system wherein a solid-state cache memory used in conjunction with long-term magnetic disk storage devices of varying types can be subdivided into blocks or "domains" of sizes chosen so as to insure efficient cache memory utilization despite varying demands placed on the system.

The present invention relates to a solid-state cache memory subsystem designed to be used primarily as an adjunct to long-term magnetic disk storage media. The cache memory is connected outboard of a conventional data transmission channel not specially designed for use with such a cache memory sub-system, such that the cache memory system is plug-compatible with and software transparent to a host computer. In particular, the invention is designed to operate in accordance with the cache memory subsystem which is the subject matter of copending patent application Ser. No. 325,346 filed Nov. 27, 1981 incorporated herein by reference. That application describes a cache memory subsystem which is operatively connected to storage director(s) which in turn are connected between a host computer at one point and the control modules at another. In turn the control modules connect to the actual disk drive. As is known in the art, the directors serve to interface the host with the control modules which in turn interface the directors to the disk drives. The control modules are selected in accordance with the particular type of disk drive used while the directors may be used with various types of control modules and hence with various types of disk drives. According to the invention of that application, the cache is adapted to be used with varying sorts of disk drives by connection to the director(s).

The function of the solid-state cache memory subsystem in the invention of the copending application referred to above is to store certain data which has been determined to be likely to be called for by the host in advance of an actual host request for that certain data. This determination is done in accordance with the invention disclosed in another copending application, Ser. No. 325,350 filed Nov. 27, 1981, also incorporated herein by reference. In a preferred embodiment of that invention, once it has been determined that a particular block of data is likely to be called for by the host, the entire disk track containing that block of data is brought into the cache memory for temporary storage in anticipation of its being called for by the host computer. Inasmuch as the cache memory subsystem must be adapted to cooperate with varying types of disk drives which in turn have varying amounts of data stored on a given track, means must be provided whereby the cache memory is as useful as is possible despite the fact that disk tracks of varying sizes will be stored therein track by track. For example, if it were assumed that the total cache memory was to contain 200 kilobytes (hereinafter 200 kb) of data, and the track length of a particular disk drive was 20 kb, 10 such tracks would fit into the cache. Stated slightly differently, the cache could be divided into ten 20-kb "frames". When, for example, all ten frames were full and it was desired to cache another track, a cache manager could then simply deallocate a frame, for example a frame containing the data which had been least recently used and then allocate that frame to the new track. However, if data stored on another type of disk drive having a track size of, say, 25 kb, were desired to be staged, it would be necessary to deallocate two adjacent 20 kb frames in order to accommodate a 25 kb track thus wasting 15 kb of space. Even assuming two adjacent frames could be deallocated without loss of useful data, the waste of space alone would clearly be undesirable. If on the other hand the entire solid-state memory were divided into one subportion or "domain" adapted to store only tracks of one size, and another subportion proportioned to storage of tracks of another size, the individual domains of the memory might be efficiently allocated. However, utility of this solution presupposes that the relative activity of the two sizes of tracks is constant over time, which cannot be expected to be the case. Thus, it is desirable that all area of the solid state memory be accessible to any track size supported by the cache in order that it can be most efficiently used. Finally, one could divide all track sizes into much smaller uniformly-sized "pages", e.g., 2 kb, which would fit fairly uniformly within the frames. However, this would require complex hardware to implement and would additionally reduce the efficiency of the cache as necessarily an individual disk track would tend to be stored on varying portions of the cache, which would entail substantial additional support, e.g. recall of the numerous storage locations of the portions of a given track.

It is accordingly an object of the present invention to provide a means whereby the storage area available in a solid-state cache memory can be most efficiently used.

It is a further object of the invention to provide a method whereby a solid-state cache memory subsystem can be operated efficiently to store data records of varying sizes.

It is a further object of the invention to provide a means for "adaptively" or "dynamically" reconfiguring a cache memory previously divided into domains of plural classes in accordance with actual use, so as to fully utilize the memory space available.

The above needs of the art and objects of the invention are satisfied by the present invention according to which a solid-state memory is subdivided into domains. The size of the domains is chosen to be a substantially integral multiple of each of the track sizes desired to be stored therein, such that, for example, three 50-kb tracks from one class of device will fit within the same 150-kb domain as two 75-kb tracks from another class. Each individual domain is thereafter assigned to storage of tracks from a given type of device, and then divided into track-sized frames, to which individual tracks are written. Statistics are maintained on domain usage. If the indications are that the data staged from a particular device type predominates over another, a given domain is reallocated from the second device type to the first device type so as to insure efficient usage of the total memory space available. In this way, fragmentation due to intermingling of varying track sizes in the same area of memory is avoided.

The invention will be better understood if reference is made to the accompanying drawings, in which:

FIG. 1 represents a schematic block diagram view of the system of the invention;

FIG. 2 shows how individual domains can be subdivided according to the track sizes desired to be stored therein;

FIG. 3 shows a domain switching decision table used in determining which domains to switch at a particular time; and

FIG. 4 shows a diagram indicating how the domain switching determination decision is utilized in actually performing the switching operation and how the control tables are updated as the scheme proceeds.

As discussed above, the adaptive domain partitioning methods of the invention are designed to be used in a solid-state cache memory subsystem operated to contain data being stored on disk memory in anticipation of its being called for by a host computer.

FIG. 1 shows a schematic diagram of such a system. The host computer 10 is connected by means of a conventional channel interface 12 to a storage director 16. The storage director in turn interfaces to plural control modules 18 which themselves control disk drives 14 which as shown may be of differing types. In particular, the amount of data stored on a given track of one of the disks 14 can vary. When a decision is made by a cache manager 24, e.g. in accordance with copending application Ser. No. 325,350 filed Nov. 27, 1981, that it would be desirable to store the entire contents of a track in a solid-state cache memory array 22, a number of locations in the cache corresponding in size to the length of the track on the disk 14--i.e., a "frame" of locations in the cache--must be assigned to the data. When it is desired by the cache manager that a particular track be staged into a frame in the cache array 22, a frame is assigned to that track and the operation proceeds. The present invention is concerned with efficient division of the cache array into subportions or "domains". The size of the domains is chosen so that approximately integral numbers of disk tracks of varying lengths fit into "frames" in each domain. Thus, after allocation of the domain to a particular track length, it is divided into "frames" of sizes corresponding to the track lengths.

It can be expected that at some point all the domains allocated to tracks of a particular length would be heavily in use, while those allocated to disk tracks of other type(s) were not. According to an important aspect of the present invention, the domains are chosen to all be of the same size so that they can be efficiently reallocated to contain data stored on a different type of disk memory device. The domains are "dynamically" repartitioned into frames of a different size according to usage. For this reason, the domain size is chosen such that the domains contain integral numbers of frames of various sizes. FIG. 2 shows an example of how this can be done. There the domain size is approximately 256 kilobytes (kb). In a preferred embodiment, the cache contains some 12 of such domains for a total capacity of approximately three megabytes. Three possible domain subdivision schemes are shown in FIG. 2.

FIG. 2A shows a scheme which might be adopted for a domain to be used with the Model 8650 disk drive sold by Storage Technology Corporation, the assignee of the present invention, in which each track contains some 19,069 bytes plus some indentifying "header" information. Thirteen such tracks with their headers will fit within the 256 kb domain. Storage Technology Corporation's Model 8375 disk drive has a track size of approximately 37,000 bytes. As shown in FIG. 2b, six such 37,000 kb tracks with headers will fit within the 256 kb domain. Finally, Storage Technology Corporation's Model 8380 disk drive has a 47,476 byte track size. FIG. 2c shows that five such frames together with headers fit well into the 256 kb domain size.

According to the present invention, dynamic repartitioning of cache space allocation is provided in accordance with usage. This allows high storage efficiency without severe fragmentation. Fragmentation is wastage of space due to lack of correspondence between the size of available frames and the sizes of the tracks that are to be stored in the available frames. Severe fragmentation is avoided because a domain may be reallocated from one device type to another by reframing whenever desirable. Reallocation occurs in only one domain at a time, thus providing stability and simplicity by eliminating the need to invalidate the entire cache design. A "domain use count" is kept to keep track of which class of domains is experiencing heavy usage. This makes it possible to determine whether the current allocation of the domains is the most efficient with respect to usage of the system at any particular time. This is discussed below in connection with FIGS. 3 and 4.

A preliminary problem which must be addressed is frame assignment, that is, picking the best frame to assign to a given track to be staged to the cache array. The "best frame" is one which is not currently assigned to any track. However, if no frames are free then a frame must be "detached" from the track to which it is currently assigned and reassigned to the new track which is to be staged. The best frame to detach is one which is no longer being read by the host. Frames which are still being read by the host should not be detached because overhead spent in staging the track would not be recovered if it is detached before the data is read. In a presently preferred embodiment of the invention, frame assignment is performed using a "least recently used" (LRU) scheme. The LRU scheme is implemented by means of a list of frames. The list has a top, at which is located the frames which have been most recently used, and a bottom, which corresponds to the frames which have not been used recently and those which are free. Thus when a frame is used, that is, is staged to or read from, its name is placed atop the list. When a frame is detached it is moved to the bottom of the list. This occurs when the host directs a write operation to the area on disk from which the contents of that frame were staged, thus rendering the data in that frame invalid, or when all records have been read from that track. Thus, the bottom of the list contains a free frame or the least recently used frame, and the frame on the bottom of the list will typically be the best to assign to a track to be staged. Of course, when the frame is assigned, it is moved to the top of the list, indicating that it is the most recently used.

Even given the LRU scheme, however, it may well be that if two or more classes of domains of different sizes are present in the cache memory, a first type will be much more heavily used than a second type so that the LRU list for the first domain type will have, for example, fewer free frames on it than the list of the second domain type. System performance, in general, can be improved by reallocating domains from the second type of storage to the first. This domain reallocation method allows for efficient use of cache space. Since the need for frames suited to a particular device will vary with time, it is particularly desirable to dynamically allocate and deallocate domains to device type based on demand. An important aspect of the present invention relates to an adaptive or dynamic domain allocation scheme; in a preferred embodiment, the invention supports two types of disk storage devices having differing track storage capacities and can reallocate domains from, e.g. device type A to device type B, although it would be possible to provide three or more class of domains in a given cache. In this way the cache space is allocated in proportion to the input/output activity.

According to the presently preferred embodiment of the invention, three questions are considered by the domain allocation decision-making process:

How often to switch domains?

When to switch a domain?

Which domain to switch?

With respect to the first question, domain switching is considered in the presently preferred embodiment on the order of once every 1000 stage operations, i.e., every 1000 times a data track is read into the cache memory. In a relatively large embodiment of the invention this is expected to occur approximately every 30 to 60 seconds.

The second problem is determining when to switch a domain. This decision should be based on an imbalance between the cache input/output activity ratios for device type A with respect to device type B. A suitable method of measuring activity is to count the stages to domains assigned to each device type. This stage count ratio should be equal to the ideal "frames available" ratio. In turn, the frames available ratio can be used to compute an ideal "domains allocated" ratio since the number of frames per domain is known. If the actual domains allocated ratio is different from the ideal domains allocated ratio it is time to switch domains. In this way the relative size of the various frames is removed from the consideration, as opposed to merely measuring domain type activity.

The third and most difficult problem is to determine which domain to switch. Domains with many recently used frames are clearly not good candidates for switching. The best domains to switch are instead those that have many old and free frames. Two alternatives may be considered. First, since the LRU list described above shows the relative age of the frames of each domain, the list can be used to determine which domain to switch. A scan of a predetermined number of frames on the list described above can be made, incrementing a counter for each domain when a frame belonging to that domain is found on the list. Accordingly, the domain having the lowest count at the end of the scan is that which is currently undergoing the least activity. The scan should be limited to the top portion of the list, because this is where activity occurs. It might seem that the lower end of the list could be most profitably looked at because this is where the old and invalid frames will congregate. However, if but a single active frame is in the same domain with a large number of old frames it would invalidated as well by the domain reallocation operation which would be undesirable. The second alternative is to count the stages to each domain rather than examine the least recently used frame list. This would again provide an indication of relative activity.

Thus, it will be clear that the domain switching scheme according to the invention has two portions: the decision making portion, which decides whether or not it is desirable to switch domains from one class to another, and the actual switching process, in which it is determined which domain is to be switched. In a particularly preferred embodiment, the decision making process is table driven in order to save execution time; that is to say, when the device is turned on, a look-up table indicating the ideal allocations of the domains for varying relative activity levels is generated and stored so that subsequent reallocation decisions can then simply be look-up processes not requiring an elaborate computation. Such a domain switch decision table usable in a preferred embodiment of the invention where the domain switching occurs between two device types is shown in FIG. 3.

The table is built by making all the possible domain switching decisions at post-IMPL time (that is, upon initial loading of the device) thus saving computation time when the decision is actually made. The decision thus becomes a simple table look up based on, e.g. the number of domains allocated to device type A and "A domain use"--a number representing actual staging operations from devices of the type A--at a particular time. In the embodiment of FIG. 3, the table shows numbers between zero and 15 (i.e., F, in hexadecimal) along the horizontal axis indicative of the relative number of stages to A device type domains. Each column thus represents the A-use figure. The table is built one horizontal "row" at a time with each row representing the number of domains allocated to device A at a given time. For each row, that is, for each number of domains allocated to A-type devices, there is thus an ideal target A-use. The table thus provides a correlation between the number of domains assigned to device A and the relative number of A stages in a given time. If the correspondence is not found then it is evident that domain switching should be performed. Target A use is calculated based on the assumption that the ideal cache partitioning is achieved when ##EQU1## One can usefully define a term called "Target A Stages", this being the number of stages which would occur to domains assigned to device type A if the staging activity was in balance with cache allocation. This will lead to a target A frame use which can be compared with the actual A frame use to determine how close the actual stage activity compares with what would be ideal. Thus, in the table of FIG. 3 each of the horizontal rows represents a given number of domains allocated to a device. The target A use is then identical for that row. One of three indications is then filled in for each position of that row according to the actual use. A small a, indicating that a B to A domain reallocation is pending, i.e. desirable, is filled in if actual A use is greater than target A use, thus indicating that switching of a frame from B to A is an acceptable action if a request for an A frame assignment cannot be satisfied based on current allocation. Similarly, a small b indicating a pending switch from A to B is filled in if A use is less than target A use. Otherwise an n, indicating a "no switching required" condition is filled in.

As will be observed from Table III, several, typically 3, of the columns for each row have n's which indicate that no switch is pending. This is desirable in order to prevent unnecessary domain switching due to statistical variation in the number of stages per domain type over a short period of time which can be expected to occur, while not indicating that domain reallocation is called for. The plurality of n's in the columns thus provide a quasi-damping effect on the domain switching according to the invention.

As noted above, domain switching is initiated based on demand for a frame and on the value of a "domain switch pending" flag generated in the statistics keeping process. Thereafter, if data is to be cached and no frame for the proper device type is free, and a domain switch operation is pending which would provide a frame of the proper device type, the domain switching operation is initiated.

The first task performed in the domain switching operation is to decide which domain would be the best to switch. The least recently used list of the "from" device type is scanned to determine which domain has the fewest frames on the top half of the least recently used list. If this domain in turn has no frames marked "In use" or "Stage in Progress", it is presumed to be the best one to switch. Otherwise, the domain with the next fewest frames in the top of the least recently used list is chosen. Once a domain is chosen for switching, the domain is converted to the new device type. All valid frames belonging to the domain are deallocated. All frames in the domain are removed from the "from" device type least recently used list. The number of new frames that will fit into the new domain are linked onto the bottom of the new device type least recently used list. Finally, new pointers to the cache array are assigned for each frame according to the frame size of the new device type. Thus, when the domain conversion routine is finished, the new frames are at the bottom of the new device type least recently used frame list, and are marked free.

FIG. 4 shows a schematic view of the decision-making process according to the invention. As discussed above, at initial program load, "IMPL time", the table is written with all the possible decisions as at 32, the table then appearing as in FIG. 3. This information is then available to domain switch determination module 34. This decision making process is also supplied with domain switch statistics as at 36 which indicate whether domain usage is presently corresponding to domain allocation. Therefore, upon a stage request being validated as at 38 by the cache manager 24, (FIG. 1) as discussed in copending application Ser. No. 325,350 filed Nov. 27, 1981, the domain switch statistics may be examined to determine which, domain if any, should be switched. Typically, the operation is performed every, e.g., 1000 stages. If an imbalance is detected by accessing the table of FIG. 3 as at 32, a flag is set indicating that a switch is pending as at 40. Thereafter, upon a request that a frame be allocated to a data record at 42, if no free frame is available, and if an appropriate switch is pending, the least recently used domain is switched as described above as indicated at 44. The domain thus converted at 46 has now been allocated to the new device type, e.g. type A, and the table is updated to reflect this fact at 48, while the directory of frames available is likewise updated as at 50.

Attached hereto as Appendix A and incorporated herein by reference is a document entitled "Module: Domain Switching" which is a detailed description of methods used to perform the apparatus of the invention. This document is largely written in a form of convenient shorthand language referred to as pseudocode, the grammar of which is explained in a document also attached hereto and incorporated by reference, marked Appendix A-1, entitled "Chinook Pseudo Code Conventions". Those skilled in the art supplied with the domain switching document of Exhibit A together with the interpretative pseudocode convention document would have no difficulty implementing the methods of the invention.

Those skilled in the art will recognize that there has been described a domain partitioning system for dividing up a solid-state memory which fulfills the needs of the art and objects of the invention mentioned above. Specifically, use of domains having a size into which commonly accessed quantities of data, e.g., tracks, fit conveniently provides efficient storage of data without fragmentation as old records are overwritten by new data. More particularly, the provision of an adaptive domain switching scheme allows efficient utilization of memory to be achieved even when relative domain utilization changes from time to time. Furthermore, it will be appreciated that the method of the invention has applicability beyond the two device embodiment specifically described, and that therefore the description of the invention given above should not be considered as a limitation on its scope but only as exemplary thereof; the scope of the invention is more properly limited by the following claims. ##SPC1## ##SPC2##