Method for Selecting Computer Storage Devices for Computer Applications

1. Field of the Invention

The present invention relates to a method of rating a storage system by reducing a set of attributes associated with the storage system and an application to a single quantified number.

2. Description Of The Prior Art

In an analysis of performance related to a particular system or algorithm, various techniques of benchmarking are used to provide the evaluation. These various prior art benchmarking techniques are then used to provide a comparative analysis of related devices, systems or software by measuring key attributes of interest.

For example, a benchmarking technique for analyzing the floating-point performance is known as the Whetstone Benchmark. The Whetstone Benchmark, described in IBM Technical Disclosure Bulletin, Volume 28, No. 12 (see reference.1, listed below) is actually a mix of the 10 Whetstone kernels wherein each kernel represents a different type of numerical operation. The Whetstone Benchmark is used as a standard by which the floating-point performance of the different systems can be compared.

A second example is a technique for evaluating advanced time sharing systems. This technique, disclosed in IBM Technical Disclosure Bulletin, Volume 14, No. 5 (see reference 2) describes an analytical model which correlates various parameters, such as resource utilizations, mean queue lengths, response times, and throughput rates for closed systems having a finite quantity of requestors competing for the system's resources. A model is provided for each system and variables assigned to each of the system parameters are analyzed to calculate a benchmark value. Then the benchmark values of each of the competing systems are compared to evaluate the performance of each time sharing system.

In some instances, a single figure is determined from a calculation involving a multitude of parameters. Two well-known prior art techniques to arrive at a single performance figure are the Gibson Mix and the Dhrystone Benchmark. Both of these are concerned with measurement of the speed of a central processor unit (CPU). The Dhrystone Benchmark measures the speed of executing a given number of program statements on a CPU (see reference 3). The Gibson Mix refers to the mix of instructions used by a computer while executing scientific programs. The Gibson Mix is used as a workload model for a CPU (see references 4-7). The Gibson Mix provides a weighted sum as the mix of a set of instructions. Although various benchmarking techniques are known in the prior art, most of these techniques apply to CPUs or to other processing systems.

Storage systems, such as disk drives, are functionally different from processors and require a different set of parameters to benchmark attributes relating to the performance of the storage systems In the instance where a number of storage media or application alternatives are available, it would be advantageous to derive a method of comparing the performance of the various storage systems and applications. Further, if the parameters associated with the various storage systems and applications can be benchmarked to a single figure, similar to the Dhrystone and Gibson Mix benchmark figures described above, then the benchmark values can be compared to evaluate the various alternatives for the purpose of choosing the optimal storage system for each specific application or choosing the optimal file system for a specific storage system.

The present invention describes a method for rating storage systems by correlating a set of parameters associated with attributes of the storage system and computing a single figure of merit as a product of selected parameter factors. To evaluate the performance of a given storage system, such as a hard disk, a set of parameters is established and values assigned to these parameters. The establishment of the parameters also considers the associated file system and the application requirement of the storage system.

Once the parameters are established, values are assigned to the parameters depending on the attributes of the storage system in its environment. Then, using these parameter values, performance factors are calculated to derive the figure of merit. These factors measure various performance abilities such as addressability, growth reserve, efficiency and recoverability of null space relating to the storage system. Then these factors are normalized wherein the minimum accepted level is given a value of unity (1.0). Then the performance measures which are to be used in the calculation of the figure of merit are multiplied to provide the final product. This product represents a relative indication of the overall performance of the storage system. Because the factors are normalized to a value of unity for minimum acceptance, the figure of merit also is normalized to unity for minimum acceptance. If a factor cannot meet a mandatory requirement, then a default value of zero is used to represent the factor. In such a case, the resulting product of zero disqualifies the system.

Once the figure of merit is calculated for a given storage system, the same calculations can be applied to other systems. Then, the various figures of merit can be ranked to provide an ordinal ranking of the various storage systems. Because of the ordinal ranking, the figure of merit having the highest numerical value is chosen as the optimal product for the specific application.

In addition, derivatives can be calculated for each of the factors to determine which parameter element has the greatest impact in changing the overall figure of merit value.

The foregoing and other objects, aspects and advantages of the invention will be better understood from the following detailed description of the preferred embodiment of the invention with reference to the accompanying drawing, in which

FIG. 1 is a flow diagram showing the steps of the algorithm for deriving the figure of merit of the present invention.

FIG. 2 is a block diagram showing a computer system in conjunction with which the claimed invention may be implemented.

The present invention describes a method of rating a storage system by correlating a set of parameters and computing a single figure of merit for the storage system. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be obvious to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods have not been described in detail in order not to unnecessarily obscure the present invention.

The present invention provides a method for assigning a single numerical quantity, known as a "figure of merit", to any system composed of a disk storage device, the file system which accesses the device, and the dedicated application, including the organization and the extent of the application data as stored on the device.

The single numerical quantity provides for an easy ordinal ranking of, and selection among, such disk storage devices and associated file systems. The convenience results from the reduction of a large number of factors to a single numerical quantity per system.

Parameters are established to measure a number of attributes associated with each storage device, its file system, and its data, as organized on the disk. Then, for each storage system, the parameters are reduced to the single figure of merit quantity per system.

The reduction of the various, multiple parameters to a single quantity allows the selection of the most optimal storage system (disk, file system, and data organization) from a set of alternative storage devices and file systems.

As implemented in the preferred embodiment, the method for rating the storage system is performed by an algorithm. The algorithm receives input values for the various parameters and computes the figure of merit for each of the storage systems. Further, in the example below, the derivation of the figure of merit is discussed in relation to the selection of a hard disk. However, the example is for illustrative purposes only and is not to be interpreted to limit the present invention. It is to be appreciated that the method for rating the performance of hard disks described below using the figure of merit of the present invention can be applied to most storage devices This rating technique addresses filler space required to reach disk boundaries, as well as reserve space on the disk. Further, it not only addresses adequacy of space, but also considers failure to re-utilize storage that no longer contains valid data. An emphasis is placed on disk capacity and optimal usage of that capacity for the given application requirement which uses a given associated file system. One such application requirement is a need for a specific amount of bytes of data.

The operation and use of a hard disk is well-known in the prior art, and therefore, elaboration on hard disk devices is not provided here. The algorithm of the preferred embodiment considers the following attributes in calculating the disk capacity figure of merit for each of the hard disk devices under consideration. Such attributes include the size of the file space required by the software, the amount of physical (formatted) capacity of the hard disk, the address space recognized by the software and the firmware, the assumed number of bad tracks on the disk (worst case distribution of bad tracks is assumed if there are no alternate track schemes), whether there is an alternate track scheme, and the size of the allocation unit (such as floppy, track or sector).

In the method of determining the disk capacity figure of merit of the preferred embodiment, certain notions related to the hard disk are to be presumed The hard disk is comprised of an integral number of tracks, wherein each formatted track supports an integral number of sectors. The exact number of sectors per track is not essential to the practice of the present invention, however, the number of sectors per track is to be fixed. For each sector, an integral number of bytes is assigned, such that each sector has the same number of bytes. The disk is capable of storing files, which may also be called volumes. These files are comprised of a number of segments, which sometimes are called extents. Typically, extents begin and terminate on track boundaries and no logical gaps exist in the track numbers within an extent. However, if an alternate track scheme is used for remapping the bad tracks, then physical gaps can exist within an extent. These physical gaps will normally not be noticeable to the application software and at most will only impact arm movement of the disk drive. Therefore, in a system without bad track remapping, extents must be composed of contiguous tracks.

For definition purpose, a bad track is any track that has at least one unreadable or one unwriteable byte. That is, if any portion of a track is unreadable or unwriteable, then the entire track is presumed unusable Further, an "alternate track scheme" is defined to be any scheme that remaps bad tracks so that the good tracks appear to the file system software as a consecutive sequence of tracks without gaps. Also, with the method of the preferred embodiment, a worst case analysis is used to determine the pattern of the bad tracks. If the number of addressable tracks is less than the physical capacity of the hard disk, then all bad tracks are assumed to fall within the addressable space. This analysis for bad track distribution is rather conservative because it assumes that bad tracks appear in the worst possible pattern. To maintain consistency, such assumptions are made uniformly to each system analyzed. In most instances, file sizes are multiples of an allocation unit, which originally was exactly the size of a floppy disk (six hundred and eight, 512 byte sectors), and an assumption is made that every volume extent is a size which is an integral multiple of the allocation unit plus the additional sectors (if necessary) to complete the volume on a track boundary.

Taking these various attributes for each hard disk, parameters are defined from these various attributes. It is to be appreciated that other hard disk attributes can be considered, such as arm movement, rotational speed, latency (response time), as well as others, and they can be readily incorporated into the calculation for the figure of merit of the present invention.

For clarity, the notion of a "floppy disk" is elaborated on before proceeding to the elaboration of the parameters and units of measurement used to derive the figure of merit.

A floppy disk, is a single, usually pliable, medium of storage, which resembles a soft phonograph record of small dimensions encased in a square envelope. Floppy disks are currently used on most personal computers (PC). The floppy disk to be mentioned in the elaboration of the preferred embodiment has a storage capacity of 608 sectors of 512 bytes each, or 608.times.512=311,296 bytes per floppy disk.

A "floppy", as used in the preferred embodiment, is simply an allocation unit of 311,296 byte size. Its relation with the physical floppy used in the PC is historical only.

The floppy disk is distinct from the disk systems for which the figure of merit is to be computed. The floppy disk is not to be analyzed; its interest lies in the fact that many hard disk systems are loaded from data that is transported via sets of floppy disks (much as data can be transported on reels of tape.) Since floppy disks have a precise capacity in bytes, many file systems assign space in terms of floppy disk units. In particular, a hard disk of one common variety, with 16 sectors per track, will require 38 consecutive tracks to accommodate the data on one floppy. Current hard disks are formatted to provide 17 sectors per track; 36 consecutive tracks are required to accommodate the data on one floppy with 4 sectors left unutilized on the 36th track. The method of the preferred embodiment provides for file space to be assigned in multiples of allocation units such as floppy disks. The preferred embodiment will also reflect the consequences of using smaller units of file allocation, such as multiples of tracks or sectors. These smaller units will generally increase the computed figure of merit since they enable an easier fitting of data between defective tracks.

The various parameters used in the calculation of the figure of merit of the preferred embodiment, as well as the values for the various parameters are provided below:

Once the parameters are set and values assigned to these parameters, then calculations can be made to determine the disk capacity figure of merit. The disk capacity figure of merit of the present invention is derived by computing a product of a number of factors. The factors are derived from the parameters and can also be regarded as parameters. However, because they are used to compute a product, these parameters are referred to as performance factors for ease of explanation. Because the figure of merit is a product of the factors, it is possible that the factors can provide different weights and provide significantly biased results In order to give equal weight to each of the factors, each factor is normalized to a value of 1.0. The normalized value of unity (1.0) designates a value for a minimal acceptance level for each factor. That is, it is desirable to have a disk configuration in which each of the factors has a rating of 1.0 or higher. Further, because the figure of merit is derived as the product of the factor terms, the figure of merit is also normalized to unity for a minimally acceptable value. That is, a figure of merit for a minimum acceptance has a value of 1.0. Conversely, a figure of merit less than unity is not acceptable.

By comparing the figures of merit for the different storage systems, an assessment can be made in ranking the systems. The figures of merit are arranged in an ordinal ranking for comparison purposes. For example, a figure of merit of 1.5 provides for a more desirable device than one which has a rating of 1.0. It is to be appreciated that this ordinal ranking of the various figures of merit is to determine the system having the highest value It is possible that in some instances a factor may have a value of less than unity, but still result in a figure of merit which is greater than unity or even to have the highest figure of merit. The selection of the optimum hard disk depends on the highest figure of merit and not necessarily the values of each individual performance factor. Where a factor may have a value of less than unity, other factors can be sufficiently above unity to "counterbalance" the low value factor, such that the overall figure of merit is above unity and, therefore, acceptable. It is also to be noted that the values for the figures of merit provide for an ordinal ranking only and does not provide an absolute quantified comparison determining the extent of superiority. For example, a figure of merit of 1.5 is not necessarily 50% better than a system having a figure of merit of 1.0.

An advantage of using a product calculation in determining a figure of merit is that a disqualification of a system for failure to meet a mandatory requirement will result in a value of 0. If a factor cannot meet a mandatory requirement, then a default value of 0 is used to represent the factor, resulting in a value of 0 for the figure of merit and, thereby, disqualifying the system from consideration. Further, the algorithm for calculating the disk capacity figure of merit provides high flexibility in its application. Factors can be readily individualized and/or modified with respect to acceptability targets. For example, percentage values for the measurement of the reserve of free disk space available for growth can be readily altered. Further, additional factors can be readily defined and inserted in the calculation of the product. As long as each of the factors is normalized to unity, the method determining the product remains unchanged. Also, if individual factors are piecewise differentiable, then the figure of merit is also piecewise differentiable. By taking the partial derivatives, it is possible to identify the factors, or parameters, for which small changes would yield the most improvement in the figure of merit. In the preferred embodiment, three factors are used to determine the disk capacity figure of merit. Measure of addressability (A), measure of reserve for growth (G) and efficiency (E) are the three factors included in the algorithm of the preferred embodiment. The definition and computation of the three factors A, G and E are described below.

Proportion of formatted good track that is utilizable. Must be computed with respect to all good tracks. To be counted, tracks must be assignable to a volume extent. It must be possible to assign all tracks simultaneously, that is, the address space must be at least as large as the number of tracks that are counted.

Range: 0.0-1.0

Target: 1.0

Computed as: U/D

Proportion of total disk space that is not assigned to any volume and is available for additional use. Let k=the proportion of reserve space required, e.g. 0.2 (or 20%)

Range: 0.0-1.0

Target: 1.0

Computed as: (1/k) * (U-F)/D for k>0 and F.ltoreq.U

Computed as: 1.0 for k=0 and F.ltoreq.U

Computed as: 0.0 for F>U.

The coefficient, (1/k) is chosen in order to normalize the target to unity (1.0).

Note that k=0 signifies that a reserve for growth is not a requirement. In general, setting a factor to unity (1.0) is equivalent to ignoring the factor.

Proportion of bytes in assigned volume space that contains information.

Range: 0.0-1.0

Target: 1.0 (100%)

Computed as: (F-S)/F

Although the example of the preferred embodiment calculates the product of three factors A, G and E, other factors can be incorporated in the calculation of the figure of merit as was described in the description above. Another such factor is the recoverability of null space (R.sub.N).

Proportion of Nulled space that can be reutilized without running compression utilities.

Range:

0.1-1.0

Target:

1.0

Computed as:

1.0 if no file is appended until all its nulled space has been re-utilized.

0.0 if null space is never reutilized.

0.0<R.sub.x <1.0 where there is some strategy for recovering null space and R.sub.x is the minimum reutilization under worst case conditions.

Set R.sub.N =Max(0.9, R.sub.x) The value 0.9 sets a floor on the value of the factor for null space recoverability, and thus a cap on the penalty for a poor null space recovery scheme. The value of 0.9 is a sample value. A value of 1.0 for the cap essentially states that null space recovery is not an issue.

Note that a value of 0.0 for the cap is equivalent to defining R.sub.N =R.sub.X. In this case, complete inability to recover null space (without running compression utilities) would automatically disqualify a system. That is R.sub.N, and hence, the Figure of Merit would be set to zero.

A typical value for the cap is 1--(N.sub.b /F) where N.sub.b denotes the maximum amount of null space (in bytes) that is likely to accumulate between successive executions of compression utilities. Here the penalty for inability to recover nulled records is capped by the maximum expected amount of null space. Poor performance here can be compensated by a larger reserve for growth.

It is to be appreciated that other factors such as arm movement, rotational speed and latency can be calculated and included in the calculation of the disk capacity figure of merit. FIG. 1 shows a basic flow diagram for the calculation of the disk capacity figure of merit as described above.

The product of the three performance factors of addressability (A), growth reserve (G) and efficiency (E) determine a single numerical figure for the disk capacity figure of merit. A figure of merit having a value greater than or equal to unity (1.0) is acceptable, unity being the minimum acceptable value. A figure of merit less than unity represents less than the minimum level of acceptance.

Once the figure of merit has been calculated for each of the storage systems under review, then the system having the highest numerical figure of merit value is chosen as the optimal product for the specific application Then, if an improvement to the figure of merit is desired, calculations can be made to determine the impact of changing various parameters which affect the calculation of the final figure of merit. Derivatives provide the rate of change of various parameter values and, therefore, can evaluate the impact of changing a given parameter for causing a corresponding change in the figure of merit. The impact of changes in physical disk size, file system, and/or space utilization by software can be estimated from the partial derivatives listed below:

if U fixed.

if U=D.

if S.ltoreq.F.ltoreq.U.ltoreq.D.

if S=t*F.

if S fixed.

Positive only when (F/U<(S/F)

The above described method for determining a figure of merit is illustrated in the process flow shown in FIG. 1 In this illustration, the selection process is used to rank various disk drives with respect to a given application and file system. It is clear that the application and/or file system could also be varied, as alternate embodiments of this invention. In step 1, a storage device is selected for rating. In step 2, parameters are established for the specific application of the storage device Next, in step 3, values are assigned to the various parameters established in step 2. These values are based on the specific storage device selected In step 4, various performance factors are selected for the figure of merit calculation. The selected performance factors are derived from the parameters and their values of steps 2 and 3.

Then, if necessary, the performance factors are normalized in step 5 so that the normalized value of 1.0 designates a value for minimal acceptance. In step 6, the normalized performance factors are multiplied to derive a product, which is the figure of merit value of the storage device. A figure of merit of 1.0 being indicative of minimal acceptance and a value of 0 being a disqualification of the storage device.

Finally in step 7, the figure of merit for this storage device is compared to other figures of merit, which were derived equivalently using the same steps for other storage devices. The storage device having the highest numerical value of at least 1.0 for the given application is selected as having the best performance.

A PASCAL program that calculates the figure of merit is provided below. Parameters, which are comprised of variables (VAR) and constants (CONST), are established and defined. Then, calculations are made to determine the performance factors A, G and E. This calculation also accounts for an alternate track scheme. The figure of merit calculation (Merit:=A*G*E) is performed at the end of the calculation phase. Finally in the last phase, the results are provided for display.

Thus, by determining various parameters attributable to storage devices under analysis, values can be assigned to these parameters and by using these parameter values, factors can be calculated These factors are then used to produce a product representing the figure of merit of a given storage device.

The figure of merit of each storage device can be compared, and the device having the highest figure of merit value is selected as the optimum system for the specific use and for the specific application chosen. As requirement constraints change, the factors can take into account other parameters to drive a different figure of merit value.

The normalized product method offers benefits of convenience and flexibility beyond the weighted sum method. In contrast to the weighted sum method (such as used in computing the Gibson Mix in the prior art), the normalized product method allows for easy modification of a computed figure of merit through the introduction of, and multiplication by, additional performance factors. In the weighted sum method, it would generally be necessary to recompute the previously defined weights and to resume the previously defined terms.

Further, the algorithm for determining the figure of merit of the present invention is implemented on a computer. The various parameters, parameter values, and algorithm are stored, such as in a memory coupled to the processor. Calculations are performed in the processor, and all parameters, values and calculated results are generated in an output format including video screen and printed spreadsheets.

The aforementioned method can be used in a repetitive fashion to cyclically process a variety of storage devices and applications in an effort to find the optimal match.

FIG. 2 is a block diagram representation of a computer system which may be used to implement the present invention. In particular, computer 11 includes a processor, memory and operating system capable of executing the Pascal program set forth above for calculating the figure of merit. Coupled to the computer are a keyboard 13 for a user to enter commands and a monitor 15 to display the results of calculations performed by computer 11. Storage system 17a, 17b and 17c are hard disc drives or storage device with a file system and data. Each of the elements shown in FIG. 2 are readily available, off the shelf components, the only requirement being that the computer system must be capable of compiling and executing the Pascal program or comparable program written in another programming language.

While the invention has been described in terms of a preferred embodiment in a specific environment, those skilled in the art will recognize that the invention can be practiced, with modification, in other and different environments within the spirit and scope of the appended claims.