Method and apparatus for estimating a percentile for a value

[0002] 1. Field of the Invention

[0003] The present invention relates to the field computation of statistics for populations and more specifically to computation of percentiles of samples of a given population which is changing over time.

[0004] 2. Description of the Related Art

[0005] It is often interesting to be able to express, for a given sample of a population, its position in the population relative to other samples. Often this is expressed as a percentile. For example, if height measurements were taken of all the children in a particular grade school, the student who had as many children taller than him as shorter than him could be said to be in the 50^thpercentile of the overall population. The population of students in a grade school is a relatively trivial example at least because the population is relatively static over time. If the population was rapidly changing (and, thus, the percentile ranking for each student changing), calculating the percentile for any particular sample becomes non-trivial especially as the rate of change approaches the time it takes to compute the percentile. Typically, computing the percentile of a sample requires obtaining a list of all samples and then sorting the list. This process, especially the sorting of large sets of samples, is compute intensive.

[0006] Thus, what is needed is an efficient method and apparatus for computing percentiles of samples of populations which change over time.

[0007] A method and apparatus for computing the value at a given percentile and the percentile of a given value is described. In the described system, computing efficiencies are achieved by storing received values in “buckets” and performing percentile calculations based on values in the buckets.

[0008] FIG. 1 is a diagram illustrating a conceptual level diagram of a system implementing an embodiment of the present invention.

[0009] FIG. 2 is a diagram illustrating a conceptual level diagram of a system for maintaining distributions implementing an embodiment of the present invention.

[0010] FIG. 3A is a flow diagram illustrating a method for seeding and computing “bucket” ranges as may be utilized by an embodiment of the present invention.

[0011] FIG. 3B is a flow diagram illustrating a method for updating the “buckets” with new values as may be utilized by an embodiment of the present invention.

[0012] FIG. 3C is a flow diagram illustrating a method for redistribution of buckets as may be utilized by an embodiment of the present invention.

[0013] FIG. 3D is a flow diagram illustrating a method of merging distributions as may be utilized by an embodiment of the present invention.

[0014] FIG. 4A is a diagram illustrating a conceptual level diagram of processes for computing percentiles and values at given percentiles.

[0015] FIG. 4B is a flow diagram illustrating a method for finding values at a given percentile.

[0016] FIG. 4C is a flow diagram illustrating a method for determining the percentile for a given value.

[0017] For ease of reference, it might be pointed out that reference numerals in all of the accompanying drawings typically are in the form “drawing number” followed by two digits, xx; for example, reference numerals on FIG. 1 may be numbered 1xx; on FIG. 3, reference numerals may be numbered 3xx. In certain cases, a reference numeral may be introduced on one drawing and the same reference numeral may be utilized on other drawings to refer to the same item.

[0018] The present invention provides a computationally efficient method and apparatus for computing the percentile for a given sample in a population where the population changes over time. The invention may be particularly useful in environments where the population is changing relatively rapidly over time. For example, in data networking applications, it has been observed that round trip time measurements and server latency measurements do not follow conventional statistical distributions (such as Poisson or Gaussian). As a result, mean latency is generally not a good indicator of the typical behavior of a component in a network. It has been observed that a more appropriate measure of typical behavior is the median. And, the percentile of a given sample can be used to provide the probability that an observation of the sample size or less will be seen in the operation of the network. However, in network monitoring systems, data collection may be an ongoing process and, thus, the population is rapidly changing as new data is collected. As was discussed, in one prior art approach, a list of all samples may be maintained and sorted. When a new sample is observed, it is inserted into the appropriate location in the list (or the list is resorted). This is clearly computationally intensive for a rapidly changing population such as the gathering of statistics in a network monitoring system.

[0019] The present invention proposes maintaining the samples in a set of groups of “buckets”. For example, a set of buckets of heights of people. Each bucket maintains a counter indicating the number of samples within that bucket.

[0020] The percentile ranking for any given sample can then be approximated by observing the count in the various buckets using the methods which will be described below.

[0021] FIG. 1 is a high level conceptual diagram of certain aspects of the present invention. The present invention maintains a distribution database 103 which maintains the counts for each “bucket”. A process 101 initializes the buckets, receives samples and increments the bucket counters as appropriate for the samples received. This process will be described in greater detail below. A second process 102 computes percentiles for any given value or alternatively provides a value for any requested percentile.

[0022] FIG. 2 is a high level conceptual diagram further illustrating the process of maintaining distributions (block 101 of FIG. 1). There are three functions performed in maintaining distributions. The distribution database 103 is initialized by process 201. This will be described in greater detail in connection with FIG. 3A. The distribution database is updated as new samples are received by process 202. This will be described in greater detail in connection with FIG. 3B. Finally, the distribution database may be redistributed under certain condition or may be merged with other distribution databases by process 204. This will be described in greater detail by FIGS. 3C and 3D.

[0023] Turning now to FIG. 3A, a process for initialization of the distribution database is described. An initial value V is received for the initialization process, block 301. In the exemplary application in which student height measurements are distributed, the initial value V may be the height of the first student measured. In another exemplary application, in which network round trip time is measured and distributed, the initial value may be the first reported round trip time. Assume for purposes of our student height example that the first student is measured at 50 inches. Therefore V=50.

[0024] A value X is then computed, block 302, where:

[0025] X=V/N

[0026] Where N is the number of buckets to be used in the process, block 301. The number of buckets may be user selectable or may be set dependent on the application. An increased number of buckets leads to increased accuracy of the percentile estimates. However, it also increases the computational requirements. Assume for purposes of our example that N is 10. Thus,

[0027] X=50/10=5

[0028] The value X is then used to compute a value 2^m, block 303, such that:

[0029] 2^m−1<X≦2^m

[0030] Thus, in our example, 2^mis 8. It should be noted that using 2^mrather than X in the rest of the process leads to certain computational efficiencies which will be discussed in greater detail below. However, in certain embodiments, X may be used directly in place of 2^m.

[0031] An initial range for the buckets is then computed as:

[0032] Initial Range=MinRto ((N*2^m)+MinR−1)

[0033] where MinR is the minimum range (which is typically user selectable or set dependent on the application), block 304. MinR is also selected to be the lower boundary of a range—e.g., in our example MinR might be 8 inches. In the described embodiment, it has been found that it is advantageous to set MinR to a value which is a power of 2.

[0034] The initial bucket size (BSIZE) is set to 2^m, block 305. Thus, the initial range is from 8 inches to 87 inches and the 10 buckets will have ranges as shown in Table I:

[0035] The initial value (50) is then placed in bucket 6.

[0036] Turning now to FIG. 3B, a process for updating the distribution database 103 with new values as they are received is described. Assume that a new value 42is received. First, a check is done to determine if the new value is within the initial range, block 311. In this case, the value is well within the initial range and a bucket (B_i) is identified corresponding to the value, block 312. In the example, B_iis bucket 5. The bucket value (BVALUE_i) for B_iis incremented, block 313.

[0037] Assume that another new value is received which is outside the initial range. For example, assume that a value of 92is received. New buckets are then computed and the values in the old buckets are redistributed to the new buckets as will now be described. First, the new value is used as the value V and X is recomputed as X=92/10=9.2, block 314. A new range and bucket sizes are then computed, block 315, as was described above (where 2^mis now 16). The new range is 8 to 167. The buckets ranges are as shown in Table II.

[0038] The values of the old buckets are then redistributed to the new buckets as is described in greater detail with reference to FIG. 3C. First, a value i is initialized to 1, block 321. Then for old bucket B_i, the corresponding new bucket BNEW_zis identified, block 322. In our example, old bucket 1 corresponds to new bucket 1 (because old bucket 1's range of 8 to 15 fits within new bucket 1s range of 8 to 23). Old bucket 2 corresponds to new bucket 1, old bucket 3 corresponds to new bucket 2, etc. It will be appreciated that one advantage of using 2^mrather than X, to compute bucket ranges is that, all old bucket ranges fit within a single new bucket during the distribution process. Values from old bucket B_Iare moved to new bucket BNEW_z.

[0039] The BVALUE of new bucket BNEW_zis then incremented by the value of old bucket B_I, block 323 and i is incremented, block 324 and the process is repeated until i is greater than N (the number of buckets), block 325.

[0040] It may be useful to be able to merge distributions. For example, in some applications such as network monitoring, distributions may be maintained to cover a certain time span. For example, a distribution may be maintained for each hour and it may be desirable to merge the 24 hourly distributions in order to provide a daily distribution.

[0041] In order to merge distributions, initially one distribution database 103 is designated as a source distribution database and an other is designated as a target distribution database. A check is made to determine if the minimum value (MinR), range and number of buckets in the source and the target are equal, block 331.

[0042] If they are not equal, a check is made of whether it is necessary to redistribute the target distribution database, block 335. The target distribution database must be redistributed if its range does not at least span the full range of the source distribution database. The process for redistributing the target database are described in FIG. 3B, blocks 314-316 with V being equal to the highest end point in the last bucket in the source distribution.

[0043] After completing the redistribution (if necessary), bucket, the values from each of the source buckets are moved to the corresponding target buckets, block 336. The corresponding target bucket is the target bucket which covers the range of the source bucket (note that at this stage of the process, source bucket ranges will always fit within exactly one target bucket).

[0044] The BVALUE for the target buckets is also incremented by the BVALUE for the source bucket, block 337.

[0045] If the range and number of buckets for the source and distribution were equal, the process is simplified because the source bucket B_icorresponds to target bucket B_i.

[0046] If there are more source distributions to be merged (e.g., 23 more hours in the day), the process is repeated for each source distribution, block 338.

[0047] What has been described thus far is how distributions are maintained. Two questions of interest may be asked:

[0048] 1) For a given percentile (P), what is the value? (e.g., what is the height of someone at the 50^thpercentile of the population?)

[0049] 2) For a given value (J), what is the percentile? (e.g., if someone is 72 inches tall, what percentile of the population is the person in?)

[0050] As is illustrated by FIG. 4A, a process for answering the first question is described in connection with FIG. 4B and a process for answering the second question is described in connection with FIG. 4C.

[0051] Turning first to FIG. 4B, the total number of samples T in the distribution is determined, block 411. In certain embodiments, a total sample count is simply kept as samples are received by incrementing and saving a counter. In an alternative embodiment, the total sample count may be computed when queries are made on the distribution database by accumulating the counts for each of the individual buckets.

[0052] The bucket in which the desired percentile is found is determined, block 412, by accumulating the counts from each bucket (BVALUE_i), starting at bucket 1 until P is greater than or equal to the accumulated count (A_i). In other words:

[0053] A_i−1/T<P≦A_i/T

[0054] The desired value is within bucket B_i. In the described embodiment, linear interpolation, block 413, is then used to compute a particular value within the bucket. For example, if P=48^thpercentile and bucket B_Iwas determined to include values from the 43^ndpercentile to the 52^ndpercentile (i.e., A_i−1=43 and A_i=52), the value that is chosen is the mid-value in the bucket. As will be appreciated, this method provides an estimation of the percentile value and not necessarily the exact value. However, for purposes of many calculations, an estimate is sufficient and worth the reduction in computing resources required for calculation by using the described method.

[0055] Turning now to the second question and with reference to FIG. 4C, initially, the bucket B_iwhich includes the desired value J is identified, block 421.

[0056] Next, the lower percentile boundary A_i/T for the bucket B_Iis calculated as was described in connection with FIG. 4B, block 422.

[0057] Finally, linear interpolation is used to determine the position in the bucket B_Iof the desired value J and the percentile value is calculated as:

[0058] A_i/T+linearly interpolated value.

[0059] For example, if A_i/T is 43and the linearly interpolated value shows that the value J falls at the mid-point of the bucket B_Iand if bucket B_Icovers values from the 43^rdto 52^ndpercentiles, then the value J represents the 48^thpercentile.

[0060] Thus, what has been described is a method and apparatus for estimating a percentile for a value and a value for a percentile.