Robust codebooks for vector quantization

[0002] 1. Field

[0003] This disclosure relates to vector quantization for the compression of image, video, speech, audio, or other data types, more particularly to methods for developing codebooks for vector quantization.

[0004] 2. Background

[0005] Data compression techniques attempt to reduce the amount of information necessary to reconstruct the original entity, while still providing enough information to reconstruct the original entity. For example, image compression compresses the amount of data necessary to reconstruct an original image. Speech compression compresses the amount of data needed to compress speech. These of course are examples as compression can be applied to any kind of data.

[0006] Vector quantization is a lossy compression technique. Vector Quantization technique (or VQ) partitions the entire data space into a series of representative regions. Within each region, approximations are designated, referred to as codevectors. The regions and codevectors are developed through a training procedure, using typical data sets, such as typical speech patterns or typical images. A typical training procedure was originally proposed in 1980 by Linde, Buzo and Gray and is therefore sometimes referred to as LBG algorithm.

[0007] LBG algorithm uses relative occurrences of the patterns in the training images. Typically a large number of training data sets are used for training. Generally, this approach works well for typical patterns. However, rare data combinations may occur that are completely missed by the training set. The resulting codebook will perform very badly when those data combinations occur.

[0008] One solution is an approach referred to as Lattice VQ. This approach mathematically partitions the data space into equal regions and includes the rare data patterns. This will have a reasonable performance if the source is also uniformly distributed in the vector space. However, it performs very poorly if the source has a skewed distribution.

[0009] Therefore, an approach is needed that performs well with typical data sets, and also with the rare patterns as with the Lattice VQ.

[0010] One aspect of the disclosure is a method for data compression. An encoder receives data vectors from the original data to be compressed. The encoder uses a vector quantization codebook to encode the data vectors into encoded vectors. The codebook is produced using a training set having a compound data set, where the compound data set includes real data vectors and artificial data vectors. The encoded vectors are indexed in the codebook and the indexes are transmitted across communication channels or transmitted to storage.

[0011] Another aspect of the encoder is the artificial data set. The artificial data set may include a uniformly distributed data set, a diagonal data set, or both.

[0012] The invention may be best understood by reading the disclosure with reference to the drawings, wherein:

[0013] FIG. 1 shows one embodiment of a vector quantization encoder/decoder pair, in accordance with the invention.

[0014] FIG. 2 shows a flowchart of one embodiment of a method to produce a VQ codebook, in accordance with the invention.

[0015] FIG. 3 shows one embodiment of footprints used to divide an image into blocks, in accordance with the invention.

[0016] FIG. 4ashows one embodiment of Voroni cells corresponding to actual data used in vector quantization.

[0017] FIG. 4bshows one embodiment of Voroni cells corresponding to a compound data set used in vector quantization, in accordance with the invention.

[0018] FIG. 5 shows a flowchart of one embodiment of a method of data compression using a VQ codebook, in accordance with the invention.

[0019] Vector quantization is one of the popular compression algorithms, which can be applied to both speech compression and image compression. It is generalized from scalar quantization to the quantization of a multi-dimensional vector, an ordered set of real numbers. The jump from one dimension to multiple dimensions is a major step that allows many new ideas, concepts, techniques, and applications.

[0020] A vector can be used to represent any type of ordered data, such as a segment of a sampled speech waveform or a block of image pixels. Vector quantization can be viewed as a form of pattern recognition where an input vector is “approximated” by one of a predetermined set of standard patterns, or in other words, the input vector is matched with one of a stored set of templates or code words. This predetermined or stored set of patterns, templates, or code words is called a ‘codebook.’ In compression applications, the index of the matched pattern in the codebook is stored or transmitted. This index may or may not be compressed by some lossless compression methods, such as Huffman coding. When a decoder receives the index, the decoder looks up the corresponding pattern in the codebook and outputs the pattern as decoded results. Therefore, a VQ decoder usually has very low complexity and can be implemented by a single-table look-up operation for each reconstructed vector.

[0021] A VQ encoder is usually much more complex than the VQ decoder. A straightforward way to implement a VQ encoder may involve a full search of the closest pattern in the codebook for each input vector, comparing the input vector with the code words in the codebook one by one. Since the full-search VQ requires a lot of computations in the encoder, many efforts have tried to simplify the encoder. For example, using a tree-search concept, lead to tree-structured VQ. Dividing the vector into smaller subvectors, resulted in a hierarchical VQ (HVQ). However, most of those simplifications are sub-optimal as the matched codeword may not be the closest one, but only a codeword close to the optimal one.

[0022] A block diagram of VQ encoder and decoder are shown in FIG. 1. The encoder 10 receives input vectors computed from the source data. The encoder then approximates the input vectors using the vectors in the codebook and an index identifying the particular vector in the codebook is either stored or transmitted at 14. The most common applications for data compression are to reduce the amount of space required to store data or the amount of bandwidth needed to transmit it.

[0023] When desired, the decoder will reconstruct the original vectors using a decoder 16. The decoder takes the indexes stored or transmitted at 14 and uses a codebook 18 that contains the same information as the codebook used by the encoder. This allows the decoder to reconstruct the original input vectors and allows the original data to be reconstructed.

[0024] This is a lossy compression process, where the reconstructed data is not the exact same as the original. The difference between the original and the reconstructed data is referred to as the error. It is left to the system designer to determine the acceptable error rate versus the acceptable compression percentage.

[0025] An important issue for VQ is construction of the codebook. A common method is to first collect a very large set of “typical” vectors, called training data or training set, and then feed this training set through Linde, Buzo, and Gray algorithm, referred to as the LBG algorithm or the Generalized Lloyd algorithm. An optimal codebook can be obtained from the results of the LBG algorithm. The LBG algorithm is an iterative algorithm with an initial codebook. One way of generating the initial codebook can be just randomly selecting the desired number of vectors in the data space. LBG algorithm can then refine this codebook iteratively, based on the characteristics of the input training data.

[0026] However, since the entire space is being represented by fewer representative code vectors, it becomes very important to carefully choose the code vectors in the training stage. VQ tries to capture the characteristics of the data and develop code vectors based on the frequency and properties of the data in the training stage. Usually the code vectors are trained using the typical data from the target environment. The VQ codebooks then get “tuned” to the target environment in the training stage, so that when similar data vector occurs in the encoding stage, it can output the index of a code vector that will be very similar to the original data vector.

[0027] The performance of VQ depends on the design of a good codebook, and to obtain a good codebook depends on the judicious choice of the training data. Some of the vectors in data space may have very low probability to occur in the target environment, and they may never occur in the training set. These regions in space are not accounted for properly in the training set and in the codebook resulting in error between the original and the reconstructed data. This can be more noticeable if fewer numbers of bits, and thus a small codebook, are available for encoding. This is especially noticeable in image data in the chrominance part of the image, as color shifts are more easily detectable by the eye than shifts in luminance or gray scale.

[0028] One way to alleviate the above problem is to increase the number of typical data in the training process. However, this can make the training set impractically large. Training in VQ may take a lot of time. Further, even if more training data are added, it can never be certain as to whether all of the data vectors are accounted for in a way proportional to their occurrence.

[0029] As mentioned previously, the other approach is to use Lattice VQ where the space is mathematically partitioned into equal regions. This approach may quantize the vectors well if the data is uniformly distributed, but in typical applications this is not suitable since the real data usually have a very skewed distribution.

[0030] Another factor noticeable in the training set, especially to image data, is that the elements of the vectors often have similar values. The codebooks emphasize the diagonals, where the entries of a vector are all similar, of the multidimensional space in code-vector creation. Emphasizing the diagonals more than the other parts of the multidimensional space will have high errors in the physical pixel regions with high variations. This data will be mapped to codes lying on the diagonal, nearest to them.

[0031] However, this problem can be overcome using a combination of some artificial data, which uniformly covers the entire space and the real data from the characteristic training set. In order to account for the distribution of real data later, it seems wise to use an approach that differs from the approach in the lattice VQ that computes the codes mathematically. Instead, one may feed uniformly distributed artificial data to the training system, and the system can automatically generate the code vectors that uniformly cover the whole space. The artificial training set may have vectors of different combinations of values, and may repeat them sufficient number of times if necessary. This artificial training set will be referred to as the uniformly distributed data set.

[0032] This data set, for example, in two dimensions, looks like: (0,0), (0,1), (0,2) . . . (0,255), (1,0), (1,1), (1,2) . . . etc. Repetition may be necessary if the number of all the possible combinations of data vectors is still smaller than the number that needs to account for the desired population of the artificial data. This would yield an almost multi-dimensional, hexagonal-like partitioning of space. In higher dimension space, it would be more practical to feed in uniformly distributed artificial data, say for example, as generated by a suitable random number generator.

[0033] However, this has the potential problem discussed above that typical images may have a high correlation between adjacent pixels, since these are usually similar. It may be better to give more emphasis to the vector patterns where all elements in a vector are of same value. In training, the system may have another artificial set where all elements in a vector are of the same value and the value is uniformly distributed. This artificial data set will be referred to as the diagonal data set.

[0034] For example in the two-dimensional case, this artificial training data set may look like (0,0), (1,1) . . . (255,255) etc., which can also be repeated enough times to reflect the influence desired. If it is impossible to enumerate all the possible values, e.g., if the elements are real numbers, not integers, the diagonal data set can be generated by a random number generator such that the element values in different vectors are uniformly distributed.

[0035] This will perform better than the case of the pure lattice VQ, since it has more chances of having adjacent pixel regions the same. In this case, the code regions will lie along the higher-dimension diagonal. Therefore the more robust option seems to be a “mix” of the actual training set and the above-mentioned two artificial sets into one set, which will be referred to as the mixed set or the compound set. The uniformly distributed set will account for the insufficient percentage of the vectors with high variations. The diagonal data set can be used to make the overall artificial data set have closer characteristics as the target environment.

[0036] The mixing is flexible, in that one may vary the percentages of the actual data and the artificial training data. The “mixed” codes retain most of the characteristics of the real data mostly along the diagonal. It has some codes in the peripherals to account for the details, where the pixels vary greatly. If these were not there, the codes would have been mapped to some point near the diagonal. In the mixed or compound set, it may be seen that the hexagonal codes account for the details, thus reducing the error as compared to the original. Also the vectors that may not occur in the original, real data set, for example, corresponding to two ends of the main diagonal in the multidimensional space, are accounted for in a better way with regards to the partitioning of space.

[0037] FIG. 2 shows one embodiment of a method to train a VQ codebook that takes these considerations into account. At 34, the process selects a compound training data set of a real vector data set and at least one other artificial vector data set as was described in the previous section. In one embodiment, the artificial data set includes a uniformly distributed data set. In an alternative embodiment, the artificial data set includes a diagonal data set. In yet another alternative, the compound set includes both. The compound training data set is used to produce the codebook, not in the actual compression process.

[0038] At 20, the initial codebook is defined. For example, the random initialization method mentioned previously may be used to define an initial codebook with a desired number of code vectors. At 22, each of the training vectors is assigned to the closest code vector. Closeness is typically defined as a distance measure of some sort. After the assignment is made, all of the training vectors that are assigned to one code vector are formed into sets.

[0039] At 24, the centroid of each set is found. Typically, this uses the number of occurrences of the training vector as the mass and the distance measure of the vector as the distance. This newly calculated centroid is then used to replace the code vector about which that set was formed at 26.

[0040] Once the centroids are replaced, an error value is computed based on the difference between the new code vectors, E_i, and the previous code vector, E_i−1. The significance of the error can be defined based on a threshold; and the difference, E_i−1−E_i, is compared to the threshold to determine if the next iteration has to be performed or not. If the error difference is no longer greater than the threshold, the process ends. If it is greater than the threshold, the process returns to the assignment process with using the new collection of code vectors. In the following example, the data used is image data in a scanner-printer application. This is only for example only, no limitation to image data is intended nor should it be implied as a limit on the scope of the invention, as it is equally applicable to many different types of data. This example uses HVQ (Hierarchical VQ) for fast compression required to store and transmit image data from the scanner side to the printer side in a scanner-printer application. The resolution of the scanned image is 600 dpi, and there is lot of redundancy in the pixel patterns. Also, the correlation between the pixels is high. This enables the use of HVQ for compression of these scanned documents.

[0041] It is well known that the YCbCr color space is better than the RGB color space for compression. In this example the system converts the RGB raw image received from the scanner to YCbCr image data. The human visual system (HVS) has about a half of the spatial bandwidth for the chrominance components, the Cb and Cr components here, than the bandwidth for the luminance component, the Y component. This results in an ability to down-sample the Cb and the Cr data to compress data with almost no quality degradation. The Y/Cb/Cr components are compressed separately. As for the Y component, it can be divided across the entire image into blocks of size 2 by 4 pixels, as shown in FIG. 3.

[0042] Using HVQ, the 2 by 4 block are encoded:

[0043] Code=Table₃[Table₂[Table₁[a][b]][Table₁[e][f]]][Table₂[Table₁[c][d]][Table₁[g][h]]]

[0044] As for the Cb/Cr components, there are just two pixel values corresponding to the physical 2 by 4 image block. This just requires coding:

[0045] Code for Cb=Table′₁[p][q]

[0046] Code for Cr=Table″₁[r][s]

[0047] Since the footprint used here just corresponds to two pixels, only 5-6 bits are necessary for encoding the chrominance part. If lattice VQ were used to handle this problem considering the entire space, this would yield an almost hexagonal partitioning of space. Since the dimension of the vector here is 2, these can be visualized as hexagonal cells partitioning the entire plane. Similar partitioning would result if the process used the uniformly distributed data set as the training set.

[0048] To account for the fact that there is a high correlation between adjacent pixels it may be more desirable if more emphasis is given to the codes where the adjacent pixels are of same value. As discussed previously, the training set may have another artificial set where the adjacent pixel values are the same, and it contains all the possible values, considered as a pair. In this case, the code regions will lie along the main multi-dimensional diagonal.

[0049] A judicious choice in the relative percentages of the above-described artificial sets and the real data set, in the codebook has to be made. Looking at the Voronoi diagrams of the space partitions corresponding to the real data, uniform quantization and the diagonal quantization, the relative percentages of the 3 sets can be decided. For example in the above case, a Voronoi diagram as in FIG. 4aand FIG. 4bwas used. FIG. 4ashows one embodiment of a Voronoi using real data. FIG. 4bshows one embodiment of a Voronoi diagram using a compound set.

[0050] The mixing is flexible, for example, the above corresponds to a mixing of the real data, and one each of the artificial data sets in a ratio of 10:1:1. As seen above, the “mixed” Voronoi diagram retains most of the characteristics of the real data along the diagonal, mainly. It has some codes in the peripherals to account for the details. If these were not there, the codes would have been mapped to some point near the diagonal. In the compound set, it can be seen that the hexagonal codes account for the details, thus reducing the error as compared to the original. Also the bright colors corresponding to the edges of the diagonals are accounted for in a better way with regards the partitioning of space.

[0051] This approach can be used for any types of VQ and for similar target environments. In scanned images, a lot of the regions are “flat” in that the pixel values do not vary much in these regions. Therefore, most of the codes may pertain to these flat regions. If one considers PSNR (pixel signal-to-noise ratio) as a measure of quality in the image case, it can be seen that the PSNR would be higher if the entire image is allowed in training. The system may lose on the pixel regions with “details” generally regions of high frequency. Overall the PSNR may be high since the flat regions, which outnumber the detail regions, are encoded with higher precision.

[0052] It must be noted that such a high precision is not required for the flat regions in typical images because of the nature of the human visual system. This high precision pays the price of losing codes in the detail regions, thus causing the images to be hazier. It would be more desirable to keep less precision in the flat zones, only enough to accommodate the human visual system, to have sufficient codes in the detail regions.

[0053] Another factor to be considered is that the chrominance changes or artifacts are more easily noticed than luminance changes. The scanned images, such as those used in the example, might not have some colors, such as bright colors. If the training set had only those non-bright colors, there may have been noticeable color mismatches if the system encountered those colors in encoding. Having all of the colors would require the huge training sets discussed above. However, selection of suitable training sets to include the artificially generated sets discussed above, provides all of the colors.

[0054] A flowchart of one embodiment of a method for data compression using a VQ codebook is shown in FIG. 5. The input vectors are received at 30. As mentioned above, the input vectors are ordered pairs of input data. The input data could be one of several different types of data, including image data or speech data. The input vectors are encoded at 32 using the codebook 36. The codebook 36 was computed offline from a compound data set of a real vector data set and at least one other artificial vector data set. In one embodiment, the artificial data set includes a uniformly distributed data set. In an alternative embodiment, the artificial data set includes a diagonal quantization data set. In yet another alternative, the compound set includes both. The compound data set is used to produce the codebook, not in the actual compression process.

[0055] These indices from codebook 36 are that which are transmitted at 38. Note that transmission in this example includes both transmissions across a communications channel and transmission to storage. The decoder has the same codebook and can use the indexes to determine the encoded vectors. These encoded vectors then become the approximation of the original data set, as discussed in detail above.

[0056] In many instances, this method will be implemented in software in a machine-readable form, such as in the encoder software. The machine-readable code includes the code that, when executed, implements the methods of the invention.

[0057] Thus, although there has been described to this point a particular embodiment for a method and apparatus for robust VQ encoding using compound sets, it is not intended that such specific references be considered as limitations upon the scope of this invention except in-so-far as set forth in the following claims.