Method and Apparatus for Optimizing Storage of Compressed Images in Memory

The invention relates generally to an image processing system and method for image data compression. In particular, the invention relates to an image processing system with improved utilization of available memory for storing compressed images.

Image compression techniques reduce the memory required for storage of large or complex images, permitting storage of images in devices with limited memory. For example, image compression techniques used in digital copiers and scanners assist in storage of complex images for subsequent printing. Similarly, image compression benefits communication links where bandwidth limitations would otherwise render transmission of image data impractical. Image compression also offers substantial benefits for archiving large image libraries.

JPEG (Joint Photographic Experts Group) is a standardized image compression technique that can be used with either full-color or gray-scale images. This technique performs a Discrete Cosine Transform (DCT) on each block of pixels in an image. The result of the DCT for each block of pixels is an array of frequency components. Low frequency components correspond to slowly varying intensities between neighboring pixels. High frequency components represent rapidly varying intensities across small clusters of pixels. Information represented by the higher frequency components is reduced by quantization, i.e., the restriction of actual values to a limited set of values. The compressed image size is related to the quantization level specified by the user. The frequency components are encoded in order from the lowest frequency to the highest. If there are no frequency components beyond a specific frequency, then no further data from that block is encoded. Instead, an end of block symbol is emitted to indicate to a decoder that the block is complete.

Application of the above technique to blocks of pixels having high spatial frequencies may result in data represented by more bits than can be stored in a memory device or transmitted over a communication link in a limited time. One technique used to overcome this problem is to count the total number of bits used to represent the block of pixels and compare the total with a maximum allowable value. If the total exceeds the maximum value, the highest non-zero frequency component is decreased in absolute value and the total number of bits is recalculated and compared with the maximum. This process is repeated until the total number of bits is no greater than the maximum value. Generally, human perception of images is more affected by amplitude changes in the lower frequency components than higher frequency components, therefore this technique typically has negligible impact on perceived image quality. However, an image stored under the JPEG standard and subsequently recalled is reduced in quality from the original image. Multiple applications of JPEG compression on an image can result in a significant reduction in the image quality. It is therefore desirable to store as many of the higher frequency components as possible in the available memory.

One known storage technique for JPEG compressed images, described in U.S. Pat. No. 5,321,522, provides fixed memory allocation for each block of pixels. Therefore, a block of pixels that has no high frequency components may not require all the bits available in a block of memory. As a result, there can be "gaps" in memory. Subsequent processing, however, may include blocks of pixels with high frequency information that exceeds the available memory allocated to individual blocks, thereby requiring the loss of some information.

A method for optimizing the storage of compressed images has been developed which is useful for a wide variety of applications, including, but not limited to, storage of digital images in limited memory devices.

The method requires the image to be compressed be divided into a series of smaller images or pixel blocks. The values corresponding to the pixel blocks are then processed, one block at a time, by a processor module. Processing can include performing a transform function on each pixel block. The transform function can be one that generates frequency space coefficients representative of the pixel block data (e.g., a discrete cosine transform (DCT)). Processing can include further operations such as ordering the frequency space coefficients, recalling quantization values from memory and creating quantized frequency space coefficients by dividing the frequency space coefficients by the corresponding quantization values. Additional processing can be used to create a data packet for each quantized frequency space coefficient and to encode each data packet (i.e., a symbol or sequence of bits).

A cumulative memory allocation, which defines the current allowable memory, is determined each time a new pixel block is processed. The cumulative memory allocation can be based on the number of pixel blocks currently processed and an average memory allocation for each pixel block. The processor module generates a series of data packets for each processed pixel block. The bits contained in each data packet are counted and the result is added to the number of bits previously stored in memory to yield a current memory total. The current memory total is compared to the cumulative memory allocation to determine whether the current data packet should be written to memory. If the current memory total does not exceed the cumulative memory allocation, the current data packet is written to memory. If the current memory total exceeds the cumulative memory allocation, no further data packets for the present pixel block are written to memory and processing begins on the next pixel block.

This method avoids the "memory gaps" that can result from existing methods, such as described in U.S. Pat. No. 5,321,522, which provide a fixed memory allocation for each block of pixels. The present invention is more flexible in its allocation of memory. If earlier processed blocks do not fill the average memory available for each block, later processed blocks containing more information can take advantage of the excess memory for the earlier blocks. In effect, each block of pixel data is packed into memory serially rather than being restricted to a block of memory, thereby permitting more information to be stored for a given image. This method can be implemented as the data packets are generated, thereby avoiding the need to later process each data block.

The invention also features an apparatus for optimally compressing a digital image into a memory module. The apparatus includes a processor module to process pixel blocks and generate data packets. The processor can perform various functions, including transforms (e.g., a DCT), on each pixel block. Processor output can include spatial frequency coefficients for each pixel block which can be quantized and formatted into data packets. The apparatus includes a counter which receives the processor output, determines the number of bits in each data packet and adds the result to generate a current memory total. A comparator compares the current memory total to a reference signal representative of the cumulative memory allocation. The cumulative memory allocation, for example, can be based upon the number of pixel blocks processed for the image and an average memory allocation per pixel block. An output device, such as a buffer, temporarily stores the data packet generated by the processor. The comparator output enables the output device to write the data packet to memory when the current memory total does not exceed the cumulative memory allocation.

FIG. 1 is a functional block diagram of an apparatus for practicing an optimized method for storage of compressed images.

FIG. 2 is a detailed block diagram of an apparatus for practicing an optimized method for storage of images compressed using the JPEG standard.

FIG. 3 is a flow chart of the iterative method for compressing a processed image.

FIG. 1 shows a functional block diagram of an apparatus for practicing the method incorporating the present invention. A digital image to be compressed is divided into a series of sub-images or pixel block data by a pixel block generator 12 and transferred to a processor 14. The processor 14 performs a transfer function on each pixel block to produce transformed pixels blocks each consisting of a series of data packets.

The number of bits in each data packet is determined by a counter 16 and added to the number of bits for all previously stored data packets, yielding a current memory total. A comparator 18 compares the value generated by the counter 16 with a reference value 20. The reference value 20 represents a cumulative memory allocation for memory 22 usually based upon the number of pixel blocks previously processed for the current image. Typically, the reference value 20 is updated each time a new pixel block is processed. If the comparator 18 determines that the current memory total does not exceed the cumulative memory allocation, the data packet is written to memory 22. If the comparator 18 determines that the current memory total exceeds the cumulative memory allocation, no further data packets for the current pixel block are written to memory and a new pixel block is processed.

Referring to FIG. 2, an apparatus for practicing an optimized method for storage of compressed images employing a JPEG compression standard is shown. Digital image data is transferred to a pixel block generator 12 and divided into pixel blocks. The pixel blocks are transferred, one block at a time, to a processor 14. First, the processor 14 performs a discrete cosine transform (DCT) 24 on each pixel block, generating frequency space coefficients. Next, the frequency space coefficients are quantized by a divisor 26 which divides each frequency space coefficient by a corresponding quantization value from a quantization table memory 28. The non-integer portion of the resulting values are discarded and the resulting quantized frequency space coefficients are provided to a frequency orderer 30 to be arranged in substantially increasing frequency order.

Quantized frequency coefficients are encoded at encoder 32 into symbols which are passed to an end of block symbol inserter 34 and a bit counter 36. The bit counter 36 determines the number of bits in the symbol and adds the result to the number of bits already stored in memory 22, yielding a current memory total. The current memory total is compared with a cumulative memory allocation at a comparator 40. If the current memory total does not exceed the cumulative memory allocation, the last created symbol is written to memory 22 and the next symbol from the encoder 32 is examined by bit counter 36. If the current memory total exceeds the cumulative memory allocation, the end of block symbol is written to memory 22 and the pixel block generator 12 provides a new block to the processor 14. If the last non-zero frequency coefficient for the block is detected by the frequency orderer 30, the end of block symbol inserter 34 appends an end of block symbol to the current symbol. Both symbols are then written to memory 22 and the pixel block generator 12 provides a new block to the processor 14.

Referring to the flow chart in FIG. 3, one implementation of the method of the invention is described as a series of steps. At step 100, pixel block counter, N, and the current memory total, MEM, are set to zero. At step 102, data packet counter, I, is set to one and N is incremented by one to indicate the start of processing of a new pixel block in step 104. Next, the cumulative memory allocation, CMEM, is calculated at step 106. In this example, CMEM is incremented by the average allowable block size, ABSIZE, at the start of each new block. However, CMEM may be determined by other means. At step 108, the number of bits (COUNTS) in the Ith data packet generated by the processor is determined.

In step 110, the current memory requirement, MEM+COUNTS, is compared with CMEM. If the current memory requirement exceeds the cumulative memory allocation, an end of block symbol is written to memory at step 120. N is then incremented by one and I is reset to one at step 102 and the next pixel block is provided to the processor at step 104. If the current memory requirement does not exceed the cumulative memory allocation, the Ith symbol for the Nth pixel block is written to memory in step 112, CMEM is updated in step 114, and a determination as to whether the last non-zero symbol has been processed for the current pixel block is made in step 116. If the symbol written to memory was not the last non-zero symbol for the pixel block, I is incremented by one in step 118 and the procedure returns to step 108 to examine the next generated symbol. If the last non-zero symbol for the pixel block was detected according to step 116, an end of block symbol is written to memory at step 122. N is then incremented by one and I is reset to one at step 102 and the next pixel block is processed at step 104.

While the invention has been particularly shown and described with reference to specific preferred embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.