Content-Based Indexing of Images by Coding Levels Defined as a Function of Reduced Entropy

The present invention relates to moving picture frame, video frame and still frame image storage and transmission and to the encoding of digital information filtered to compress data in a manner for prioritized retrieval, transmission, and reception.

It has previously been difficult to identify and retrieve selected moving picture, video or still images from a collection of image frames without human viewing of each of the frames to determine its content. As a result, retrieval of such images can be extremely time-consuming.

Another alternative is that the collection of images can be reviewed and provided with hand annotated designations as to the content of the images. However, the annotations must be entered on another media and further complicates the indexing of the data already accumulated in the images.

Moreover, while previous compression techniques can compress the image data, the compression techniques are often unrelated to the contents of the images and thus may not maximize the compression of the data. As a result, the compression does not minimize the storage or provide any information about content.

The present invention overcomes some of the above-mentioned disadvantages by providing an identification of accumulated data in storage for reaccessing the images by content. The present invention indexes the accumulated data by processing the data in a compression filter that employs a set of basis functions which minimize the entropy of the resulting generalized Eigenvalues during compression of the data. This filtering also provides the most effective compression of the data.

In the preferred embodiment, the data is prioritized by characteristic spatial frequencies. For example, since cloud images viewed from space have very characteristic spatial frequencies, an image that is compressed with a set of basis functions and optimized for compressing clouds will have a pattern of Eigenvalues with most of the energy concentrated in a small number. Accordingly, that characteristic set of Eigenvalues may be used to retrieve, transmit and receive non-cloud-covered aerial photographs without prolonged review of each frame image or the separate media annotations that identify such photographs.

As a result, the present invention provides the ability to browse databases for general subject matter content rather than imposing a complete review of the database. Moreover, only relevant identifying information need be reviewed, and does not require that the entire collection be reviewed in the order that it has been accumulated or transmitted. In addition, filtering according to the present invention improves compression of the data since it relates particularly to content of the images.

The present invention will be more clearly understood by reference to the following detailed description of a preferred embodiment when read in conjunction with the accompanying drawing in which like reference characters refer to like parts throughout the views and in which:

FIG. 1 is a block diagram of a communication system with an encoder for a vision transmission system according to the present invention; and

FIG. 2 is a diagrammatic view of the sub-band decomposition used in a filter in the system of FIG. 1.

Referring first to FIG. 1, the transmission system 10 according to the present invention performs a method for indexing video images by content by prioritizing the compression of content according to a reduced entropy of resulting Eigenvalues. The system permits browsing of the image content accumulated, retrieved, transmitted or received. Thus, the output of the image detector 12 is processed at a filter 14, preferably in the form of a hierarchical sub-band coder, to transform each of the images and categorize the data.

The categorized data is stored in a storage device 16 and selectively retrieved according to its content. Accordingly, the transmitter can then send selected data to a receiver 20, initially providing images that match a category which contains the desired content in each image. Moreover, an optimized compression of the accumulated image data can be obtained. Accordingly, the invention minimizes storage requirements and transmission time.

In addition, improved resolution transmission of image data can be obtained by employing a predictor 22 to create residual images that are calculated and stored for recreation of the high resolution images regardless of losses occurring in the compressed images. The stored or transmitted image data is compared with the actual image and subtracted to produce a residual image that can then be coded and stored for re-creating a lossless image. As a result, this reliance on a priori knowledge, as well as calculation and determination of the best of the basis functions and quantization methods in the compression process of the original frame, preferably simultaneously with the sub-band coding minimizes the number of bits required to represent the image losslessly. The coding and quantizing at filter 14 is preferably done at the same time as residual images are created at the predictor 22.

Referring now to FIG. 2, the filter of the preferred embodiment includes a three-level Mallat sub-band decomposition of the image signal. This image compression performs multi-resolution decomposition using a wavelet transform on the original image. By so processing the signal, the energy and information in the image is redistributed into a small number of coefficients, which can be represented by fewer bits and subsequently provide data compression and compaction.

The wavelet transform maps the original image into a set of orthogonal subimages. The transform decomposes the original image at different resolutions using a pyramidal algorithm architecture, as shown in FIG. 2. The decomposition on images is along the vertical and horizontal directions and entails filtering and then decimating appropriately. Since most natural images tend to have a high correlation amongst nearby pixels, most of the energy in these natural images are found in the low frequency subimages. Relatively little energy is contained in the high frequency regions after the multi-resolution analysis.

As a result, most of the energy would reside in the LL3 subimage 30 containing the low frequency horizontal, low frequency vertical subimage at resolution level 3. Little energy would reside in the LH1 subimage 52 containing the low frequency horizontal, high frequency vertical subimage at resolution level 1, the HL1 subimage 54 that contains the high frequency horizontal, low frequency vertical subimage at resolution level 1, and HH1 subimage 56 containing the high frequency horizontal, high frequency vertical subimage at resolution level 1.

Indexing of the image data is based on which basis functions require the least number of bits to represent the image and the resulting generalized Eigenvalues for that set of basis functions. Retention of residual images permits lossless reconstruction of the original image with substantially less image data than previously known compression systems.

As a result, the present invention provides a method for indexing lossless images. The invention reduces the time for identification as well as storage or transmission of the desired images. Thus, the present invention calculates and determines the best set of basis functions and quantization method in the compression process so as to minimize the data required to represent the image and to identify the image by relevant content.

Having thus described the present invention, many modifications thereto will become apparent to those skilled in the art to which it pertains without departing from the scope and spirit of the present invention as defined in the appended claims.