Image processing apparatus and method

[0002] This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 11-257704, filed Sep. 10, 1999, the entire contents of which are incorporated herein by reference.

[0003] The present invention relates to an image processing apparatus and method having the function of executing color transformation processing on image data depending on an I/O device or the like.

[0004] In general, an image I/O apparatus such as a color printer, a color display, a color scanner cannot directly use RGB signals or the like from an original image due to the variation of the physical characteristics of these apparatuses, so that color transformation must be performed before input or output depending on the characteristics of each apparatus.

[0005] Color transformation is also executed simply to make a user's face look attractive, for example, to add a tinge of red to the flesh color so that the face appears beautiful. Actually, in most cases, color space transformation is further performed, for example, to transform RGB (Red, Green and Blue) into CMY (Cyan, Magenta, Yellow).

[0006] In general, simple linear transformation is insufficient to obtain high image quality by means of the above transformation, so that a combination of a color transformation table and a multi-dimensional equation is used. Color transformation methods are roughly divided into the following three types:

[0007] (a) transformation based on an equation,

[0008] (b) transformation based on transformation table look-up (LUT), and

[0009] (c) transformation based on a combination of transformation table look-up and equations.

[0010] The simplest example of the transformation based on an equation is a one-dimensional transformation equation as shown below: $\begin{array}{cc}\left[\begin{array}{c}{R}^{\prime }\\ G^{\prime }\\ B^{\prime }\end{array}\right]=\left[\begin{array}{ccc}a\mathrm{rr}& a\mathrm{rg}& a\mathrm{rb}\\ agr& a\mathrm{gg}& a\mathrm{gb}\\ a\mathrm{br}& a\mathrm{bg}& a\mathrm{bb}\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]& \left(1\right)\end{array}$

[0011] That is, a one-dimensional equation for the value of each color (R, G, B) in an original pixel is multiplied by a transformation coefficient to obtain corrected pixel values (R′, G′, B′).

[0012] Although the amount of calculations in Equation (1) is 9 in terms of the number of multiplications, these calculations must be applied to all (N) pixels, thereby requiring 9N multiplications in total.

[0013] To meet actual needs for color transformation that is sufficient for all points on a color space, high-order items must be introduced because the above one dimensional transformation provides an insufficient image quality, resulting in a significant increase in the amount of calculations required.

[0014] For example, even the simple introduction of a two-order item requires 27 multiplications per pixel. $\begin{array}{cc}\left[\begin{array}{c}{R}^{\prime }\\ G^{\prime }\\ B^{\prime }\end{array}\right]=\left[3\times 9\mathrm{matrix}\right]\left[\begin{array}{c}R\\ G\\ B\\ R^2\\ G^2\\ B^2\\ \mathrm{RG}\\ \mathrm{GB}\\ \mathrm{BR}\end{array}\right]& \left(2\right)\end{array}$

[0015] In practice, this is often still insufficient, and the color transformation based only on an equation is thus rarely employed.

[0016] (b) Table Look-up Method

[0017] The table look-up method references a table in which color transformation is individually defined for all colors.

[0018] This method has the advantage of enabling every transformation, while having the disadvantage of requiring an enormous table size. For example, a 24-bit color space requires 2²⁴×24 bits, that is, 48 MBytes of table.

[0019] (c) Transformation Based on Transformation Table Look-up and Equations

[0020] This method provides a transformation table for representative colors sampled on a color space (for example, points that evenly divide each of R, G, and B into 16 pieces or the like) and calculates the other colors from several representative colors in the neighborhood using an equation.

[0021] This method requires a small table size and a small area to be interpolated by an equation, thereby enabling appropriate color transformation with a relatively simple equation.

[0022] This method is actually often employed and various methods have been proposed including a color space division method, a neighborhood point selection method, and an interpolation method. This method, however, also involves at least several multiplications/divisions for all pixels and thus does not eliminate the disadvantage of requiring a large amount of calculations. Accordingly, a faster method is desired.

[0023] Recently, a compression method such as the JPEG has been used, for example, to reduce the image data size. The JPEG (Joint Photographic Coding Expert Group) is a method for compressing multi-gradient images (including color images) which has been standardized by ISO (International Standards Organization) and ITU (International Telecommunication Union).

[0024] However, image compression and color transformation are treated as totally different techniques, and in practical applications, these techniques are sequentially used for processing as follows:

[0025] Output system: compressed image→(expansion)→ original image→(color transformation)→output image

[0026] Input system: input image→(color transformation)→ original image (compression)→compressed image

[0027] For example, printers perform such sequential processing by (1) expanding a JPEG-compressed image and (2) executing color transformation to this image for output.

[0028] If image compression and expansion is performed separately from color transformation, the color transformation process requires a large amount of time because it is used for raw image data obtained before compression or after expansion and having a large number of pixels. Image I/O processing further includes an amount of time required for the compression and expression processes, thereby requiring a vast amount of time.

[0029] The present invention is provided in view of these circumstances, and it is an object of the present invention to provide an image processing apparatus and method that can reduce the number of color transformation target pixels without significantly degrading image quality and that can enable very fast processing.

[0030] In order to attain this object, according to a first aspect of the present invention, there is provided an image processing apparatus comprising reduced image production means for producing reduced image data from original image data; color transformation means for transforming color of the reduced image data obtained by the reduced image production means; first frequency transformation means for executing frequency transformation to the reduced image data subjected to color transformation by the color transformation means in order to determine low-frequency coefficient of the original image data; second frequency transformation means for executing frequency transformation to the original image data to determine high-frequency coefficient of the original image data; means for combining the low-frequency coefficient determined by the first frequency transformation means and the high-frequency coefficient determined the second frequency transformation means in order to generate frequency coefficient of the original image data; and coding means for coding the generated frequency coefficient of the original image.

[0031] The processing executed by this image processing apparatus is divided into a process for determining low-frequency coefficient and a process for determining high-frequency coefficient. The process for determining low-frequency coefficient comprises executing color transformation on a reduced image obtained by sub-sampling and having a smaller number of pixels and then subjecting the color-transformed image to frequency transformation (orthogonal transformation) based on the DCT (discrete cosine transformation) or Wavelet transformation so that frequency coefficient of the reduced image obtained by the frequency transformation are determined as the low-frequency coefficient. On the other hand, the process for determining high-frequency coefficient comprises directly executing transformation to original image data without executing the color transformation process so that frequency coefficient obtained by the frequency transformation are determined as the high-frequency coefficient of the original image. Then, by combining the low- and high-frequency coefficient together, frequency coefficient of the original image are determined which covers both the low and high frequencies. This combine process can be easily implemented, for example, by executing frequency transformation to original image data without color transformation to obtain 8×8 DCT coefficients and replacing a frequency coefficient obtained from a reduced image for some of the DCT coefficients and which constitute the low-frequency component.

[0032] With this simultaneous compression and color transformation processing of separately subjecting the low- and high-frequency component sides to the frequency transformation process and executing color transformation only to the low-frequency components, which more significantly affect image quality, without executing color transformation on the high-frequency components, the number of color transformation target pixels can be reduced to increase the processing speed without seriously degrading image quality.

[0033] In addition, according to a second aspect of the present invention, there is provided an image processing apparatus comprising means for decoding compressed and coded image data to obtain frequency coefficient of the compressed and coded image data; first image data generation means for executing inverse frequency transformation to low-frequency coefficient of the obtained frequency coefficient of the compressed and coded image data to generate reduced image data; color transformation means for executing color transformation on the reduced image data obtained by the first image data generation means; image data size reproduce means for reproducing the reduced image data subjected to the color transformation by the color transformation means, to a size of the original image data; second image data generation means for executing inverse frequency transformation to the frequency coefficient of the compressed and coded image data other than the low-frequency coefficient to generate image data; and means for combining the image data obtained by the image size reproduce means with the image data obtained by the second image data generation means in order to generate a decoded image of the compressed and coded image data.

[0034] Thus, in expanding a compressed and coded image, the number of color transformation target pixels can also be reduced to increase the processing speed without degrading image quality, using the above simultaneous expansion and color transformation processing method of using the separate inverse frequency transformation processes for the low- and high-frequency component sides and executing color transformation only to the low-frequency components, which more significantly affect image quality, without executing color transformation on the high-frequency components.

[0035] The boundary for dividing the frequency coefficient of the image data into the low- and high-frequency sides may be fixedly determined beforehand. When, however, it is determined whether or not any of the high-frequency components has a higher level than a predetermined threshold and if so, then the function of dynamically changing the boundary depending on the spatial position of this frequency coefficient is desirably provided. This function can reduce the effects of color transformation errors originating from the lack of the color transformation of the high-frequency components, thereby providing higher-quality images.

[0036] In addition, in decoding image data coded by hierarchical mode, the processing speed can be increased without degrading image quality, using simple processing for executing color transformation to decoded image data obtained from low-resolution image data without the color transformation of decoded image data obtained from high-resolution image data.

[0037] Additional objects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out hereinafter.

[0038] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate presently preferred embodiments of the invention, and together with the general description given above and the detailed description of the preferred embodiments given below, serve to explain the principles of the invention.

[0039] FIG. 1 is a block diagram showing an example of a configuration of an image processing apparatus according to a first embodiment of the present invention;

[0040] FIG. 2 is a diagram for describing example of base functions for the DCT for use in the image processing apparatus according to the first embodiment;

[0041] FIGS. 3A and 3B are diagrams for describing base functions for the Wavelet transformation for use in the image processing apparatus according to the first embodiment;

[0042] FIG. 4 is a diagram for describing the characteristics of the Wavelet transformation for use in the image processing apparatus according to the first embodiment;

[0043] FIG. 5 is a diagram for describing a color transformation and compression procedure in the image processing apparatus according to the first embodiment;

[0044] FIGS. 6A to 6E show a first example of division between low and high frequency components in the image processing apparatus according to the first embodiment;

[0045] FIGS. 7A and 7B show a second example of division between low and high frequency components in the image processing apparatus according to the first embodiment;

[0046] FIG. 8 is a block diagram showing an example of a configuration of an image processing apparatus according to a second embodiment of the present invention;

[0047] FIG. 9 is a diagram for describing a color transformation and expansion procedure in the image processing apparatus according to the second embodiment;

[0048] FIG. 10 is a block diagram showing an example of a configuration of an image processing apparatus according to a third embodiment of the present invention;

[0049] FIG. 11 is a diagram showing the structure of hierarchically coded image data to be expanded according to the third embodiment;

[0050] FIG. 12 is a block diagram showing an example of a configuration of an image processing apparatus according to a fourth embodiment of the present invention;

[0051] FIGS. 13A to 13C is a diagram for describing a low/high frequency boundary determination processing method according to the fourth embodiment;

[0052] FIG. 14 is a block diagram showing an example of a configuration of an image processing apparatus according to a fifth embodiment of the present invention; and

[0053] FIG. 15 is a diagram for describing a low/high frequency boundary determination processing method according to the fifth embodiment.

[0054] The embodiments of the present invention will be described below with reference to the drawings.

[0055] FIG. 1 shows a configuration of an image processing apparatus according to a first embodiment of the present invention. A color scanner system will be illustrated and described which is comprised of a scanner 11 and a host computer 12. Color image data read by a mechanical section 13 of a color scanner 11 is transmitted to a color transformation and compression module 14 as original image data to be processed, where a color transformation and compression process is executed for simultaneous color transformation and compression. The color transformation process includes color transformation for color corrections that are compatible with the device characteristics of the color scanner 11, color transformation for obtaining images that meet a user's liking. In addition, the compression process comprises compressing and coding image data by means of a compression method based on the JPEG or the like.

[0056] Compressed image data obtained by the color transformation and compression module 14 is transmitted to a host computer 15 via a recording or communication medium, where the data is stored in a disk device 15 or the like in the host computer 15. The compressed image data recorded on the disk device 15 is output to an output device. The color transformation and compression module 14 may reside on the host computer 12 side.

[0057] With the color transformation and compression process executed by the color transformation and compression module 14, in compressing an image using frequency control (the DCT, Wavelet transform, or the like), the following method is used: When high-frequency components are calculated, the value of an input image is not subjected to color transformation or very simple transformation is used. On the other hand, when low-frequency components are calculated, the number of pixels is reduced using an LPF (Low Pass Filter) and sub-sampling for reducing the number of pixels, sub-sampled reduced image is subjected to the color transformation process. Finally, typical calculations are performed on the image obtained.

[0058] Then, the principle of the image compression process using frequency transformation (the DCT, Wavelet transform, or the like) will be described using FIGS. 2, 3A, and 3B. FIG. 2 shows base functions for the DCT (in the case of one dimension and eight pixels), FIG. 3A shows base functions for the Wavelet transformation (in the case of one dimension and eight pixels) as an example, and FIG. 3B shows a base frequency spectrum as an example.

[0059] As described above, the JPEG is a compression method for multi-gradient images (including color images) which employs as a basic method the DCT (Discrete Cosine Transform), which is a frequency transformation (an orthogonal transformation). Frequency transformation transforms an original image that is a collection of levels relative to spatial “positions” into a collection of levels for each “degree of change”. That is, this method divides the original image into the sum of “low-frequency components” representing slower changes (more leftward functions in the upper part of FIG. 2) and “high-frequency components” representing rapid changes (more rightward functions in the lower part of FIG. 2) and expresses the image as a collection of coefficients applied to each group of frequencies.

[0060] The following two points are principle reasons for the use of frequency transformation for image compression:

[0061] (a) When a natural image has its frequency transformed, high levels concentrate on low-frequency components, while high-frequency components have a level close to zero level. By using this nature to code continuous zeros in a run length manner, a compression effect can be obtained.

[0062] (b) As a visual characteristic, human beings do not virtually notice missing high-frequency components. Accordingly, no practical problem occurs even if high-frequency components are substantially quantized to reduce the amount of information therein. In addition, this quantization converges all the values close to zero at zero to further improve the effect in (a). (Hierarchical mode)

[0063] The JPEG defines a hierarchical mode as an option for improving image quality.

[0064] In order to reduce the amount of calculations, the JPEG divides an image into blocks each consisting of 8×8 pixels and applies the DCT to each block. Consequently, an increase in compression rate may result in block noise, which in turn makes one block appear to be one pixel. In the hierarchy mode, compression is performed by using following procedure to suppress the block noise:

[0065] (1) An appropriate LPF is applied to an original image, which is then sub-sampled in the ratio of 4:1 both in the vertical and horizontal direction to produce a reduced image of a {fraction (1/16)} size.

[0066] (2) The resulting {fraction (1/16)} reduced image is divided into blocks of 8×8 pixels each, which are then subjected to the DCT for compression. As a result, a low-resolution (4:1) compressed image is obtained.

[0067] (3) A differential image is formed from the original image and an image obtained by reproducing the {fraction (1/16)} reduced image to the original size. This differential image is then subjected to an appropriate LPF and sub-sampled in the ratio of 2:1 both in the vertical and horizontal directions to produce a reduced image of a ¼ size. Thus, a 2:1 compressed image is obtained.

[0068] (4) A differential image is formed from the original image and an image obtained by reproducing the ¼ reduced image to the original size. This differential image is then similarly subjected to the DCT for compression. As a result, a high-resolution (1:1) compressed image is obtained.

[0069] The above process substantially processes blocks of 32×32 pixels, thereby reducing the block noise.

[0070] Although not supported by the current JPEG, it is studied that the Wavelet transformation will employ instead of the DCT.

[0071] The Wavelet transformation selects, instead of cosine waves, functions having a high localizability in terms both of the space and frequency as shown in FIGS. 3A and 3B, and uses base functions comprised of scaling and shifts in the selected functions.

[0072] Coefficients for the DCT are characterized to have only frequency components, while coefficients for the Wavelet transformation are characterized to have a large amount of frequency information on low-frequency components and a large amount of space information (locational information) on high-frequency components. FIG. 4 shows a diagram for intuitively understanding the abilities of an original image, the DCT, and the Wavelet transformation to resolve space and frequency information.

[0073] The Wavelet transformation allows the compression rate to be improved and restrains an increase in the amount of calculations for transformation even with an increase in target pixels, compared to the DCT, thereby eliminating the need to explicitly divide the image into blocks.

[0074] Next, the configuration of the color transformation and compression module 14 in FIG. 4 will be explained.

[0075] As shown in FIG. 1, the color transformation and compression module 14 is comprised of a low-quality color transformation section 21, a high-frequency component calculation section 22, an LPF 23, a sub-sampling section 24, a high-quality color transformation section 25, a low-frequency component calculation section 26, a combine section 27, and a code generation section 28. A frequency transformation process is divided into a high-frequency component side and a low-frequency component side. The low-quality color transformation section 21 and the high-frequency component calculation section 22, which are shown in the upper part of the figure, constitute a section for processing high-frequency components. The low-quality color transformation section 21 is provided as required and may be omitted. The LPF 23, the sub-sampling section 24, the high-quality color transformation section 25, and the low-frequency component calculation section 26, which are shown in the lower part of the figure, constitute a section for processing low-frequency components.

[0076] An original image is passed both to the section for processing high-frequency components (upper part) and to the section for processing low-frequency components (lower part). In the high-frequency component processing section, the low-quality color transformation section 21 applies a very simple (meaning a small amount of calculations) color transformation to the original image as required and the high-frequency component calculation section 22 then applies frequency transformation thereto.

[0077] In the low-frequency component processing section, the LPF (Low-Pass Filter) 23 and a sampling process executed by the sub-sampling section 24 cooperate in sub-sampling the original image to obtain a reduced image with a small number of pixels. The high-quality color transformation section 25 applies an advanced (meaning a large amount of calculations) color transformation to the reduced image. Then, the low-frequency component calculation section 26 applies frequency transformation to the reduced image, which has been subjected to the color transformation. The values obtained by this frequency transformation are sent to the combine section 27 as low-frequency coefficient.

[0078] The combine section 27 combines high-frequency coefficient, which have been obtained by the high-frequency component calculation section 22, with the low-frequency coefficient, which have been obtained by the low-frequency component calculation section 26, to obtain frequency coefficient of the original image including both the high- and low-frequency coefficient. Then, the code generation section 28 applies quantization and variable coding to the obtained values to generate a compressed image.

[0079] FIG. 5 shows a specific example of a color transformation and compression process using the DCT as frequency control.

[0080] An original image is divided at each 8×8 pixels block, one block is cut out from the divided blocks, and the cut out block A which consists of 8×8 pixels is sent to each of the high- and low-frequency component processing sections. The high-frequency processing section applies the DCT to the block A of 8×8 pixels to determine 8×8 DCT coefficients. In the low-frequency component processing section, the block A of 8×8 pixels is subjected to a filtering process of the LPF and the sub-sampling process to produce, for example, a 4×4 reduced image C. The 4×4 reduced image C is then subjected to a color transformation process D to produce a color-transformed 4×4 reduced image E. The DCT is applied to the color-transformed 4×4 reduced image E to determine 4×4 DCT coefficients F. The 4×4 DCT coefficients F are embedded in the 8×8 DCT coefficients B as 4×4 DCT coefficients for the upper left (low frequency part) of the block which constitute low-frequency components within the 8×8 DCT coefficients B. This combine process produces an 8×8 DCT coefficients G of the original image which cover both the high and low frequencies.

[0081] This process is repeated for all divided blocks of 8×8 pixels in the original image.

[0082] With this simultaneous compression and color transformation processing of separately subjecting the low- and high-frequency component sides to the frequency transformation process and executing color transformation only to the low-frequency components, which more significantly affect image quality, without executing color transformation on the high-frequency components, the number of color transformation target pixels can be reduced to increase the processing speed without seriously degrading image quality.

[0083] FIGS. 6A to 6E shows an example of division into low- and high-frequency components in the case of the DCT.

[0084] If a block of 8×8 pixels is transformed by the DCT, even a frequency area is divided into 8×8 components, as described above. In FIGS. 6A to 6E, each unit square indicates one frequency component, and in the horizontal direction, more rightward components indicate higher frequencies, while in the vertical direction, more downward components indicate higher frequencies.

[0085] In FIGS. 6A to 6E, the shaded components indicate low-frequency components for color transformation.

[0086] FIGS. 6A and 6B show a method of using a square to divide an image into low and high frequencies. FIG. 6A corresponds to the example described in FIG. 5. Both in the vertical and horizontal directions, components of frequencies lower than or equal to a maximum frequency are selected as color transformation targets. In calculating these components, an original image is sub-sampled in the ratio of 2:1 to obtain a reduced image of a ¼ size, which is then subjected to color transformation, followed by the DCT. In this case, the LPF preferably passes frequencies lower than or equal to the maximum frequency both in the vertical and horizontal directions. In this manner, by executing color transformation to an image sub-sampled in the ratio of 2:1 and then calculating frequency components, the amount of calculations required for color transformation is reduced to ¼ (½ in each of the vertical and horizontal directions). Likewise, in FIG. 6B, 4:1 sub-sampling can be provided to reduce the amount of calculations to {fraction (1/16)}.

[0087] Although boundaries can in principle be provided at locations corresponding to the square of two, the sampling ratio in this case is 4:3 and the sampling process requires interpolation (the values of coordinates at x=¾ must be calculated by means of interpolation using the values x=1 and x=2), preventing a high efficiency from being achieved.

[0088] FIG. 6D shows a method of using a rectangle for selection, which is effective in the case where, for example, horizontal information has a special meaning. In this case, the amount of calculations for color transformation is diminished to ¼ in the vertical direction ad to ½ in the horizontal direction, that is, to ⅛ in total.

[0089] FIG. 6E shows a method of using a triangle for selection. With this method, the theoretical amount of calculations is the same as in FIG. 6A, and image quality is slightly degraded. This method, however, may be effective for implementations because the components are coded in a run length manner in this order during JPEG code generation.

[0090] FIGS. 7A and 7B show an example of division into low- and high-frequency components in the case where the Wavelet transformation is used.

[0091] In the figures, the solid lines separate frequencies from one another, while the broken lines separate spatial positions in the frequency from one another. The figures show the case of 8×8 pixels.

[0092] Since the Wavelet transformation does not necessarily require block division, the coefficient actually handles larger squares or rectangles.

[0093] The selection of low-frequency components is essentially the same as in the DCT and may involve squares, rectangles, triangles, etc. Basically, the solid line portions define the boundaries. However, it is not totally meaningless that the broken line portions define the boundaries, and this is effective when an image partly having a high quality is desired. For example, in the example in FIG. 7B, a central portion of the image has a higher quality.

[0094] The image quality in this case is slightly lower than in the conventional all-pixel color transformation for the following two reasons:

[0095] (1) Since color transformation is not applied to high-frequency components, corresponding correction errors may occur.

[0096] (2) When a sub-sampled image is reproduced to the size of an original image, missing pixels are interpolated by the LPF. The color transformation, however, is not linear, so that accurate correction colors are not always provided. For example, in a 2:1 sampled image, the RGB values of adjacent pixels are assumed to be (100, 0, 0) and (110, 0, 0) before color transformation and (102, 0, 0) and (114, 0, 0) thereafter, a pixel to be reproduced at the intermediate point therebetween has, for example, a RGB value of (108, 0, 0) according to the proposed method, but strictly speaking, a transformed color for (105, 0, 0) should be placed at this portion and does not always correspond to (108, 0, 0).

[0097] For (1), however, since “high-frequency components inherently have small values” and “errors concerning high-frequency components are less perceivable to human beings”, the correction error poses no practical problem. In addition, for (2), neighborhood colors remain as neighborhood colors after transformation, whereby the above error virtually poses no problem unless the variation in color between samples has a large absolute value.

[0098] FIG. 8 shows a configuration of an image processing apparatus according to a second embodiment of the present invention. A color printer system will be illustrated and described which is comprised of a host computer 31 and a color printer 32. A disk device 33 in the host computer 31 saves images produced by an application program or image data input by an input device such as a scanner or a camera, after these images or data have been compressed and coded based on the JPEG or the like. This compressed image data is transmitted to a color expansion module 34 in a color printer 32 via a recording or communication medium. The color transformation and expansion module 34 may reside on the host side.

[0099] The color transformation and expansion module 34 executes a color transformation and expansion process on compressed image data for simultaneous color transformation and expansion. The color transformation process includes color transformation for color corrections that are compatible with the device characteristics of the color printer 32, color transformation for obtaining images that meet a user's liking, or color transformation for transforming a color space between RGB and CMY. In addition, the expansion process decodes image data compressed by means of a compression method based on the JPEG or the like in order to produce a printed image. This printed image is sent to a printer mechanical section 35 to print it out.

[0100] With the color transformation and expansion process executed by the color transformation and expansion module 34, in expanding an image compressed using frequency transformation, the following method is used: When low-frequency components are expanded, inverse frequency transformation (inverse DCT) is performed to obtain a sub-sampled image, which is then subjected to color transformation. When high-frequency components are expanded, color transformation is not executed.

[0101] The color transformation and expansion module 34 is comprised of a decode generation section 41, a high-frequency component reproduce section 42, a low-frequency component reproduce section 43, a high-quality color transformation section 44, an image size reproduce section 45, and a combine section 46.

[0102] Compressed and coded image data is subjected to variable decoding and inverse quantization by the decode generation section 41 so as to be decoded into frequency coefficient (DCT coefficients or the like). The frequency coefficient are passed to each of a high-frequency reproduce processing section (in the upper part of the figure) and a low-frequency reproduce processing section (in the lower part of the figure). The high-frequency component reproduce section 42 excludes low-frequency components from the frequency coefficient and then performs inverse frequency control to the remaining frequency coefficient to obtain an image corresponding to the high-frequency coefficient. The low-frequency component reproduce section 43 first extracts low-frequency components from the frequency coefficient and then performs inverse frequency control to the low-frequency coefficient. This reproduce operation provides a reduced image with a smaller number of pixels. The reduced image is subjected to an advanced (meaning a large amount of calculations) color transformation by the high-quality color transformation section 44 and then sent to the image size reproduce section 45. In the section 45, the sent image is subjected to interpolation processes by using an appropriate LPF so as to reproduce the size of the original image.

[0103] A final reproduced image is obtained by using the combine section 46 to combine the images produced through the two flows. The method of division into low- and high-frequency components and its effects are similar to those in Embodiment 1.

[0104] FIG. 9 shows a specific example of a color transformation and expansion process using the DCT as frequency transformation.

[0105] Eight×eight DCT coefficients A′ of compressed and coded image data are transmitted to each of the high- and low-frequency component processing sections. The high-frequency component processing section first replaces, for example, all the 4×4 DCT coefficients in the upper left of the 8×8 DCT coefficients A′ with zero to generate 8×8 DCT coefficients B′ free from the low-frequency components. The inverse DCT is then applied to the 8×8 DCT coefficients Be free from the low-frequency components to obtain a block image C′ of 8×8 pixels. The low-frequency component processing section extracts low-frequency components from the 8×8 DCT coefficients A′ to obtain 4×4 DCT coefficients D′ in the lower left of the image data. The inverse DCT is applied to the 4×4 DCT coefficients D′ to generate a reduced image E′ of 4×4 pixels. A color transformation process F′ is executed on the 4×4 reduced image E′, and the color-transformed 4×4 reduced image is then enlarged by means of interpolation to obtain a block image G′ of 8×8 pixels. The block images C′ and G′ of 8×8 pixels each are added together to obtain a final reproduced image H′.

[0106] This process is repeated for each block of 8×8 DCT coefficients.

[0107] Thus, in expanding a compressed and coded image, the number of color transformation target pixels can also be reduced to increase the processing speed without degrading image quality, using the above simultaneous expansion and color transformation processing of separately subjecting the low- and high-frequency component sides to the inverse frequency transformation process and executing color transformation only to the low-frequency components, which more significantly affect image quality, without executing color transformation on the high-frequency components.

[0108] Next, as a third embodiment, a method will be described which is used to execute color transformation and expansion to image data hierarchically coded using a hierarchical mode.

[0109] In this case, the color transformation and expansion module 34 comprises three decode and inverse DCT sections 51, 52, 53, an image size reproduce section 54, a combine section 55, a high quality color transformation section 56, an image size reproduce section 57 and a combine section 58 as shown in FIG. 10.

[0110] Coded and compressed image data transmitted and received in the hierarchical mode is comprised of, e.g. as shown in FIG. 11, coded and compressed images which are sampled at a plurality of sampling rates.

[0111] In FIG. 10, low resolution 4:1 compressed image data is decoded by the decode and inverse DCT section 51 to reproduce low resolution 4:1 compresses image data having quarter size of the original image. Medium resolution 2:1 compressed image data is decoded by the decode and inverse DCT section 52 to reproduce difference image data having half size of the original image. Furthermore, high resolution 1:1 compressed image data is decoded by the decode and inverse DCT section 53 to reproduce difference image data having same size of the original image. A size of the low resolution 4:1 compressed image data output from the decode and inverse DCT section 51 is enlarged to a size thereof by 2 times to output it into the combine section 55. In the combine section 55, the image data of which a size is enlarged by 2 times and medium resolution 2:1 difference image data output from the decode and inverse DCT section 52 are combined with each other. The image data combined by the combine section 55 is subjected to highly color transformation processing in the high quality color transformation section 56. And then, the image data which is subjected to color transformation processing is transformed such that a size thereof is same size of the original image in the image size reproduce section 57. In the combine section 58, high resolution 1:1 difference image data output from the decode and inverse DCT section 53 and image data output from the image size reproduce section 57 are combined with each other, thereby generating a complete reproduce image.

[0112] With the above described embodiment, 4:1 compressed image data and 2:1 compressed image data are subjected to color transformation processing as an example, but this invention is not limit to this. For example, only 4:1 compressed image data may be subjected to color transformation processing. In this case, in FIG. 10, a structure should be adopted in which the combine section 55 should be deleted, and the image data, of which a size is enlarged to a size by 2 times, output from the image size reproduce section 54 and medium resolution 2:1 difference image data, of which a size is enlarged to a size by 2 times by image size reproduce section (not shown), output from the decode and inverse DCT section 52 should be input to the high quality color transformation section 56.

[0113] Next, a sampling rate switching method applied to the color transformation and expansion module 34 according to the second embodiment will be explained as a fourth embodiment.

[0114] In this case, the color transformation and expansion module 34 additionally has a frequency component division determination section 61 as shown in FIG. 12. The frequency component division determination section 61 essentially divides the frequency coefficient of compressed and coded image data at a certain boundary into a low- and a high-frequency areas, but this example changes this boundary depending on a processed image.

[0115] That is, in the second embodiment, image data is fixedly divided into low- and high-frequency components areas using a method similar to that of first embodiment. Although color transformation errors may occur due to the lack of the color transformation of high-frequency components as described above, the color errors tend to increase linearly with the magnitude of high-frequency components.

[0116] Thus, in this embodiment, a method of increasing the number of color transformation target pixels, for blocks (8×8 blocks or the like) with high frequency components of a large magnitude is adopted.

[0117] A specific example of a procedure is shown below.

[0118] First, in a default, an area of low-frequency components are selected as shown by the shaded portion in FIG. 13A. If an area of high-frequency components, which is not shaded portion, have no large DCT coefficient value, the image data is divided in this condition. In this case, the image is subjected to color transformation in the ratio of 4:1 both in the vertical and horizontal directions, and the amount of calculations corresponds to {fraction (1/16)} of the image data.

[0119] If the filled area in FIG. 13B has a large DCT coefficient value, then the shaded area in FIG. 13B is classified into the area of low-frequency components. In this case, the image is subjected to color transformation in the ratios of 4:1 in the vertical direction and of 2:1 in the horizontal direction, and the amount of calculations corresponds to ⅛ of the image data. That is, an area of low-frequency components is selected so as to define an area having large DCT coefficient value by a rectangle. The sampling interval varies depending on the boundaries shown by the thick lines in FIG. 13C, a rectangle may be selected based on these boundaries.

[0120] Determination of the magnitude of high-frequency components includes:

[0121] comparison of each component with a threshold and

[0122] comparison the sum (or square sum) of values in a partial rectangle with a threshold value.

[0123] When a color image is used, 8×8 pixels exist in each RGB so that these pixels may be added together.

[0124] In this manner, the frequency component division determination section 61 dynamically determines a boundary position for each block.

[0125] Next, a sampling rate switching method (Wavelet) applied to the color transformation and expansion module 34 according to the second embodiment will be described as a fifth embodiment.

[0126] FIG. 14 shows a configuration of the color transformation and expansion module 34 to which the sampling rate switching method (Wavelet transformation) is applied. The color transformation and expansion module 34 has a component division rate determination section 62 in addition to the structure of the second embodiment.

[0127] The component division rate determination section 62 carries out essentially the same processing as in Embodiment 4 and provides such control as to increase a color transformation target area if any high-frequency component has a large value. According to the Wavelet transformation, fixed spatial division is avoided in contrast to the DCT, which uses blocks of 8×8 pixels for such division, and components individually correspond to a certain area on a space, so that the components can be divided more flexibly than in the DCT.

[0128] FIG. 15 shows an example of component division. For simplification, this figure shows an example of one dimension and sixteen pixels.

[0129] Standard low-frequency components are indicated by the shaded portion (//) and a high-frequency component of a large value exists in the filled portion. The shaded components are passed to the low-frequency reproduce section, where they are subjected to color transformation at the sampling rate of 4:1 and then reproduced to the original image size. Further, the coefficient of the filled portion and lower-frequency components including the same position as this coefficient (the reversely shaded portion) are also passed to the low-frequency reproduce section, where they are subjected to color transformation (the color transformation cannot be calculated correctly without the reversely shaded portion) and the relevant portion is reproduced to the original image size. This partial image is replaced for the corresponding portion of the recovered low-frequency image. (In fact, the range of the relevant portion varies depending on the selection of Wavelet base functions.)

[0130] The other portions (white portions) are passed to the high-frequency reproduce section, where they do not undergo color transformation.

[0131] In this manner, if any high-frequency component has a large value, the boundary is changed so as to include the frequency component value of the corresponding spatial position within the color transformation target. Consequently, a more accurate color transformation can be applied to important portions.

[0132] The color transformation and compression process and color transformation and expansion process described in each of the above embodiments can be implemented either by hardware or by software. If the processes are implemented by software, the processing procedure corresponds to the flow of processing described in the block diagrams. In addition, color transformation parameters may be determined beforehand depending on the characteristics of the device, but color transformation can be performed to a user's liking by providing a function of changing the parameters in accordance with the user's operations.

[0133] As described above, according to the present invention, the image compression or expansion process and the color transformation process are simultaneously executed to enable the number of color transformation target pixels to be reduced without significantly degrading image quality, thereby realizing vary fast processing.

[0134] Additional advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described herein. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.