Automatic image enhancement of halftone and continuous tone images

[0002] 1. Field of Invention

[0003] The invention relates to the field of image processing. In particular, the invention relates to automatic image enhancement of halftone and continuous tone images.

[0004] 2. Description of Related Art

[0005] Automatic image enhancement (AIE) is a technology that operates on sampled images and invokes image processing operations on the sampled images based on analysis of the images. Automatic image enhancement automatically corrects image deficiencies through adjustments of exposure, contrast, sharpness, color balance and saturation. After analysis of the image is performed to determine what action, if any, to perform on the image, the appropriate image processing is applied to the image. In the past, automatic image enhancement was done almost exclusively on continuous tone (contone) images, such as photographs.

[0006] In the office environment, it is much more likely to encounter a halftone original than a contone original. Halftone originals are often degraded when scanned or reproduced, for example, by photocopying.

[0007] This invention provides a system and method for automatically enhancing degraded halftone images as effectively as degraded contone images can be enhanced.

[0008] This invention provides a system and method that automatically enhances halftone images using a spatial filter.

[0009] This invention provides a low-pass spatial filter as the spatial filter.

[0010] This invention provides a pyramid filter as a two-dimensional embodiment of the spatial filter.

[0011] This invention provides a triangular filter as a one-dimensional embodiment of the spatial filter.

[0012] Due to the human eye's inability to individually resolve the pixels of a halftone image at normal viewing distance, a halftone image appears to have tone gradations like a contone image. However, pixel-by-pixel analysis, or statistics collection, of the halftone image may produce a different result than the same pixel-by-pixel analysis of a similarly-degraded contone image. This difference usually produces disappointing results when automatic image enhancement is applied to a degraded halftone image.

[0013] To avoid the disappointing results of applying automatic image enhancement to a degraded halftone image, the system and method of this invention smooth the scanned halftone image using a low-pass spatial filter prior to collecting statistics, to obtain more accurate information about the scanned halftone image. Although many low-pass spatial filter configurations will improve conventional automatic image enhancement methods, a 7×7 two-dimensional pyramid low-pass filter is an example of a low-pass spatial filter which provides satisfactory results. A 15×1 one-dimensional triangular lowpass filter can also be used in the invention. Compared to two-dimensional filtering, one-dimensional filtering is less expensive since it does not need scan line buffers to store multiple scan line image data. Using a 15×1 one-dimensional filter in the invention provides acceptable results so that it is preferable in a cost sensitive application.

[0014] By applying the low-pass spatial filter to the scanned halftone image, the image density of each pixel is adjusted to eliminate large image density differences between adjacent pixels. As a result, the halftone variation is “smoothed out”, which provides a better starting point for the statistics collection.

[0015] The apparatus and method of the invention can also be applied to continuous tone images and it is not necessary to designate whether the original image is a halftone image, a continuous tone image or a combination of halftone and continuous tone images.

[0016] These and other features and advantages of this invention are described in or are apparent from the following detailed description of the preferred embodiments.

[0017] The invention will be described in relation to the following drawings in which like reference numerals refer to like elements, and wherein:

[0018] FIG. 1 is a block diagram of an automatic image enhancement system according to an embodiment of the invention;

[0019] FIG. 2 illustrates an exemplary 7×7 pyramid low-pass filter;

[0020] FIG. 3 is a three-dimensional bar graph representing the weights of the filter shown in FIG. 2;

[0021] FIG. 4 shows a portion of an exemplary scanned halftone image having 81 pixels and the scanned image density of each pixel;

[0022] FIG. 5 is a three-dimensional bar graph representing the image densities of pixels (D)(4)-(F)(6) shown in FIG. 4;

[0023] FIG. 6 shows the product of the image densities and the corresponding filter weights for pixels (A)(1)-(G)(7) shown in FIG. 4 when the filter of FIG. 2 is centered over the pixel (D)(4);

[0024] FIG. 7 shows the resulting image densities for pixels (D)(4)-(F)(6) after filtering;

[0025] FIG. 8 is a three-dimensional bar graph representing the resulting image density of pixels (D)(4)-(F)(6) shown in FIG. 7;

[0026] FIG. 9 illustrates an exemplary 15×1 triangular low-pass filter;

[0027] FIG. 10 is a three-dimensional bar graph representing the weights of the filter shown in FIG. 9;

[0028] FIG. 11 shows a portion of an exemplary scanned halftone image having 72 pixels and the image density of each pixel;

[0029] FIG. 12 is a three-dimensional bar graph representing the image densities of pixels (H)(1)-(K)(4) shown in FIG. 11;

[0030] FIG. 13 shows the product of the image densities and the corresponding filter weights for pixels (A)(1)-(O)(1) shown in FIG. 11 when the filter of FIG. 9 is centered over the pixel (H)(1);

[0031] FIG. 14 shows the resulting image densities for pixels (H)(1)-(K)(4) after filtering;

[0032] FIG. 15 is a three-dimensional bar graph representing the resulting image densities of pixels (H)(1)-(K)(4) shown in FIG. 14;

[0033] FIG. 16 is a flowchart outlining one embodiment of the halftone image automatic image enhancement method according to this invention;

[0034] FIG. 17 is an example of a halftone image before automatic image enhancement;

[0035] FIG. 18 is an example of the halftone image shown in FIG. 17 after automatic image enhancement without using a low-pass filter and method according to this invention; and

[0036] FIG. 19 is an example of the halftone image shown in FIG. 17 after automatic image enhancement in accordance with the invention.

[0037] The system and method for automatic image enhancement of halftone images of this invention allows conventional automatic image enhancement methods, designed for continuous tone images, to be usable with scanned halftone images. U.S. Pat. Nos. 5,363,209, 5,371,615, 5,450,217, 5,450,502, 5,414,538, 5,347,374, 5,357,352, 5,045,952 and 5,581,370 and pending U.S. patent application Ser. No. 08/854,279, each assigned to the same assignee as this application and each incorporated herein by reference in its entirety, describe such continuous tone automatic image enhancement methods and systems that automatically enhance continuous tone images.

[0038] Thus, a detailed description of these continuous tone automatic image enhancement systems and methods will be omitted from the following description of the system and method according to this invention. However, it should be appreciated that once a scanned halftone image has been processed according to the system and/or method of this invention, any of the continuous tone automatic image enhancement systems and methods described above, and any other known or later developed continuous tone automatic image enhancement systems or methods, can be used with or applied to the resulting processed image.

[0039] FIG. 1 shows a functional block diagram of an automatic image enhancement processing system 100 according to this invention. As shown in FIG. 1, the automatic image enhancement processing system 100 includes an input/output interface 120, a controller 130, a low-pass filter circuit 140, an image analyzer module 150, an image processor module 160, a memory 170, and a block 180 representing any other image processing modules that may be implemented in the automatic image enhancement processing system 100 when programed to perform the automatic image enhancement processing system and method according to this invention. Each of the input/output interface 120, the controller 130, the low-pass filter circuit 140, the image analyzer and image processor modules 150 and 160, the memory 170, and the block 180 are connected by all internal control and data bus 190. A number of image data sources, such as a scanner 210, a host computer 240 and a memory 230, and a number of image data links, such as the host computer 240, the memory 230, an image forming device 220, and a display device 250 are connected to the automatic image enhancement processing system 100. Each of the scanner 210, the image forming device 220, the host computer 240, the memory 230, and the display device 250, are connected to the automatic image enhancement processing system 100 through the input/output interface 120.

[0040] In operation, a halftone image 200 formed on an image recording medium is scanned by the scanner 210 to generate electronic image data of the halftone image 200. Although this example uses a halftone original 200, the system can also be applied to a continuous tone image or a combination halftone/continuous tone image. The electronic image data representing the halftone image 200 is output from the scammer 210 to the input/output interface 120. The electronic image data representing the halftone image 200 received by the input/output interface 120, is transmitted under the control of the controller 130 over the data-control bus 190 to the memory 170. Once all of the electronic image data representing the halftone image 200 is stored in the memory 170, and any other desired pre-processing is applied to the electronic image data stored in the memory 170, blocks of the electronic image data surrounding a current pixel of interest are output to the low-pass filter circuit 140 on a pixel by pixel basis from the memory 170. However, it should be appreciated that the electronic image data representing the halftone image 200 can be directly input to the low-pass filter circuit 140 from the input/output interface 120. The low-pass filter circuit 140 “smoothes out” the electronic image data by eliminating the high frequency components of the electronic image data. The smoothed electronic image data output from the low-pass filter circuit 140 is input to the memory 170 where it is stored separately from the original electronic image data representing the halftone image 200. The smoothed electronic image data stored in the memory 170 is then output to the image analyzer module 150, where it is analyzed by the image analyzer module 150 to determine what, if any, processing should be performed on the original electronic image data. It should also be appreciated that the smoothed electronic image output from the low-pass filter circuit 140 can also be directly input to the image analyzer module 150.

[0041] After analysis, or statistics collection, by the image analyzer module 150, the original electronic image data is processed by the image processor module 160 based on the results of the analysis performed by the image analyzer module 150. Both the image analyzer module 150 and the image processor module 160 perform functions well known in the art, such as the various automatic image enhancement processes disclosed in the incorporated patents and applications. The processed electronic image data output from the image processor module 160 can be output to the image forming device 220, the other image processing modules represented by the blocks 180, and/or the memory 170.

[0042] It should be appreciated that the sampled image automatic image enhancement system 100 shown in FIG. 1 is preferably implemented using a general purpose computer. However, the sampled image automatic image enhancement system 100 can also be implemented using a special purpose computer, a programmed microprocessor or microcontroller and peripheral integrated circuit elements, an ASIC or other integrated circuit, a digital signal processor, a hardwired electronic or logic circuit such as a discrete element circuit, a programmable logic device such as a PLD, PLA, FPGA or PAL, or the like. In general, any device, capable of implementing a finite state machine that is in turn capable of implementing the flowchart shown in FIG. 16, can be used to implement the sampled image automatic image enhancement system 100.

[0043] It should further be appreciated that the sampled image automatic image enhancement system 100 can be incorporated into the image forming device 220 or the scanner 210, such as incorporating the sampled image automatic image enhancement system 100 into a scanner, a facsimile device, a digital photocopier or a printer. In the first case, a previously sampled halftone image may be received by the low-pass filter from the memory 230 or the host computer 240. For example, the host computer 240 may be a remotely located personal computer connected to the sampled automatic image enhancement system 100 over a local area network, a wide area network, an intranet, the Internet or any other distributed processing and storage network. Similarly, the memory 230 may be a memory of a remotely located server connected to the sampled image automatic image enhancement system 100 over a local area network, a wide area network, an intranet, the Internet or any other distributed processing and storage network.

[0044] In the second case, the processed halftone image output by the sampled image automatic image enhancement system 100 may be output to the host computer 240, the memory 230 and/or the display 250. Each of these devices may be connected to the sampled image automatic image enhancement system 100 over a local area network, a wide area network, an intranet, the Internet or any other distributed processing and storage network.

[0045] It should also be appreciated that the scanner 210, the sampled image automatic image enhancement system 100 and the image forming device 220 can be combined into a single device, such as a digital photocopier.

[0046] FIGS. 2 and 3 show one example of a 7×7 pyramid low-pass filter 300 that can be used in the invention. FIG. 2 shows the weight distribution of the filter 300. The center position of the filter 300 is weighted with a value of 16 and each of the corner positions of the filter 300 are weighted with a value of 1. The values of the remaining positions change smoothly. FIG. 3 is a three-dimensional bar graph representing the weight distribution of the filter 300 shown in FIG. 2. The weight distribution of the 7×7 filter shown in FIGS. 2 and 3 is only an example of one appropriate filter usable with the automatic image enhancement system 100 of this invention. Other low-pass filters of different dimensions and different weights can be appropriately used in the system and method of this invention.

[0047] FIG. 4 shows a portion of an exemplary scanned halftone image having 81 pixels referenced by column designators (A)-(I) and row designators (1)-(9). The number in each pixel represents the image density of that pixel. In this example, the image density of each pixel is in the range of 0-255. FIG. 5 is a three-dimensional bar graph representing the image densities of pixels (D)(4)-(F)(6) within the box 310 of FIG. 4.

[0048] In applying the 7×7 filter 300 to the image data of the halftone image, the filter 300 is successively centered on each pixel of the image. For example, the value of pixel (D)(4) after filtering is determined by centering the filter 300 on the pixel (D)(4) so that the filter 300 covers pixels (A)(1)-(G)(7). The image density of each pixel (A)(1)-(G)(7) is then multiplied by the corresponding filter weight. For example, the image density of pixel (A)(1), 234, is multiplied by 1, the image density of pixel (B)(1), 220, is multiplied by 2, the image density of pixel (B)(2), 210, is multiplied by 4 and the image density of pixel (D)(4), 180, is multiplied by 16. FIG. 6 shows the resulting weighted image densities for all 49 pixels (A)(1)-(G)(7). The sum of the 49 weighted image densities shown in FIG. 6 is then divided by the sum of the 49 weights of the 7×7 pyramid filter 300 shown in FIG. 2 to determine a resulting image density after filtering for the pixel (D)(4). For the 7×7 pyramid filter used in this example, the sum of the filter weights is 256. The sum of the 49 weighted image densities shown in FIG. 6 is 14,113. Therefore, the resulting image density after filtering for pixel (D)(4) equals 55 (14,113 divided by 256).

[0049] FIG. 7 shows the same sample image area shown in FIG. 4 after applying the 7×7 low-pass filter 300 shown in FIG. 2. The box 320 of FIG. 7 contains pixels (D)(4)-(F)(6). FIG. 7 shows only the resulting image densities after filtering of pixels (D)(4)-(F)(6). When an actual image is filtered, the 7×7 low-pass filter 300 is applied to all of the pixels of the image, except for the outermost three rows and three columns of pixels along each edge of the image.

[0050] FIG. 8 is a three-dimensional bar graph representing the resulting image density, after filtering pixels (D)(4)-(F)(6), within the box 320 shown in FIG. 7. FIG. 8 corresponds to FIG. 5, in that FIG. 5 shows the image densities of pixels (D)(4)-(F)(6) before filtering, while FIG. 8 shows the image densities of the same image pixels after filtering. FIG. 8 shows a smoother transition between image pixels, due to smaller differences between adjacent pixels, than does FIG. 5. For example, the difference between the image densities of pixels (D)(4) and (D)(5) in FIGS. 4 and 5 is 155 (180-25) whereas the corresponding difference in FIGS. 7 and 8 is 10 (55-65).

[0051] As discussed above, a one-dimensional filter 400 may also be used in the invention. FIGS. 9 and 10 show one example of a 15×1 low-pass filter 400. FIG. 9 shows the weight distribution of the filter 400. The center position of the filter 400 is weighted with a value of 8 and each of the end positions of the filter 400 are weighted with a value of 1. The values of the remaining positions change smoothly. FIG. 10 is a three-dimensional bar graph representing the weight distribution of the filter shown in FIG. 9. The weight distribution of the 15×1 filter 400 shown in FIGS. 9 and 10 is only an example of one appropriate filter 400. Other low-pass filters 300 and 400 of different dimensions and different weights can be appropriately used in the system and method of this invention.

[0052] FIG. 11 shows a portion of an exemplary scanned halftone image having 72 pixels referenced by column designators (A)-(R) and row designators (1)-(4). Similarly to FIG. 4, the number in each pixel represents the image density of that pixel. In this example, the image density of each pixel is in the range of 0-255. FIG. 12 is a three-dimensional bar graph representing the image densities of pixels (H)(1)-(K)(4) within the box 410 in FIG. 11.

[0053] The one-dimensional filter 400, in this example a 15×1 filter 400, is applied to the image in a similar manner as the two-dimensional filter 300 discussed above. FIG. 13 corresponds to FIG. 6 in that it shows the product of the image density of a pixel and the corresponding filter weight for each of the 15 pixels processed in one application of the 15×1 filter 400. The image density after filtering of, for example, the pixel (H)(1), is determined by dividing the sum of the 15 weighted image densities shown in FIG. 13 by the sum of the filter weights of the 15×1 filter 400 shown in FIG. 9. In this example, the resulting image density after filtering for pixel (H)(1) is 79 (5,029 divided by 64).

[0054] FIG. 14 shows, in box 420, the image density after filtering of pixels (H)(1)-(K)(4). FIG. 15 is a three-dimensional bar graph representing the resulting image density after filtering of pixels (H)(1)-(K)(4) within the box 420 shown in FIG. 14.

[0055] A comparison of FIGS. 5 and 8 shows the smoothing effect of the two-dimensional pyramid filter. Before filtering, FIG. 5 shows large differences in image density between adjacent pixels. In contrast, after filtering, FIG. 8 shows much smaller differences in the image density between adjacent pixels. A comparison of FIGS. 12 and 15 shows a somewhat similar smoothing effect as explained above in reference to FIGS. 5 and 8. However, because FIGS. 12 and 15 correspond to the use of a one-dimensional filter, the smoothing of the image density of pixels is only along one direction (the x direction in FIG. 15). Because FIG. 8 corresponds to the use of a two-dimensional filter, smoothing takes place in both the x and y directions in FIG. 8.

[0056] By smoothing, i.e., decreasing the difference between the image densities of any two adjacent pixels in the image, low-pass filter 300 or 400 creates a filtered image that is more like a contone image than the original halftone image before filtering. Because it is more like a contone, the filtered image is usually a better input image for image analysis than is the halftone before filtering.

[0057] FIG. 16 is a flow chart outlining one method for preparing a scanned halftone image for automatic image enhancement according to this invention. Starting at step S100, control continues to step S110, where the electronic image data representing the degrading halftone image is input. Next, in step S120, a first pixel of interest is selected from the electronic image data input in step S110. Then, in step S130, a set of pixels is determined that includes the pixel of interest and pixels neighboring the pixel of interest. Control then continues to step S140.

[0058] In step S140, the image density C_iof each pixel in the set of pixels determined in step S130 is multiplied by a filter weight F_icorresponding to the spatial position of that pixel relative to the pixel of interest. A sum S is then set equal to the sum of these products. Next, in step S150, an image density I is set equal to the sum S divided by the sum o f the filter weights F_i. Then, in step S160, the image density of the pixel of interest is set equal to I. Control then continues to step S170.

[0059] In step S170, the control system determines if there are any more pixels that need to be filtered. If there are more pixels to be filtered, control continues to step S180. Otherwise control jumps to step S190.

[0060] In step S180, a next pixel is selected as the pixel of interest. Control then jumps back to step S130. In step S190 the filtered electronic image data representing the degraded halftone image is stored and/or output. Then, in step S200, the process stops.

[0061] Automatic image enhancement of degraded halftone images according to the invention results in an improved halftone image compared to the original degraded halftone image and/or the original degraded halftone image after conventional automatic image enhancement. FIG. 17 shows an original halftone image. FIG. 18 shows the image of FIG. 17 after automatic image enchancement without using a low-pass filter according to this invention. FIG. 19 shows the image of FIG. 17 after automatic image enhancement using a low-pass filter according to this invention. The improvement in image quality that results from using low-pass filtering according to this invention is apparent from a comparison of FIGS. 18 and 19. Although a benefit of the invention is shown using a monochrome halftone image as an example, it should be recognized that the invention can also be applied to multicolor halftone images and continuous tone images.

[0062] While the invention has been described in conjunction with the specific embodiments outlined above, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, the preferred embodiments of the invention as set forth above are intended to be illustrative and not limiting. Various changes may be made without departing from the spirit and scope of the invention as defined herein.