Image Processing Method to Remove Halftone Screens

This invention is related to the following applications, filed on even date with the present application: Ser. No. 07/894,858, for Classification to Change Exposure Within a Cell of Different Pixels; Ser. No. 07/894,857, for Halftone Dot Arrangement in Gray Level Halftone Printing; Ser. No. 07/894,8595, for A Method and Arrangement For Providing a Default Mode in Digital Copying; Ser. No. 07/895,555, for A Method and Arrangement For Locally Switching Gray Dot Types to Reproduce an Image With Gray Level Printing; Ser. No. 07/895,985, for Multi-Bit Rendering Method and Arrangement for Continuous Tone Picture Representation and Printing; Ser. No. 07/895,554, for A Method and Apparatus For Imbedding Controlled Structure For Gray Scale Rendering; and Ser. No. 07/895,988, for Line Screen Design for Gray Scale Rendering. Each of these related applications is hereby expressly incorporated by reference into this application.

The present invention relates to the field of encoding pictorial imagery for reproduction on display or printing systems, and more particularly, to method and apparatus for removing halftone screens while maintaining the contrast of text in a digital image reproduction environment.

In the area of digital printing (the term "printing" is used to encompass both printing and displaying throughout), gray level has been achieved in a number of different manners. The representation of the intensity, i.e., the gray level, of a color by binary displays and printers has been the object of a variety of algorithms. Binary displays and printers are capable of making a mark, usually in the form of a dot, of a given, uniform size and at a specified resolution in marks per unit length, typically dots per inch. It has been common to place the marks according to a variety of geometrical patterns such that a group of marks when seen by the eye gives a rendition of an intermediate color tone between the color of the background (usually white paper stock) and total coverage, or solid density.

Continuous tone images contain an apparent continuum of gray levels. As an approximation to continuous tone images, pictorial imagery has been represented via binary halftone technologies. In order to record or display a halftone image with a scanning system, one picture element of the recording or display surface consists of a j.times.k matrix of sub-elements where j and k are positive integers. A halftone image is reproduced by printing the respective sub-elements or leaving them blank, in other words, by suitably distributing the printed marks.

Halftone image processing algorithms are evaluated in part, by their capability of delivering a complete gray scale at normal viewing distances. The capability of a particular process to reproduce high frequency rendition (fine detail) with high contrast modulation makes that procedure superior to one which reproduces such fine detail with lesser or no output contrast.

Another method of producing gray levels is provided by gray level printing. In such a method, each pixel has the capability to render several different dot sizes. The dot size for a pixel is a function of the exposure time provided an LED element corresponding to that pixel. The longer the exposure time, the more toner is attracted to that particular pixel. See, for example, U.S. Pat. No. 4,680,645 for a method of rendering gray scale images with variable dot sizes.

There are two major concerns in rendering a continuous tone image for printing: (1) t he resolution of image details, and (2) the reproduction of gray scales in a binary halftone representation scheme, these two fundamental factors compete with each other. The more gray levels that are rendered, the larger is the halftone cell. Consequently, coarse halftone line screens are provided, with the attendant poor image appearance. Hence, a compromise is made in rendering between the selection of line resolution and gray scales in binary halftone printing. However, in gray level halftone printing, one can satisfy both resolution and gray level requirements. In gray level printing, the same number of addressable dots are present, and there is a choice of dot sizes from one dot-size of 1 bit/pixel to 16 different dot-sizes of 4 bit/pixel. An image could then be rendered with 133 line screens and 128 gray scales of higher quality image. Although providing higher image quality with respect to line resolution and tonal scales, gray level halftoning presents its own dot rendering issues.

A problem exists in the application of a gray level rendering technique to a document that contains different types of images: text, halftone, and continuous tone. These different types of images create different rendering problems, based on a trade-off between tone scales and detail resolution. For example, with text, the number of tone scales is not as important as providing a smooth text edge, whereas the opposite holds true for continuous tone images. Providing a single type of gray level halftone rendering technique to a document that contains two or more types of images may lead to the production of a document in which one or more of the different types of images are reproduced unsatisfactorily.

When scanning a document, image processing techniques have been applied to convert a gray scale image into an image representation which a printer can accept (either binary format or gray level format). In this scanning process, text areas, line drawing and halftone pictures are indistinguishable from each other, and all appear to be a gray scale image. Improper conversion process create artifacts in the hardcopy such as a jagged boundary in the text area, or a Moire pattern in the halftone region. To overcome this, intelligent processes have been developed to segment the image into different regions of text, line drawing, and picture. Different conversion processes for the individual segments were then applied to these segments to restore the original document. However, these segmentation and conversion processes unduly complicate the digital copying process.

In the digital copier environment, a document is scanned into a digital pixel value and stored for later rendition process. The document may consist of different image types such as: text, halftone picture, and continuous tone picture. Each type has its own characteristics. All of them are needed to be rendered into 1-bit binary or multi-bit gray scale before printing. If all different types are treated the same in the rendition, a text may be screened and a halftone picture may be full of beating patterns (ie. Moire patterns). This results in a poor quality of print. Therefore, it would be preferred to remove screens with some sacrifices on the contrast of the text before rendering. It needs a special filter design to accomplish this goal.

This and other needs are met by the present invention which provides a method of reproducing an original image, comprising scanning an original image to produce a digitized image, performing a local structure analysis of the digitized image, and selectively applying a descreening filter to a region of the digitized image based on the results of the local structure analysis to remove a specific frequency of the digitized image caused by halftone screens. The descreening filter bypasses regions of the digitized image such that these bypassed regions remain unfiltered. The filtered regions and the bypassed regions are printed as a single, unified reproduction of the original image.

In certain embodiments of the invention, the step of performing the local structure analysis includes the steps of establishing a moving window for the digitized image around a central pixel, determining if the moving window is a low contrast window or a high contrast window, determining if there is a consistent transition in the moving window around the central pixel, and causing the application of the descreening filter to the moving window when the moving window is a high contrast window and there is not a consistent transition in the moving window around the central pixel.

The earlier stated needs are also achieved by another embodiment of the invention which provides an arrangement for reproducing an original image, comprising a scanner which scans and digitizes the original image, and a controller coupled to the scanner to receive the digitized original image. The controller performs a local structure analysis of the digitized original image, selectively applies a descreening filter to a region of the digitized original image based on the results of the local structure analysis to remove a specific frequency of the digitized image caused by halftone screens, bypasses the descreening filter with regions of the digitized image such that these bypassed regions remain unfiltered, and produces a first signal containing filtered regions and unfiltered regions, this first signal corresponding to the digitized original image. The arrangement includes a printer coupled to the controller to receive the first signal and produces a gray level halftoned reproduction of the original image.

The method and arrangement of the present invention, which selectively applies a descreening filter, removes halftone screens from images while maintaining the contrast of text when reproducing an original image.

Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.

FIG. 1 shows a block diagram of an arrangement for reproducing an image, constructed in accordance with an embodiment of the present invention.

FIG. 2 shows an example of a 4.times.4 cell with indications of gray level for each pixel.

FIG. 3 illustrates an exemplary 3-bit gray halftone dot layout according to a full dot type embodiment of the present invention.

FIG. 4 shows a cell with dots that have been formed.

FIG. 5 illustrates an exemplary halftone dot mask.

FIG. 6 shows a 3-bit gray halftone dot layout according to a partial dot type embodiment of the present invention.

FIG. 7 shows a 3-bit gray halftone dot layout according to a mixed dot type embodiment of the present invention.

FIG. 8 shows a 4-bit gray halftone dot layout according to a mixed dot type embodiment of the present invention.

FIG. 9 shows a basic flow diagram of a method of the present invention.

FIG. 10 shows the design of the descreening filter of the present invention.

FIG. 11 is a flow chart of an embodiment of the method of the present invention.

FIG. 12 is a flow chart of a unified rendering method that can be used in the present invention.

FIG. 1 illustrates an arrangement which reproduces a document. The document 10 can contain different types of images on the same document. For example, document 10 may contain both text and continuous tone areas, and may also contain halftone areas.

The document 10 is scanned in and digitized by a conventional scanner 12, which operates to provide digital signals representative of the densities of the areas of the document 10 corresponding to a pixel. These signals are sent to a memory (or buffer) 14. Under the direction of a controller 16, these signals may be modified and provided as gray level signals through a framestore 17 to a printer 18 and/or a display 20 for each pixel. The printer 18 and/or display 20 will then reproduce the document 10 by energizing each of the individual pixels according to the gray levels as modified (or not modified) by the controller 16. The memory 14, the controller 16, the framestore 17, the printer 18 and the display 20 are of conventional hardware design.

In gray level printing, each pixel has the capability to render several different dot sizes, and thus different gray levels. However, instead of simply providing each pixel with an independent gray level, several pixels may be organized together to form a superpixel, or cell. Each of the pixels in a cell is then provided with a gray level. The human visual response integrates the various gray levels of the individual pixels in the cell to a single perceived gray level for the cell. This is similar to the basic concept of binary halftoning. The number of tone scales for a cell is increased greatly, however, due to the number of different gray levels available for each pixel. For example, instead of only the two levels provided in binary halftoning for each pixel, eight levels can be provided with gray level printing for each pixel in a cell (3 bits/pixel). When the cell is made up of 4.times.4 pixels, for example, the gray level printing allows 121 different gray shades to be rendered for that cell. An example of a 4.times.4 cell 28 with numbers that represent gray levels for each pixel is shown in FIG. 2.

The formation of the dots in the pixels of a cell can be performed in a number of different manners to achieve different desired results. The dots can be formed as "full" dot, "partial" dot, or "mixed" dot to provide gray level halftoning.

FIG. 3 illustrates an example of a 3-bit gray halftone dot layout for a full dot type formation. Also illustrated are seven different pixel-dot sizes, corresponding to the sizes that each individual pixel-dot can obtain. There are 57 possible gray levels for the exemplary eight element cell 30 shown here. An example of the formation of a cell that is at gray level 12 will now be given.

The pixel circled in level 1, reference numeral 1, is formed to dot-size 1 in level 1. (Only one cell will be described, although the pixels of other cells will be changed according to the same layout, as shown in FIG. 3). The dot at this pixel grows larger and larger as the levels increase from level 1 to level 2 all the way to level 7. One can see that this circled pixel increases in value from 1 to 7 as the levels increase. If the desired gray level for the cell 30 was 7, then the formation of dots would be completed once the circled pixel has reached the dot-size of 7 in level 7. In this example, however, the gray level for the cell 30 is desired to be 12. At gray level 7, the circled pixel has reached its maximum dot-size, so that a dot at another pixel must now start forming. This dot starts forming at the pixel indicated with a square around it in level 1, with the numeral 8.

The dot formation process continues, with the dot at this second pixel growing larger and larger as the levels again increase from level 1 to level 5. The formation process stops at level 5, since the pixel has now reached the value of 12. The halftone cell 30 now contains, as seen in FIG. 4, a dot of dot-size 7, and a dot of dot-size 5. The extension of this formation process to 57 gray levels is easy to see from this example.

The full dot type process thus involves forming dots at the highest priority pixels to their maximum allowable dot-size before beginning the formation of the dots at the next highest priority pixels. An exemplary halftone dot mask 32 with pixel priorities indicated is shown in FIG. 5. Different matrix sizes, cell shapes and priorities can be used for the cells than that illustrated in FIG. 3, without departing from the spirit and scope of the present invention.

In the electrophotographic process, the full dot type formation process is favored because it forms stable dots and exhibits less granularity (halftone printing noise). Another method which carries more information detail than full dot, but at the cost of less stable dots, is the partial dot type, described below.

A 3-bit gray halftone dot layout for the partial dot type formation process is shown in FIG. 6. In this process, the cell 34 is built by providing a dot of the same size to each pixel in the cell to the extent possible, before building up the dot at any particular pixel to the next larger size. Thus, for a gray level of 6 for the cell 34, the circled pixel in level 1 would have a dot formed at that pixel with a dot-size of 1. For larger gray levels, for example gray level 13, each of the pixels in the cell 34 would be built up to at least dot-size of 1. The pixels indicated with a square around them in level 2 would be built up to have a dot-size of 2.

The partial dot formation process can thus be seen to spread out the information over the cell, and therefore carries more information detail than the full dot. It does suffer from less stable dots and more granularity, however.

The mixed dot type, discussed below, combines the merits of both the full dot and the partial dot types in gray level halftoning. A number of different processes can be provided to combine the full dot type and the partial dot type, with the specific mixed dot type being chosen based on which renders an image with more smoothness, less graininess, and more image details. Suggested strategies are: 1) build small stable dots in the highlight (toe) region; 2) keep tone response linear in the mid-tone region; 3) reduce dot structure in the shadow (shoulder) region and render more details. Based on these considerations, a specific mixed dot type can be chosen by one of ordinary skill in the art to optimize stable dots, more image detail and less graininess.

An example of a specific mixed dot type 3-bit gray halftone dot layout is illustrated in FIG. 7. As can be seen, until gray level 41 is reached, the pixels are constrained from growing beyond dot-size of 5. The pixels grow in a full dot type process, with the pixel circled growing to a dot-size of 5, with the pixel that is squared then starting to grow in size. Once all of the pixels in the cell have attained a dot-size of 5, corresponding to gray level 40, the cell then increases in gray level by using a partial dot type process. In other words, each of the pixels in the cell must grow to a dot-size of 6 before any of the pixels begins growing to a dot-size of 7.

An example of a 4-bit gray halftone dot layout for mixed dot type is illustrated in FIG. 8. The formation of the dots is the same in concept to that illustrated in FIG. 7. Because there are 15 dot sizes available for each pixel, 121 gray levels for an eight element cell are obtainable.

Although any one of the three dot types could be used to produce a satisfactory continuous tone image, the mixed dot type is the best choice for continuous tone rendering. For scanned text and halftone, the full dot type creates a screen structure in the background of text and a Moire pattern in the halftone. The mixed dot type also creates a screen structure in the background of text and creates a Moire pattern in the halftone, though weaker than that created by the full dot type. By contrast, the partial dot renders well on both text and halftone.

The unified rendering technique used in the present invention uses both partial dot and mixed dot types according to local image content so that text, halftone and continuous tone images are all reproduced well.

An exemplary embodiment of a method for unified image rendering is illustrated in FIG. 12. A document 10 is scanned in step 80, using the scanner 12 as shown in FIG. 1. The controller 16 collects statistical information from each dot region (e.g. 4.times.4 pixels) in step 82. This statistical information can include contrast, variance, variation or roughness, for example. The contrast is defined as the difference between local maximum intensity and local minimum intensity. The variation is defined as the average of the intensity difference between the nearest neighboring pixels. Techniques for finding the contrast or the variation are well known to those of ordinary skill in the art.

The present invention determines in step 83 whether the contrast or variance of a dot region is greater than a specified value. If the contrast or variance is greater than the value, a partial dot is selected for that dot region (step 84), otherwise, a mixed dot is selected (step 86). Finally, the image is reproduced by the printer 18 in step 88, with the specific dot region rendered in either the selected partial or mixed type dot.

Thus, a partial dot is rendered for text and halftone, since text and halftone are usually higher contrast in nature. This will not cause the Moire pattern in the halftone and will produce a smooth text boundary. The mixed dot, the best of the different dot types for continuous tone images, is rendered for the continuous tone region.

Collecting the statistical information and locally selecting the appropriate dot type from between a partial dot type and a mixed dot type provides superior reproduction of an image that contains different types of image regions, such as text, halftone and continuous tone regions.

The unified rendition method described above achieves many quality goals to render a mixed type document. However, there are some noise problems associated in the halftone printing due to its rendered weak dots in the halftone picture. To stabilize the rendered dots in the halftone picture, a different screen structure is necessary to impose on the original halftone screen. However, this enhances the Moire pattern in the halftone picture. A compromise between the noise issues and the Moire pattern is necessary to minimize the problem in printing halftone pictures and other mixed types of images.

The approach of the present invention, a basic flow diagram of which is illustrated in FIG. 9, further enhances the quality of the unified rendition method in dealing with a mixed type document. A document is scanned in step 40, and this scanned document is processed through a descreening filter in step 42 which removes only a certain frequency caused by the halftone screen. The descreening filter can be resident in the scanner 12 or the controller 16. The text blurring effects of this filter have been minimized for the specific designed filter characteristics.

A local structure analysis of the image is performed by the controller in step 44. The filter operation is applied selectively to the image region based on the results of this local structure analysis. The controller 16 will not alter those pixels along the text border (see bypass 43) while it filters those pixels in a low contrast region and in a halftone dot region. In this way, the text contrast is maintained and the noise and the halftone screen are smoothed out. The processed image is then stored (step 46) and rendered later through the unified method (step 48) and printed (step 50).

The design of the filter will now be described with reference to FIG. 10. First, a frequency spectrum is selected in which the frequency response needs to be achieved. For example, in the present invention, it is desired to boost a certain frequency region while smoothing out another frequency region in the 256.times.256 size spectrum because of the scanner modulation transfer function which needs to be corrected in this manner. This frequency spectrum has a real number representation instead of a conventional complex number (a real part and an imaginary part) representation.

Second, the filter modifies the frequency strength around some frequency spots of the desired frequency spectrum to a minimum (i.e., approaching zero response) in a continuous but sharp slope change. These frequency spots are the locations of the halftone screen frequency. Other frequencies will not be altered.

Third, this modified frequency spectrum is then applied to an inverse FFT (Fast Fourier Transform) back to the spatial domain where image processing is operated. It has now a complex number representation in the spatial domain. An mxn window near the center (i.e., low frequency response region) of the real part of the spatial spectrum is cropped out. This window size is the designed filter size, for example a 5.times.5 filter kernel size. Due to this window cropping operation, certain regions in the spatial domain were excluded, in other words, coefficients are set to zero outside of the window. This cropping will modify some of the response shape in the desired frequency spectrum. Hence several feedback loops between the second step and the third step of previous described method are performed to enforce the strength of the screen frequency to be removed to the minimum. A final mxn window in the spatial domain is then chosen as the kernel of the descreening filter. This filter, when operated on the image, will then remove the specific frequency generated by the halftone screens. However, this filtering operation will also smooth out some contrast of text (the blurring effects) to a certain extent.

In the image descreening operation (step 42 of FIG. 9), the filter is convolved with image pixels to produce a result. During the convolution operation, a structure analysis in the current window (for example, 5.times.5 sizes) is taken to make a decision whether it will convolve the image pixels or not. The original pixel is output instead of the convolved result if there is a text structure in the operated window. Hence, some regions with text structures are maintained in the original pixel values while other regions with halftone structures are convolved with a descreening filter.

The present invention makes a determination that a region is halftone or text, or is low or high contrast. Both text and halftone structure are high contrast, but their local structures are different. For example, text as normally presented has a high contrast transition across its border. One side has a high pixel value while the other side (in vertical direction or in horizontal direction) has a low pixel value. If the pixel value difference from one side to the other side is greater than a certain threshold, it is determined to be a high contrast window. Otherwise, it is determined to be a low contrast window. If there is a consistent transition in the high contrast window such that one side has high values while the other side has low values, it is assumed that a text structure is in this window. If there is no consistent transition in the window and it causes a saddle point in the window, a halftone structure is assumed. This classification strategy based on the image structure makes about 95%-98% of correct decisions on text. Only a few percentage errors are made which is due to a small font of text whose body has a similar size to the window. In this situation, the structure may not be clearly defined. However, this small percentage error does not cause a problem of blurring in the normal size of text. Hence, an original image can be reproduced so as to maintain its text contrast in the text region while its screen structure is smoothed out in the halftone region and some noise is filtered out in the low contrast region.

FIG. 11 shows a more detailed flow chart of the process of an embodiment of the present invention. In step 60, a moving window (such as a 5.times.5 window) is established for the image, with the pixel being processed at the center of this image. In step 62, it is determined whether there is a large intensity difference in the window. If there is not a large intensity difference, then the windowed region is a low contrast region, so that a smoothing filter operation to remove noise is applied in step 64, and these filtered pixels are provided to step 74 to be stored (compressed), rendered, and printed. A simple smooth filter that can be used is a weighted average operated with 3.times.3 or 5.times.5 window size.

When a large intensity difference is detected in step 62, the window is determined to be a high contrast window. The consistency of the transition within the window around the central pixel is then evaluated in step 66. If there is a consistent transition, then the window is considered to contain text structure and the original pixel is provided as an output in step 68.

When there is not a consistent transition in the window around the central pixel, the structure in the window is considered to be halftone structure, and the 5.times.5 descreening filter is applied to remove the screen in step 70. Briefly stated again, in the filtering operation, the image is transformed into the Fourier domain and then some specific frequency content (such as 133, 150, or 200 line screens) are removed. This modified Fourier image spectrum is then transformed back to the spatial image domain to complete the removal of the halftone screen. After being descreened, the image pixels are provided to step 74 for final processing.

The design of the descreening filter can be performed "on-line" as indicated by step 72, or it can be already provided. However, this is computationally slow, and it is preferred to operate the filter design off-line and only perform convolution in the halftone window structures on-line.

Although the invention has been described and illustrated in detail, it is to be clearly understood that the same is by way of illustration and example, and is not to be taken by way of limitation. The spirit and scope of the present invention are to be limited only by