Method to obtain consistent image quality measurements from different image input devices

[0002] 1. Field of Invention

[0003] This invention relates to an image quality analysis system that obtains consistent image quality measurements regardless of the particular image input device used.

[0004] 2. Description of Related Art

[0005] It is well known that customer satisfaction can be improved and maintenance costs reduced if problems with copiers and printers can be fixed before they become serious enough to warrant a service call by the customer. While current technology exists to enable printers and copiers to call for service automatically when sensors detect certain operating parameters outside of permissible ranges, there is not a very comprehensive manner of detecting incipient system failure or automatically diagnosing when problems with image quality reach a level where human observers perceive a reduction in quality. This is caused not only by the large number of operating parameters that would need to be tracked, but also because these parameters are strongly coupled to one another. That is, a given parameter at a certain value may or may not be a problem depending on the values of other parameters. While existing systems provide some level of image quality analysis, these systems have been found less than satisfactory as image quality determination is machine dependent and may be inconsistent with perceptions of image quality as judged by human users.

[0006] Systems which can perform image analysis on printed test samples can be used in a variety of ways to provide solutions and value to users of digital printers and copiers, for example as the analysis engine for automatic or remote diagnosis of print quality problems, or for monitoring quality as part of a print quality assurance system.

[0007] The system would typically use an input scanner, either stand-alone or part of a multi-function printer/scanner/copier, to scan the printed test sample, and then perform a series of analyses on the scanned image. Alternatively, a CCD camera could be used in place of the scanner. It is important to have consistent behavior of such systems, when used with different input scanners. That is, the results of the analysis from one input scanner should be essentially identical to results from other input scanners. Only if this is the case, can the IQ measurements from one such system be compared with other systems. In the case where the system is used by the printer/copier user, for example for IQ assurance purposes, it is imperative that the results be consistent in order to compare with industry standard measurements.

[0008] However, the input scanner is likely to be part of a multi-function printer/scanner/copier, and each system would therefore use a scanner with different characteristics, especially in terms of spatial resolving power.

[0009] This invention overcomes problems with differences in analysis results caused by use of different image input devices by providing a computational process that allows absolute image quality measurements to be performed consistently from scans of print samples, largely independent of the resolving power of the scanner. Absolute image quality metrics reflect quality as seen by a human observer. By determining the resolving characteristics of the input scanner, the scanned image can be processed, with little or no artifacts, to resemble the image as perceived by a human observer, while at the same time eliminating differences that would arise when using a scanner having a different spatial resolving power, provided that the resolving power of each scanner is sufficiently greater than the resolving power of the human visual system.

[0010] The invention will be described with reference to the following illustrative drawings, wherein like numerals refer to like elements and wherein:

[0011] FIG. 1 shows a typical digital copier machine having a user interface suitable for use with the invention;

[0012] FIG. 2 is a schematic diagram of a digital copier having a user interface for communicating with a remote diagnostic computer;

[0013] FIG. 3 is a flow chart showing an image analysis method according to the invention;

[0014] FIG. 4 is an alternative image output device and image analysis system according to the invention;

[0015] FIG. 5 is a flow chart showing a more detailed image analysis method according to the invention.

[0016] FIG. 6 shows examples of a human visual transfer function (VTF), as well as several MTF for different scanners; and

[0017] FIG. 7 shows applying a simple processing according to the invention to images scanned with two different devices.

[0018] An exemplary device to which automatic image quality analysis is to be performed will be described with reference to FIGS. 1-3. FIG. 1 shows an image output device, in particular a digital copier machine 10, comprising a plurality of programmable components and subsystems which cooperate to carry out copying or printing jobs programmed through a touch dialog screen 42 of a user interface (UI) 11. Internal operating systems of the digital copier 10 are disclosed in U.S. Pat. Nos. 5,038,319, 5,057,866, and 5,365,310, owned by the assignee of the present invention, the disclosures of which are incorporated herein by reference in their entirety. As such, no further detailed description thereof is necessary. Digital copier 10, however, is merely representative of a preferred printing system to which the image quality determination is made. It should be understood that a loosely coupled printing or reproducing system is also applicable for use with the invention described herein, such as a printer or facsimile device. Moreover, while there may be benefits to use of the image quality analysis on a reproduction system, such as a digital copier having an integral scanner component, the invention also is applicable to a printer used in conjunction with a stand-alone scanner, such as a flatbed type scanner.

[0019] Referring to FIG. 2, operation of the various components of exemplary digital copier 10 is regulated by a control system which uses operating software stored in memory in the system controller 16 to operate the various machine components in an integrated fashion to produce copies and prints. The control system includes a plurality of printed wiring boards (PWBs), there being a user interface module (UIM) core PWB 18, a scanner/imaging core PWB 20, an input station core PWB 22, a paper handling core PWB 24 and an output station core PWB 26, together with various input/output (I/O) PWBs 28. A shared line (SL) 30 couples the core PWBs 18, 20, 22, 24 and 26 with each other and with the electronic data node core 32, while local buses 34 serve to couple the PWBs to the respective cores and to stepper and servo PWBs. Programming and operating control over digital copier 10 is accomplished through touch dialog screen 42 of UI 11. The operating software includes application software for implementing and coordinating operation of system components.

[0020] Floppy disk port 38 provides program loading access to UIM core PWB 18 for the purpose of entering changes to the operating software, loading specific programs, such as diagnostic programs, and retrieving stored data, such as machine history data and fault data, using floppy disks. Hard disk 36 is used as a non-volatile memory (NVM) to store programs, machine physical data and specific machine identity information. One of the programs hard disk 36 may store is image quality analysis software that forms an image quality analysis module 70 used by the invention. Module 70 may also reside on a floppy disk used in floppy disk port 38.

[0021] UIM core PWB 18 communicates with video engine 40 for driving a suitable visual display 42, such as a CRT or flat screen of the user interface 11. The UIM core 18 also has connected thereto a control panel I/O processor 44 and a generic accessories interface I/O processor 46. The interface I/O processor 46 is in turn connected to a modem PWB 48. The modem 48 provides communication between digital copier 10 and a communications channel, such as a public switched telephone network 50 to facilitate information transfer to and from a remote diagnostic computer 60, which may also include image quality analysis module 70 as well as other diagnostic modules.

[0022] The information from the subsystem cores flows to and from the UIM core PWB 18, which embodies software control systems including a user interface system manager and a user interface manager. The UI system manager includes a UI display manager subsystem for controlling the display of messages on the display 42. A data manager subsystem provides data management to the UI system manager.

[0023] In a first embodiment of the invention, image quality analysis is performed by the process set forth in the flow chart of FIG. 3. The process starts at step S300 and advances to step S310 where at least one specific digital test pattern, which can either be in hardcopy original form or a digital image stored in memory 36, is provided. Preferably, multiple different test patterns are used to analyze various components relevant to a determination of image quality. Flow then proceeds to step S320 where a corresponding hardcopy output of the test pattern is generated. This can be by outputting a printed hardcopy output from output station 26 using the digital test pattern as an input when the test pattern is stored in digital form, such as in hard disk 36 or floppy disk 38. Alternatively, an accurate original hardcopy test pattern may be placed at scanner 20 and scanned into the digital copier 10 to form a digital test pattern, which can be used as an input to output station 26 to form the hardcopy output. Then, flow advances to step S330 where the hardcopy output is scanned by scanner 20 to form a digital raster image for analysis purposes.

[0024] After step S330, flow advances to step S340 where the digital image is preferably acted on by pattern recognition software, which can be located within hard disk 36 or floppy disk 38 and is associated with image quality analysis module 70, to determine a precise location of various test elements within the scanned digital raster image. This software uses a Hough or similar transform to automatically detect locator marks on the image. A suitable pattern recognition system for use with the invention can be found in U.S. Pat. No. 5,642,202 to Williams et al., owned by the assignee of the present invention, the disclosure of which is incorporated herein by reference in its entirety. Alternatively, or in conjunction therewith, the test pattern may include a script that signifies a particular test pattern. The copier machine 10 may hardware/software to decipher the particular script embedded into the test pattern. The memory of the copier 10 may be provided with a file corresponding to each possible script detailing the contents of the script and associated test pattern, as well as detailing the particular image quality analysis routine to be used to measure a particular part of overall image quality. A more detailed description of such a scripted test pattern can be found in co-pending U.S. Ser. No. 09/450,182 to Rasmussen et al., filed concurrently herewith, entitled “Method to Allow Automated Image Quality Analysis of Arbitrary Test Patterns”, the subject matter of which is incorporated by reference herein in its entirety.

[0025] After step S340, the process flows to step S350 where image quality analysis is performed on the test image using image quality analysis module 70. From step S350, flow advances to step S360 where a determination is made by the image quality analysis module 70 whether the image quality for this particular test image is acceptable. If it is, flow advances to step S380 where the process stops. However, if the image quality is not acceptable, flow advances from step S360 to step S370 where a call can be made to a diagnostic facility. This call may be an automatic service call made through modem 48 for scheduling an actual service visit by a service technician to correct the noted problems. Alternatively, it may be a call to a more sophisticated diagnostic module 80 located locally or at the remote facility that can further analyze the image quality problem along with values from various sensors and settings on the copier 10. This would provide corrective feedback to the digital copier 10, such as through modem 48 when module 80 is remotely located, allowing the digital copier 20 to adjust itself within acceptable parameters.

[0026] Alternatively, the image quality analysis module 70 may be remote from image output device 10. An example of which is illustrated in FIG. 4 where image output devices are in the form of printers 10A, 10B which are associated with a personal computer 60 through appropriate data cables. A flat bed scanner 20 is also associated with personal computer 60 and image quality analysis module 70 is in the form of software provided in personal computer 60. This embodiment operates as the previous embodiment in that the printers 10A, 10B (which ever is being tested) are given a test pattern to generate a hardcopy output from. This hardcopy output is then placed in scanner 20 to generate the digital test image. This digital test pattern is then analyzed to determine image quality of the printer. In the case where the image quality analysis system is used not for machine diagnosis, but for example for quality assurance, the results from step S350 would be provided to the user of the system.

[0027] While shown in FIG. 4 to be loosely associated, the invention can also be practiced with completely discrete components, such as a separate printer, scanner and computer or other source for containing image quality analysis module 70. In this case, the hardcopy output from the printer can be provided to a non-associated scanner for scanning. Then, the digital test image from the scanner can be stored or converted onto a portable recording medium, such as a floppy disk and provided to a non-associated computer having the image quality analysis module.

[0028] The test pattern used can be one of several test patterns designed to provide evaluation of one or more parameters relevant to image quality analysis of the output of the printing system, such as color registration, motion quality, micro and macro uniformity, color correction, and font quality. This overall analysis is typically performed using human visual perception models so that only those differences that would be perceived by a human observer are determined. That is, rather than having the analysis merely compare a scanned image to an original or to determine some level of variation or deviation from a given norm that may or may not rise to the level of a perceived image quality issue when viewed by a human observer, the image quality analysis utilizes models of human visual perception. At a simple level, this can be achieved by passing the image through a band-pass filter known to be similar in resolution to that achieved by a human vision system. More sophisticated modeling can also be used.

[0029] This particular invention relates specifically to providing consistent image quality analysis of an image output device, such as a printer or copier, regardless of the analysis equipment used. For a more detailed description of the overall image quality analysis system, see co-pending U.S. Ser. No. 09/450,185 to Rasmussen et al., filed concurrently herewith, entitled “Virtual Tech Rep By Remote Image Quality Analysis”, the disclosure of which is incorporated herein by reference in its entirety.

[0030] The ability of a scanner to resolve fine details in an image can be characterized in terms of its sensitivity at different spatial frequencies. This is usually referred to as the scanner's MTF (modulation transfer function).

[0031] For many applications it is necessary that image quality (IQ) metrics directly reflect quality as seen by a human observer. The human visual system (HVS), operating at normal viewing distances of 30 cm or more, typically has worse resolving power than image input devices such as flatbed scanners. Therefore, analysis of scanned images can falsely identify print quality problems. Standard image quality metrics for graininess, for example, take into account the sensitivity of the HVS at different spatial frequencies. The sensitivity of the HVS at different spatial frequencies is expressed through a so-called Visual Transfer Function (VTF), which is the human equivalent of the MTF of a scanner.

[0032] The ratio between the human VTF and the scanner MTF expresses, at each spatial frequency, the extent to which the human visual system is less sensitive than the scanner. We call this the differential transfer function (DTF).

[0033] Image quality metric that quantify quality as perceived by a human observer, typically start with a scan of a printed test sample, and then utilize a sequence of image processing steps, one or more of which correspond to human visual processing as described by human visual models. For an example of such processing see co-pending U.S. Ser. No. 09/450,180 to Rasmussen et al., filed concurrently herewith, entitled “Image Processing Algorithm For Characterization of Uniformity of Printed Images”, the disclosure of which is incorporated herein by reference in its entirety. Since the scanned image has already been degraded (blurred) to some extent by the scanner, the image processing steps corresponding to human visual perception, should utilize the DTF, rather than the VTF, in order to further transform the image to the point where it resembles what is perceived by the HVS.

[0034] Finally, various image analysis techniques can be applied to calculate specific image quality measures, such as graininess and line darkness, from the transformed image that is independent from the individual scanner used to obtain the digital image. For an example of such processing see co-pending U.S. Ser. No. 09/450,180 to Rasmussen et al., filed concurrently herewith. See also co-pending U.S. Ser. No. 09/450,177 to Rasmussen et al., filed concurrently herewith, entitled “Outline Font For Analytical Assessment of Printed Quality Text”, the disclosure of which is incorporated herein in its entirety, relating to image quality analysis of text.

[0035] An exemplary method for performing consistent image quality analysis according to the invention will be described with reference to FIG. 5. Before this process can start, the MTF of the specific scanner must have been determined. This needs be done only once for the given scanner, and in the case where the scanner is part of the multifunction device that is being measured, the MTF of the scanner can be pre-determined by the manufacturer, and the results incorporated as a data file with the image quality analysis software. Alternatively, a special, characterized hardcopy test pattern, which is suitable for determination of scanner MTF, can be provided with the image quality analysis system, and one of the functions of the image quality analysis system can be to determine the MTF of the particular scanner used, based on a scan of this hardcopy test pattern. The resulting scanner MTF data can then be stored in a data file for subsequent use by the image quality analysis system with the specific scanner.

[0036] The process starts at step S500 and flow advances to step S510 where at least one specific test pattern, which can either be in hardcopy original form or a digital image stored in memory 36, is provided. Preferably, multiple different test patterns are used to analyze various components relevant to a determination of image quality. Flow then proceeds to step S520 where a corresponding hardcopy output of the test pattern is generated. This can be by outputting a printed hardcopy output from output station 26 using the digital test pattern as an input when the test pattern is stored in digital form, such as in hard disk 36 or floppy disk 38. Then, flow advances to step S530 where the hardcopy output is scanned by scanner 20 to form a digital raster image for analysis purposes. Each scanner has its own sensitivity and spatial resolution. After step S530, flow advances to step S540 where a particular test image is detected. Then, flow advances to step S550 where the pre-determined modulation transfer function (MTF) of the particular scanner being used is obtained. This transfer function helps to provide analysis that is device independent. It is also important for the subsequent image quality analysis to reflect characteristics of image quality that are perceivable by human viewers. Accordingly, a transform according to human visual perception is also applied to the image. Several human vision models are known from the literature, and all involve steps corresponding to application of a Visual Transfer Function (VTF) that blurs the image to some extent. From step S550, flow advances to step S560 where by dividing the VTF by the MTF, a differential transfer function (DTF) can be derived that reflects both the particular characteristics of the human visual system and the particular characteristics and resolving power of the scanner to provide a device independent compensation value. From step S560, flow advances to step S570 where the compensation function DTF is applied to the digital image from step S530 to achieve an image that corresponds to that perceived by a human viewer. Such an image is, as described above, largely device independent such that consistent comparisons can be made and generated regardless of the particular scanner used. From step S570, flow advances to step S575 where image quality analysis is performed on the compensated image.

[0037] At step S580, if the results of the image quality analysis indicate that a problem with image quality exists, flow may advance to step S590 where a service call can be made to a service facility. If no problem exists, flow advances from step S580 to step S595 where the process stops.

[0038] The invention is useful in providing consistent image quality measurements regardless of the particular image input device used and its particular resolving powers. For example, as shown in FIG. 6, three different scanners are shown, each having differing MTFs (labeled MTF1, MTF2 and MTF3). Also shown is a human VTF. Even though the MTFs vary greatly from scanner to scanner, by using the inventive methods, a consistent image quality analysis can be performed regardless of which is used.

[0039] FIG. 7 shows a simple processing according to the invention applied to images scanned from two different devices. Scanner SM-4000 and scanner UMAX were both used to scan a same printed halftone sample. The RMS of the pixel value deviation from the image average can be taken as a simple measure of uniformity, with a lower average being better. When this was applied directly to the scanned images, as in (A) and (B), the RMS values were 26 and 15, respectively. The large difference is caused by the significantly better resolving power of the SM-4000 device. However, when a filter that represents the residual between the human VTF and the device MTF is applied before RMS is calculated, as in (C) and (D), the agreement between the measurements using the two devices becomes excellent (4.0 versus 4.1).

[0040] Alternatively, two other methods could be used to avoid measurements that are affected by the resolving power of the instrument used to acquire the image data. However, neither of these alternatives are practical for one or more reasons. The first can be used when the scanning equipment includes high resolution scanning microdensitometers and CCD cameras. As these scanners have very high resolving powers (resolution), their MTF can be neglected for this type of application. Thus, DTF becomes equal to the VTF, and the IQ analysis module can simply use the VTF to obtain measurements that correlate with human visual perception. However, such instruments are very expensive and difficult to operate. As such, these types of scanners will not typically be used for routine image quality analysis outside dedicated image quality labs. While these instruments could be used in a stand-alone environment, such as that shown in FIG. 2, they would not be feasible for self-diagnosing multipurpose devices, such as copiers and facsimile machines as such scanners are cost-prohibited for such commercial products. As such, this alternative method is primarily limited in application.

[0041] The second alternative method may be useful when image scanning is performed by scanners with significant degradation caused by their particular MTF (i.e., not case 1 above). Many scanners can be “corrected” based on knowledge of the MTF. For example, typical flatbed scanner systems include options to sharpen the image in order to compensate for the loss of sharpness caused by the limitations in the MTF. However, such algorithms which compensate for the MTF and try to reestablish the image as it would have been recorded by a perfect scanning device are prone to generate artifacts in the image. These artifacts may significantly impact the results of the IQ analysis. As such, this alternative also has its disadvantages and is not suitable for the purposes discussed here.

[0042] With the present invention, relatively inexpensive image scanning devices can be used for the image quality analysis, despite their limitations and differences in terms of MTF. In particular, scanners that are part of a multi-function printer/scanner/copier can be used, and yet provide consistent IQ measurements. The image processing required to apply the DTF can be performed without introduction of artifact, since it does not attempt to directly correct for the MTF of the scanner, but essentially only blurs the image further, to the point where it resembles the visually perceived image.

[0043] Several straight-forward extensions and/or modifications of the invention are possible, including:

[0044] 1. In addition to the correction for spatial resolving power of the devices, a color calibration can be applied. This is further described in co-pending U.S. Ser. No. 09/450,185 to Rasmussen et al., filed concurrently herewith, entitled “Virtual Tech Rep By Remote Image Quality Analysis”.

[0045] 2. The invention has been described mainly in terms of frequency-domain characterization of scanner and human visual perception, that is, in term of MTF and VTF which provide sensitivity at different spatial frequencies. However, the actual processing of images is often better done using real-space convolution kernels, which correspond to the frequency-space MTF or VTF. Therefore, the invention could equivalently have been described in terms of convolution kernels. The critical point is that, when the image is processed (whether in frequency—or real-space) to represent the humanly perceived image, the scanner resolving power characteristics are factored out.