Color Separating Method and Apparatus Using Statistical Techniques

The present invention relates generally to color separating systems, and in particular to a method and an apparatus for separating the colors of an original print painted with a number of predetermined colors. More particularly, the invention relates to a method and an apparatus for separating the colors using probability and statistical data handling techniques for eliminating errors arising from the ambiguous color spectral information.

In a prior art color separator, the spectral information of a color is spectroscopically separated into a number of predetermined spectral components and compared with a set of reference values to determine the degree of similarity therebetween. However, when the original color print is painted by color samples prepared by artisans, there is a certain degree of irregularities in the thickness of color print, uneven quantities of compositions in each color sample and possible color overlapping in the print. This would result in ambiguity in the tint, hue and brightness information, and as a result the separator tends to recognize erroneously when encountered with such ambiguous colors.

Therefore, the primary object of the present invention is to provide an improved method and apparatus for separating the colors of a print in which probability and statistical data handling techniques are used to recognize the colors with a high degree of certainty.

The present invention is based on the assumption that the colors of a print observed in terms of their spectral reflectivities can be identified with the highest degree of certainty by two probabilities: one is the probability of occurrence of each color as the print is scanned across its surface and the other is the probability that an observed color spectral component corresponds to a particular color sample. Assume that the color print is painted with "n" distinct color samples (#1, #2 . . . #n), and let "C.sub.i " denote each color sample where "i" ranges from unity to "n" and "x" denote the observed value of spectral reflectivity of a given color of the print, the former probability is denoted by P(C.sub.i), while the latter is a conditional probability density function p(x.vertline.C.sub.i). According to Bayes' Rule the product P(C.sub.i)p(x.vertline.C.sub.i) equals the joint probability density p(x, C.sub.i) which is the probability of simultaneous occurrence of "x" and "C.sub.i ". Therefore, the observed elemental or dot area can be regarded as being painted with a color which belongs to color sample "C.sub.i " with the highest degree of certainty when a color sample C.sub.i is detected which maximizes the value of P(C.sub.i)p(x.vertline.C.sub.i).

The apparatus of the invention generally comprises a microscopic color scanner, a macroscopic color separator and a data processing unit. The microscopic color scanner scans along a plurality of line paths within the print to collect light from an elemental or dot area at a given instant of time and separates the collected light into a plurality of spectral components to generate set of electrical signals each representing the intensity of each separated spectral component. The macroscopic separator collects light from the entire area or at least a part of the area which encompasses a plurality of such elemental areas and separates the collected light into a plurality of spectral components to generate a set of electrical signals each representing the intensity of each spectral component. Each of the macroscopically generated signals is an integration of each of the microscopically generated signals. The average intensity of a given dot area is given by ##EQU1## where .mu..sub.j.sup.(i) is the mean value of a spectral component "j" of a color sample "i". A set of mean values .mu..sub.j.sup.(i) is a set of spectral reflectivities exhibited by a color sample "i" and the intensity of each spectral component of the actual color dot found along the scanned line paths varies in accordance with the conditional probability density function p(x.vertline.C.sub.i). The macroscopically generated signals are divided in magnitude by the size of the area from which the signals are obtained in order to generate a set of signals each representing the intensity of a spectral component per unit area. Since each of the per-unit area representative signals is of equal value to the average intensity of each spectral component of a given dot area, the probability P(C.sub.i) can be obtained from the set of mean values .mu..sub.j.sup.(i) and the per-unit area representative signals. The processing unit executes the computation of P(C.sub.i) from the inverse matrix of .mu..sub.j.sup.(i) multiplied by a set of per-unit area representative signals. The conditional probability density function p(x.vertline.C.sub.i) is an unbiased estimator which can be used for any color pattern as long as the same color samples are employed. The data processor then accepts the microscopically obtained data as the print is scanned to compute the product P(C.sub.i)p(x.vertline.C.sub.i) and detects C.sub.k which maximizes the value of the product to generate an output indicating that the dot area being scanned is painted with the color which belongs to color sample C.sub.k.

Therefore, an object of the invention is to provide a method and an apparatus for detecting a color sample that maximizes the joint probability using statistical data collected from an elemental area of the print and a larger area encompassing a plurality of such elemental areas within a short interval of time before the point of observation is shifted successively to the next.

These and other objects, features and advantages of the invention will be understood from the following description taken in conjunction with the accompanying drawings, in which:

FIG. 1A and 1B show an embodiment of the invention in schematic form;

FIG. 2 is a color print painted with "n" different colors useful for description of the invention;

FIG. 3 is a graphic illustration of the spectral reflectivity distribution of a given sample point on the print of FIG. 2;

FIG. 4A is a graphic illustration of the spectral reflectivity distributions obtained from "n" color samples;

FIG. 4B is a graphic illustration of the mean spectral reflectivity distribution multiplied respectively by the probability P(C.sub.i);

FIG. 4C is a graphic illustration of the summation of the multiplied spectral distributions of FIG. 4B;

FIGS. 5A and 5B are a modification of the embodiment of FIG. 1;

FIGS. 5C and 5D are examples of a converter which may be employed in the circuit of FIG. 1;

FIG. 6 is a specific embodiment of the invention;

FIG. 7A is a flow chart showing the process steps required to perform computation of the data used in the embodiment of FIG. 6;

FIG. 7B is an analog computing circuit used in the circuit of FIG. 6 to compute P(C.sub.i)p(x.vertline.C.sub.i);

FIG. 8 is a modification of the embodiment of FIG. 6; and

FIG. 9A and 9B show details of the scanners used in the embodiments of the invention.

Referring now to FIG. 1, an embodiment of the present invention is schematically shown. In FIG. 1A, the apparatus of the invention comprises generally a light source 10 for illuminating a color print 11, a microscopic color scanner 12, a macroscopic color separator 13, and drive means 14 which drives the microscopic color scanner 12 to scan across the surface of the print 11. Scanner 12 comprises a lens system 15 which collects light from an elemental area along the scanned line and passes the collected light to a spectroscopic beam splitter 16 which in turn separates the collected light beam into spectral components .lambda..sub.1, .lambda..sub.2 . . . .lambda..sub.r each of which may be a spectral line or a cluster of spectral lines. The beam splitter 16 is comprised of a plurality of color filters f.sub.1, f.sub.2. . . f.sub.r arranged as shown in FIG. 9A and passes the separated light beams to photoelectrical devices 17 equal in number to the separated light beams to detect the spectral intensity of each light beam producing a set of corresponding electrical signals (x.sub.1, x.sub.2. . . x.sub.r).

The print 11 is painted with "n" different colors C.sub.1, C.sub.2 . . . C.sub.n by artisans as shown in FIG. 2. Because of the ambiguous nature of the spectral information derived from each elemental area of the print, the observed spectral reflectivities are considered as a set of random variables which distribute about a set of mean values .mu..sub.1.sup.(i), .mu..sub.2.sup.(i) . . . .mu..sub.r.sup.(i) as shown in FIG. 3 where i ranges from unity to n. Color samples C.sub.1, C.sub.2 . . . C.sub.n are attached to the print. These samples are carefully painted with respective colors such that each possesses a set of means values, for example, color sample C.sub.1 possesses a set of mean values .mu..sub.1.sup.(1), .mu..sub.2.sup.(1) . . . .mu..sub.r.sup.(1), and so on, as clearly shown in FIG. 4A.

As the microscopic color scanner 12 scans across the surface of print 11, each of the colors C.sub.1 to C.sub.n is assumed to occur with the probability P(C.sub.i), which is, otherwise stated, a probability that the color encountered belongs to a color sample C.sub.i.

Consideration is now given to the conditional probability density function p(x.vertline.C.sub.i) of a set of observed values x.sub.1 to x.sub.r given that the observed values belong to a color sample C.sub.i. Let x.sup.(i) denote a color signal vector belonging to a color sample C.sub.i, then x.sup.(i) is written as a vector of "r" dimensions

then the mean vector of x.sup.(i) is given by

The observed spectral intensities x.sup.(i) therefore have a distribution in accordance with the conditional probability density function p(x.vertline.C.sub.i) with a set of mean values .mu..sup.(i).

The macroscopic color separator 13, on the other hand, comprises a lens system 18 which collects light from the entire area of the print 11 and passes the collected light to a spectroscopic beam splitter 19 having a plurality of color filters F.sub.1, F.sub.2 . . . F.sub.n equal in number to the color samples, so that the beam splitter 19 separates the beam into "n" spectral components at wavelengths .lambda.'.sub.1, .lambda.'.sub.2, . . . .mu.'.sub.n (FIG. 9A). A plurality of photoelectrical devices 20 receives the separated light beams to convert into corresponding electrical signals X.sub.1, X.sub.2, . . . X.sub.n. It may be noted that each of these signals X.sub.1 to X.sub.n is an integration of the intensities of the same components collected from the elemental areas.

Referring to FIG. 1B, an attenuator 21 is connected to the output of the macroscopic analyzer 13 to receive the signals X.sub.1 to X.sub.n to provide demultiplication of the input signals by the entire area (S) of the print 11 to provide a set of output signals X'.sub.1, X'.sub.2 . . . X'.sub.n each representing the spectral intensity per unit area of the print. Therefore, ##EQU2## where j ranges from unity to n. ##EQU3##

From the foregoing discussion it follows that the mean value of the spectral intensity of a given point of observation is given by

The signals X'.sub.1 to X'.sub.n from the attenuator 21 are applied to the first inputs of a computing circuit 22 which may be a general-purpose computer MODEL-II manufactured by Data General Inc., to which is also applied through the second inputs a set of mean values .mu..sub.1.sup.(i) to .mu..sub.n.sup.(i) which have been derived from each color sample and stored in memory 23. The circuit 22 first computes the inverse matrix of .mu. values and then computes P(C.sub.i) in accordance with Equation 9.

The probability P(C.sub.i) can thus be obtained uniquely from Equation 9 since the simultaneous linear equations of Equation 7 can be regarded as being linearly independent. Otherwise stated, the mean values .mu..sub.j.sup.(i) are so selected that they satisfy the regular matrix requirement: ##STR4##

It is important to note that since there must be a set of "n" linearly independent equations to determine P(C.sub.i), the beam splitter 19 must separate the incident light into a set of "n" spectral components.

The microscopic spectral data x.sub.1, x.sub.2 . . . x.sub.r are fed into a computing circuit 24 which also receives a signal representing the conditional probability density function p(x.vertline.C.sub.1) and the output from the circuit 22. The conditional probability density function is primarily determined by the compositions of paint, the material of the paper on which the image pattern is painted, and the thickness of the paint material which may vary from point to point, all of which may be collected as microscopic statistical data from the scanner 12 before the computing circuit 24 receives signals x.sub.1 to to x.sub.r from the scanner 12. Therefore, the conditional probability density function can be considered as being an unbiased estimator so that it can be used for any design pattern as long as the same color samples are employed once the mean values are determined for a set of color samples. Therefore, the signal representing p(x.vertline.C.sub.i) is supplied from an external source (not shown) in which the mean values are prepared and stored in advance using the statistical data handling technique.

The circuit 24 first provides substitution of x.sub.1 to x.sub.r for x and then provides computation of the products P(C.sub.1)p(x.vertline.C.sub.i), P(C.sub.2)p(x.vertline.C.sub.i), . . . and so on. The results of the computation are applied to a comparator 25 which detects a particular color sample C.sub.k which maximizes the product P(C.sub.i)p(x.vertline.C.sub.i) so that the apparatus recognizes that a set of signals x.sub.1 to x.sub.r derived from a given point of observation from the microscopic scanner 12 belongs to the color sample C.sub.k and presents a signal on its output to indicate that the sampled point is painted with color C.sub.k.

FIGS. 4A to 4C are presented for better understanding of the present invention. FIG. 4A shows the mean spectral reflectivity distribution of color samples #1, #2 . . . #n and FIG. 4B illustrates the product of the means distribution of FIG. 4A and P(C.sub.i) as given in Equation 5. FIG. 4C depicts the integration of the terms of Equation 5 which corresponds to the signals X.sub.1 to X.sub.n supplied from the macroscopic separator 13.

A set of mean values shown in FIG. 4A can be obtained by placing the color print 11 relative to the microscopic color scanner 12 and operating it to collect spectral data from color samples C.sub.1 to C.sub.n if they are attached thereto The collected data from each sample is selectively filtered to produce a set of "n" spectral components of the same light wavelength as the spectral components X.sub.1 to X.sub.n obtained from the macroscopic color separator 13. Since the color samples are carefully painted so as to serve as a standard, the spectral intensity of the filtered components is assumed to take the mean calues so that a set of mean values .mu..sub.1.sup.(1) . . . .mu..sub.1.sup.(n) to a set of mean values .mu..sub.n.sup.(1) . . . .mu..sub.n.sup.(n) are obtained from the color samples C.sub.1 to C.sub.n respectively. If color samples are not provided, the microscopic scanner 12 is operated so as to collect spectral data from a plurality of areas painted with a same color. The collected data are filtered to produce a set of "n" spectral components. Since the spectral data collected from such painted areas tend to differ from the mean values as indicated by the dottle-line curve of FIG. 3, the data are mathematically manipulated with the aid of a computer to produce a set of mean values for the associated color. This process will be repeated a number of times equal to the number of colors.

To provide accurate macroscopic data to determine the probability P(C.sub.i), an arrangement shown in FIG. 5 is appropriate. In FIG. 5A, the print 11 is rolled around a revolving drum 30 which is driven by the drive means 14 in synchronism with each line scan to successively shift the scan line to the next. The lens system 18 of the macroscopic color scanner 13 is adjusted so that it collects light from a limited area which encompasses a plurality of elemental areas from which microscopic data is obtained. This narrowing of the macroscopic area serves to collect more valid statistical data for the events in the neighborhood of the sample point, and is particularly advantageous for such prints where the image pattern is remarkably different from one area to another. In the illustrative embodiment of FIG. 5, the macroscopic scanner 13 is driven by the drive means 14 instead of being held stationary as illustrated in the previous embodiment, so that the area from which the macroscopic data is collected is shifted successively until the entire area of the print is scanned. The signals from the macroscopic color scanner 13 are then applied through a set of mode selector switches 31-1, 32-2 . . . 31-n to an attenuator 33 which provides demultiplication of the input values by the size of the area (S') from which the data is collected macroscopically (FIG. 5B). The apparatus can be operated on a real-time basis or on a non-real time basis. In the real-time operation, the two scanners 12 and 13 are simultaneously driven to generate the macroscopic data simultaneously with the generation of the microscopic data. In this case the signals from the attenuator 33 are directly applied to the computing circuit 22. In the non-real time operation, the macroscopic scanner 13 is driven first to generate the data required to obtain P(C.sub.i) in advance of the generation of the microscopic data. In the latter case, the signals from the attenuator 33 are stored in a storage device 34 from which the data is retrieved in synchronism with the line scanning. For this purpose the drive means 14 is supplied with clock pulses from a clock pulse generator 35 to move the scanning point in step therewith. The clock pulse is also fed to a counter 36 which provides an output when a number of line scans is completed by counting the clock pulses. The output from the counter 36 is connected to the storage device 34 to instruct it to shift the stored location to another to read out the macroscopic data appropriate for the area of the print being scanned by microscopic scanner 12. When the print image is a uniform pattern, an integrator 37 may be employed to integrate the input signals as the macroscopic scanner 13 is scanned across the surface of the print to generate data which is analogous to that obtained from the entire area of the print as provided by the macroscopic color separator 13 of the previous embodiment. In this mode of operation, switches 31 and 32 are operated to switch the paths of incoming signals to the integrator 37 which feeds its outputs to an attenuator 38 to demultiply the input values by the area (S) of the print 11, and thence to the computing circuit 22.

The elemental (dot) area representative signals from scanner 13 may be applied to logarithmic amplifiers 50 as shown in FIG. 5C to generate output signals which are a logarithmic function of the input signals, and thence to analog-digital converters 51. The digitally converted signals are multiplexed through a multiplexer 52 and applied to the computer 24. FIG. 5D shows a circuit which eliminates noise contaminating the dot-area representative signals. The input signals are applied to a group of differential amplifiers 53. The amplifier 53-1 has its noninverting input connected to receive signal x.sub.1 and its inverting input connected to receive signal x.sub.2 to generate an output which represents the difference between the two input signals so that the noise which is common to the input signals are cancelled. Similarly, the second amplifier 53-2 has its noninverting input connected to receive signal x.sub.2 and its inverting input connected to receive signal x.sub.3 so that the color difference output therefrom is representative of the difference x.sub.2 and x.sub.3. An adder 54 is provided which has its noninverting input connected to receive all the signals x.sub.1 to x.sub.r while its inverting input connected to a reference or ground potential. The output from the amplifier 54 is a summation of all the input signals to represent luminance value of the dot area. The outputs from the differential amplifiers 53 and the adder 54 are applied to a group of analog-to-digital converters 55 and 56, and applied to the computer 24 through multiplexer 57. The effect of the luminance signal is to separate the colors of the same hue, but of different luminosity.

FIG. 6 illustrates a specific embodiment of the invention wherein the same numbers are used to designate parts employed in the previous drawings. In this embodiment, the conditional probability density function is assumed to follow normal density function (Gaussion distribution) of multiple random variables which is given by Equation 11: ##EQU5## where, .SIGMA.i is the matrix of covariance of x belonging to C.sub.1, .SIGMA.i, the inverse matrix of said covariance, and "t" represents the transposition of vector (x-.mu..sup.(i)).

By substituting the Bayes' formula P(C.sub.i)p(x.vertline.C.sub.i) and taking its logarithm, the Bayes' formula can be given by

where, ##EQU6## if,

"x" can be regarded as belonging to color sample C.sub.i.

In order to compute Equation 13, the matrix of covariance "x" and its inverse matrix should be computed, which are given as follows: ##STR5## where, the elements s.sub.lm are given by ##EQU7## where, "k" is the "k"th sample collected from the color samples or from the original print, and "N" is the number of such samples. Then, ##STR6##

In FIG. 6, the apparatus has three mode selector switches SW1 to SW3. In the first mode of operation, SW1 is operated to connect the dot-area representative signals to a computer 40 to first generate the mean values .mu. and applies them to a storage device 42 and then proceeds to perform computation of Equations 15 to 17 through a number of processing steps as illustrated in the flow chart of FIG. 7A. In the latter execution, computer 40 first executes the step 1 to generate covariance of "x", and then arranges the generated covariances in a matrix form in step 2. This matrix of covariance is used in step 3 to compute the determinant .vertline..SIGMA.i.vertline. and applies it to a logarithmic amplifier 41. The computer 40 then proceeds to determine the inverse matrix of covariance of "x" (.SIGMA.i.sup.-1) and feeds the result of computation to the storage device 42.

In the second mode of operation, switch SW2 is depressed to supply the output from the macroscopic scanner 13 to the computer 22 through attenauator 21. Computer 22 also receives the mean value data .mu. from the computer 40 so that it can proceed to execute computation of P(C.sub.i) in a manner as previously described, and feeds its resultant output to the logarithmic amplifier 41 and thence to the storage device 42. It is obvious that computers 22 and 40 may be provided by a single computer unit. The storage device 42 is thus loaded with the data required to perform calculation of Equation 13 and is ready to transfer the stored information to a computing circuit 44 in synchronism with clock pulses supplied from the clock pulse generator 35. Since the data on probability P(C.sub.i) may vary from area to area, the storage device 42 identifies the stored P(C.sub.i) data with the signal from the counter 36 so that in the transfer mode the counter output serves to shift the storage locations.

In the third mode, switch SW3 is operated to connect the output from the microscopic scanner 12 to the computing circuit 44 via a logarithmic amplifier 43, and at the same time the data stored in the storage device 42 are transferred to the circuit 44. FIG. 7B illustrates the computing circuit 44 in detail. The signals x.sub.1 to x.sub.r from the microscopic scanner 12 are each applied to one input of substractors S.sub.1 or S.sub.r through corresponding input terminals designated by the same characters x.sub.1 to x.sub.r. The .mu..sup.(i) data transferred from the storage device 42 are each applied to the other input of the subtractors through corresponding digital-to-analog converters (D/A) such that subtractor S.sub.1 provides an output representing x.sub.1 - .mu..sub.1.sup.(i), S.sub.2 providing an output x.sub.2 - .mu..sub.2.sup.(i), and so on. The outputs from the subtractors S.sub.1 to S.sub.r are connected to a matrix of multipliers M.sub.11 to M.sub.rr which are arranged in a plurality of rows and columns in triangular form since the inverse covariance matrix is symmetrical. The inverse matrix data derived from Equation 17 are applied to one input of the corresponding multipliers through corresponding analog-to-digital converters (not shown). The output circuits of the subtractors are each connected to the other input of those multipliers arranged in a row to provide multiplication of (x - .mu..sup.(i)) .SIGMA.i. The outputs from those multipliers arranged in a column are connected to a respective one of adders A.sub.1 to A.sub.r-1 to deliver the result of the multiplication to the next stage. The outputs from the adders A.sub.1 to A.sub.r-1 are then coupled respectively to one input of multipliers M.sub.1 to M.sub.r-1 and the output from each of the subtractors S.sub.1 to S.sub.r is also connected to the other input of multipliers M.sub.1 to M.sub.r-1. A multiplier M.sub.r has its one input connected to the output of multiplier M.sub.rr and the other input connected to the output of subtractor S.sub.r. An adder 60 is provided to collect data from the multipliers M.sub.1 to M.sub.r. The output from the adder 60 is a representation of (x - .mu..sup.(i)) .SIGMA.i.sup.-1 (x - .mu..sup.(i))t, and thus the first term of Equation 13 is completed. The log .vertline..SIGMA.i.vertline. data in analog quantity is supplied from the storage device 42 to an adder 61 which provides summation of this input data with the output from the adder 60 to compute Equation 13. The output data from the adder 61 is then connected to one input of an adder 62 for summation with an analog data representing 2 log P(C.sub.i) supplied from the storage device 42. The polarity of output from the adder 62 is then inverted by an inverter 63 to provide an output proportional to f.sub.i (x) + log P(C.sub.i) to the maximum selector 25 through output terminal 64 to detect C.sub.k which maximizes f.sub.i (x) + log P(C.sub.i).

It is appreciated from the foregoing description that the dot area representative sepctral data is processed in the computing circuit 44 in analog values before the scanner 12 is moved to the next dot area. It is to be noted that the logarithmic amplifier 43 may be replaced with a linear matrix converter providing Hadamard conversion or Karhunen-Loeve expansion depending upon the statistical nature of the input signals. The mean values .mu..sup.(i) can also be prepared directly by scanning the color samples if available. Furthermore, if probability P(C.sub.i) has a constant value such as "1/n", the minimax principle may be used to identify the color of the elemental area instead of using the Bayes' Rule.

The arrangement of FIG. 6 can be simplified by the use of a variable focusing spectroscopic scanner 70 as illustrated in FIG. 8. At a first range focus, the scanner 70 collects data from the elemental area to serve as the microscopic scanner and at a second range focus, the data is collected from a larger area operating as the macroscopic scanner. The scanner 70 may comprise as shown in FIG. 9B a variable range focus lens system 17 for collecting light onto a plurality of color filters 72 equal in number to the color samples (in this case "n") to separate the incident light into "n" arbitrarily selected light wavelengths. The output from the photoelectrical devices 73 is therefore a set of signals x.sub.1 to x.sub.n when the scanner 70 is set at the first range focus, or a set of signals X.sub.1 to X.sub.n when the scanner is set at the second range focus.

The foregoing description shows only preferred embodiments of the present invention. Various modifications are apparent to those skilled in the art without departing from the scope of the present invention which is only limited by the appended claims. Therefore, the embodiments shown and described are only illustrative, not restrictive.