Image processing method and apparatus

[0002] 1. Field of the Invention

[0003] The preset invention relates to an image processing method and apparatus, and particularly to formation of un-sharp signals equivalent to a blurred image, and also to image processing using such un-sharp signals.

[0004] 2. Description of the Related Art

[0005] Un-sharp masking processing is well-known as a means for emphasizing images, and is often employed in the art of photography, and further has recently come to be used as means for emphasis processing of digital images in medical X-ray systems (“Newest Movements in Image Processing Algorithms”; Edited by Takagi, Toriwaki, and Tamura; Shin-Gijutsu Communications).

[0006] Now, with the input image as f(x, y), the resultant image g (x, y) obtained by un-sharp masking processing can be expressed as follows in Expression (1):

[0007] g(x, y)=f(x, y)+c×{f(x, y)−f_av(x, y)}  (1)

[0008] Now, f_av(x, y) represents the local average value for point (x, y), and is obtained from the n×m pixel area surrounding the point (x, y), and generally can be calculated using a simple average pixel value as shown in the following Expression (2): $\begin{array}{cc}{f}_{\mathrm{av}}\left(x,y\right)=\left\{1/\left(n\times m\right)\right\}\times \sum _{j=0}^{m-1}\sum _{i=0}^{n-1}f\left(x-i,y-j\right)& \left(2\right)\end{array}$

[0009] This local average value f_av(x, y) represents the blurred image formed by blurring the input image f(x, y), and the greater the surrounding pixel area of which the local average is obtained, the more blurred the image becomes. Further, the second term in Expression (1) includes high-frequency components of the input image due to difference, the un-sharp masking process being to add high-frequency components multiplied by coefficient c to the input image.

[0010] Also, a photography method using an analog filter is known in medical X-ray radiography, for allowing for better observation of the mediastinum in simple chest photography and chest tomography, which can be realized by processing the image captured with normal radiography, even without using analog filters. (“Development of self-compensating digital filters with CR”; Ohtani et al; Japan Radiation Technology Association Magazine, Volume 45, Issue 8, p.1030, 1989)

[0011] The resultant image g′(x, y) from processing by this self-compensating digital filtering process can be expressed by the following Expression (3):

[0012] g′(x, y)=f(x, y)+F{f_av(x, y)}  (3)

[0013] wherein (x, y) represents the input image, and also wherein f_av(x, y) represents the local average value for point (x, y) and indicates the blurred image of which the input image f(x, y) has been blurred, as with Expression (1). Also, F{*} is a function representing an analog filter.

[0014] However, with the above-described un-sharp masking processing and self-compensating digital filtering process in the known example, both are image processing using a blurred image represented by f_av(x, y), so the amount of calculations for creating the blurred image has been the predominant factor in the processing time thereof.

[0015] Also, there is a problem in the creation of the blurred image, in that enlarging the n×m pixel area for calculating the local average value to obtain an image with a greater amount of blurring substantially decreases processing speed. For example, in the event that the n×m pixel area for calculating the local average value is multiplied by k (an integral wherein k>0), the area for calculating the local average value becomes:

[0016] Kn×Km=K²×(n×m)

[0017] so that the amount of calculation for calculating the local average value for a certain point (x, y) is multiplied by k².

[0018] The present invention has been made in order to solve problems such as those described above, and accordingly, it is an object of the present invention to provide an image processing method and apparatus which is capable of speedily creating un-sharp signals.

[0019] In order to achieve this object, an embodiment of the image processing method and apparatus employs a process wherein the number of pixels of the original image is temporarily reduced, following which un-sharp processing is performed on the image of which the number of pixels has been reduced, and the number of pixels of image signals to which un-sharp processing has been performed is increased by means of interpolation, thus regaining the original number of pixels.

[0020] Other objects and characteristics of the present invention will become clear from the following detailed description of the embodiments of the invention with reference to the drawings.

[0021] FIG. 1 is a block diagram illustrating the overall configuration of an image processing apparatus according to an embodiment of the present invention;

[0022] FIG. 2 is a flowchart for describing the operation of the apparatus shown in FIG. 1;

[0023] FIG. 3 is a diagram for describing an example of the step of reducing the number of pixels by extraction of average pixels executed in the image reduction step shown in FIG. 2;

[0024] FIG. 4 is a diagram for describing the bilinear interpolation method executed in the image enlarging step shown in FIG. 2;

[0025] FIG. 5 is a diagram for describing another example of the step of reducing the number of pixels by extraction of average pixels executed in the image reduction step shown in FIG. 2; and

[0026] FIG. 6 is a block diagram illustrating the overall configuration of an image processing apparatus according to another embodiment of the present invention.

[0027] The following is a detailed description of the embodiments of the present invention, with reference to the drawings.

[0028] FIG. 1 is a block diagram illustrating the overall configuration of an image processing apparatus according to the present invention, and in the FIG. 1 denotes a central processing unit (CPU) for centrally executing control of the overall image processing, and the outputs of the storage device for work 2 which serves as the main storage area and work area for this central processing unit, image input devices 3 such as scanners, cameras, etc., image output devices 4 for outputting the results of processing to printers or film, operation input devices 5 for inputting the amount of blurring to the main apparatus by keyboard or mouse, and operation display devices 6 such as CRTS, liquid crystal monitors, etc. for displaying the values input to the main apparatus or the state of the apparatus, are mutually connected via a system bus 7. also, according to the present embodiment, image data input from the image input devices 3 or image data stored in an external storage device 8 may be processed as image data.

[0029] FIG. 2 is a flowchart for describing the operation of the apparatus shown in FIG. 1, wherein processing is performed to an input image S0 input at the image input step P0. First, a certain amount of blurring S1 is input at the step for input of amount of blur P1, and the amount of blurring S1 input at the step for input of amount of blur P1 is used for calculating a rate of reduction S2 for reducing the image in the step for determining reduction ratio P2. The amount of blur S3 for performing blurring to the reduced image is determined in the step for determining amount of blur for reduced image P3, based on the amount of blurring S1 input at the step for input of amount of blur P1 and the rate of reduction S2 for reducing the image in the step for determining reduction ratio P2. Incidentally, the term “image reduction” here refers to the later-described process of reducing the number of pixels provided to the subsequent processes.

[0030] Next, in the step for reducing image P4, the input image S0 is reduced based on the rate of reduction S2 determined in the step for determining reduction ratio P2, thus obtaining a reduced image S4. In the step for blurring reduced image P5, a reduced blurred image S5 is created based on the amount of blur S3 determined in the step for determining amount of blur for reduced image P3 and the reduced image S4 reduced in the step for reducing image P4. Next, the reduced blurred image S5 created in the step for blurring reduced image P5 is enlarged in the step for enlarging image P6 until the image is the same size as the input image S0, thus outputting an enlarged blurred image S6. Incidentally, the term “image enlarging” here refers to the later-described process of increasing the number of pixels so as to equal the number of pixels of the input image.

[0031] Now, with the amount of blurring S1 to be input in the step for input of amount of blur P1, i.e., the size N×M of the pixel area for calculating the local average value, as 15×15, and with the size of the input image as x×Y (X>N, Y>M), the rate of reduction h for reducing the input image to 1/h should be selected from an integer which does not exceed the desired amount of blurring N and M, and also whereby N and M are divisible. That is, since N=M=15, selection is made from integers whereby N and M as dividends are divisible and do not exceed N and M, e.g., h=5 in this case.

[0032] Next, in the step for determining amount of blur for reduced image P3, the amount of blur S3 to be applied to the reduced image is determined. That is, based on the rate of reduction h determined in the step for determining reduction ratio P2, the amount of blur S3 for the reduced image is determined to be (N/h)×(M/h), and since 15/5=3, the amount of blur S3 for the reduced image is 3×3.

[0033] Next, in the step for reducing image P4, a reduced image is created based on the rate of reduction h determined in the step for determining reduction ratio P2. That is, since the rate of reduction is h=5, the pixel value of a 5×5 pixel area in the input image is averaged to serve as the pixel value for the reduced image. FIG. 3 represents the average value of all pixels (25 pixels) within the 5×5 pixel area (the area enclosed by solid lines) being used as the pixel value for the reduced image.

[0034] Next, in the step for blurring reduced image P5, the input image data S0 is blurred by certain means, such as Expression (2). Subsequently, in the step for enlarging image P6, the blurred image S5 of the reduced image created in the step for blurring reduced image P5 is enlarged by a factor inverse to the rate of reduction h determined in the step for determining reduction ratio P2, i.e., h=5. Enlarging is performed by using a bilinear interpolation method, e.g., such as described in (“Introduction to Computer Image Processing”; compilation supervision by Tamura; Soken Shuppan).

[0035] FIG. 4 shows an explanatory diagram of the linear interpolation method, wherein a 3×3 pixel area is enlarged to 15×15 pixels using the bilinear interpolation method.

[0036] The pixel value of (i+s, i+t): 0≦s, t≦1 surrounded by four points, i.e., coordinates (i, j), (i+1, j), (i, j+1), and (i+1, j+1), is calculated by the bilinear interpolation method, and can be expressed as shown in the following Expression (4): $\begin{array}{cc}\begin{array}{c}f\left(i+s,j+t\right)=f\left(i,j\right)\times \left(1-s\right)\times \left(1-t\right)+\\ f\left(i+1,j\right)\times s\times \left(1-t\right)+\\ f\left(i,j+1\right)\times \left(1-s\right)\times t+\\ \left(i+1,j+1\right)\times s\times t\end{array}& \left(4\right)\end{array}$

[0037] Although there are various enlarging methods other than the bilinear interpolation method, such as zero-order interpolation or cubic convolution interpolation, such methods are not appropriate when the object of creating a blurred image is considered, since these methods generate high-frequency components in the processing itself.

[0038] Next, a calculation will be made regarding the degree of reduction in the amount of computation. The number of calculations C_orgin the event that the input image is calculated with the amount of blurring for 15×15 pixels within any change can generally be calculated by the following Expression (5). This Expression (5) indicates that the number of calculations MN for creating one pixel in a blurred image is performed for processing the number of XY pixels.

[0039] C_org=MN×XY=15²XY=225XY  (5)

[0040] Then, the number of calculations C_minwith the creation of a reduced image is expressed in the following Expression (6): $\begin{array}{cc}\begin{array}{c}{C}_{\min }=h^2\times \left(\mathrm{XY}/h^2\right)+\left(\mathrm{NM}/h^2\right)\times \left(\mathrm{XY}/h^2\right)+\\ 16h^2\times \left(\mathrm{XY}/h^2\right)\\ =17\mathrm{XY}+\left(\mathrm{NMXY}/h^4\right)\\ =17\mathrm{XY}+\left(9\mathrm{XY}/25\right)\\ =17.4\mathrm{XY}\end{array}& \left(6\right)\end{array}$

[0041] The first term in Expression (6) represents performing calculation h²times from the rate of reduction in order to obtain one pixel of the reduced image, performing this for the total number of pixels XY/h²in the reduced image. The second term represents the number of calculations for the process of blurring the reduced image, wherein the average value in the area of (N/h)×(M/h) within one pixel of the reduced image is calculated, and that this calculation is performed for (X/h)×(Y/h) pixels.

[0042] Also, the third term represents the number of calculations for enlarging the reduced image (X/h)×(Y/h) to x×Y pixels, wherein the pixel value for interpolation is calculated from the four surrounding pixels based on Expression (4) in order to obtain one pixel by the linear interpolation method, and that this calculation is performed for h²×(X/h)×(Y/h) times. Thus, it can be understood from Expressions (5) and (6) that the amount of calculations can be reduced to about 1/12.

[0043] As described above, in order to create a blurred image, a reduced image of the input image is created, blurred processing is performed with the reduced image, and the reduced blurred image is enlarged to create the desired blurred image, whereby a blurred image equivalent to the cases wherein blurring processing is performed to an input image without change can be obtained with only a fraction of the amount of calculation required therefor.

[0044] The enlarged blurred image data is thus used as un-sharp signals in the step for image correction P7. Now, this step for image correction P7 uses image data which has been subjected to bilinear interpolation in the above image enlarging step, to perform image correcting processing such as described in the following example.

[0045] As a first example, it is possible to perform processing for emphasizing the edge (high-frequency component) in order to adjust the image quality of this image correction processing. That is, with the image data input in the above image input step P0 represented by f(X, Y), and the un-sharp signal represented by f_av(X, Y), the processed image g(X, Y) such as shown in the above Expression (1) is calculated by an expression f(X, Y)+k×{f(X, Y)−f_av(X, Y)} (wherein k is a coefficient), and the high-frequency components of the required portion are emphasized.

[0046] As a next example, it is possible to perform filtering processing with the above un-sharp signal, for raising the image contrast of a certain portion, by using f_av(X, Y), as given in Expression (3).

[0047] Further, as a next example, it is possible to compare the above un-sharp signal with a separately-provided threshold value, and using this for area division of the photographed image. For example, in order to segment and distinguish between the lung portion and mediastinum, a histogram is made of input image signals, the value at the low point in the histogram is calculated, the un-sharp signals are digitized with this calculated value as the threshold, and the image is thus separated into two areas. Then, performing separate processing to each of the two separated areas corrects the overall image to an image which is appropriate for diagnosis.

[0048] In the above-described embodiment, an averaging manipulation method was used for reducing the input image in the step for reducing image P4, but in the event that the input image has the high-frequency components thereof sufficiently removed, simple sub-sampling of the image data will pose no problem for creating a blurred image.

[0049] FIG. 5 shows a diagram for describing the method of simply sub-sampling the input image. With a pixel within the 5×5 pixel area (surrounded by bold lines) as the pixel value for the reduced image, the number of calculations C_minshown in Expression (6) can be expressed in the following Expression (7): $\begin{array}{cc}\begin{array}{c}{C}_{\min }=\left(\mathrm{XY}/h_2\right)+\left(\mathrm{NM}/h_2\right)\times \left(\mathrm{XY}/h_2\right)+16h_2\times \left(\mathrm{XY}/h_2\right)\\ =\left(\mathrm{XY}/h_2\right)\times \{\left(1+\left(\mathrm{NM}/h_2\right)\right\}+16\mathrm{XY}\\ =\left(\mathrm{XY}/25\right)\times \left(1+9\right)+16\mathrm{XY}\\ =16.4\mathrm{XY}\end{array}& \left(7\right)\end{array}$

[0050] Using this method, the amount of calculations can be reduced to approximately 1/13 as compared with Expression (5), thus improving the calculation speed.

[0051] As described above, regarding a method wherein the number of pixels of an original image is temporarily reduced, following which blurring processing is performed on the image obtained by reducing the number of pixels, and the number of pixels of the blurred reduced image is increased to the original number of pixels, the image processing apparatus according to the above embodiment of the present invention allows for fewer calculations and higher speed, so that blurring image processing such as un-sharp masking or self-compensating filtering can be performed more efficiently.

[0052] Although in the above embodiment, image processing is carried out by software based on the CPU shown in FIG. 1, but the idea of the present invention can be similarly applied to apparatuses which perform image processing using hardware, as well.

[0053] FIG. 6 is a diagram illustrating an image processing apparatus according to another embodiment of the present invention, and in the Figure, 11 denotes an input terminal for the digital signal to be processed, and 12 denotes a block-forming circuit for separating the digital signals into blocks each comprised of (5×5) pixels, for example. More specifically, the block-forming circuit 12 is comprised of a buffer memory capable of storing at least five lines worth of image data, and is a circuit which changes the order of the image data being input to the buffer memory in the order of raster so that the data is sequentially output in units of blocks each comprised of (5×5) pixels.

[0054] The output of the block-forming circuit 12 is input into an average value calculating circuit 13, and the average of each 25 pixels, i.e., each block, is calculated in this circuit 13. Accordingly, the number of samples output by the average value calculating circuit 13 is 1/25 of the number of pixels input. 14 denotes a blurring circuit to which the average values output by the average value calculating circuit 13 are input, and which performs further filtering to the image of which the number of samples has been reduced so as to blur the overall image.

[0055] Of course, the number of samples output by this blurring circuit 14 is 1/25 of the number of samples of the input image, so that high-speed processing can be realized even if a quite complex digital filter is used for this blurring circuit 14, and the scale of the circuit can be substantially reduced. Next, the output of this blurring circuit 14 is stored in memory 15. Regarding writing to this memory 15, the addresses of the pixels to which interpolation is to be performed are skipped, a bilinear interpolation circuit 16 accesses the pixel data written to this memory 15, the interpolated pixels are subjected to linear calculation according to the aforementioned method, and the interpolated pixels are written to the remaining addresses.

[0056] Thus, the un-sharp image signals for one image stored in memory 15 are read out synchronously with the image signals for one image stored in memory 17, and input to an image quality adjusting processing circuit 18. This image quality adjusting processing circuit 18 performs image correction processing according to the calculating expressions such as the aforementioned Expressions (1) and (3), or performs different processing to the input signals according to the digitized un-sharp signals. The output of the image quality adjusting processing circuit 18 is supplied to an image output device such as a printer or display, via an output terminal 19.

[0057] Thus, even in an embodiment comprised of hardware, the processing speed of the overall apparatus can be improved by reducing the scale of the of the blurring circuit 14 and increasing the processing speed thereof at the same time.