Image processing apparatus

[0002] This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 11-284415, filed Oct. 5, 1999, the entire contents of which are incorporated herein by reference.

[0003] The present invention relates to an image processing apparatus and more particularly to an image processing apparatus which accurately separates a shadow and the background, and selectively suppresses a noise component of the background without damaging a contrast of the shadow in a medical image including a shadow potentially dispersed in the background noise, for example, in an X-ray image including a linear shadow such as a vascular shadow and a guide wire and an edge-like shadow.

[0004] Catheter therapy under fluoroscopy as a medical technology utilizing X-rays is frequently performed. However, as X-ray fluoroscopy reduces a dose of X-ray compared to that of conventional X-ray radiography so that it may reduce an amount of exposure of the patient, the problem that noises superimposed over an image are so large and a guide wire, a catheter or vascular shadow contrasted is hindered from seeing clearly by the background noise will occur. On the other hand, when performing fluoroscopy by increasing the dose, the background noise becomes relatively small, but it cannot be actually applied because the amount of exposure of the patient and medical concerns is increased. Thus, up to this time, the technologies for reducing noises and enhancing contrasts using image processing have been continued to be studied.

[0005] The most generally used image processing technology for reducing the noises is a method of averaging images which are inputted sequentially in time. This technology is already known, and regression filter, which referred to a recursive filter, as its applied technology is also generally used. Addition in time has a noise reducing action for an area without movement, however, with respect to an object moving just like a guide wire inserted into the heart vascular system, it has a drawback to reducing its contrast or resulting in afterglow. Therefore, the weight of averaging is changed per image so that image addition effect can be changed by detecting the movement of an object (See, for example, Japanese Patent Publication No. 2508078, Japanese Patent KOKOKU Publication No. 6-69447, Japanese Patent KOKAI Publication No. 3-198836, Japanese Patent KOKAI Publication No. 6-47035, Japanese Patent KOKAI Publication No. 7-79956, and Japanese Patent KOKAI Publication No. 8-255238).

[0006] However, since detection of the movement of an object is essentially performed by a method based on the difference between the present image and the prior image, time changing of pixel density due to the background noise as well as the movement of the object is falsely recognized.

[0007] Therefore, when precision for detecting a movement of these such as a catheter whose diameter is minimized as medical treatments progress recently, and a guide wire originally having a small contrast is enhanced, the effect of reducing noises results in its limit. This is because when processing parameter enlarging the effect of noise reduction is set, an image is not clearly seen by generating afterglow instead.

[0008] On the other hand, for the purpose of reducing noises and enhancing contrast, linear spatial filter is also used in general. This is a processing for obtaining weighted average of pixels distributed in space and emphasizing contrast by smoothing or by having spatial differential action. However, it has a drawback in damaging a contrast by blurring out a guide wire and a catheter to be linear shadows or edges of organs, and a drawback in noises also being emphasized when contrast is emphasized. Thus, an apparatus for spatially performing pattern recognition of vascular shadows (Japanese Patent KOKAI Publication No. 4-122355), or an apparatus for detecting an edge area of an object, and performing emphasizing contrast in an edge area and smoothing in other areas (Japanese Patent KOKAI Publication No. 60-245084) has been developed.

[0009] Since the former requires many averaging circuits and comparator circuits per one pixel unit and it requires a large number of circuits in order to perform a real-time calculation such as fluoroscopy processing and the like, it is not capable of producing the effect of reducing noises. Moreover, since the latter distinguishes an edge area from the background area with only information of one bit, if a false recognition is occurred, a noise which is located adjacently to an area being extremely performed smoothing and falsely recognized, for example, is emphasized and outputted, and it easily damages the quality of an image because the false recognition sensitively appear as an artifact of an output image.

[0010] Furthermore, besides those, in Japanese Patent KOKAI Publication No. 9-248920, an image processing method of identifying a linear shadow of an image by determining whether it is a true linear shadow or the background using information of 2 bits is disclosed, whereby resulting in identifying precision being enhanced, but some artifacts are occurred by a false recognition. Moreover, a problem that it cannot effectively perform noise reduction to the linear shadow itself is occurred.

[0011] As another method for image processing, a method for determining directivity of a linear shadow or edge, and emphasizing its directional component or separating an area having directional component from other area and separately filtering those areas has been reported. These technologies refers to Directional Adaptive Filters, and some of these technologies are known.

[0012] A description of a method for detecting a direction using wavelet transformation with respect to an X-ray image is disclosed in U.S. Pat. No. 5,799,100, or U.S. Pat. No. 5,717,791. Moreover, a method for applying a filter intensified for performing noise reduction at the place where a linear shadow or edge is absent, by analyzing the difference between the original MRI image and the MRI image subjected to 4-direction filtering is disclosed in U.S. Pat. No. 5,602,934. However, for these methods, its algorithms and calculations are too complex to perform real time processing. Specifically, in the case of a method using wavelet transformation, even when performing just only wavelet transformation, it is difficult to perform real time processing. Moreover, a method as described in the specification of U.S. Pat. No. 5,602,934, actually performing a filter having directivity of angles of 4 directions. In this case, however, besides disadvantage in precision when a linear shadow or an edge is present at the angles other than those angles, it has also a problem that the number of filter calculation times and amounts of calculations between images are large. Therefore, more high speed and precision enhancement are required.

[0013] Thus, in conventional image processing technologies, it was difficult to suppress not only selectively a noise component of the background, but also in real time without damaging contrast of a linear shadow and an edge-like shadow in a image including a linear shadow or an edge-like shadow. It should be noted that this problem can occur similarly, without limiting to an X-ray image, with respect to the images in general which a random noise is superimposed such as CT image, MRI image and the like.

[0014] Accordingly, it is an object of the present invention to provide an image processing apparatus for enhancing the precision of separation between a linear shadow or an edge-like shadow and the background noise, suppress the background noise without damaging the contrasts of the shadows, and in addition to the above, being capable of performing a real time processing.

[0015] An image processing apparatus according to the present invention, comprises means for detecting a direction of a linear shadow or an edge-like shadow included in the image and its intensity, and filter means for performing a spatial filtering operation having directivity corresponding to the direction detected by the detecting means and its intensity.

[0016] Therefore, it can separate a linear shadow or an edge-like shadow potentially dispersed in the background of the image from the background noise and suppress the background noise without lowering the contrasts of the shadows.

[0017] Additional objects and advantages of the present invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the present invention.

[0018] The objects and advantages of the present invention may be realized and obtained by means of the instrumentalities and combinations particularly pointed out hereinafter.

[0019] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate presently preferred embodiments of the present invention and, together with the general description given above and the detailed description of the preferred embodiments given below, serve to explain the principles of the present invention in which:

[0020] FIG. 1 is a block diagram showing the configuration of a first embodiment of an image processing apparatus according to the present invention;

[0021] FIG. 2 is a diagram showing one example of an image subject of processing of the present invention;

[0022] FIGS. 3A and 3B are diagrams showing examples of images whose directions are less intensified than that of FIG. 2 of the present invention;

[0023] FIGS. 4A, 4B and 4C are diagrams showing examples of spatial filters of the first embodiment of the present invention;

[0024] FIGS. 5A and 5B are diagrams showing a principle for detecting a direction and its intensity of a linear shadow or an edge by applying principle component analyzing method to a direction detection circuit of the first embodiment of the present invention;

[0025] FIG. 6 is a circuit diagram of a part of a direction detection circuit executing product-sum calculation for analyzing a direction and its intensity;

[0026] FIG. 7 is a concrete circuit diagram of a direction detection circuit of the first embodiment of the present invention;

[0027] FIG. 8 is a schematic diagram showing an operation of a vector filter circuit of the first embodiment of the present invention;

[0028] FIG. 9 is a detailed diagram showing an operation of a vector filter circuit of the first embodiment of the present invention;

[0029] FIG. 10 is a diagram showing a spatial filter circuit of the first embodiment of the present invention;

[0030] FIGS. 11A and 11B show an operation providing a plurality of small type spatial filters instead of providing one large type spatial filter in the first embodiment of the present invention;

[0031] FIG. 12 is a diagram showing a principle of detecting a direction of a linear shadow or an edge of a second embodiment of an image processing apparatus of the present invention;

[0032] FIG. 13 is a block diagram showing the configuration of a fourth embodiment of an image processing apparatus according to the present invention; and

[0033] FIGS. 14A and 14B are diagrams showing the configuration of the flat type detector of direct conversion method employed in a fifth embodiment of an image processing apparatus according to the present invention.

[0034] A preferred embodiment of an image processing apparatus according to the present invention will now be described with reference to the accompanying drawings.

[0035] First Embodiment

[0036] An X-ray image as an example of an image subject to deal with is exemplified and described herein. In general, it is known that a noise included in a digital X-ray image is proportional to square root of the number (N) of X-ray incident photons injecting at one pixel. For example, when 100 number of photons are incident at one pixel, expected value L of outputted signal and standard deviation a are expressed as follows:

[0037] L=a×100×E

[0038] where E represents average energy of X-ray photon, “a” represents a gain obtained when X-ray energy is converted into an outputted signal.

[0039] σ=a×10×E

[0040] Therefore, an image obtained by irradiating X-ray to a uniformed subject is, on average, one on which noises having the standard deviation σ are distributed on the image having level of L. A fine and thin guide wire or catheter used in interventional radiology (IVR) is depicted as a linear shadow in the noise. It is an object of the present invention to lower noises with respect to such a linear shadow and an edge-like shadow embedded in the background noise without lowering a sharpness of the contrasts.

[0041] It should be noted that requirements are also present to clearly observe a linear shadow of vascular shadow and the like, and a tumor shadow or an edge of an organ and the like in other medical images as well as in an X-ray image. The present invention is applicable to such images, and furthermore, not limiting to medical images, but to images in general including a linear shadow and an edge.

[0042] FIG. 1 is a diagram showing schematic configuration of an image processing apparatus involving with a first embodiment of the present invention. The basic method of the present invention is to provide noise lowering filter along the direction of a linear shadow and an edge. An input image is inputted into a pre-process circuit 10 and a spatial filter circuit 12. The pre-process circuit 10 is used for either emphasizing a linear shadow or an edge of an image to some extent, or lowering noises in general and facilitating to detect a direction of the linear shadow or the edge and its intensity. Hence, the subjects to be observed in an X-ray image are medical tools such as catheter and a guide wire, or a contrast blood vessel, but usually these shadows have a pixel value smaller than that of the background. It is because these tools are constituted of an object having a high X-ray blocking ability, for example, a metal whose specific gravity is larger than that of water. In the latter processing, it is convenient that these shadows have a pixel value larger than that of the background to the contrary. Therefore, in the pre-process circuit 10, an operation may be also added so that the shadow has a pixel value larger than that of a background.

[0043] Following pre-processing, the image signal is supplied to a direction detection circuit 14, and the direction detection circuit 14 detects a direction and its intensity of a linear shadow or an edge. This processing is performed at each area of interest in an image, and in a pattern having a linear shadow or an edge as shown in FIG. 2, a direction to which linear shadow or edge is running is detected. As a result of detection, a direction and its intensity are outputted. Since a direction to be detected is a direction of a certain pattern within an area of interest, even if a shadow running complicatedly depicts, once an area of interest is properly defined, the direction of the shadow can be assumed as a straight line within the area of interest, and it is easy to detect the direction. Moreover, an intensity of the direction is such an index that indicates more intensified when a shadow is clearer, and weaker when a shadow is more blurred. For example, the linear shadow and the edge illustrated in FIGS. 3A and 3B are ones whose intensity are less intensified compared to those of the linear shadow and the edge illustrated in FIG. 2.

[0044] Two outputs from the direction detection circuit 14 are modified and processed by a vector filter circuit 16. The object of this modification and processing is to make an error detection less influential by forcing the two outputs correlate to the resulting outputs from peripheral area of interest when a signal embedded in the noise is detected and determined as an erroneous direction and intensity. For example, even if a pattern just like a linear shadow is formed incidentally in a noise pattern and a direction is detected as intensified one, in the case where directions of peripheral area are varied with each other, the intensity of the detected direction is weakened by a certain degree. To the contrary, in the case where noise forces a part of an area of a linear shadow to accidentally lose a contrast as if the linear shadow is cut off, an erroneous direction is detected in the area, but when outputs of peripheral area are detected as a certain direction as a whole, direction is modified and corrected even in the area.

[0045] Two outputs from the vector filter circuit 16 (i.e., direction and intensity) are supplied to the spatial filter coefficient generation circuit 18, and a filter coefficient of the spatial filter circuit 12 is determined based on the direction and the intensity, whereby the image signal is processed by a certain spatial filter according to a direction and an intensity of a shadow within the area of interest based on an output from the vector filter circuit 16.

[0046] Examples of coefficient of the spatial filter circuit 12 are shown in FIGS. 4A, 4B and 4C. Spatial filters to be applied are a spatial filter shown in FIG. 4A in which a shadow having an intensified directivity in a horizontal direction is detected, a spatial filter shown in FIG. 4B in which a shadow having a weak directivity in a horizontal direction is detected, and a spatial filter shown in FIG. 4C in which a shadow having an intensified directivity in a direction which inclined approximately 30 degrees with regard to the horizontal direction, respectively. Although exemplified spatial filters described above are made as the sum of coefficients being 10, it is modulated that each coefficient is multiplied by {fraction (1/10)} and the sum of them is 1 so that the level of the image signal is not changed after filtering. These spatial filters have different frequency reduction effects along different directions. For example, in the case of FIG. 4A, an example has a frequency reduction effect toward horizontal direction, but a linear shadow running in the vertical direction is not made blurred since no operation involving with a filter is performed in the vertical direction.

[0047] Thus, the principle of the present invention is to perform noise reduction along the running direction of a linear shadow or an edge-like shadow according to the direction and its intensity of the shadow in each area of interest of an image.

[0048] Detailed operation of each portion in FIG. 1 will be described below.

[0049] The object of the pre-process circuit 10 is to reduce erroneous recognition of a shadow due to a noise in the latter processing steps by facilitating to detect a linear shadow or an edge-like shadow, or by reducing noise. For example, it is commonly known that a linear filter having all of coefficients of conventional 3×3 matrix used in general set at {fraction (1/9)} is a low-pass filter. If the lower frequency component of the image signal is extracted by the pre-process circuit 10, noise reduction is realized to some extent and erroneous recognition is reduced in the latter processing steps even on an image having a lot of noises. Moreover, in the case where an image of an edge being blurred is inputted, if the image is processed by a band pass filter having an edge emphasis action and the like, a linear shadow or an edge-like shadow can be depicted more clearly, the precision of the latter direction detection processing steps is enhanced. Moreover, on an X-ray image, it is frequently observed that a catheter or a guide wire is constituted of an object made of metal or some other materials having an X-ray blocking ability, a shadow is formed by a smaller pixel value. In the pre-process circuit 10, an image processing is performed such that this shadow has a larger pixel value than that of the background. This is because pixel value has to be larger in a shadow area so that calculating such a spatial expansion of a shadow is performed, and processing is performed as a largely expanding direction being a direction in the latter processing step of the direction detection circuit 14. It should be noted that if this processing is not performed, a direction having smaller spatial expansion should be made as the direction in the direction detection circuit 14. However, herein, description about the situation in which a shadow of this observation subject (a linear shadow, an edge) has a larger pixel value than that of the background is mentioned. The concretely simplest method is a processing of solely inverting code of a pixel value in the pre-process circuit 10 in the case where a shadow density of the observation subject is smaller than that of the background. It should be noted that the processing of performing reduction of a value of the relevant input image from the largest pixel value among the whole input image will also be good in a processing apparatus which does not perform signed calculation. For example, when an input image is expressed in 12 bits, the largest value of the input image is, 2¹²−1=4,095. Thus, when 4,095−(pixel value of the input image) is calculated in the pre-process circuit 10, it results in a pixel value of the linear shadow included in the input image being larger than that of the background.

[0050] The most important portion of the present invention is the direction detection circuit 14 for detecting a direction of the shadow and its intensity. A method in accordance with the principle component analyzing method as one of this detecting means will be described below.

[0051] First, assume an area of interest of an image as shown in FIG. 5A. This is, for example, a part image of a guide wire depicted when it is inserted into a human body under X-ray fluoroscopy. Assume that a pixel density distribution within this area of interest is a distribution in which data is dispersed only in the number of pieces of pixel value D (i, j) in coordinate point (i, j) fixed as a center of (0, 0), the direction of a linear shadow is considered as a direction of a linear pattern in which spatial distribution of the data is mostly dispersed. An example shown in FIG. 5A is an example in which a linear pattern of pixel density 10 is running on pixel density 0, in this case, it is interpreted that coordinate positions of solid round marks, for example, D(2, −1)=10 pieces of data are present at the position of address (i, j)=(2, −1) on this image. When an image is interpreted as a distribution of the data of D (i, j) pieces at the address (i, j), the principle component analyzing method is applicable, and then in the case of expressing covariance matrix as follows, S₁₁, S₁₂, and S₂₂become as follows: $\begin{array}{cc}A=\left[\begin{array}{cc}{S}_{11}& S_{12}\\ S_{21}& S_{22}\end{array}\right]S_{11}=S_{11}^{\prime }-{\mathrm{Xg}}^{2}z& \left(1\right)\\ S_{12}=S_{12}^{\prime }-\mathrm{XgYgZ}& \left(2\right)\\ S_{22}=S_{22}^{\prime }-{\mathrm{Yg}}^{2}Z& \left(3\right)\\ S_{11}^{\prime }=\sum _{i=-M}^M{i}^2\sum _{j=-M}^{M}D\left(i,j\right)& \left(4\right)\\ S_{12}^{\prime }=\sum _{i=-M}^{M}i\sum _{j=-M}^M\mathrm{jD}\left(i,j\right)& \left(5\right)\\ S_{22}^{\prime }=\sum _{j=-M}^M{j}^2\sum _{i=-M}^{M}D\left(i,j\right)& \left(6\right)\end{array}$

[0052] where M represents a range of an area of interest. In an area exemplified in FIGS. 5A and 5B, M=3. Moreover, Xg, Yg, or Z indicates the sum of center of gravity of horizontal direction, the sum of center of gravity of vertical direction, and the sum of the number of pieces of data, respectively, and are expressed as following expressions. It should be noted that S′₁₁, S′₁₂, or S′₂₂is a secondary moment surrounding the origin, respectively. $\begin{array}{cc}{X}_g=\sum _{i=-M}^{M}i\sum _{j=-M}^{M}D\left(i,j\right)/Z& \left(7\right)\\ Y_g=\sum _{j=-M}^{M}j\sum _{i=-M}^{M}D\left(i,j\right)/Z& \left(8\right)\\ Z=\sum _{i=-M}^M\sum _{j=-M}^{M}D\left(i,j\right)& \left(9\right)\end{array}$

[0053] In this calculation, Z is the sum of pixel values within the area of interest, Z/(2M+1)²represents the average value of pixel values of the areas of interest. Therefore, if the average value of an input image is determined by other method, Z is not required to be determined. For example, in a modified example shown in FIG. 13 which will be described later, since the average value of the image is determined in the pre-process circuit 10, Z can be estimated from the result determined thereof.

[0054] Thus, the eigen value of covariance matrix A is determined as follows: $\begin{array}{cc}{\lambda }_1=\left(1/2\right)\left[\left(S_{11}+S_{22}\right)+\sqrt{\left\{{\left(S_{11}-S_{22}\right)}^2+4S_{{12}^2}\right\}}\right]& \left(10\right)\\ {\lambda }_2=\left(1/2\right)\left[\left(S_{11}+S_{22}\right)-\sqrt{\left\{{\left(S_{11}-S_{22}\right)}^2+4S_{{12}^2}\right\}}\right]& \left(11\right)\end{array}$

[0055] Assume that orientation of eigen vector to the eigen value is θ₁, θ₂, the followings can be obtained:

[0056] tan(θ₁)=(λ₁−S₁₁)/S₁₂  (12)

[0057] tan(θ₂)=(λ₂−S₁₁)/S₁₂  (13)

[0058] Specifically, since the orientation of eigen vector corresponding to the larger value out of eigen values λ₁and λ₂is the orientation of the principal axis which makes dispersion the maximum (a direction of larger spatial expansion), λ₁is employed and θ₁can be determined from the equation (12). It should be noted that θ₁can be set in a range between 0 and 180 degrees, since θ₁is the orientation of the principal axis.

[0059] Moreover, the most general method among definition methods of direction and its intensity is expressed as:

[0060] C=λ₁/(λ₁+λ₂)  (14)

[0061] by which a contribution rate is determined, and it can be determined between C=0 to 1. Specifically, S₁₁, S₂₂, and S₁₂are calculated for pixel value D within an area of interest of an image (i, j) (i=−M, −(M−1), . . . , M; j=−M, −(M−1), . . . , M)), θ₁and C are determined, and then an index of a direction and its intensity of a running direction pattern can be determined, respectively.

[0062] Up to this point, the principle for determining an index of a direction of the shadow and its intensity has been described. Embodiments in which actual calculations are performed will be shown below. For example, in order to determine center of gravity Xg in the horizontal direction, first, the sum of the pixel values along the vertical direction is calculated per each column, and after values of (2M+1) pieces are determined, and then the values of (2M+1) pieces are accumulated along the horizontal direction. This calculation will be capable of being performed by the same calculation method as that of one-dimensional linear filter in general, and the filter coefficients are determined according to the distance from the origin. Specifically, center of gravity Xg in the horizontal direction is determined by performing the following operations for pixels within an area of interest:

[0063] (1) An output is fetched by applying the one-dimensional filter having all filter coefficients of 1 along the vertical direction.

[0064] (2) This operation is performed for all columns, and (2M+1) pieces of values arranged in the horizontal direction are determined.

[0065] (3) Assuming that filter coefficients are made as −M, −(M−1), −(M−2), . . . , (M−1), M for values of (2M+1) pieces, one-dimensional filter is applied along the horizontal direction, and then this output is assumed as Xg.

[0066] In a similar manner, filter coefficients of the vertical and the horizontal direction are exchanged, center of gravity Yg of the vertical direction can be determined.

[0067] Assuming all filter coefficients are 1.0, the sum of data pieces Z can be determined by the same hardware.

[0068] Furthermore, for S′₁₁, S′₁₂, and S′₂₂, product-sum calculation will be applied when filter coefficients are changed, S′₁₁can be determined as follows:

[0069] (1) An output is fetched by applying the one-dimensional filter having all filter coefficients of 1 along the vertical direction.

[0070] (2) This operation is performed for all columns, and (2M+1) pieces of values arranged in the horizontal direction are determined.

[0071] (3) Assuming that filter coefficients are made as M², (M−1)², (M−2)², . . . , (M−1)², M²for values of (2M+1) pieces, one-dimensional filter is applied along the horizontal direction, and then this output is assumed as S′₁₁.

[0072] Furthermore, S′₂₂can be determined when filter coefficients are changed between the horizontal direction and the vertical direction. Then, S′₁₂can be determined as follows:

[0073] (1) An output is fetched by applying the one-dimensional filter having filter coefficients of −M, −(M−1), −(M−2), . . . , (M−1), M along the vertical direction.

[0074] (2) This operation is performed for all columns, and (2M+1) pieces of values arranged in the horizontal direction are determined.

[0075] (3) Assuming that filter coefficients are made as −M, −(M−1), −(M−2), . . . , (M−1), M for values of (2M+1) pieces, one-dimensional filter is applied along the horizontal direction, and then this output is assumed as S′₁₂.

[0076] Summarizing the above described calculations, calculations are carried out in the configuration as shown in FIG. 7. Three vectors V₀(i), V₁(i), V₂(i) are determined by a product-sum calculation for the vertical direction, and then by a product-sum calculation of these for the horizontal direction, each value from the equation (4) to the equation (9) can be determined.

[0077] After these numerical values are determined, high speed calculation can be carried out using ROM or high speed CPU and the like because only numerical values are calculated.

[0078] Calculation of any size involving with an area of interest is possible, but a size of more than 7×7 is desirable. It is expected that an intensity of the direction is extremely low in an area not including a linear pattern or an area consisting of only noise. That dispersion of the horizontal and vertical directions is equal, i.e., S₁₁=S₂₂, or S₁₁and S₂₂are capable of being approximately equal with each other is a prerequisite to it. For realizing it, it is necessary to have enough statistics, generally speaking, more than 50 pieces are desirable, therefore, a size of more than 7×7 is desirable.

[0079] Now, for an image as a whole, while the center position of an area of interest is gradually changed, a direction and it intensity of local pattern is determined. For example, directions and their intensity are going to be determined while center position of an area of interest is shifted per 2 pixels to the horizontal direction on an image consisting of pixels of 500×500 matrix, information on directions and their intensity per an area divided in 250 pieces in the horizontal direction are obtained. In a similar manner, also in the vertical direction, scanning is performed while center position of an area of interest is shifted per 2 pixels whole over the image, information of 250×250 matrix is obtained. In the case where shift amount of this area of interest is small, since a certain area of interest and another area of interest which is shifted from the former have many superimposed pixel information between these areas, directions and their intensity which are determined are highly correlated each other. That is to say, in this case, redundancy is large. For example, when center position of an area is shifted per 1 pixel, information of 500×500 matrix is obtained, but since the results are mutually redundancy predominantly occupying ones in consideration of the large number of calculations for the whole image. Therefore, these calculations are wasted to some extent. To the contrary, when shift amount is so large, the number of calculations is small, but the independency of the local information is significant, and is easy to be influenced from the noise on the image. Maintaining the correlation with neighboring areas is also necessary to provide a vector filter on the next occasion, and it is desirable that shift amount is less than half of the size of an area of interest. For example, as a preferable example of the present invention, a method of scanning the whole image and locally determining directions and their intensity while an area of interest having a size of 7×7 is shifted per 2 pixels is recommended.

[0080] After determining a direction θ₁and its intensity C by the direction detection circuit 14, a processing that determines more accurate directions and their intensity is performed on the basis of neighboring information by the vector filter circuit 16. An example shown in FIG. 8 is one of the vector filter circuits 16. When the directions of the areas of interest are determined per position area as shown in FIG. 8, even if the direction of the center pixel of 3×3 pixels is determined as a wrong direction due to the noise, a procedure in which the direction of the center of pixel is modified and corrected from information of a direction of neighboring pixels by supplying the image signal to the vector filter circuit 16.

[0081] This vector filter circuit 16 is constituted of the followings:

[0082] The first preferable example of configuration is a method of classifying angles of direction vector of neighboring small areas, and determining by majority. In FIG. 9, a conceptual diagram of classifying per 22. 5 degrees is shown. For example, in this case, since directions entering in a range of 45 degrees±11. 25 degrees is most common, it results in an output of the vector filter 16 being 45 degrees. On the other hand, intensity of direction can be decided by the sum of intensity of vectors selected by majority only divided by the sum of intensity of all-direction vectors. For example, assume that directions θs and their intensity Cs of a small subject area #0 and its 8 neighboring areas #1 to #8 are as the followings:

[0083] Since areas having directions entering in a range of 45±11. 25 degrees decided by majority are 6 areas of #2, #3, #5, #6, #7, and #8, intensity is decided according to these following calculations: $\begin{array}{c}C=\left(0.1+0.9+0.8+0.8+0.5+0.6\right)/\\ \left(0.3+0.5+0.1+0.9+0.7+0.8+0.8+0.5+0.6\right)\\ =3.7/5.2\\ =0.7\end{array}$

[0084] The second preferred example of configuration of the vector filter 16 is a method of calculating matrix Ag which averages all of the neighboring small areas of S₁₁, S₂₂, S₁₂and determining direction vector and its intensity on the basis of Ag matrix.

[0085] Matrix Ag is given as follows: $A=\left[\begin{array}{cc}{\mathrm{Sg}}_{11}& {\mathrm{Sg}}_{12}\\ {\mathrm{Sg}}_{21}& {\mathrm{Sg}}_{22}\end{array}\right]$

[0086] Sg₁₁=Σ(the sum of S₁₁of neighboring small areas)/(number of neighboring small areas) Sg₂₂=Σ(the sum of S₂₂of neighboring small areas)/(number of neighboring small areas) Sg₁₂=Σ(the sum of S₁₂of neighboring small areas)/(number of neighboring small areas)

[0087] When the sum of S₁₁of the neighboring small area or the like is calculated, not a simple average value but a weighted average value may be used.

[0088] The object of vector filter 16 described above by two preferred examples of configuration is to estimate a pattern direction and its intensity in a wide range to a certain extent including the small subject area, whereby direction of the shadow can be modified and corrected by information obtained from peripheral areas even if a pattern direction is erroneously recognized as a specific direction due to noise and the like.

[0089] After vector is modified and corrected on the basis of the results of direction detection in neighboring areas of the subject area by the vector filter circuit 16, a frequency reduction filter specific for this direction is applied. Examples of these filters are already shown in FIGS. 4A to 4C, but these filter coefficients are changed for every pixel by a spatial filter coefficient generation circuit 18 on the basis of information of directions and their intensity determined locally. In FIG. 10, an input image, an example of an output of the vector filter circuit 16, and a spatial filter of 5×5 to be provided on the basis of the information of the former ones are exemplified for 3 local areas. It is described that a linear shadow and an edge portion in an input image have intensified direction component locally, a spatial filter for reducing spatial frequency along that direction is applied to such portions and an all-directional spatial filter is applied in the background portion.

[0090] Thus, in the present embodiment, a spatial filter capable of setting a coefficient per pixel unit is applied so that it is difficult to carry out real time calculation when size of the spatial filter is large. Therefore, calculation circuit can be simplified by sequentially applying a spatial filter of smaller size instead of applying one larger filter. For example, product-sum calculations of 25 pieces is required for a spatial filter of 5×5, but if a spatial filter of 3×3 is provided 2 times in series, 18 pieces of product-sum calculations are enough to provide, and circuit scale can be minimized. In FIGS. 11A and 11B, an embodiment is shown. An output ζ1 of spatial filter of 5×5 applied once (FIG. 11A) and an output ζ2 of spatial filter of 3×3 (FIG. 11B) applied twice results in the identical output.

[0091] As described above, according to the present embodiment, the noise reduction can be achieved without blurring the linear shadow or the peripheral portions of an edge portion or lowering a contrast by providing a spatial filter according to a direction of pattern of a linear shadow or an edge portion included in an image and its intensity along its direction.

[0092] Other embodiments of an image processing apparatus according to the present invention will be described below. The portions identical with those of the first embodiment are attached with identical reference numerals, and its detailed description will be omitted.

[0093] Second Embodiment

[0094] Since the block diagram of the second embodiment of the present invention is identical with the block diagram of the first embodiment of the present invention, an illustration thereof will be omitted. The second embodiment of the present invention is an embodiment involving with a modification of processing of detecting a direction.

[0095] In the first embodiment of the present invention, S₁₁, S₁₂, and S₂₂are determined by the equations (1) to (3). However, in general, a center of gravity is changed by average value of pixel values of an image. Therefore, when average level of an image is high, center of gravity is determined near the origin of an area of interest, and when average level of an image is low, center of gravity is located far from the origin. Moreover, in the case of an image in which the difference between two certain images are taken, for example, such as DSA (digital subtraction angiography) which is obtained in the X-ray diagnostic apparatus whose pixel average value is close to zero, it is recognized that the sum of data pieces may be zero, calculation using the equation (1) may occur the errors. Thus, a method of not depending on average value of pixel values as means for enhancing the durability of the present invention will be shown as the second embodiment of the present invention.

[0096] In the second embodiment of the present invention, S₁₁, S₁₂, and S₂₂are determined by the following equations instead of the equations (1) to (3).

[0097] S₁₁=S′₁₁−Xg′²(−2Z′)  (19)

[0098] S₁₂=S′₁₂−Xg′Yg′(−2Z′)  (20)

[0099] S₂₂=S′₂₂−Yg′²(−2Z′)  (21)

[0100] $\begin{array}{cc}{X}_g^{\prime }=\sum _{i=-M}^{M}i\sum _{j=-M}^{M}D\left(i,j\right)/Z^{\prime }& \left(22\right)\\ Y_g^{\prime }=\sum _{j=-M}^{M}j\sum _{i=-M}^{M}D\left(i,j\right)/Z^{\prime }& \left(23\right)\\ Z^{\prime }=\sum _{i=-M}^M\sum _{j=-M}^M\left|D\left(i,j\right)-D_{\mathrm{av}}\right|& \left(24\right)\end{array}$  D_av=Z/(2M+1)²  (25)

[0101] The significance of this calculation is in defining center of gravity for component of difference from average value of pixel values within an area of interest as (X′g, Y′g).

[0102] Center of gravity is determined as a value not depending on average value by determining center of gravity with such a definition, because center of gravity position is determined through the procedure subjecting only the component of difference from average value even if average value D_avtakes any value.

[0103] In order to perform this calculation by actual hardware, the following calculation will be performed by the pre-process circuit 10.

[0104] Output of the pre-process circuit=(Input image)−(Low pass filter image of input image(=D_av))  (26)

[0105] According to this calculation, since an image inputted into the direction detection circuit 14 is an image in which average value is zero locally, calculation of the equation (24) can be performed as D_av=0, hardware configuration is simplified.

[0106] Third Embodiment

[0107] A third embodiment of the present invention is also an embodiment of involving with a modification of a processing of detecting a direction. In the present embodiment of the present invention, S₁₁, S₁₂, and S₂₂are simply represented by simple secondary moment around the origin, they are determined as follows:

[0108] S₁₁=S′₁₁  (27)

[0109] S₁₂=S′₁₂  (28)

[0110] S₂₂=S′₂₂  (29)

[0111] When an eigen value of matrix A is determined using these equations, the farther a linear shadow is left from the center of an area of interest, the weaker intensity of directivity gradually is, and the still farther, another direction other than a direction of a linear shadow may be erroneously detected. However, because the directivity of an area of interest near the linear shadow is properly determined, it can be put to practical use. However, at the same time, calculation of Xg, Yg should be performed, and in the case where it is found too large shift of center of gravity is present, which means that the center position of the area of interest is far from the linear shadow, it is necessary to perform an operation such as one forcing the intensity of the directional be zero.

[0112] Fourth Embodiment

[0113] A fourth embodiment of the present invention is also an embodiment of involving with a modification of a processing of detecting a direction. Although a direction θ₁is mathematically determined without depending on average value of an image, contribution rate C which indicates an intensity has a mathematical nature that it gradually becomes smaller when the average value is larger. Moreover, if average value of an image is zero, it may result in λ₁+λ₂=0, and may not determine correctly the directional.

[0114] For solving this problem, an amount P not depending on average value of an image is defined as P as follows:

[0115] P=λ₁−λ₂  (30)

[0116] where the measured P is compared with λ₁−λ₂previously measured under various conditions can be carried out, and an index C′ which indicates intensity of a direction can be obtained.

[0117] In the case where only noise is distributed on an image without any linear shadow or edge-like pattern, P complies with probability distribution decided by a size of an area of interest and standard deviation a of noise of an image. This formulation is complex, but it mostly complies with normal distribution. On the other hand, the standard deviation σ of noises can be determined from imaging conditions of an image and the like. As described already, in the case of an X-ray image, since the standard deviation a of noises of an image is in proportion to a root of X-ray photon number N, assume that average value of an image is L, the number of photon of X-ray can be estimated, and σ value is also determined. For example, when L₀of a pixel value is generated for an X-ray photon, the average value L of the image is obtained by:

[0118] N=L/L₀

[0119] Thus, $\sigma =\sqrt{\left\{L_0\left(L/L_0\right)\right\}}$

[0120] That is to say, by obtaining average value of the image, probability distribution of P can be calculated in the case where only noise is present. Assume that this distribution is f(P) (f(P) is a distribution in which it is to be 1.0 if integrated), probability can be calculated between 0 and 1 by determining where Px=λ₁−λ₂is located in this distribution, the Px=λ₁−λ₂being actually determined by direction detection processing within the area of interest. FIG. 12 represents an example of f(P) which was previously determined and Px which is actually obtained, if using integral value η of oblique line portion in the figure, the following equation can be obtained:

[0121] C′=1−η

[0122] where C′ represents intensity of a direction.

[0123] In FIG. 13, a flow of signal of the fourth embodiment of the present invention is shown. In the pre-process circuit 10, a low-pass filter 10asuppresses the high frequency component noise. The subtracter circuit 10b outputs the high frequency component of the image signal. f(P) is generated based on the output signal from the low-pass filter 10a,i.e., a standard deviation a of noises corresponding to its average value. On the other hand, the direction detection circuit 14 determines P_x=λ₁−λ₂. Moreover, η is calculated from a position of P_xof f(P) distribution, an then C′ is determined.

[0124] Fifth Embodiment

[0125] Next, as the fifth embodiment of the present invention, an X-ray diagnostic apparatus of a concrete application of an image processing apparatus of the present invention will be described. Conventionally, an X-ray diagnostic apparatus is a useful apparatus by which a subject is examined in a non-destructive manner, it is desirable for an X-ray diagnostic apparatus to obtain its high grade and fine moving image. Moreover, in order to avoid upsizing and expanding the depth of the apparatus, a flat type detection apparatus instead of an image intensifier tube is needed. For achieving these needs, a flat type detection apparatus of active matrix type is proposed. These flat type detection apparatus includes an indirect conversion method and a direct conversion method, the direct conversion method which directly detects electron-hole pair generated by an X-ray irradiation can obtain more high grade and fine image comparing to that of the indirect conversion method. This direct conversion method is mentioned, for example, in U.S. Pat. No. 5,319,206; “A New Digital Detector for Projection Radiography”, Denny L. Lee et al., SPIE. vol. 2432, pp 237-249, 1995; and Japanese Patent KOKAI Publication No. 11-4821. In FIGS. 14A and 14B, a section view and circuit a diagram of the detection apparatus described in Japanese Patent KOKAI Publication No. 11-4821 are shown.

[0126] Description is given with reference to FIGS. 14A and 14B. First, an X-ray which injects into a photo-conductor 301 generates an electron-hole pair 309, and electric charge is distributed by the signal capacity 305, and a capacity formed by a pixel electrode 304 and an opposing pixel electrode 300. An electric charge of the X-ray injection is collected through the drains D1 to Dn into the integrator by opening the gate G of transistor by a signal from a gate signal line G1. In a direct conversion method, a high grade and fine image can be realized by making a pixel electrode shape be finer and more precise as it is characterized in that all information injected into the photoconductor 301 is collected as the electron hole-pair 309. Herein, the reference numerical 308 indicates the power supply.

[0127] Now, although catheter therapy is frequently performed under fluoroscopy as a medical technology utilizing an X-ray, since an X-ray fluoroscopy reduces the dose in order to reduce exposure amount to a patient comparing with an X-ray radiography, it has a problem that noise is significantly superimposed on an image, linear shadows of a catheter, other guide wires and blood vessels and the like are hindered from the background noise and it is difficult to be seen clearly. In consideration with it being performed in an X-ray diagnostic system having flat type detection apparatus utilizing a direct conversion method described above, because a fine and high grade image can be formed according to its principle, this phenomenon is significantly occurred due to more frequent emergence of quantum noise of an X-ray at lower doses.

[0128] However, if an X-ray diagnostic system having a flat type detection apparatus of the direct conversion method is combined with an image processing apparatus according to the present invention, especially excellent results in which the background noises can be selectively suppressed without damaging the contrast of a linear shadow and edge-like shadow, while essentially making the best use of characteristics of the flat type detection apparatus excellent in an ability with respect to high resolution and high grade and fine imaging function are provided for the flat type detection apparatus of the direct conversion method. Specifically, an X-ray image obtained by detecting an X-ray transmitted through the subject body with a flat type detection apparatus of the direct conversion method as shown in FIGS. 14A and 14B is inputted into an image processing apparatus according to the present invention as an input image of FIG. 1, a spatial filter processing having a directivity according to a direction of a linear shadow and edge-like shadow and its intensity is provided, and then an X-ray diagnostic apparatus in which a high grade and fine image can be obtained is realized. Therefore, during catheter therapy using lower doses, background noises can be selectively suppressed without damaging the contrast of a linear shadow. An X-ray diagnostic apparatus of the fifth embodiment is applicable to any embodiment from the first to fourth embodiments described above.

[0129] Additional advantages and modifications will readily occur to those skilled in the art. Therefore, the present invention in its broader aspects is not limited to the specific details, representative devices, and illustrated examples shown and described herein. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. For example, in the above mentioned description, an X-ray image is exemplified, but not limited thereto, the present invention is similarly applicable to an image in general in which random noises are superimposed, such as a CT image, an MRI image and the like. Each embodiment is solely described, but multiple embodiments are combined at their convenience.

[0130] According to the present invention, as described above, the following effects are obtained:

[0131] (1) An image processing apparatus according to the present invention comprises means for detecting a direction and an intensity of a linear or an edge-like shadow included in an image; and means for processing the image using a spatial filter having a directivity according to the direction and intensity detected by said detecting means.

[0132] According to this image processing apparatus, the precision of separation between a linear shadow or an edge and the background noise is enhanced, and background noises can be suppressed without lowering the contrast.

[0133] (2) An image processing apparatus according to the present invention is an image processing apparatus described in (1), and in which a pixel value of the shadow is larger than a pixel value of a background of the image; said detecting means detects a direction based on an orientation of eigen vector corresponding to a larger eigen value out of two eigen values of covariance matrix of an area of interest of the image; and said processing means processes the image using a digital spatial filter whose frequency characteristic of the direction detected by said detecting means is different from a frequency characteristic of other directions.

[0134] (3) An image processing apparatus according to the present invention is an image processing apparatus described in (1), and in which a pixel value of the shadow is smaller than a pixel value of a background of the image; said detecting means detects a direction based on an orientation of eigen vector corresponding to a smaller eigen value out of two eigen values of covariance matrix of an area of interest of the image; and said processing means processes the image using a digital spatial filter whose frequency characteristic of the direction detected by said detecting means is different from a frequency characteristic of other directions.

[0135] According to the image processing apparatus described in (2) and (3), a direction and its intensity can be detected in real time and high speed processing can be carried out as well as the precision of separation between a linear shadow or edge and the background noise is enhanced, and background noises can be suppressed without lowering the contrast.

[0136] (4) An image processing apparatus according to the present invention is an image processing apparatus described in one of (1), (2) and (3), and in which said detecting means sequentially detects a direction and an intensity of the shadow per an area of interest of the image while shifting the area of interest to a horizontal direction and a vertical direction; and said processing means processes the image using a spatial filter with a size of the area of interest while shifting the filter over the image and changing the directivity.

[0137] (5) An image processing apparatus according to the present invention is an image processing apparatus described in (2) or (3), and in which said area of interest comprises at least 5×5 pixel arrays; and said detecting means performs a two-dimensional product-sum calculation between predetermined coefficients and pixel values in the area of interest and calculates a covariance matrix of the area of interest based on the result of the two-dimensional product-sum calculation.

[0138] An image processing apparatus described in (2), (3), (4) or (5) can reduce the number of product-sum calculations for detecting directions and their intensity, hence the scale of the hardware may be smaller and high speed processing can be carried out.

[0139] (6) An image processing apparatus according to the present invention is an image processing apparatus described in (1), and in which said detecting means estimates dispersion of random noises within an area of interest based on an average value of pixel distribution within the area of interest, estimates probability distribution which is determined by the estimated dispersion of random noises and a matrix size of the area of interest with respect to an index representing intensity, and calculates a probability that intensity calculated based on pixel values within the area of interest can be taken under the estimated probability distribution, the calculated probability being the index representing intensity.

[0140] (7) An image processing apparatus according to the present invention is an image processing apparatus described in (2) or (3), and in which said detecting means detects a ratio of two eigen values of a covariance matrix as an index of intensity of a direction.

[0141] (8) An image processing apparatus according to the present invention is an image processing apparatus described in (1), and in which said detecting means calculates a new covariance matrix by performing a weighted average to elements of an old covariance matrix of an area of interest set up and elements of a covariance matrix of a peripheral area of the area of interest and determines a pattern direction vector based on a direction and an eigen value of an eigen vector of the new of covariance matrix as indices of a pattern running direction and an intensity of the pattern direction of the area of interest; and said processing means processes the image using a spatial filter having a directivity according to the pattern direction vector determined by said detecting means.

[0142] Since an image processing apparatus described in (6), (7) or (8) can determine center of gravity position without depending on average value of a pixel value of an area of interest, the precision of detection of directions and their intensity is enhanced.

[0143] (9) An image processing apparatus according to the present invention is an image processing apparatus described in (8), and in which said detecting means calculates new covariance matrixes of peripheral areas of the area of interest, calculates directions of eigen vectors of the peripheral areas, and determines a direction of eigen vector which is appeared in the largest number of areas as a pattern direction of the area of interest.

[0144] (10) An image processing apparatus according to the present invention is an image processing apparatus described in one of (1) to (3), and in which said detecting means processes the image using a frequency reduction filter, and performs direction detection processing to a difference image between an output image from the frequency reduction filter and an input image to the frequency reduction filter.

[0145] (11) An image processing apparatus according to the present invention is an image processing apparatus described in one of (1) to (3), and in which said detecting means performs direction detection processing after inverting a code of a pixel value of the image.

[0146] (12) An image processing apparatus according to the present invention is an image processing apparatus described in one of (1) to (3), and in which said detecting means performs direction detection processing after subtracting the image from the maximum value decided by the maximum bit number of the image.

[0147] (13) An image processing apparatus according to the present invention is an image processing apparatus described in one of (1) to (3), and in which said detecting means performs pre-processing for suppressing noise spectrum while emphasizing a band having signal spectrum of the image.

[0148] (14) An image processing apparatus according to the present invention is an image processing apparatus described in (6), and in which the index representing intensity is an eigen value obtained based on a covariance matrix around a position decided by a center position of an area of interest and a center of gravity of pixel distribution within the area of interest.

[0149] (15) An image processing apparatus according to the present invention is an image processing apparatus described in (6), and in which the index representing intensity is a difference or a ratio of eigen values obtained based on covariance matrixes around positions decided by a center position of an area of interest and a center of gravity of pixel distribution within the area of interest.

[0150] According to an image processing apparatus described in (12), (13), (14) or (15), the precision of detection of directions and their intensity is enhanced.

[0151] (16) An X-ray diagnostic image processing apparatus according to the present invention comprises means for obtaining an X-ray image of a subject using an X-ray detection apparatus of direct conversion method; and an image processing apparatus comprising means for detecting a direction and an intensity of a linear or an edge-like shadow included in the X-ray image, and means for processing the X-ray image using a spatial filter having a directivity according to the direction and intensity detected by said detecting means.

[0152] (17) An X-ray diagnostic apparatus according to the present invention is an X-ray diagnostic apparatus described in (16), and in which an image processing apparatus utilizing an image processing apparatus described in any one of (2) to (15) may be employed.

[0153] According to an X-ray diagnostic apparatus described in (16) or (17), an small sized and light-weighted X-ray diagnostic apparatus by which a high resolution and highly graded fine dynamic image is obtained can be realized, background noises can be selectively suppressed without damaging the contrast of a linear shadow even during a catheter therapy.