Automated Method and System for the Detection of Gross Abnormalities and Asymmetries in Chest Images

1. Field of the Invention

The invention relates to a method and system for the computerized automatic detection of gross abnormalities and asymmetries in chest images, and more specifically to a method and system for detection in digital chest radiographs. Asymmetries are detected by multiple stages of global and local gray-level thresholding along with contour detection. Abnormalities are detected based on deviation from expected symmetries between the left and right lungs, using such features as size and density of the aerated lung regions.

2. Discussion of the Background

Computer-aided diagnosis (CAD) has potential to become a valuable tool for detecting subtle abnormalities in chest radiographs. It would be useful for a CAD scheme to detect more large-scale abnormalities, which commonly cause abnormal asymmetry on the radiograph. In general, asymmetric abnormalities appear as a substantial decrease in the area of aerated lung in one hemithorax. This would include interstitial infiltrates, dense air space infiltrates, pleural effusions, large masses, or unilateral emphysema.

Most CAD schemes currently employed in digital chest radiography are specific to one particular and often localized pathology, for example lung nodule, interstitial infiltrate, pneumothorax or cardiomegaly. These schemes often utilize a priori information regarding the "normal" appearance of the ribcage, diaphragm and mediastinum in a digital chest radiograph. A potential problem arises when the nature of the thoracic abnormality is such that it substantially affects the volume of the lungs. An abnormality of this type will usually cause a decrease in the aerated lung region (i.e. the high optical density associated with the normally low attenuation of the lungs) as projected onto the radiograph. This can substantially alter the overall morphology of the chest, resulting in potential failure of such CAD schemes. Detection of these abnormalities may also prove useful in prioritizing abnormal cases in a picture archiving and communication system (PACS).

The present application discloses a technique for the automated detection of abnormal asymmetry in digital chest radiographs. The method consists of an iterative global thresholding technique in conjunction with a contour detection algorithm to construct an initial set of contours around the two projected aerated lung regions in a chest image. In order to identify the lungs more accurately, a local thresholding technique is then applied within regions of interest (ROIs) centered along the contours that result from the global thresholding. The areas and densities of the two lung regions identified in this manner can be compared in order to determine whether an asymmetric abnormality is present.

Accordingly, an object of this invention is to provide an automated method and system for detecting and displaying gross abnormalities and asymmetries in medical images of the chest.

Another object of this invention is to provide an automated method and system for the iterative gray-level thresholding of lung regions in a chest image.

Another object of this invention is to provide an automated method and system for the local thresholding of lung regions in a chest image.

Another object of this invention is to provide an automated method and system for defining the edge of the aerated lung regions by using features based on the anatomic structure of the lung and its surround.

Another object of this invention is to provide an automated method and system for the detection of lung boundaries within a radiographic image of the thorax.

Another object of this invention is to provide an automated method and system for the extraction of objects (features) within the lung regions in chest images of the thorax by using size and shape of the aerated lung regions.

Another object of this invention is to provide an automated method and system for distinguishing abnormal regions from normal anatomy based on asymmetries between the left and right lung regions.

These and other objects are achieved according to the invention by providing a new and improved automated method and system in which an iterative, multi-level gray level thresholding is performed, followed by a local thresholding and an examination of regions based on anatomically-based features, and then followed by an analysis of the possible asymmetries. For example, asymmetric abnormalities may present on the chest radiograph as a decrease in the area of aerated lung in one hemithorax and in different degrees of subtlety due to size and composition. In order to allow the various abnormalities to be detected, the aerated areas are analyzed and compared with the opposing hemithorax.

A more complete appreciation of the invention and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:

FIG. 1 is a schematic diagram illustrating the automated method for detection of asymmetric abnormalities according to the invention;

FIG. 2 is a schematic diagram illustrating the automated method for the iterative, global thresholding for the detection of the boundary of the thorax according to the invention;

FIG. 3 is a graph illustrating the gray-level histogram of a chest image indicating the computer-determined location of the lung peak and the minimum between the lung and mediastinum peaks; This represents the range of gray values used in the iterative thresholding technique.

FIGS. 4A-4E shows binary images created by thresholding a chest image at different gray values;

FIG. 5 is a graph showing the empirically-determined centroid limits plotted on image-based coordinates, where contours with centroids outside this limit are eliminated from the image;

FIG. 6 is a schematic diagram illustrating the automated method for the local thresholding for the detection of the boundary of the thorax according to the invention;

FIG. 7 is a schematic diagram showing the placement of ROIs for local threshold analysis along the initial contours that result from the iterative global threshold process;

FIGS. 8A and 8B are diagrams illustrating local threshold analysis;

FIGS. 9A and 9B are schematics illustrating the features (area and density, respectively) extracted for use in the comparison of the left and right aerated lung regions;

FIG. 10 is a schematic diagram illustrating the automated method for comparison of left and right lung regions and the determination of the presence of an abnormality;

FIG. 11 is a graph of the left lung area plotted as a function of right lung area for 70 cases. The regression line shown is based on true normal cases only;

FIG. 12 is a graph illustrating the ROC curve showing the performance of the method in detecting gross abnormalities and asymmetries; and

FIG. 13 is a schematic block diagram illustrating a system for implementing the automated method for the detection of gross abnormalities and asymmetries in chest images.

Referring now to the drawings, and more particularly to FIG. 1 thereof, a schematic diagram of the automated method for the detection of gross abnormalities and asymmetries in chest images is shown. The overall scheme includes an initial acquisition of a radiograph of the thorax and digitization to produce a digital chest image (step 10). Detection of aerated lung boundaries is performed initially using an iterative, global thresholding technique (step 11) which includes a centroid test. After which, the initial contours are used for positioning of the ROIs for a local thresholding technique (step 12). After the local thresholding procedure, there is a correction for the costo- and cardiophrenic angle (step 13). The lung contours are then determined (step 14). By analyzing the resulting contours and the projected lung areas (step 16), the chest image is assigned a likelihood of having a gross abnormality or asymmetry (step 16).

FIG. 2 shows a schematic diagram illustrating the automated method for the iterative global thresholding for the detection of the boundary of the thorax. Initially, horizontal gray-level profiles in the image are calculated (step 20) and used to determine the location of the mediastinum and the lung apices. This information is used throughout the scheme in order to differentiate the right hemithorax from the left and to identify an upper bound in the image that prevents the contours from growing along the neck. This information is required for each image to compensate for differences in patient positioning.

A global gray-level histogram is used to initially segment the lungs in the chest image (step 21). In an effort to obtain more uniform histograms, the calculation of the histogram is effectively centered over the thorax. In this example, a 181.times.141 pixel region centered 70 pixels from the top of the image, i.e. over the thorax, was chosen. Step 22 of the method shown in FIG. 2 where the lung and mediastinum peaks are located will now be described. The typical histogram (FIG. 3) resulting from such a region exhibits two peaks 30 and 31: one centered over lower gray values that corresponds to pixels predominantly within the lungs, and another centered over higher gray values that corresponds to pixels predominantly in the mediastinum, in the ribcage edge, and, presumably, in any asymmetric abnormality that may be present, respectively.

The goal with regard to performing global threshold analysis, therefore, is to use the histogram in order to determine an appropriate gray value that separates the gray values belonging to pixels within the aerated lung region from those that are located outside the lungs. The task of determining an appropriate threshold proved to be impossible based on the selection of a single gray value. Values were either too low and the resulting binary image insufficiently captured the lung regions, or the values were too high and the lung regions merged with regions outside the lungs; in most cases, a certain threshold value resulted in the former condition in one area of the lung region and in the latter condition in another area.

This problem is overcome by introducing an iterative global threshold scheme. Instead of choosing one gray-level threshold value, a range of values is used in succession. The running slope (first derivative) of the global gray-level histogram is used in order to identify the gray value at which the peak (30 in FIG. 3) corresponding to the lung regions occurs and the gray value at which the minimum between the lung and mediastinum peaks occurs (32 in FIG. 3). A number of equally-spaced gray values between this peak and minimum are then each used in a threshold iteration. In this example 7 gray values were detected. A typical histogram along with the range of gray-level thresholds used during the iterative global gray-level thresholding is shown in FIG. 3.

The iterations are used to create a binary image (step 23). The first iteration creates a binary image by using the smallest of the seven gray values (i.e. the highest optical density of the range) as the threshold limit. Pixels are turned "on" in the binary image if the corresponding pixel in the image has a gray value greater than a certain level above the background level, such as 10, but less than this threshold limit. FIGS. 4A-4E illustrates, schematically, the binary images that result from applying four different gray-level thresholds to a chest image.

In FIG. 4A, the result of thresholding at a smaller threshold is shown. The actual lung boundary 40 is shown for reference only. The thresholding produces a number of regions 41 within the lung boundary 40 and a region 42 outside of the boundary 40. The region 42 is eliminated by a centroid check described below.

The resulting binary image is sent to the contour detection routine, which utilizes a connectivity scheme (such as 8-point) to identify contours representing the boundaries of groups of contiguous "on" pixels (step 24). Connectivity schemes are described, for example, in "Automatic segmentation of liver structure in CT images," by K. Bae, M. L. Giger, C. T. Chen, and C. Kahn, (Medical Physics 20, 71-78 (1993)). The routine also calculates important geometrical properties of these contours, such as the centroid of the contour, contour compactness, contour length (in terms of pixels), and the area enclosed within the contour (in terms of pixels).

The centroid of a contour is used to determine whether the pixels within that contour are within a region that is likely to contain lung (step 25). A "centroid limit" can be empirically constructed by analyzing the centroids of all contours resulting during all of the threshold iterations. The limit, shown in FIG. 5, is based on the spatial distribution in the image plane of the centroids of contours falling within the lung regions and the centroids of contours external to the lungs for 28 chest images. If the centroid of a contour falls outside this limit, the pixels enclosed by this contour are turned "off" (such as region 42 in FIG. 4A). These external regions are thus prevented from merging with regions within the lungs at later iterations where the threshold gray value is greater and the likelihood of such a merge is increased. This allows for the most complete identification of the lungs without also incorporating extraneous area.

This process of thresholding to create a binary image, identifying contours, and turning pixels "off" based on a centroid check is repeated for each of the iterations, with the threshold gray value used to produce the binary image increasing at each iteration. FIGS. 4B-4D show the results of subsequent iterations. In FIG. 4B, larger regions 43 are determined within the lung boundary 40 along with a region 44 which is also eliminated by the centroid check. Larger regions 45 are determined within the lung boundary 40 shown in FIG. 4C. Also shown in FIG. 4C is a region 46 which contains pixels outside of the lung boundary 40, but is not eliminated by the centroid check as it has a centroid within the limit 51 as shown in FIG. 5. FIG. 4D shows an iteration at a higher pixel value where the regions 47 and 48 are now closely approximating the lung boundary 40. The iterative thresholding technique can be thought of as finding the perimeter of a mountain range at various heights, with lower threshold values being closer to the top of the mountain range.

It should be pointed out that a more strict centroid limit could be employed during the later iterations, since the spatial distribution of lung contours typically becomes more confined as the threshold level is increased. The stricter centroid limit shown as 51 in FIG. 5 could eliminate regions such as region 46.

A morphological open operation with a 3.times.3 kernel is also applied during each of the final two iterations (step 26). The morphological open operation is described in, for example, Image Analysis and Mathematical Morphology by J. Serra (Academic, New York, 1982). This combination of an erosion filter followed by a dilation filter eliminates many of the slender artifacts that remain "on" in the binary image as a result of the process that turns "off" regions of the image based on the centroid check. Thus as shown in FIG. 4E, the morphological open operation can eliminate the slender portion 49 of region 48 to give contours 50. The final result of the global threshold iterations is an initial set of contours representing the aerated lung regions in the image (step 27).

It was found that these initial contours tended to under-represent the actual lung regions. Since increasing the largest threshold gray value produced more artifacts in the resulting contours, a local thresholding scheme is applied to the output of the global thresholding scheme. FIG. 6 illustrates schematically the method for the local thresholding. Overlapping ROIs pixels are centered along the initial contours (step 60). In this example ROIs of dimension 31.times.31 were centered at every thirtieth point. Other sizes and spacing are possible. FIG. 7 schematically shows the placement of ROIs 70 along the lung contours 71. The pixels within each ROI are then turned "on" or "off" to form a binary image based on a threshold gray value determined individually for each ROI.

As shown in FIG. 6, it is determined in step 61 whether the ROIs are located on a lateral aspect of the lung. For ROIs along the medial and diaphragmatic aspects of each lung region, the mean pixel value within the ROI is selected as the threshold (step 62). For ROIs along the lateral aspect, a gray level histogram is constructed (step 63), and the initial choice of threshold is set equal to the gray value at which the histogram minimum with the largest gray value occurs. The threshold value actually used for the lateral ROIs is then an average of the initial threshold values of the ROI and its two neighboring ROIs. Thus the threshold values for each of the ROIs is determined by repeating the operation for all ROIs (step 64).

The local thresholding is described in more detail in FIG. 8A and 8B. FIG. 8A shows a histogram of a ROI such as 70 (see FIG. 7), while FIG. 8B shows a histogram of a ROI such as 74. The bimodal distribution results from the overlap of the contour, which produces valleys 80 and 81. In the local thresholding, the gray value p.sub.i of the center pixel of the ROI is replaced with one of two selected gray values depending on whether p.sub.i<p.sub.valley or p.sub.i.gtoreq.p.sub.valley. Or, ##EQU1## Instead of 1, p(x,y) could be left unchanged. This local thresholding thus provides a binary image.

The method, however, may under-represent the costophrenic angle and over-represent the cardiophrenic angle. To accommodate these important anatomical landmarks, two additional ROIs (72 and 73 in FIG. 7) are added to better capture the costophrenic angle, while the criterion for setting the threshold within the ROI overlaying the cardiophrenic angle was altered to yield a lower threshold value when necessary (step 65).

The binary image constructed by thresholding individual ROIs in this manner is sent to the contour detection routine, the morphological open operation is again applied, and the resulting final contours (step 66) are superimposed on the original image (step 67).

The analysis of the projected lung areas (step 15 in FIG. 1) will now be described. The calculated areas and densities of the final contours, corresponding to the projected areas of the aerated lung regions, can then used to make a decision regarding the presence of an asymmetric abnormality (step 16 in FIG. 1). These features are schematically described in FIGS. 9A and 9B.

In FIG. 9A, the determined contours 90 are shown along with the actual lung boundaries 40. The area of contours to the left and the right of the mediastinum (steps 100 and 101) (the area within each of the left and right lung contours) is calculated (i.e. the pixels are counted). The decision criterion is based on a comparison of the right lung area and left lung area (step 102), which are defined as the sum of the areas of all contours to the right and left of the mediastinum, respectively. FIG. 10 shows schematically the method for the comparison of the left and right aerated lung regions. Presumably, all contours outside the lung region are suppressed, and large abnormalities may result in the aerated portion of the lung being divided into two or more regions.

The scheme was applied to a database of 35 normal cases, and an average right lung area to left lung area ratio (referred to as the area ratio measure) was calculated along with the standard deviation of the ratio. In addition, a regression line was constructed based on the relationship between the left lung area and the right lung area for these normal cases (step 93), as shown in FIG. 11. The average perpendicular distance from the regression line of each point corresponding to a normal case in this plot was calculated along with the standard deviation of that distance. A distance threshold from the regression line is used to determine the finding of abnormality. FIG. 12 shows the performance of the method with a ROC curve.

Also shown in FIG. 10 is the area analysis. The average pixel density is calculated for a number of regions in each lung contour (steps 104 and 105). The number and actual locations of the division of the areas can be empirically derived. The lung contours 90 of FIG. 9A have be divided to obtain lung contours 91 in FIG. 9B. In this example the number of regions was selected as three, and the location of the divisions are as shown. A density ratio is determined and compared to the density of normal cases (step 106). A regression line analysis similar to that described with respect to the area analysis is performed (step 107). The abnormality decision is based on a finding of abnormality in either of the area or density analysis (step 108).

FIG. 13 is a more detailed schematic block diagram illustrating a system for implementing the method of the invention. Referring to FIG. 13, radiographic images of an object are obtained from an image acquisition device 130, such as an x-ray device and laser digitizer, and input to the system. Each chest image is digitized by device 130 and put into memory 131. The image data is first passed through the iterative global thresholding circuit 132, which utilizes temporary image memory 133 during the iterations, and then the image data undergoes centroid checking centroid analysis circuit 134 in order to determine the initial boundary of the aerated lung regions. The data is passed to the local thresholding circuit 135, which uses temporary memory 136 during the thresholding, and the correction circuit for the costo- and cardiophrenic angle 137 in order to determine the final boundary of each lung region. Image data within each lung region is passed to the sizing circuit 138 and the density-analysis circuit 139 in order to determine the area and density features for input to the comparison circuit 140. During the comparison of the aerated lung regions, the data is retained in image memory 141. In the superimposing circuit 142 the results are either superimposed onto chest images or given in text format. The results are then displayed on the display system 144 after passing through a digital-to-analog converter 143. The system according to the method can be implemented in both software and hardware, such as a programmed microprocessor or computer.

Obviously, numerous modifications and variations of the present invention are possible in light of the above technique. It is therefore to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described herein. Although the current application is focused on gross abnormalities and asymmetries in chest, the concept can be expanded to the detection of abnormalities in other organs in the human body.