US 6,813,373 B1Grant
Image segmentation of embedded shapes using constrained morphing
Issue Date:2004-11-02
•21 Claims
•20 Drawing Sheets
Abstract
An imaging system and method enables 3-D direct segmentation from a series of spatially offset 2-D image slices in a volume scan. The algorithm first smoothes and preserves the interface edges of the image volume using Bottom-Hat gray scale morphological transform followed by 3-D segmentation using fast 3-D level sets by preserving topology constraints, for example, cortical thickness in a brain volume. The method inputs opposite polarity spheres (contracting and expanding spheres) which morph into shapes within the volume using a surface propagation technique. The speed of propagation is controlled by the likelihood statistical component derived under constraints. During the propagation polygonalization extracts the zero-level surface set. The field distribution is computed using the improved shortest distance method or polyline distance method. The morphing algorithm then morphs the input concentric spheres into interface surfaces such as WM-GM and GM-CSF with cortical constraint. The system is optimized by computing the 3-D field in the narrow band on the morphing spheres.
Metadata
Assignee
- Koninklijke Philips Electronics, N.V.
Inventors
- Jasjit S. Suri
- Kecheng Liu
- Laura M. Reden
Application Information
Application Number:US 09/825,028
Filing Date:2001-04-03
Priority Date:2001-04-03
Art Unit:7
Classifications
IPC:
G06K 300
Field of Search:
382128382171382173382308382385382154382106324306345419345420345422378 4378 41378 57600117600407600408600459606130700112
Patent Drawings (20 sheets)
Description
Background of the Invention
[0002] The present application relates to diagnostic medical imaging. The invention finds particular application in segmenting and quantifying three-dimensional medical image data. It is to be appreciated however, that the present application finds further utility in quickly and accurately converting a series of two-dimensional images into a 3-D representation with accurately defined interfaces between distinct layer.
[0003] Quantification of the brain's gray matter, cortical thickness, the atrophy index of sulci in the cortex, and the principal curvatures of the cortex are important and clinically significant for analyzing brain diseases such as Alzheimer's, epilepsy, band heterotopia, dysplasia, schizophrenia, and dementia. The measurements can be computer estimated by accurately segmenting the cortex of the human brain. The above quantification tools also help in analyzing functional brain information such as in functional MRI (fMRI).
[0004] Typically, volumetric images are acquired and manipulated as a series of closely spaced, parallel two-dimensional image slices. Rendering this large database of sequential images into three dimensions is prohibitively expensive in computational power and in time. For example, the cerebral sheet at the interface between gray matter (GM) and white matter (WM) in the human brain is about 2500 cm2. The shape and convolutedness of the interface is important in many diagnostic procedures. For example, in neurological surgical planning/navigation and image guided surgery, an objective is to help the neuro-surgeon to plan the neuro-surgery in an operating room to identify the location of the tumor(s) in 3-D coordinates and the critical trajectories through which the tumors are accessed and removed. In surgical navigation, the objective is to navigate using the tools (such as the Y-probe) towards the target (such as a tumor in the brain) while avoiding critical brain structures and their functions during surgery. In Image Guided Surgery, the reconstructed patient's brain anatomy helps surgeons to interactively guide the surgery.
[0005] However, while the importance of cortical segmentation has been identified, it has not been easy to develop robust and high speed 3-D image creation and manipulation techniques.
[0006] The present invention contemplates an improved method and apparatus which overcomes the above-referenced problems and others.
Summary of the Invention
[0007] In one embodiment of the present invention, a medical imaging method includes retrieving image data for a volume of interest, where the image data comprises a plurality of two-dimensional image slices. The plurality of image slices are transformed via gray scale mathematical morphology open-close and Bottom Hat edge preserving smoothing system into a three-dimensional volume. Interfaces within the three-dimensional volume are then identified.
[0008] In accordance with another aspect of the present invention, the transforming comprises applying a mathematical morphology transform to a selected one of the plurality of two-dimensional image slices.
[0009] In accordance with another aspect of the present invention, the mathematical morphology transform comprises performing an open smoothing operation and a closing smoothing operation on the selected two-dimensional image slice yielding an open slice and a closed slice. The open slice and the closed slice are then averaged using shifting.
[0010] In accordance with another aspect of the present invention, the mathematical morphology transform further comprises applying a Bottom-Hat smoothing system to the averaged slice.
[0011] In accordance with another aspect of the present invention, Bottom-Hat edge preservation and smoothing system comprises of a combination of histogramming the gray scale slices and applying a set of image algebra, logical Boolean operators and recursive mathematical morphology.
[0012] In accordance with another aspect of the present invention, recursive mathematical morphology comprises of thinning operation on binary images using skeletonization of binary shapes.
[0013] In another with another aspect of the present invention, the complete system comprises of linking the edge-preserved image denoising system with image derived constrained morphing system.
[0014] In accordance with another aspect of the present invention, the identifying comprises deriving constrained image forces which navigate the deformation process from the three-dimensional volume.
[0015] In accordance with another aspect of the present invention the identifying further comprises estimating fields within the three-dimensional volume contained by respective narrow bands.
[0016] In accordance with another aspect of the present invention the identifying further comprises under the derived image force constraints and within the narrow bands, propagating the estimated fields yielding updated fields, and checking the updated fields for convergence.
[0017] In accordance with another aspect of the present invention, the identifying further comprises estimating an isosurface under a morphological edge-preserved level set framework for each of the updated fields.
[0018] In accordance with another aspect of the present invention, the identifying further comprises rendering the updated fields.
[0019] In accordance with another aspect of the present invention, the estimating comprises generating a signed distance transform from initial fields or spheres.
[0020] In accordance with another aspect of the present invention, the generating comprises obtaining a vertex on one of the spheres, and computing polyline distances in the narrow band to yield accurate distances from the vertex.
[0021] In accordance with another aspect of the present invention, a medical image processing method comprises non-linear smoothing and edge-preserving a volume of interest represented by a series of spatially offset images, and segmenting embedded shapes in the smoothed volume of interest.
[0022] In accordance with another aspect of the present invention, the smoothing comprises sequentially selecting one of the series of spatially offset images. The selected image is smoothed with an open-close method and transformed with an ensemble of operators: morphology, logical operators, image algebra operators that constitute a bottom-hat morphological transform.
[0023] In accordance with another aspect of the present invention, the dynamic segmenting comprises estimating initial surfaces in the volume of interest, propagating the initial surfaces into revised surfaces, extracting an iso-surface from the revised surfaces, reinitializing the surfaces, checking for completion, and if incomplete, replacing previous surfaces with the reinitialized surfaces and repeating.
[0024] In accordance with another aspect of the present invention, the segmenting comprises deriving constrained morphing forces, computing an initial distribution model, morphing the distribution model by applying the derived forces, and extracting a volume from the morphed distribution model.
[0025] In accordance with another aspect of the present invention, the deriving comprises estimating a three-dimensional field distribution from initial spheres, computing data such as normals and offsets for selected voxels in the estimated three-dimensional field distribution, and determining a likelihood probability for a distribution such as WM or GM.
[0026] In accordance with another aspect of the present invention, the computing comprises computing a perpendicular distance between a point on an inner sphere and a corresponding point on an outer sphere.
[0027] In accordance with another embodiment of the present invention, a medical imaging apparatus comprises means for open smoothing and close smoothing a selected image slice to yield an open slice and a closed slice, means for averaging the open slice and the closed slice, and means for smoothing the averaged slice into a volume image representation.
[0028] In accordance with another aspect of the present invention, the imaging apparatus further comprises means for estimating spheres in the volume image representation, means for propagating the estimated spheres constrained by image force constraints and narrow bands surrounding the spheres to yield propagated spheres, means for reinitializing and repropagating until the spheres converge.
[0029] In accordance with another aspect of the present invention, the imaging apparatus further comprises means for revising the volume image representation from the propagated spheres.
[0030] One advantage of the present invention resides in a robust system capable of handling variability in the input volumes, such as low signal to noise ratio, data corruption due to system noise, poor edge interface description and the like.
[0031] Another advantage of the present invention resides in the way the interface edges are given high scores, such as WM/GM interface and GM/CSF (Cerebral Spinal Fluid) interface. These edges help in building the convergence of the deformation process.
[0032] Another advantage of the present invention lies in the cleaning of the cavities between convoluted shapes, for example, between two sulci and two gyri, the cleaning is done using the morphologic system.
[0033] Another advantage of the present system lies in the design of the morphologic system using a combination of binary and gray scale morphological tools, logical operators and image algebra. Thus it is compact system adjusting the parameters automatically.
[0034] Another aspect of the present invention resides in creating a pipeline link for histogramming the gray scale images based on moments, applying the binary morphology and then using thinning operators for preserving the convoluted edges of the twisted shapes.
[0035] Another advantage of the present invention resides in the way that the constraints are applied in the level set framework for the deformation process to extract the embedded medical or non-medical shapes.
[0036] Another advantage of the present invention resides in the dynamic process of shape deformation based on image forces, which are computed dynamically.
[0037] Another advantage of the present invention resides in control of the speed function which is based on the likelihood model of the gray scale image smoothed and edge-preserved volume.
[0038] Another advantage of the present invention resides in control of the speed function automatically by checking the distances between the multiple deforming shapes.
[0039] Another advantage of the present invention resides in control of the speed function by mapping the gray scale likelihood function and controlled by the constrained distance between the propagating surfaces.
[0040] Another advantage of the present invention resides in the very accurate capturing of the 3-D topology of a volume such as a brain using the morphing algorithm.
[0041] Another advantage of the present invention resides in the system's adaptability to various shapes within the imaged volume.
[0042] Yet another advantage of the present invention resides in accurate distance computations such as the improved shortest distance method (or polyline distance method) as opposed to a single shortest distance method.
[0043] Another advantage of the present invention is speed since the propagation of the 3-D surface takes place in the narrow band.
[0044] Another advantage of the present invention lies in the flexibility of changing the mathematical functions or decay functions which controls the speed of the convergence process.
[0045] Another advantage of the present invention lies in the estimation of the geometrical features and quantification of segmented shapes based on the differential geometry.
[0046] Still further advantages will become apparent to those of ordinary skill in the art upon reading and understanding the following detailed description.
Brief Description of the Drawings
[0047] The invention may take form in various components and arrangements of components and in various steps and arrangements of steps. The figures are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention.
[0048] FIG. 1 is an illustration of a section of a convoluted 3-D surface;
[0049] FIG. 2 is a system diagram of an apparatus according to the present invention;
[0050] FIG. 3 is an object-process flow chart overview of a morphological smoothing and 3-D segmentation process;
[0051] FIG. 4 is an object-process flow chart of the morphological smoothing system as seen in FIG. 3;
[0052] FIG. 5 is an object-process flow chart detailing a process from FIG. 4;
[0053] FIG. 6 is an object-process flow chart illustrating a process in FIG. 5;
[0054] FIG. 7 is an object-process flow chart illustrating another process from FIG. 5;
[0055] FIG. 8 is an object-process flow chart of a 3-D segmentation process as seen in FIG. 3;
[0056] FIG. 9 is an object-process flow chart of a process as seen in FIG. 8;
[0057] FIG. 10 is an object-process flow chart of another process as seen in FIG. 8;
[0058] FIG. 11 is a stylized representation of the cortical constraints between inner and outer spheres as used in the morphing system;
[0059] FIG. 12 is an object-process flow chart of a 3-D signed distance transform;
[0060] FIG. 13 is an object-process flow chart of a process from FIG. 8;
[0061] FIG. 14 is a graphical progression showing the 3-D morphing process according to the present invention;
[0062] FIG. 15 is an image matrix illustrating an isocontour;
[0063] FIG. 16 is an object-process flow chart to extract an isosurface during the morphing procedure;
[0064] FIG. 17 is an object-process flow chart of a process of FIG. 16;
[0065] FIG. 18 is an object-process flow chart of other processes in FIG. 16;
[0066] FIG. 19 is an object-process flow chart of a process seen in FIG. 13;
[0067] FIG. 20 is the object-process flow chart showing how one can invoke the segmentation engine from the script mode;
[0068] FIG. 21 is the object-process flow chart of the process showing how one can compute the RGB color images once the segmented output is displayed; and
[0069] FIG. 22 shows how one can compute the shape indices of the objects using the concept from differential geometry.
Detailed Description of the Preferred Embodiments
[0070] With reference to FIG. 1, a volume of interest is comprised of several tissue types, for example, white matter (WM) 110, gray matter (GM) 120, and cerebral spinal fluid (CSF) 130. WM 110 and CSF 130 are separated by an area of GM 120 of assumed uniform thickness called the cortical thickness 150. Analysis has shown that employing a cortical thickness 150 between 3 mm and 5 mm results in satisfactory images. Indeed, a thickness of 3-4 mm typically produces optimal results.
[0071] The robust coupled surface idea (that is, the correspondence between edge enhanced surfaces of the WM 110 and GM 120) is motivated by the nearly constant thickness of the cortical mantle and thus uses this coupling as a image constraint during the morphing process. The algorithm starts with two or more embedded surfaces in the form of concentric sphere sets (more fully developed below) or concentric regular shapes. The inner and outer surfaces of these concentric shapes are then morphed, driven by their own attractive image-derived information from the edge-preserved volume, respectively, while maintaining the coupling in-between through the prior specified constraint in the form of the thickness 150.
[0072] With reference now to FIG. 2 an imaging system 200 includes a data collection unit 210 used to acquire image slice data over a volume of interest. Exemplary data collection units 210 include magnetic resonance systems, computed tomography devices and the like. In the case of a magnetic resonance system, the data collection unit collects the MR image volume based on the type of weighting and image parameters 215. For example, T1-weighted images may use the following parameters: Volume data collection using RF-FAST technique, TE=6.7 msecs, TR=14 msec, BW=31.3 kHz, FOV=25.6 cm, Flip angle=20, NSA=1, Thickness=1 mm, phase matrix=256, read matrix=256, number of slices=182, scan time=13.08 minutes. Accordingly, the data collection unit 210 at least somewhat controls the slice thickness and resolution of the pixels. The output of the data collection unit 210 is the imaged volume 220 captured as a series of 2-dimensional slices 220a, 220b, . . . 220n. The volume 220 is processed by an image processor 500 which receives inputs including prior knowledge or constraints 502, such as the cortical thickness 150 (FIG. 1). Other inputs include statistical tools 504, visualization tools 506 and quantification tools 508 which the processor 500 uses to manipulate and show the volume on a display unit 510.
[0073] With reference now to FIG. 3 the image processor 500 includes two major systems. The first is the morphological smoothing system 600 and the second is the 3-D segmentation processor 700 using constrained deformation system in level set framework. The morphological smoothing system and edge preservation 600 cleans and smoothes a 3-D surface from the plurality of 2-D image slices 220a, 220b, . . . 220ncomprising the imaging volume 220. The morphological smoothing system 600 cleans and smoothes the surface as well as cavities yielding a smooth volume 620. This morphological smoothing system is based on gray scale morphology and a combination of logical operators and image algebra techniques that will be further developed below. Continuing with FIG. 3, the smoothed volume 620 is segmented in the 3-D segmentation processor 700 yielding a 3-D segmented volume 710 for display 510.
[0074] With reference now to FIG. 4, an exemplary morphological smoothing system 600 selects an image slice 220x. Open-close and Bottom-Hat morphological transform methods 610 are performed on the slice 220x. Mathematical morphology is used for noise reduction, edge preservation and volume enhancement. This method is based on four fundamental gray scale mathematical morphological operations called: Dilation, Erosion, Opening and Closing. Using these operations, the smoothed images slice by slice, are generated from a Bottom-Hat transform.
[0075] The strength of mathematical morphology in edge preservation and image denoising is tremendous. This is implemented using the convolution operation of the image with kernels. This kernel is also called a structuring element (SE) The structuring element can be of various shapes and sizes and depends upon the type of structure to be smoothed and segmented. An example of the SE would be a disk which is a 3×3 window in which all the pixels used except the corners. Another example is a square where the SE is a 3×3 window with all pixels are used. Usually the center of the SE is the center of the shape, but one can have the center of the SE any other pixel in the window. We will represent the SE as B and represent the original image as A.
[0076] Given a structuring element B and the original image A, one can mathematically define the four fundamental equations for morphological transform as:
[0077] The binary dilation is defined as:
[0078] A⊕B={c|c=a+b, for every a∈A, b∈B} (1)
[0079] The binary erosion is defined as:
[0080] AΘB={c|c+b∈Afor every b ∈B} (2)
[0081] The binary closing is mathematically defined in terms of binary dilation and binary erosion as:
[0082] A●B=(A⊕B)ΘB (3)
[0083] The binary opening is mathematically defined in terms of binary dilation and binary erosion as:
[0084] A∘B=(AΘB)⊕B (4)
[0085] The gray scale dilation is given as under:
[0086] Let “f” be the image signal and “g” be the structuring element. Then the gray scale dilation is defined as:
[0087] Gray scale dilation has a property to enlarge geometric structures in an image. To understand the gray scale dilation for medical images, we explain the Umbra and Top of a surface first.
[0088] Let F∈EN-1f:F→E, then the Umbra of a function f, denoted by U[f], U[f]∈F×E, then Umbra is mathematically defined as:
[0089] U[𝑓]={(x,y)∈F×E|y<=𝑓(x)} (6)
[0090] and the Top of Surface is defined mathematically as:
[0091] T[A](x)=max{y|(x,y)∈A} (7)
[0092] where A is a set.
[0093] Thus the gray scale dilation using the Top and Umbra (as defined above) is given as:
[0094] 𝑓⊕k=T{U[𝑓]⊕U[k]} (8)
[0095] The gray scale erosion is defined as:
[0096] Gray scale erosion has a property to shrink geometric structures in an image, which is explained using the Top and Umbra as:
[0097] 𝑓Θk=T{U[𝑓]ΘU[k]} (10)
[0098] In the same manner as the binary opening and binary closing, we present the gray scale opening and gray scale closing. The gray scale closing is mathematically defined in terms of gray scale dilation and gray scale erosion as:
[0099] A●B=(A⊕B)ΘB (11)
[0100] The gray scale opening of an image is mathematically defined in terms of gray scale erosion and gray scale dilation as:
[0101] A∘B=(AΘB)⊕B (12)
[0102] These eight equations (1, 2, 3, 4, 5, 9, 11, 12) are the fundamental equation necessary for building edge preserving and image denoising. Another important operation is median filtering based on the gray scale opening and closing, and is mathematically defined as:
[0103] We also use the skeletonization (or thinning) of the set A by a structuring element K by sets S0, S1, S2. . . Snand is given as:
[0104] where Snis given as:
[0105] Note that the subscript “n” is being used in the equation (15). This equation is the difference between the eroded image and the opening of the eroded image. The erosion is the recursive process.
[0106] Using the eight fundamental equations (a subset from equations 1 to 12, four binary and four gray scale dilation, erosion, opening and closing) along with the median (equation 13) and skeleton (equation 14) combined with image algebra produces a very powerful method for non-linear smoothing for medical images. This method is called the Bottom-Hat Gray scale morphological transform. There are three major features of such a system: (a) it removes the noise, (b) it cleans the cavities of the convoluted shapes and (c) it preserves the edges between the interfaces such as WM/GM and GM/CSF.
[0107] Continued reference to FIG. 4 illustrates an exemplary technique 614, 616 to repeat the process for all slices 220xbefore writing the smoothed image volume 620.
[0108] Referring now to FIG. 5, an exemplary open-close and Bottom hat morphological transformation method 610 first computes the open-close smoothing 630 of a slice followed by the Bottom-Hat morphological transform 660. The Bottom-Hat morphological transformed slice undergoes image algebra with respect to original slice (220x) to generate a smoothed slice S1 (see FIG. 5). The system also generates another set of smoothed slice S2 using the combination of Bottom-Hat Morphological transform slice and the smoothed slice obtained from open-close process (see FIG. 5).
[0109] More specifically, and with reference now to FIG. 6, the open-close morphological smoothing 630 proceeds by processing an image slice 220xin parallel paths. In the left column, the gray scale image slice 220xis dilated 632 by an image by a structuring element (SE). In the right column, the image slice 220xis eroded 634 by the SE. The output of these processes is passed to both opening 640 and closing 642 gray scale mathematical transforms to produce an opened slice 646 and closed slice 648. These are averaged 650 using a binary shift operation 652. If Iopen and Iclose are the two gray scale images, then the averaging operation is defined as: (Iopen+Iclose)>>1. In other words, if the binary values are right shifted by 1 then it is shrinking the gray scale intensities by half. An example is: Suppose the image intensities represent from 0 to 8 and they can be written in binary format as: 0000, 0001, 0010, 0011, 0100, 0101, 0110, 0111, 1000. On right shifting the 8thinteger that is from 1000 to 0100, which is equivalent to a integer value of 4, which is nothing but half of 8. Thus the right shifting brings averaging.
[0110] Referring now to FIG. 7, the Bottom-Hat morphological smoothing process 660 uses a combination of Erosion, Dilation, Skeleton and Logical ANDing and ORing process. First, the smoothed gray scale slice 654 is changed to a binary image using a thresholding process 662. This process 662 first computes the histogram of the image and then the area under the histogram. The second moment of this area will compute the threshold value 664 of the given gray scale image. This threshold 664 is used with the morphological operation to estimate the Eroded image 666. Using the cleaning and dilation process 668, we get intermediate output 670. Now, the eroded image 666 and dilated image 668 are logically ANDed to generate the ANDed image 672. The ANDed image 672 is eroded with the structuring element (binary) to yield eroded image 674. Using the same combination of filtering, dilation and logical ANDing, we generate the image 676. The thinning procedure 677 on this image generates the image 678. This image 678 is dilated to yield image 680. This is logically ANDed with the smoothed open-close input image 654 to produce the smoothed Bottom-Hat Smoothed Slice 684. This entire process is repeated for all the slices to generate the smoothed image volume 620 (FIG. 4). Note there are two key advantages of this technique: first, it cleans the image and removes the noise; and second, it opens the cavities between two sulci, which is the key in 3-D segmentation.
[0111] With reference now to FIG. 8, the 3-D segmentation system 700 (see also, FIG. 3) using constrained morphing starts with the smoothed and edge preserved 3-D volume 620. The exemplary, illustrated system 700 consists of five principal steps which are: (i) Derivation of the constrained morphing forces 720; (ii) Field estimation 740; (iii) Internal 3-D morphing or propagation 780; (iv) Isosurface extraction and estimation 800; and (v) Reinitialization 820. Each of these steps are considered below.
[0112] Derivation of the constrained morphing forces 720 is based on a likelihood generation using a Gaussian Distribution model (also called CLAMPER or regularizer because this helps in clamping for the morphing process). Initial field estimation 740 uses a signed distance transform on the surface in the narrow band (NB). Internal 3-D morphing 780 completes the morphing of initial field estimate by picking up constrained forces derived from the constrained morphing forces 720. Isosurface extraction 800 extracts the same density surface from the given volume. Exemplary algorithms are based on triangularization or polygonalization of the volume. This extraction can also be displayed by computing the shading on the surface based on a computer graphics technique. We will discuss these steps in greater detail below.
[0113] Referring now to FIG. 9, an exemplary algorithm for design of the regularizer or clamper. This means deriving the image forces 720 using differential geometry is based on a likelihood generation using a Gaussian Distribution model (called CLAMPER). While the WM-GM likelihood image is illustrated, those skilled in the art will appreciate that the same principle is applied for the GM-CSF likelihood image computation or that of another interface. The forces derived are the image forces which are also used for capturing topology information to the morphing process. They are also called as navigating forces which steers the morphing process based on the gray scale distribution of the intensities in the volume. Since the gray scale information is available, one of the best strategies would be to use the likelihood operator.
[0114] The image-derived information is obtained by using a local likelihood operator based on gray-level information, which gives the algorithm the ability of capturing the homogeneity of the tissue inside the volumetric layer. From the morphological smoothed and edge preserved volume 620, a 3-D field 722 is estimated (Signed Distance Transform) given the initial spheres 620a, 620b. This field is the distance measure. From the initial field distribution 722, the normals and offsets 724 are computed at every voxel location using differential geometry. These two offsets along with the neighboring voxels 728 are used in two (WM-GM and GM-CSF) different distributions to compute the likelihood values. For the first distribution (here, WM), the WM likelihood probability 730 is computed given a voxel and similarly, the GM likelihood probability 732 is computed given the second distribution (here, GM). From the WM probability 730 and the GM probability 732 the WM-GM likelihood image 734 is computed. Mathematically, they can be expressed as:
[0115] where e stands for the exponential, W and G are the WM and GM regions, μWand μGare the mean values of the WM and GM, respectively. σWand σGare the standard deviations of the WM and GM regions. Iwand Igare the WM and GM pixel intensities. Note, the output 734 of the WM-GM likelihood functions 730, 732 is the images which have edge information or clamping forces about the surface of WM-GM. Similarly, the WM-CSF likelihood function (not shown) is an image which has WM-CSF edge or gradient information. Note that those skilled in the art of image processing can appreciate that other regularizers and distributions could also be used for computing the likelihood.
[0116] Referring now to FIG. 10, a first estimate for the initial 3-D field 750 is made given the concentric initial spheres 620a, 620b. This initial field computation process is preferably done in parallel for both the inner 620aand outer 620bshapes as illustrated. Detailing only one of the parallel paths, those skilled in the art will be able to extend the teaching to the other path. Thus, starting with the inner volume or sphere 620a, a signed distance transform 742 is performed constrained in the narrow band 744 and by the cortical thickness 150 (FIG. 1), yielding estimated field 750.
[0117] With reference now to FIG. 11, the narrow banding 744 is graphically illustrated as surrounding the inner 620aand outer 620bspheres. Bands 744aaround the inner sphere 620adefine the narrow band in which the 3-D field is computed. Similarly, bands 744baround the outer sphere 620bdefine the narrow band for the outermost sphere. Noteworthy is that the respective narrow bands 744a, 744bcontain both the inner and outer spheres 620a, 620b.
[0118] With reference now to FIG. 12, the computation of the initial field using the polyline distance scheme is illustrated as an exemplary method to accomplish the signed distance transform 742 (FIG. 10) for the inner and outer surfaces 620. This is a very accurate method of computing the perpendicular distance between the point and the opposite segment. Generally, the goal is to find the shortest distance 748 from the given vertex to neighboring vertices 746. If the neighboring vertices are kept in an ordered list, this computes the distance between the given vertex and the vertices in the list. It is not necessary that the shortest distance 748 is the distance is the distance between the given vertex and the “vertices” in the list. One could find the shortest distance between the given vertex and the “polyline joining the vertices” in the list. This polyline is the line segment which joins any pair of two vertices in the list in an order. Thus the shortest distance 748 (also called polyline distance here) is the shortest distance between the given vertex and the opposite pairs of line segments if the segment lies in the range of the given vertex. Otherwise, it is one of the shortest distances from a given vertex to one of the vertices in the list. Suitable methods such as vector calculus are used to find if the given vertex lies in the range of the polyline. If the given vertex is in the range of the polyline, then the perpendicular is dropped from the given vertex to the ranged polyline and the point of intersection between the perpendicular and the polyline is found. The distance between the given vertex and the intersection point is the improved shortest distance measure or polyline distance measure. Since one uses the vector calculus based method to estimate the range and the intersection point, this method directly gives the distance measure, which is major advantage of this technique besides being a very stable technique.
[0119] Now referring to FIG. 13, an exemplary embodiment of the internal propagation step 780 (FIG. 8) shows the computation of the morphed surface given the initial sphere fields 620aand 620b(see FIG. 10). The internal propagation is used to solve the partial differential equations corresponding to the two or more embedded spheres in the narrow band. One of the methods for morphing could be using the level set framework, a powerful numerical scheme for computing interface motion (see Osher and Sethian, Fronts propagating with curvature dependent speed: algorithms based on Hamilton-Jacobi formulations, Journal of Computational Physics, Vol. 79, No. 1, pp. 12-49, 1988). Instead of evolving the two surfaces directly, two level functions whose zero level set corresponding to the cortical bounding surfaces were calculated. The equations of the morphing surfaces are given as:
[0120] Ψin(x,y,z,t)=CFin(x,y,z)|∇Ψin(x,y,z,t)|
[0121] Ψout(x,y,z,t)=CFout(x,y,z)|∇Ψout(x,y,z,t)| (17)
[0122] where CFin(x,y,z) and CFout(x,y,z) are the inner and outer constrained coupled speed functions which are computed from the image derived forces from step one. Note that the speed is totally controlled by the image-derived constrained forces, which is a function of the smoothed and edge-preserved input volumes. Those in the field of image processing will appreciate the power of the image-derived speed functions which have stronger information about the interfaces (e.g. WM/GM and GM/CSF). The combination of morphological processing, image algebra and logical operators provides an improved speed function, not just plain gray scale speed functions. Since the likelihood is computed on the edge perserved and smoothed volume, the speed function is able to be better controlled. The speed function should be ideally zero if the likelihood computation is 1. This means the interface has been detected. If the likelihood function is not close to 1, but instead varies between 0 and 1, then the speed function also varies between the high and low limits. This variation of speed function are constrained by the distance (150FIG. 1) between the outer and inner surfaces, which is the prior information. If the distance is out of range, say larger than the cortical thickness, then the speed function is affected or penalized. In other words, the speed function is checked by the prior knowledge of the embedded thickness of cortex and the edge preserved smoothed volume. As discussed above, if the distance is out of range, the speed needs to be checked by the checking function which is a slow moving polynomial such as exponential or Hermitite polynomial. Those in the field of statistical analysis can appreciate that there are other mathematical functions and penalize the speed-likelihood mapping function to control speed so that the morphing process can behave in the normal limits. A rapid fall in speed-likelihood mapping function will help the convergence process faster and similarly if the morphological edge preservation is stronger, the convergence is faster.
[0123] To accomplish the internal propagation or settling surface, the speed control function is multiplied to the gradient of the signed distance function. This is the Eulerian representation of the curve evolution theory or flame propagation theory. Thus, the gradient information is computed using the nabla or gradient operator given as:
[0124] Ψin(x,y,z,t) and ∇Ψout(x,y,z,t)
[0125] The partial differential equations are solved in a narrow band using the fast marching method. The coupling between the two surfaces was realized through the propagation speed terms CFout and CFin, while the distance between the two surfaces is within the normal range, the inner and outer cortical surfaces propagate according to their own image features. When the distance starts to fall out of the normal range, the propagation slows down and finally stops when the distance is outside the normal range, or the image feature is strong enough. A coupled narrow band algorithm was customized for the coupled-surfaces propagation. The correspondence between points on the two bounding surfaces falls out automatically during the narrow band rebuilding, which is required for surface propagation at each iteration. This shortest distance-based correspondence is essential in imposing the coupling between the two bounding surfaces through the thickness constraint.
[0126] Referring now to FIG. 14, a three stage progression of the morphing process 800 (FIG. 8) is illustrated. Stage one 802 shows a set of concentric initial shapes 620. Typically these are user chosen shapes, which can, for example include two concentric spheres on one half of the brain and two concentric spheres on the other half of the brain (as illustrated). Stage two 804 shows the morphing of the initial shapes closer to the shapes of the brain topology, keeping the constraints in the picture. This deformation uses the edge preserved interfaces from the morphologic smoothing system to speed up the convergence along with the image derived information gathered from the likelihood function from the morphologic smoothed volume. The constrained deformation from stage one to stage two means morphing the initialized shape towards the topological shape. Finally, in step three 806 the goal shapes are identified as the speed falls close to zero at the interfaces. Due to the property of level sets, the morphing process has the ability to spilt and merge by itself, which provides yet another advantage to the morphing system.
[0127] With attention now to FIG. 15, a 2-D graphical example of a contour extraction during the morphing process will illustrate the topological 3-D process. This figure shows how the isocontour 820 is extracted from a given image. An isocontour is a contour that has the same intensity value in the image. As seen in this figure, a “zero level” isocontour 820 is seen in the image. Similarly in 3-D, an isosurface is generated. Since we are dealing with volumes, the polygonization is computed from the volume directly. The polygonalization process is constructed using the triangulation scheme in the spirit of the marching cube (MC) technique (see Lorensen et al. Marching Cubes: A high resolution 3-D surface construction algorithm, ACM Computer Graphics, Proceedings of Siggraph, Vol. 21, No. 4, pp. 163-169, July 1987). The edge table generation and normal computation using finite difference offers an added simplicity and robustness to the system. The basic principle of the MC algorithm was to subdivide the 3-D scalar field values into a series of small cubes. The algorithm then marches through each of the cubes testing the cube corners and replacing the cube with a set of polygons. The total sum of all the polygons constitutes the surface (or isosurface) which is an approximation to the 3-D data or 3-D field values. For a 3-D volume, each cube has 8 corners which leads to 256 (2 power of 8) possible combinations of corners. To simplify the MC algorithm, others have reduced the complexity by taking into account cell combinations that duplicate under the following conditions: (1) Rotational invariance along any of the three primary axes (by any degree); (2) Mirror image of the shape across any of the three primary axes; or (3) Inversion of all the corner states and flipping the normals of the relating polygons.
[0128] By taking these conditions, one can reduce the 256 combinations down to 15 combinations. Thus, given the volume, the first step is the edge table generation, which results in the triangulation. The edge table consists of finding out which cells or voxels are cut out by the morphing surface. The edges of the cubes which get cut by the surface are noted. The cut points constitute the polygons and all the polygons constitute the polygonalization process.
[0129] With respect now to FIG. 16, an isosurface extraction algorithm 800 is illustrated which starts with a volume 790 which is the new estimated field flow (see FIG. 8). This new field is the field morphed from the initial field acted by the image derived forces controlled by the edge-preserved volume and likelihood measure. The isosurface generation consists of two steps. First is the triangularization or polygonalization of the input volume 870 yielding a corresponding shape 872. Step two computes normal components and locations by two paths: the finite difference method 886 and the vector calculus method 890. The finite difference method 886 computes the difference of densities (gradients) at each vertex (3 components) and normalized normals are computed by normalizing the gradients. In the vector calculus method 890, the normalization is computed by dividing the normals by the square root of the square of the three components. The normals and the vertices are used for isosurface generation. These shapes 872 represent the isosurface which can also be shaded.
[0130] Referring now to FIG. 17, the shape conversion 870 (i.e. triangularization or polygonalization) of the input volume includes selecting a cube or shape 873, and applying a threshold 874 to vertices of the shape. Intersected edges are obtained 876 from the thresholded vertices, which are then converted to a binary sequence 878. When all the cubes are completed, a table is fully populated with edge information 872.
[0131] FIG. 18 illustrates the normalization and shading process 886, 890. In the first way 886, the gradients are computed 894 from the voxel densities at each cube vertex. These are the normal components at each vertex. The normalization of these normals 896 is determined by dividing these gradient values by the square root of the sum of the squares of the three components. In the second way 890, we compute the vector difference method to compute the normal components 902. These normal components are then normalized 896 in a similar way as was done in the first method, where the normals are divided by the square root of the sum of the squares of the three components. The shading of these intermediate surfaces is done using the pipeline discussed below.
[0132] Referring now to FIG. 19, an exemplary user interface shell script 510a, as a component of the display 510, includes four components: an initialization of open Graphical Library interface component 561; cleaning the background 562; and setting the perspective and Lookup parameters using 563. The shading and material properties are set up using the ambient, diffusion and shine values. The normals are computed at the vertices and the output is a shaded surface.
[0133] Referring now to FIG. 20, an exemplary method of display with feed back 510balso is a component of the display 510. Segmentation engine is acted and the results can be displayed on the window. The user can interactively manipulate the segmented results using the feedback method. The volume is loaded in the memory just once and segmented results are displayed and manipulated using the feedback method.
[0134] Referring now to FIG. 21, an exemplary method of saving the displayed results in color 510cis a component of the display 510 also. The Red-Green-Blue-Alpha (RGBA) buffer is allocated 512 and displayed window contents are now stored. The stored contents are converted to unsigned characters and the matrix is written into a file.
[0135] Referring to FIG. 22, an exemplary method of geometric shape quantification 508 (see also FIG. 2) computes the mean 1002 and Gaussian 1003 curvatures. This is used for computing the maximum 1006 and minimum 1007 principal curvatures which are later used for shape index computation 1008. Since differential geometry is used in the level set framework driven by the edge-preserved morphologic system and the likelihood models, the geometric features are very clinically useful. They can tell about the curvatures of the shapes and curvatures for normal and abnormal cases can be compared.
[0136] The invention has been described with reference to the preferred embodiment. Obviously, modifications and alterations will occur to others upon reading and understanding the preceding detailed description. It is intended that the invention be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims
Having thus described the preferred embodiment, the invention is now claimed to be:
1. An apparatus for medical imaging comprising:
a means for retrieving image data for a volume of interest, where the image data comprises a plurality of two-dimensional image slices;
a means for preserving edges while transforming the plurality of image slices into a three-dimensional volume;
a means for deriving constrained image forces from the three-dimensional volume; and
a means for identifying interfaces within the three-dimensional volume based on the derived image forces.
2. The apparatus for medical imaging as set forth in claim 1, where the means for preserving edges comprises:
a means for mathematically morphing for each of the plurality of two-dimensional image slices into a smoothed slice; and
a means for applying image algebra and logical operators to each smoothed slice.
3. A method of medical imaging comprising:
retrieving image data for a volume of interest, the image data including a plurality of two-dimensional image slices;
preserving edges while transforming the plurality of image slices into a three-dimensional volume, including:
for each of the plurality of two-dimensional image slices:
performing an open smoothing operation and a closing smoothing operation on the slice yielding an open slice and a closed slice; and
averaging the open slice and the closed slice.
4. The method of medical imaging as set forth in claim 3, further including:
applying a bottom-hat smoothing process to the averaged slice.
5. A method of medical imaging comprising:
retrieving image data for a volume of interest, the image data including a plurality of two-dimensional image slices;
preserving edges while transforming the plurality of image slices into a three-dimensional volume: and
identifying interfaces within the three-dimensional volume based on a predetermined constraint, the identifying including:
deriving constrained image forces from the three-dimensional image volume.
6. The method of medical imaging as set forth in claim 5, where the identifying further comprises:
estimating fields within the three-dimensinoal volume contained by respective narrow bands.
7. The method of medical imaging as set forth in claim 6, where the identifying further comprises:
under the derived image force constraints and within the narrow bands, propagating the estimated fields yielding updated fields; and
checking the updated fields for convergence.
8. The method of medical imaging as set forth in claim 7, where the identifying further comprises:
estimating an isosurface under the edge preserved three-dimensional volume and level set framework for each of the updated fields.
9. The method of medical imaging as set forth in claim 8, where the identifying further comprises:
surface rendering the estimated isosurfaces.
10. The method of medical imaging as set forth in claim 6, where the estimating comprises:
generating a signed distance transform from initial shapes.
11. The method of medical imaging as set forth in claim 10, where the generating comprises:
obtaining a vertex on one of the spheres; and
computing distances in the narrow band to yield a shortest distance from the vertex to all vertices in a neighborhood.
12. An apparatus for processing a medical image comprising:
a means for smoothing a volume of interest represented by a series of spatially offset images while preserving interfaces in the volume; and
a means for segmenting the smoothed volume of interest based on level set framework and three-dimensional polyline differences, the segmenting means including:
a means for deriving constrained morphing forces;
a means for computing an initial likelihood distribution model over the preserved interface volume;
a means for morphing the distribution model by applying the derived forces: and
a means for extracting a volume from the morphed distribution model.
13. A method of processing a medical image comprising:
smoothing a volume of interest represented by a series of spatially offset images while preserving interfaces in the volume, the smoothing including sequentially:
selecting one of the series of spatially offset images;
smoothing the selected image with an open-close method;
transforming the smoothed image with a bottom-hat morphological transform; and
segmenting the smoothed volume of interest based on level set framework and three-dimensional polyline distances.
14. A method of processing a medical image comprising:
smoothing a volume of interest represented by a series of spatially offset images while preserving interfaces in the volume; and
segmenting the smoothed volume of interest, the segmenting including:
estimating forces in the volume of interest;
based on the estimated forces, dynamically propagating initial surfaces into revised surfaces;
extracting an isosurface from the revised surfaces;
reinitializing the surfaces;
checking for completion;
if incomplete, replacing previous surfaces with the reinitialized surfaces and repeating.
15. A method of processing a medical image comprising:
smoothing a volume of interest represented by a series of spatially offset images while preserving interfaces in the volume; and
segmenting the smoothed volume of interest based on level set framework and three-dimensional polyline distances, the segmenting including:
deriving constrained morphing forces;
computing an initial likelihood distribution model over the preserved interface volume;
morphing the distribution model by applying the derived forces; and
extracting a volume from the morphed distribution model.
16. The method of processing a medical image as set forth in claim 15, where the deriving comprises:
estimating a three-dimensional field distribution from initial spheres;
computing data for selected voxels in the estimated three-dimensional field distribution;
determining a likelihood probability for a distribution; and
manipulating the morphing forces based on a distribution of the likelihood probability.
17. The method of processing a medical image as set forth in claim 16, where the manipulating comprises:
adjusting the morphing forces at a variable speed depending on convergence of the initial spheres.
18. The method of processing a medical image as set forth in claim 15, where the computing comprises:
computing a perpendicular distance between a point on an inner sphere and a corresponding point on an outer sphere.
19. An imaging apparatus comprising:
means for open smoothing and close smoothing a selected image slice to yield an open slice and a closed slice;
means for averaging the open slice and the closed slice; and
means for smoothing the averaged slice into a volume image representation.
20. The imaging apparatus as set forth in claim 19, where the imaging apparatus further comprises:
means for estimating spheres in the volume image representation;
means for propagating the estimated spheres constrained by image force constraints and narrow bands surrounding the spheres to yield propagated spheres;
means for reinitializing and repropagating until the spheres converge.
21. The imaging apparatus as set forth in claim 20, where the imaging apparatus further comprises:
means for revising the volume image representation from the propagated spheres.
Patent Citations (21)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US4710876(A) | 1987-12-01 | Cline et al. | Applicant |
| US4729098(A) | 1988-03-01 | Cline et al. | Applicant |
| US4751643(A) | 1988-06-01 | Lorensen et al. | Examiner |
| US5005578(A) | 1991-04-01 | Greer et al. | Examiner |
| US5048095(A) | 1991-09-01 | Bhanu et al. | Applicant |
| US5185209(A) | 1993-02-01 | Zupancic | Examiner |
| US5185809(A) | 1993-02-01 | Kennedy et al. | Examiner |
| US5187658(A) | 1993-02-01 | Cline et al. | Examiner |
| US5239591(A) | 1993-08-01 | Ranganath | Applicant |
| US5332968(A) | 1994-07-01 | Brown | Examiner |
| US5433199(A) | 1995-07-01 | Cline et al. | Applicant |
| US5490516(A) | 1996-02-01 | Hutson | Examiner |
| US5568384(A) | 1996-10-01 | Robb et al. | Examiner |
| US5655028(A) | 1997-08-01 | Soll et al. | Examiner |
| US5687737(A) | 1997-11-01 | Branham et al. | Examiner |
| US5709206(A) | 1998-01-01 | Teboul | Examiner |
| US5806521(A) | 1998-09-01 | Morimoto et al. | Examiner |
| US5827499(A) | 1998-10-01 | Metz et al. | Examiner |
| US5871019(A) | 1999-02-01 | Belohlavek | Examiner |
| US5990897(A) | 1999-11-01 | Hanratty | Examiner |
| US6018499(A) | 2000-01-01 | Sethian et al. | Applicant |
Non-Patent Literature (18)
- Burdin et al., “Modeling and Analysis of 3-D Elongated Shapes with Applications to Long Bone Morphometry”, IEEE Transactions on Medical Imaging, Feb. 1996, pp. 79-91.Applicant
- Joliot et al., Three-Dimensional Segmentation and Interpolation of Magnetic Resonance Brain Images, IEEE Transactios on Medical Imaging, Jun. 1993, pp. 269-277.Applicant
- Medical Computer Vision, Virtual Reality and Robotics by Nicholas Ayache; Image and Vision Computing vol. 13, No. 4, May 1995.Applicant
- Computer-Assisted Interactive Three-Dimensional Planning for Neurological Procedures by Ron Kikinis, M.D., et al.; Neurosurgery, vol. 38, No. 4, Apr. 1996.Applicant
- The use of Computerized Image Processing for Neurological Planning by Ferenc A. Jolesz, et al.; Blackwell Scientific, 1993.Applicant
- Automatic Segmentation of the Brain from MRI-T1 Data—Chapter 2 by N. Ayache, 1996.Applicant
- Leaking Prevention in Fast Level Sets Using Fuzzy Models: An Application in MR Brain by Jasjit S. Suri, Ph.D; Magnetic Resonance Clinical Science Div., Marconi Medical Systems, Inc.; Inter. Conference in Information Technology in Biomedicine, Nov. 2000.Applicant
- A Unified Geometric Model for 3-D Confocal Image Analysis in Cytology by A. Sarti, et al.; Lawrence Berkeley National Laboratory, Univ. of California, 1998.Applicant
- Greedy Algorithm for Error Correction in Automatically Produced Boundaries from Low Contrast Ventriculograms by J. Suri, et al.; Pattern Analysis & Applications, 2000.Applicant
- Computer Vision, Pattern Recognition and Image Processing in Left Ventricle Segmentation: The last 50 Years by Jasjit S. Suri; Image Guided Surgery, Marconi Medical Systems, 2000.Applicant
- Feature Extraction for MRI Segmentation by Robert P. Velthuizen, Ph.D, et al.; American Society of Neuroimaging, 1999.Applicant
- From Retinotopy to Recognition: fMRI in Human Visual Cortex by Roger G.H. Tootell, et al.; Trends in Cognitive Sciences, vol. 2, No. 5, May 1998.Applicant
- Segmentation of Hippocampus From Brain MRI Using Deformable Contours by Amir Ghanei, et al.; Computerized Medical Images and Graphics (CMIG), vol. 22, No. 3, pp. 203-216, May-Jun. 1998.Applicant
- Marching Cubes: A High Resolution 3D Surface Construction Algorithm by William E. Lorensen, et al.; Computer Graphics, vol. 21, No. 4, Jul. 1987.Applicant
- Skeleton Climbing: Fast Isosurfaces With Fewer Triangles by Tom Poston, at al., Oct. 1997.Applicant
- Geometrically Deformed Models: A Method for Extracting Closed Geometric Models from Volume Data by James V. Miller, et al.; Computer Graphics, vol. 25, No. 4, Jul. 1991.Applicant
- Segmentation and Measurement of the Cortex from 3-D MR Images Using Coupled-Surfaces Propagation by Xiaolan Zeng, et al.; IEEE Transactions on Medical Imaging, vol. 18, No. 10, Oct. 1999.Applicant
- Interactive 3D Segmentation of MRI and CT Volumes Using Morphological Operations by Karl Heinz Höhne, et al.; Journal of Computer Assisted Tomography, Mar./Apr. 1992.Applicant