Method and apparatus for ultrasound image reconstruction

[0002] This application claims priority under 35 U.S.C. §119(e) from U.S. Provisional Patent Application No. 60/029,562, filed on Nov. 7, 1996, and U.S. Provisional Application No. 60/036,318, filed on Jan. 30, 1997, both of which are incorporated herein by this reference.

[0003] The present invention relates to a method and apparatus for image reconstruction and particularly to a method and apparatus for three- and four-dimensional image reconstruction.

[0004] Ultrasound imaging is becoming increasingly popular as a replacement for X-ray imaging techniques due to the health hazards associated with x-ray radiation. In ultrasound imaging, one or more emitting piezoelectric transducers are placed into contact with the patient's skin and energized to generate one or more ultrasound signals. The ultrasound signals are reflected by changes in tissue density, such as by the organ of interest, and the reflected signals received by one or more receiving piezoelectric transducers. The collected data, namely the time (measured from the time of signal emission) required by the signal to be received by the receiving piezoelectric transducer(s) and the intensity of the received signal, can be combined with the position (x,y,z) and orientation (alpha, beta, gamma) of the probe to generate a plurality of two-dimensional scanning or imaging planes. To form a three-dimensional image of the organ, an output volume containing a number of elements is generated. A gray-scale value is assigned to each element by sequentially and independently processing each of the scanning planes.

[0005] In designing an efficient three-dimensional ultrasound imaging system, there are a number of considerations. First, image data should not be ignored during generation of the output volume. The quality of the image can be negatively impacted by the failure to consider image data in the scanning planes. Second, the processing of the image data in all of the scanning planes should be performed quickly and with the least amount of memory space. Third, the user should have the option of utilizing a number of different algorithms to process the image data in the scanning planes. This flexibility permits the user to select the algorithm producing the highest quality image.

[0006] These and other design objectives are realized by the ultrasound imaging method of the present invention. The method can construct three- or four-dimensional image of an object, such as an organ (e.g., for medical diagnosis) or a machine part (e.g., for defect identification). The system includes at least the following steps:

[0007] (a) generating a plurality of image (or scanning) planes, each image plane containing image information describing an object; and

[0008] (b) determining the image information in each of the plurality of image planes that is within a defined portion of an output volume to define a first subset of image information corresponding to the defined portion. The defined portion of the output volume can be an image element of any number of dimensions, such as pixels (i.e., two-dimensional image elements), voxels (i.e., three-dimensional image elements) or toxels (i.e., four-dimensional image elements).

[0009] As noted, the generating step is commonly performed by manually passing an ultrasound probe over the object or by electronically pulsing differing piezoelectric transducers in a transducer array. The position of the ultrasound probe is typically determined by a position sensor attached to the ultrasound probe.

[0010] The spatial offset(x, y, z, ∝, β, γ) offset between the the position sensor or the used probe and the ultrasound-beam generating piezoelectric transducer in the transducer array must be determined. This offset can be determined by utilizing different methods. By way of example, a measurement with a three-dimensional ultrasound phantom can be performed. The offset can be determined by measuring defined landmarks in phantom images acquired from different directions.

[0011] The output volume generally includes a plurality of defined portions. The determining step is therefore repeated for each of the plurality of defined portions to define a subset of image information corresponding to each of the plurality of defined portions.

[0012] The image information typically includes location coordinates (i.e., x,y,z) relative to a reference axis, an angular orientation of the probe (alpha, beta, gamma) and gray scale value. As used herein, “gray scale value” refers to the gray scale value with or without color information, tissue doppler imaging information, or any other parameter describing the appearance of the object. The output volume is typically defined by three orthogonal reference axes. In the determining step, the image information that is located within the defined portion or that is within a selected distance of the defined portion is identified to define the first subset of image information.

[0013] The first subset of image information can be analyzed to define a second set of image information that is derived from the first subset of image information. The second subset of image information is different from the first subset of image information. The analysis can be performed using any known interpolation algorithms, including the first found algorithm, the closest distance algorithm, the weighted average algorithm, and the last found algorithm. A gray scale value is thereby assigned to the defined portion of the output volume. The gray scale value is related to a plurality of gray scale values in the first subset of image information.

[0014] There are a number of benefits associated with the imaging method of sorting through all of the image information. The gray scale value for a voxel or toxel in the output volume (or tolume) is determined by using all of the corresponding image information in the input volume. By way of example, image information is not ignored during generation of the output volume. The quality of the image is therefore relatively high. The simultaneous processing of selected image information in all of the scanning planes can also be performed quickly and with the least amount of memory space. Finally, the user has the option of utilizing a number of different algorithms to process the image data in the scanning planes. This flexibility permits the user to select the algorithm producing the highest quality image.

[0015] The present invention further includes an ultrasound imaging system for constructing three- or four-dimensional images describing an object. The system preferably includes:

[0016] (a) generating means for generating a plurality of image planes, with each image plane containing image information describing the object; and

[0017] (b) determining means (e.g., a processor) for determining the image information in each of the plurality of image planes that is within a defined portion of the output volume to define a first subset of image information corresponding to the defined portion in the output volume. The generating means is in communication with the determining means to permit image construction.

[0018] The generating means can be any suitable image data acquisition device, such as an ultrasound probe connected to an ultrasound system. A position sensor receiver can be attached to the ultrasound probe to determine the position of the ultrasound probe. By way of example, this can be achieved by an electromagnetic position sensoring system. The generating means can further include a calibration to determine a pixel size of a plurality of pixels in each image plane.

[0019] The system can further include analyzing means (e.g., a processor) for analyzing the first subset of image information to define a second set of image information that is derived from the first subset of image information. The second subset of image information is different from the first subset of image information. The analyzing means can assign a gray scale value to the defined portion of the output volume. The gray scale value is preferably related to a plurality of gray scale values in a first set of image information.

[0020] In a further embodiment of the present invention, a method is provided for constructing a three- or four-dimensional image describing tissue of a patient using temporal information. The method includes the steps of:

[0021] (a) generating image information describing the tissue and corresponding to a plurality of image planes, wherein the image information comprises a time stamp (e.g., a time at which the image information was generated). This time stamp can be an absolute time stamp and/or relative time stamp generated at a defined event. By way of example, such an event can be a defined phase in the cardiac cycle (e.g., measured with an ECG) and/or respiratory cycle of a patient;

[0022] (b) during the generating step, monitoring at least one of the cardiac and/or respiratory cycles of the patient to provide a set of temporal signals;

[0023] (c) filtering the image information based on the relationship of the image information to the temporal signals; and

[0024] (d) thereafter generating an image of the tissue.

[0025] The monitoring step can include a number of substeps. In the case of cardiac cycles, a plurality of cardiac signals (a type of temporal signal) are generated based on sampling of an ECG signal and thereafter converted into digital values. Each of the digital values is then compared to a threshhold value to identify cycles of contraction and relaxation of heart muscles. In the case of respiratory cycles, a plurality of respiratory signals (another type of temporal signal) are generated and converted into digital values. As in the case of cardiac signals, each of the digital values is then compared to a threshhold value to identify cycles of contraction and relaxation of diaphragm muscles.

[0026] In the filtering step, the time intervals between cycles of contraction and relaxation of the heart or diaphragm muscles are determined and used in the selection of an optimal time interval. The time stamps for each item of image information are individually compared to the selected time interval. If the time stamps for a given item of image information fall within the time interval, the item is considered in generating the image of the tissue. If not, the item is not considered in generating the image. Another way of using the time stamps is to generate a continuous stream of images together with the time stamps. The image information is then filtered or sorted retrospectively after the acquisition according to the time stamps.

[0027] In cardiac applications, after the filtering step the various image elements in the output volume can be sorted based upon time stamp to provide images of the heart at selected points during a cardiac cycle or at the selected point during a number of cardiac cycles.

[0028] FIG. 1 depicts the ultrasound image system and the computer for image reconstruction;

[0029] FIG. 2 depicts the assorted scanning planes produced during image acquisition;

[0030] FIG. 3 depicts the output volume and an image element in the output volume;

[0031] FIG. 4 is a flow schematic depicting the image acquisition and reconstruction steps;

[0032] FIGS. 5-7 depict various flow schematics for different algorithms that can be used during image reconstruction to determine gray-scale value for an image element;

[0033] FIG. 8 represents MinDist and MaxDist relative to an image element of interest;

[0034] FIGS. 9A and B are depictions of a two-dimensional scanning plane for a sector scanner (FIG. 9A) and a linear scanner (FIG. 9B);

[0035] FIG. 10 is a flow schematic of another embodiment of the present invention;

[0036] FIG. 11 depicts the ECG signal as a function of time;

[0037] FIG. 12 depicts a plot of digital values translated from the ECG signal as a function of time;

[0038] FIG. 13 depicts a plot of the r—r interval versus time;

[0039] FIG. 14 depicts respiration intensity as a function of time;

[0040] FIG. 15 is a histogram based on the digital respiration values;

[0041] FIG. 16 depicts the various output volumes V₁to V_nin a representative r—r interval; and

[0042] FIG. 17 depicts a plurality of two-dimensional images, each from an image scan, containing image information.

[0043] The present invention provides a method and apparatus for image generation and reconstruction that is particularly useful in ultrasound applications. The method preferably includes the steps of: (i) generating a plurality of scanning planes containing image information together with spatial and/or temporal information of an object; (ii) providing an output volume for image reconstruction containing a plurality of image elements (e.g., image elements of any number of dimensions, such as pixels, voxels or toxels); and (iii) determining the image information in each of the plurality of scanning planes that is within a selected distance of a selected image element in the output volume or is contained within the selected image element itself. The image information falling within the selected distance of the selected image element or contained within the selected image element itself (referred to as “defined portion of the output volume”) can be stored in the database and analyzed by any number of techniques at a later time. By way of example where image information from numerous scanning planes falls within the selected distance or is contained within the image element (i.e., the image information for a given image element contains a plurality of x's, y's, z's, alphas, betas, gammas, and/or other parameters (such as gray-scale value in ultrasound applications)), the image information can be analyzed by various estimation methods, such as first found, last found, closest distance, weighted average, and the like, to filter the image information as the resulting gray scale value in the output element. The above steps are repeated sequentially image element by image element in the output volume until all of the image elements in the volume contain image information.

[0044] The method is particularly useful for generating three- and four-dimensional images from two-dimensional image information. In ultrasound applications, for example, the scanning planes are segmented into a plurality of pixels (i.e., two dimensional elements) with each pixel containing image information. The scanning planes include information about the position (x, y, and z) and orientation (alpha, beta, and gamma) of the ultrasound probe in space as a function of time. The image data set for each voxel or toxel includes information regarding voxel or toxel position and the gray-scale value. As will be appreciated, the scanning planes can be substantially parallel to one another, such as where the probe is mounted on a carriage device, or rotated to one another, such as where the probe rotates around a fixed axis (e.g., around any one of the x, y, or z axes) or is arbitrarily moved where the probe is located by a suitable tracking system.

[0045] This technique permits the reconstruction of a more accurate image based on image information. The technique does not discard image information for each voxel but can have all the image information within the specified distance of a portion of the voxel and permit the user to perform a neighborhood analysis by any one of a number of techniques selected by the user. The user therefore has flexibility in selecting the analytical technique providing the most accurate image in a given application.

[0046] In another embodiment, the output volume is generated using spatial and temporal image information. The appropriate image information to be included in each image element is thus determined for each of the plurality of scanning planes by selecting image information that is within a selected distance of the element or is contained within the selected image element itself and/or within a selected time interval of a selected reference point. By way of example, the ultrasonic imaging system can “time stamp” the scan yielding image information. The time stamp indicates the time of the scan and therefore the acquisition of the image information (other than the time stamp) relative to a selected reference point. The time stamp can be measured relative to the initiation of image acquisition sequence (i.e., absolute time stamp), relative to the nearest cardiac cycle (or r wave) (i.e., time stamp relative to r-wave), relative to the nearest respiratory cycle (i.e., time stamp relative to respiratory cycle), or relative to another selected temporal reference point. The use of time stamps permits sorting of the image elements to be performed based on spatial information and/or temporal information as desired. The time stamp advantageously permits a user to perform freehand data acquisition for applications involving moving organs, such as the heart, and to perform image construction at selected points of time during organ movement.

[0047] In yet another embodiment, the image information includes the one-dimensional ultrasound beam data instead of the two-dimensional ultrasound images commonly used in image reconstruction. Ultrasound beam data is the ultrasound information along a one-dimensional ultrasound be am or ray (which are typically generated by a single piezoelectric element or an array of piezoelectric elements) resulting from reflection of the beam or ray by tissue or body structures. For example, ultrasound beam data includes “raw” RF beam data, demodulated beam data, and any signal along the beam derived from the “raw” RF data. A two-dimensional ultrasound image generally is formed by data produced by a plurality of ultrasound beams. By using ultrasound beam data as image information and not two-dimensional ultrasound images, the present invention can produce three- and four-dimensional images that are more accurate than those produced using the two-dimensional images alone. As will be appreciated, in converting the ultrasound beam data into two-dimensional images prior to three- or four-dimensional image reconstruction valuable data can be discarded. The present invention overcomes this problem by converting the beam data directly into the three- and four-dimensional images and not using the intermediate two-dimensional images.

[0048] Referring to FIG. 4, the steps for generating a three-dimensional image of a scanned object according to the present invention are illustrated. Referring to FIGS. 1 and 4 in action boxes 50 and 54, the user, in one embodiment of the invention, attaches a transmitter 22 of a position sensing system to the patient's table to provide a two- or three-dimensional electromagnetic “grid” (formed by overlapping electromagnetic fields) and a position sensor receiver to the ultrasound imaging probe 10 connected to the ultrasound system 14 to record the position and orientation of the probe 10 in the electromagnetic grid during image acquisition as a function of time. As will be appreciated, other suitable grid systems can be used for probe tracking, such as an infrared grid system. Alternatively, any other suitable probe can be employed including a probe carriage device such as that disclosed in U.S. Pat. Nos. 5,295,486 and 5,105,819 (which are incorporated herein by reference) or a fixed probe such as that disclosed in U.S. Pat. No. 5,159,931 (which is incorporated herein by reference).

[0049] In action boxes 58, 64 and 66 and decision box 62, a measurement is performed with a calibration phantom to determine the precise position of the sensor on the probe and the (x,y,z,∝, β, γ) offset and an image calibration is thereafter generated to determine the pixel size (x and y) of the frame. The calibration phantom informs the central processor in the computer 18 of the offset in space and orientation contained within a specified amount of probe displacement. If the measurement is not successful, action box 58 is repeated. If the measurement is successful, in action box 66 the pixel size (x and y) of the frame and offset are stored in the memory of the computer 18.

[0050] In action boxes 70-74, the scanning or image information acquisition is performed. After location of the object or anatomy to be scanned in action box 70, freehand scanning, or another suitable scanning technique, is performed by placing the probe over the object and moving the probe relative to the object or by pulsing different transducers in a transducer array. In freehand scanning, the user moves the probe over the object and can translate, rotate, and tilt the probe by an arbitrary amount in any arbitrary direction at any time. As shown in FIG. 2, the position (x, y, and z) and orientation (alpha, beta, and gamma) of the probe over time is recorded by the position sensor receiver attached to the corresponding scanning plane 26a-n′. As will be appreciated, to define a plane in space a point (x, y, and z) on the plane and the orientation (alpha, beta, and gamma) of the plane are required. The user terminates acquisition when n′ scanning planes have been acquired with n′ being a sufficient number of scanning planes to permit reconstruction of an image of the scanned object. The image information includes raw images that represent the ultrasound reflection for a number of specific scanning planes in the object. The image information for each of the scanning planes 26a-n′ are digitally stored together with all corresponding spatial and temporal coordinates and directions in the computer memory.

[0051] In action box 78, the coordinate system used during image acquisition is translated or shifted to the upper left of every scanning plane. This is done by calibrating the raw image(s) in each scanning plane, translating from the sensor coordinate system which originates in the sensor receiver to the upper left of every scanning plane, and setting the resolution in all dimensions of the output image to the minimal resolution of the raw image dimensions. Using the stored values of the earlier measurement with the calibration phantom, the raw images are calibrated by defining the resolution of x and y corresponding to image width and image height (measured in millimeter per pixel) of the object. To translate the sensor coordinate system, the following equation is employed for x, y and z:

[0052] X′=Tb+Rb*Ri(Ti+X)

[0053] where Tb=the translation of the position sensor, Rb=the rotation matrix calculated from position sensor angles, Ri=the matrix describing the tilting of the image plane relative to the plane defined by the receiver of the position sensor, and Ti=the translation of the upper left of the scanning plane relative to the position transmitter location. As will be appreciated, the equation can be used for y by substituting “y” for “x” and for z by substituting “z” for “x”. The resolution in all of the volume dimensions is set to the minimal resolution of raw image dimensions. In other words, the appropriate number of pixels is determined that correspond to each millimeter of the output volume in the x, y, and z directions.

[0054] In action box 82, the orientation of the coordinate system (x, y, and z) is changed so that the output volume is substantially minimized. This step minimizes the risk of exceeding the memory capacity of the computer 18 by making the volume too large. Accordingly, the orientation of the coordinate system is set to enable the volume to contain the various scanning planes using the least amount of memory capacity.

[0055] In action box 86, the orientation (x, y, and z) is changed again to make the “z” dimension the smallest dimension. Thus, the z axis is oriented such that the smallest dimension of the scanned object lies along the z axis.

[0056] Referring to FIGS. 3 and 4, in action box 90, image information in each of the scanning planes that is within the selected distance of a portion of each and every image element 94a-min the output volume 98 (e.g., voxel cube) is determined. The goal of volumetric reconstruction is the transformation of the image information corresponding to each pixel of the various scanning planes 26a-n′ to a three-dimensional orthogonal coordinate system. Thus, the collection of raw images of dimension Xin and Yin in the scanning planes 26a-n′ is transformed into an output volume 98 having dimensions x, y, and z. An output image 102 is one slice of the output volume 98. Output volume dimensions in x and y correspond to image width and image height. Output volume dimension z is described by a vector that is substantially normal to an output frame.

[0057] The reconstruction algorithm iterates sequentially through each image element in the output volume and estimates the gray-scale value from the image information in all of the scanning planes that is contained within or is within the selected distance of the image element. Thus, the output volume is “empty” of image information before application of the reconstruction algorithm and is sequentially filled image element by image element by the algorithm. In contrast, existing reconstruction algorithms sequentially fill the output volume with image information contained within the scanning planes. This is done on a scanning plane-by-scanning plane basis rather than on an image element-by-image element basis.

[0058] The reconstruction algorithm of the present invention can be performed mathematically for each image element in the output volume by reviewing for each image element all of the scanning planes and identifying for the selected image element the scanning planes which intersect with a sphere of radius “R” centered at the image element of interest. As will be appreciated, the sphere can be replaced by a rectangle or another suitable geometrical shape.

[0059] In each of the iterations, “R” is determined based on MinDist and MaxDist values. As shown in FIG. 8, MinDist and MaxDist values represent two discrete spatial distances between the output image element and the input image element. More specifically, MinDist and MaxDist are distances in space given by x,y and z coordinates related to the image element of interest 96. MinDist and MaxDist are values used as distance thresholds for “R” in the reconstruction algorithm. MinDist and MaxDist settings depend on the pixel size of an individual data set which are derived from the image calibration. Thus, pixel sizes are identical for all frames of a data set. MinDist and MaxDist are computed according to the following equations:

[0060] MinDist=Input pixel size [x];

[0061] and

[0062] MaxDist=6*input pixel size [x].

[0063] By way of example, if the pixel sizes of the acquired frames through image calibration are 0.50 mm per pixel for the x dimension (horizontal) and 0.34 mm per pixel for the y dimension (vertical), MinDist is set at 0.5 mm and MaxDist is set at 6*0.5=3 mm.

[0064] The difference, MaxDist−MinDist, is divided into n−1 steps to yield DeltaDist, with n representing the number of iteration loops in the reconstruction algorithm. Preferably, n is set at 4. Thus, DeltaDist is defined by the following equation:

[0065] DeltaDist=(MaxDist−MinDist)/(n−1)

[0066] Using the value of DeltaDist, the value for “R” is determined by one of the following equations, depending upon the iteration:

[0067] After all of image information in the neighborhood of each image element 94a-min the output volume 98 (where m is the number of image elements in the output volume) is determined by the above-noted neighbor analysis or when the neighbor analysis is being performed on an image element-by-image element basis, any number of analytical techniques can be employed in action box 90 to assign a particular gray-scale value or other parameter to each image element. Such techniques include, for example, a first found algorithm, a closest distance algorithm, a weighted average algorithm, and a last found algorithm.

[0068] FIG. 5 is a flow schematic of the first found algorithm being used at the same time as the neighbor analysis technique noted above. At most n iterations are performed. In the first iteration, the first input pixel gray-scale value that is found and that is closer to a portion of the image element 94 than the MinDist is taken as the actual gray-scale value for the particular image element 94. If no such pixel is found, MinDist is set to MinDist+DeltaDist and the operation is repeated. If no pixel is found after n repetitive operations, the pixel value is left undefined.

[0069] Referring to FIG. 5 in action box 100, the first image element 94ain the output volume 98 is selected and the first pixel in the first scanning plane in a frame stack. The frame stack is simply the collection of all of the scanning planes 26a-n′. In decision box 104, the central processor determines if the image element 94ais the last image element in the output volume 98. If so, the central processor exits the algorithm. If not, the central processor continues to decision box 108, where it determines whether the actual iteration is the last iteration “n”. If so, the central processor continues to action box 112. If not, the central processor continues to decision box 116 where it determines if the pixel under consideration is the last pixel in the frame stack. If so, the central processor continues to action boxes 120 and 124 where it increases the iteration number by “1”, and the pixel number to the first pixel and returns to decision box 104. If not, the central processor continues to decision box 128, where it determines if the distance from the center of the image element 94ato the pixel in the scanning plane 26a-nof interest is less than the selected radius (i.e., MinDist). If so, the central processor continues to action box 132 where it sets the gray-scale value for the image element 94aequal to the pixel gray-scale value. If not, the central processor continues to action box 136 where it selects the next pixel and returns to action box 108. In action box 112, the steps are repeated for the next (typically adjacent) image element in the output volume.

[0070] Referring to FIG. 6, the closest distance algorithm is depicted. In the algorithm, the gray-scale value of the pixel of the scanning planes 26a-n′ that is closest to the image element 94a-mand that is closer than MinDist is taken as the gray-scale value of the image element 94a-m. In action box 140, the central processor selects the first image element 94a. In decision box 144, the central processor determines if the image element is the last image element. If so, the central processor exits the algorithm. If not, the central processor continues to action box 148 where it sets the gray-scale value of the image element 94aequal to the gray-scale value of the closest pixel in the various scanning planes 26a-n′ to the image element 94a. If the closest pixels are equidistant from the image element, suitable mathematical techniques can be used to select the gray-scale value, including weighted average techniques. In action box 152, the central processor repeats the preceding steps for the next image element 94bin the output volume 98 and so on until the steps are performed for image element 94m.

[0071] FIG. 7 depicts the weighted average algorithm. The gray scale value of the image element 94a-mis the linear distance-weighted average of all neighbored input pixels in the scanning planes 26a-n′. As noted above, if no neighbors are found in the actual iteration step, the process is repeated up to n times with MinDist=MinDist+DeltaDist. In action box 156, the image element 94ais set as the first image element in the output volume and the actual iteration is set at n=1. The central processor then continues to decision box 160 where the central processor determines if the image element 94ais the last image element in the output volume 98. If so, the central processor exits the algorithm. If not, the central processor continues to decision box 164 where it determines if the subject iteration is the last (i.e., whether the iteration is the nth iteration). If so, the central processor proceeds to action box 168 where it sets the image element's gray-scale value to the background value and continues to action box 160. If not, the central processor continues to decision box 172, where it identifies the pixels in the frame stack closer to the image element 94athan the radius “R” (i.e., MinDist). If no such pixels are identified, the iteration number is increased by 1 in action box 176 and the central processor returns to decision box 164. If a pixel is identified, the central processor continues to action box 180, where it determines the weighted average of the gray-scale values of all of the pixels identified as being within radius “R” of the image element 94aand assigns the weighted average gray-scale value as the gray-scale value of the image element. In action box 168, the central processor repeats the preceding steps for the next image element 94bin the output volume and so on until the steps are performed for image element 94m.

[0072] Following the completion of action box 90, the central processor in action box 200 saves the image reconstruction information in memory. In decision box 204, the central processor determines if further scanning or image information acquisition are required. If so, the central processor returns to action box 66. If not, the central processor exits the algorithm.

[0073] In an alternative embodiment, ultrasound beam data is used in action box 90 to perform image reconstruction instead of two-dimensional images formed from the beam data as noted in the embodiment above. By computing the spatial position of the ultrasound beam data in combination with the spatial information for the two-dimensional scan plane containing the ultrasound beam data, this embodiment can sort oblique scan planes to form three- and four-dimensional images. As will be appreciated, three- and four-dimensional image reconstruction commonly converts ultrasound beam data corresponding to a particular scanning plane into a single two-dimensional image. As shown in FIGS. 9A and B, each scanning plane 250a,bcan contain a plurality of ultrasound beams 254, 258, with the number of beams per scanning plane typically being 64, 128, or 256. By knowing the position of the scan plane 250a,bin which the beam is generated, the number of the beam (i.e., 1. beam, 2. beam, etc.), and the type of probe (i.e., linear, curved linear, sector, etc.), the spatial position of each beam 254a-nor 258a-ncan be determined. The location of the beam and its associated data is then used in action box 90 above to complete each of the image elements in the output volume.

[0074] Yet another embodiment is depicted in FIG. 10. In this embodiment, both spatial and temporal image information are employed in the reconstruction of three- and four-dimensional images using the techniques described above. A time stamp is stored with each element of ultrasound beam data or two dimensional image produced from the beam data. Each time stamp can represent a selected period of time, such as 16.5 milliseconds. Thus, two time stamps represent 33 milliseconds after a selected datum point and so on. Typically, one time stamp ranges from about 1 to about 100 milliseconds. When image acquisition is completed, spatial information can be used to transform the beam data or series of two dimensional images derived therefrom into a three dimensional image dataset while the time stamp information is used to transform the two dimensional images or beam data into a four dimensional data set.

[0075] Referring again to FIG. 10, the computer performs the steps noted above with the addition of action boxes 260, 264, 268 and 272. Action box 260 is substantially the same as action box 90 above. Thus, an output volume can be constructed that includes all of the image information corresponding to each of the time stamps. The output volume can be employed in later steps to form a plurality of output volumes corresponding to a plurality of different time stamps or points in the cardiac cycle.

[0076] In action box 264, filtering is performed based on the temporal distribution of the image information relative to the movement of the heart or cardiac cycle. During image acquisition, the ECG signal is monitored using conventional techniques and pertinent analog values stored for each two dimensional image or item of beam data. Additionally, an absolute time stamp and/or time stamp relative to the r-wave are determined and stored with the image or beam data. This data is depicted in FIG. 17.

[0077] In action box 264, the image information is filtered based on the length of the heart cycle (i.e., the interval between adjacent r-waves). As will be appreciated, the heart is not beating at a constant rate and therefore certain images acquired during some of the heart cycles may have to be rejected depending upon the length of the heart cycle. To perform the filtering step, suitable hardware and software, as known in the art, samples the analog ECG signal of the patient (see FIG. 11) at a number of points S₁to S_nand converts the magnitude of the signal at each point into a proportional digital value. Referring to FIG. 12, the digital values above a threshold level are converted into r-waves (i.e., contractions of the heart muscles) R₁to R_nwhile the digital values below the threshold level are assumed to correspond to relaxation of the heart muscles (i.e., the interval between adjacent r-waves). The time intervals t₁to t_nbetween adjacent r-waves R₁to R_nare determined and a histogram of the type depicted in FIG. 13 is plotted. The apex 280 of the resulting curve 284 is assumed to correspond to a perfect R—R interval. An interval “I” is selected having the apex 280 of the curve as its center. The width of interval “I” is selected based on suitable criteria known in the art. Only image information falling within the selected interval “I” is used in later reconstruction steps. Image information falling outside of the interval is rejected as being anomalous.

[0078] In action box 268, the image information is filtered based on the patient's respiration phase. As will be appreciated, because the heart is moving during respiration, certain images, depending upon the respiration phase, may have to be rejected to enhance image quality. To perform this filtering step, a patient's respiration is monitored using suitable equipment during image acquisition. To perform the filtering step, suitable hardware and software, as known in the art, samples the analog respiration signal of the patient (see FIG. 14) at a number of points S₁to S_nand converts the magnitude of the signal at each point into a proportional digital value. The digital values are used to prepare a histogram of the type depicted in FIG. 15. The apex 290 of the resulting curve 294 is assumed to correspond to a perfect respiration phase relative to the R—R interval. An interval “I” is selected having the apex 290 of the curve 294 as its center. The width of interval “I” is selected based on suitable criteria known in the art. Only image information falling within the selected interval “I” is used in later reconstruction steps. Image information falling outside of the interval is rejected as being anomalous.

[0079] After the filtering step, the various image elements in the output volume can be sorted based upon time stamp to provide images of the heart at selected points during a cardiac cycle or at the selected points during a number of cardiac cycles. This is depicted in FIG. 16 for a plurality of output volumes V₁to V_nfor a representative cardiac cycle formed by R-waves R₁and R₂. In the former case, the time stamp relative to the r-wave is employed such that an output volume corresponds to each time stamp in a representative r—r interval. Stated another way, each time stamp includes image information from a number of different ultrasound scans taken at different times. In the latter case, the absolute time stamp is used such that an output volume corresponds to each absolute time stamp over a number of r—r intervals.

[0080] While various embodiments of the present invention have been described in detail, it is apparent that modifications and adaptations of those embodiments will occur to those skilled in the art. It is to be expressly understood that such modifications and adaptations are within the spirit and scope of the present invention.