US 6,320,928 B1Grant
Method of reconstruction of a three-dimensional image of an object
Issue Date:2001-11-20
•12 Claims
•2 Drawing Sheets
Abstract
A set of digital two-dimensional projected images of an object is acquired for different positions of a camera rotating around an object. The projected images are calibrated into a volume containing the object and divided into voxels, the space coordinates of which are identified in a chosen calibration frame of reference. A pretreatment is applied on each acquired image in order to elaborate a rectified image having a predetermined spatial orientation selected as a function of a chosen specific axis of the calibration frame of reference, and an iterative algorithm of algebraic image reconstruction between each rectified image and the set of voxels is applied by successively treating the voxels in a predetermined order linked to the choice of said specific axis, which makes it possible to minimize the duration of reconstruction of the three-dimensional image.
Metadata
Assignee
- GE Medical Systems, S.A.
Inventors
- Régis Vaillant
- Yves Trousset
- Romain Boucherie
- René Romeas
Application Information
Application Number:US 09/254,094
Filing Date:1999-03-01
Priority Date:1996-09-04
Art Unit:7
Classifications
IPC:
A61B 600
Field of Search:
700 98700118700182700120345419345420382133382130378 4378901
Patent Drawings (2 sheets)
Description
[0002] This is a continuation application of International Application No. PCT/FR97/01554 filed Sep. 3, 1997.
Background of the Invention
[0003] The invention concerns the reconstruction of a three-dimensional image of an object from a set of two-dimensional projected images of the object obtained for different positions of a camera around the object.
[0004] Its application is of particular interest in the medical field, in which reconstruction of the internal structures of a patient being examined is undertaken, and especially the reconstruction of angiographic images, that is, to obtain images of vascular trees opacified by injection of a contrast product.
[0005] The invention can, however, find applications in other fields, notably, in nondestructive industrial control, in which examinations of the same type as medical examinations are performed.
[0006] In the medical field two-dimensional projected images of the object, a patient's head, for example, are generally obtained by rotation of an X-ray camera turning around the object
[0007] There are essentially two types of reconstruction algorithms in X-ray imaging.
[0008] A first type provides for a calculation of back projection and filtering or even a reconstruction by Fourier transform in several dimensions.
[0009] A second type, involved in the present invention, concerns iterative methods of reconstruction also called algebraic. The principle of such an algebraic algorithm is well known to the expert and has already been the subject of numerous published papers. One can notably cite Gordon, Bender and Herman, “Algebraic Reconstruction Technic for Three-Dimensional Electron Microscopy and X-ray Photography,” Journal THEO. BIOL. 29, pages 471 to 781 (1970); Anil K. Jain, “Fundamentals of Digital Image Processing,” Prentice Hall Information and System Sciences Series, Thomas Kailath Series Edition, or French patent Applications Nos. 89 03606 or 89 16906.
[0010] After a calibration of the camera used to determine, notably, the parameters of projection in the projection planes of the acquired images, of an observed volume broken down into elementary volume elements or voxels (those calibration parameters forming projection matrices), the algebraic image reconstruction algorithm is used to reconstruct the three-dimensional volume from those two-dimensional projected images. The basic principle of the algorithm is to initialize the voxels of the volume to a predetermined initial value, a zero value, for example, and to repeat a number of times the following operations: projection of voxels in the plane of each acquired image so as to obtain a virtual image, determination of the difference between the projected volume (virtual image) and the corresponding acquired image and then back projection of the difference in volume. After a number of iterations, an estimated value representative of the density of the contrast product injected in the vessels X-rayed is obtained for each voxel, which makes it possible to visualize in three dimensions the cartography of those X-rayed vessels.
[0011] Such an algorithm necessitates determining a great number of times, typically several million times, the projection of a voxel, using the different projection matrices.
[0012] If the algorithm is not optimized, determination of the projection of a voxel requires at least nine additions, ten multiplications and one division, which is a severe drain on the calculation time of the microprocessor incorporating the algorithm and, consequently, on the duration of image reconstruction.
Brief Summary of the Invention
[0013] An embodiment of the invention optimizes the time of calculation of the image reconstruction algorithm and consequently reduces the time necessary for reconstruction of the three-dimensional image.
[0014] An embodiment of the invention is thus intended to reduce to a single addition the number of operations necessary for the determination of projection of a current voxel, knowing the projection of the previous voxel.
[0015] An embodiment of the invention therefore proposes a method of reconstruction of a three-dimensional image of an object from a set of digital two-dimensional projected images of the object obtained for different positions of a camera around the object. The embodiment of the method may include a calibration of the camera, an acquisition of the set of digital two-dimensional projected images and a reconstruction of the three-dimensional image from the projected two-dimensional acquired images and an iterative algorithm of algebraic image reconstruction. According to a general characteristic of an embodiment of the invention, the calibration stage a volume containing the object is broken down into voxels, the space coordinates of which are identified in a so-called chosen “calibration” frame of reference. A pretreatment or rectification is applied on each acquired image in order to establish a rectified image having a predetermined spatial orientation selected as a function of a chosen specific axis of the calibration frame of reference, e.g., the mean estimated axis of rotation of the camera around the object. The iterative algorithm of algebraic image reconstruction between each rectified image and the set of voxels is then applied by successively treating the voxels in a predetermined order linked to the choice of said specific axis. The duration of reconstruction of the three-dimensional image is thus minimized.
Brief Description of the Drawing(s)
[0016] Other advantages and characteristics of the invention will appear on examination of the detailed specification of a not-at-all limitative mode of use and of the attached drawings, on which:
[0017] FIG. 1 schematically illustrates a set of two-dimensional projected images around an object;
[0018] FIG. 2 illustrates in more detail the acquisition of one of those two-dimensional projected images, and
[0019] FIG. 3 is a flow chart of a mode of use of the process according to an embodiment of the invention.
Detailed Description of the Invention
[0020] Although the invention is not limited thereto, an embodiment of the invention to the reconstruction of a three-dimensional angiographic image of a patient, particularly of his/her head, is now going to be described.
[0021] According to one embodiment, the digital two-dimensional projected images are acquired from detectors placed in different projection planes corresponding to the different positions of the camera. In practice, these different detections can be obtained by the rotation of a single detector around the object. Calibration of the camera, notably, provides calibration parameters (projection matrices) relative to the projection of voxels in these projection planes. For the rectification of each acquired two-dimensional image, the acquired two-dimensional image is projected in a rectified plane parallel to the specific axis chosen, a rectified frame of reference is defined in the rectified plane, one of the axes of which is parallel to the specific axis chosen, and rectification parameters (rectification matrices) are established, making it possible to determine the coordinates of each pixel of the acquired image in the associated rectified frame of reference, in order to establish the digital rectified image for the acquired image. The calibration parameters associated with each acquired image are then modified, taking into account the rectification parameters, for example, by making a simple matrix calculation between the initial projection matrices and the rectification matrices, so as to obtain modified or rectified projection matrices. The iterative algorithm of algebraic image reconstruction is applied between each rectified image and the set of voxels, taking into account the modified calibration parameters by successively treating the voxels situated on lines parallel to said specific axis chosen.
[0022] In other words, while in the prior art, the algebraic image reconstruction algorithm was applied directly between the set of voxels of the volume and the set of acquired two-dimensional images in the different original projection planes, the image reconstruction algorithm is applied according to the invention between images rectified in relation to those acquired images and the set of voxels of the volume by successively treating the voxels in a predetermined order, taking into account the choice of the specific axis of the calibration frame of reference.
[0023] More precisely, a rectified plane parallel to that specific axis and in which the acquired image is projected is defined from the original projection plane of an acquired image. The pixels of the rectified image are then found in a frame of reference of that rectified plane, one of the axes of which (abscissa axis or ordinate axis) is parallel to said specific axis, the rectification matrix making it possible to determine, in that rectified frame of reference, the coordinates of each pixel of the acquired image. The projection of voxels of the volume in the rectified frame of reference of each rectified image will then be carried out with the aid of the modified projection matrix.
[0024] In an embodiment of the invention two of the coefficients of the matrix are nil, which makes it possible, by successively treating the voxels in a predetermined order, to determine, knowing the projection of a current pixel, the projection of the following pixel with only one addition.
[0025] More precisely, if z designates the coordinate of a voxel according to the specific axis chosen, for example, the mean axis of rotation of the camera, and if x and y designate the other two coordinates of the voxel, the voxels with constant x and y will be treated by varying coordinate z in order.
[0026] Referring, in particular, to FIGS. 1 and 2, it can be seen that the imaging system usable for applying the invention makes it possible to obtain a set of two-dimensional acquired images IA1-IAn, in this case by rotation of an X-ray source 2 around an object 1, e.g., a patient's head. In fact, as is standard in angiography, each acquired image IA1 is a subtracted image which is, for example, obtained by a standard technique of logarithmic subtraction of two X-rays taken under the same incidence before and after an injection of a contrast product in the vascular tree, the three-dimensional image of which it is desired to reconstruct.
[0027] Each acquired image, e.g., IAi is obtained from a two-dimensional radiation detector, for example, of the luminance amplifier type used in radiology, arranged opposite the X-ray tube in a so-called projection plane PPi. The different projection planes are obtained by the different angular positions of the detector in rotation around the patient's head. The normal XOi in projection plane PPi defines the optical axis of the acquired image IAi. The detector is connected to treatment means 3 containing, notably, sampling means connected to a microprocessor incorporating software in its program memory associated with the algebraic image reconstruction algorithm used and, in general, all the functional means making possible use of the method according to the invention.
[0028] In the case of an X-ray imaging system composed of an X-ray source and a two-dimensional detector, the geometric operation taking place in production of the acquired image is a conic projection of a scanned object, deployed in a three-dimensional space, on a two-dimensional space which is that of the projection plane corresponding to the detection plane. The geometric parameters completely describing the different conic projections must be known. Now, it is often impossible and too imprecise to access these parameter directly, that is, for example, by directly measuring on the acquisition system the distance between the X-ray source and the detector.
[0029] The operation resulting in a precise indirect knowledge of the geometric parameters occurring in the production of an image is called imaging system calibration. The principle, standard and well known, is based on the use of a geometric shadow known in three-dimensional space and whose two-dimensional projection is acquired. More precisely, the calibration involves the following stages:
[0030] a known object is used, the calibration shadow presenting a number of characteristic points whose position in space is known by coordinates measured in relation to a marker proper to that object;
[0031] the image of that shadow is acquired under geometric conditions from a point of view (or incidence) that it is desired to calibrate;
[0032] the projections of the characteristic points in the image are recognized. For that purpose, each characteristic point of the object is associated with its trace in the projected acquired image;
[0033] the system of equation describing the projection is inverted in the mathematical sense;
[0034] and the set of parameters is finally obtained for the given point of view.
[0035] A form of geometric calibration shadow often used is that of a cube, with eight corners, on which metal balls opaque to X-rays are arranged. The calibration being an operation known to the expert, it will not be described more in detail, especially considering that several papers published have already described the principle of a manual geometric calibration. The following articles can, notably, be cited:
[0036] (1) D. L. Parker, J. Wu, D. L. Pope, R. Van Bree, G. R. Caputp and H. W. Marshall, “Three-dimensional reconstruction and flow measurements of coronary arteries using multiview digital angiography,” New Develogments in Quantitative Coronary Arteriography, J. C. Reiber and P. W. Serruys Eds., pp. 225-247, Kluwer Academic Publishers, 1988;
[0037] (2) D. J. Hawks, A. C. F. Colchester and C. R. Mol, “The accurate 3-D reconstruction of the geometric configuration of the vascular trees from X-ray recordings,” Physics and Engineering of Medical Imaging, R. Guzzardi Ed., Nijhoff, 1987;
[0038] (3) M. Garreau, J. L. Coatrieux, R. Collorec and C. Chardenon, “A knowledge-based approach for 3-D reconstruction and labeling of vascular networks from biplane angiographic projections,” IEEE Medical Imaging, Vol. 10, No. 2, pp.122-131, June 1991.
[0039] It is also possible to use a known method of automatic geometric calibration of an X-ray imaging system, such as that described in French Patent Application No. 93 00804. In short, for such automatic calibration, a shadow is used in which the balls are gradually distributed in a succession such that ball heights, measured along the axis of rotation of the imaging system, and especially a shadow axis, are monotones increasing (or decreasing) with a serial number of balls in the sequence.
[0040] Calibration of the imaging system makes it possible, notably, to determine the estimated mean axis Ax of rotation of the camera around the object, as well as the position of the source 2 and the geometric characteristics of the optical axes of the different acquired images. Calibration also makes it possible to define a virtual volume VV (intersection of the different projection cones) surrounding the object 1 and broken down into elementary volume elements Vi or “voxels.” This volume VV, and therefore each voxel Vi, is spatially marked in a frame of reference, hereinafter called calibration frame of reference, one of the axes of which, axis Z in this case, is merged with the estimated axis of rotation Ax. It is to be noted here that the projection planes PPi in which the acquired images IAi are projected are not generally parallel to axis Z.
[0041] Calibration also makes it possible to define for each acquired image IAi a projection matrix Pi enabling the coordinates of its projection (pixel) in the corresponding acquired image IAi to be determined for each voxel Vi.
[0042] A mode of use of the method in an embodiment of the invention is described with reference, in particular, to FIG. 3. While according to the prior art the algebraic image reconstruction algorithm was applied directly on the acquired images IAi (obtained after calibration 30 and acquisition 31), in an embodiment of the invention performs a pretreatment or rectification 32 on the acquired images, so as to obtain rectified images IRi on which the three-dimensional image will be reconstructed.
[0043] More precisely, in order to obtain the rectified image IRi from the corresponding acquired image IAi, the optical axis XOi of the projection plane PPi of the acquired image IAi is projected in a plane perpendicular to axis Z and then the plane of the rectified image is defined as being the plane perpendicular to the projection of that optical axis XOi. The rectified plane in which the rectified image is going to be found is therefore parallel to axis Z. A rectified frame of reference is then defined in that rectified plane, one of the axes of which, e.g., the abscissa axis, is parallel to axis Z.
[0044] A rectification matrix making it possible to determine the coordinates of each pixel of the acquired image IAi in that rectified frame is reference is then defined. The new coordinates of those pixels thus define a digital rectified image IRi. The elementary geometric conversion calculations of the projections making it possible to pass from an acquired image to a rectified image and to elaborate the corresponding rectification matrix are standard calculations for the expert and have not been described more in detail above for the sake of simplification. This set of operations is carried out as software in the microprocessor of the treatment means.
[0045] With this rectification matrix, a modified projection matrix Qi is elaborated, obtained simply by a left-hand multiplication of the projection matrix Pi (obtained after calibration) by the rectification matrix.
[0046] The modified projection matrix Qi therefore makes it possible to determine the coordinates of the projection of each voxel Vi in the rectified frame of reference of the rectified image and to determine, consequently, whether that voxel is actually projected or not in that rectified image.
[0047] This modified projection matrix Qi is in the form:
[0048] It is therefore observed that two of the coefficients of that matrix Qi, namely, coefficients 123and 133, are nil.
[0049] The following stage of the procedure consists of reconstruction 33 of the three-dimensional image IF by applying the algebraic image reconstruction algorithm between the set of rectified images IRi and the set of voxels Vi of volume VV.
[0050] In that connection, a coefficient μ is assigned to each voxel Vi, which is initialized to an initial value, generally zero. Following the reconstruction stage, that is, after a predetermined number of iterations of the image reconstruction algorithm, a set of values will be obtained for all the coefficients μ associated with all the voxels of the volume. Each coefficient μ is representative of the position of the corresponding voxel in relation to a blood vessel. In fact, if the value μ assigned to a voxel is nil, then that voxel is not situated in a blood vessel, while an other than nil value of that coefficient signifies that the corresponding voxel is situated in a blood vessel. It will therefore be possible, by means of a standard visualization device, to reconstitute visually the three-dimensional image of the vascular tree situated in the patient's head from the set of values of the coefficients μ obtained.
[0051] In each iteration of the image reconstruction algorithm, each rectified image IRi is successively considered.
[0052] For each image IRi, one determines, first of all, the coordinates of the projection of each voxel Vi in the rectified frame of reference of image IRi by means of the modified projection matrix Qi. If the voxel Vi is actually projected in image IRi, that is, if the coordinates of the corresponding projected pixel are situated within the limits of the rectified image IRi, a gray level is then assigned to that projected pixel equal to the gray level of that pixel obtained on the previous iteration, plus the current value of the coefficient μ associated with voxel Vi. A virtual image IVi is thus defined, from which the rectified real image IRi is subtracted. A residual image is then obtained, each of the pixels of which is assigned to a residual gray level. A back projection or reverse projection is then made of that residual image in volume VV. In practice, for that back projection, the same projection calculations as those made in the previous projection stage are carried out on each voxel Vi of volume VV, but this time one modifies the value of coefficient μ associated with a voxel Vi which is actually projected in the residual image by adding to the present value μ of the voxel the residual value of the pixel corresponding to a near standardization coefficient.
[0053] Once these projection-back projection operations have been carried out for the set of voxels for a rectified image IRi, the same operations are performed for the following rectified image and all these operations are repeated a number of times, three in practice.
[0054] An embodiment of the invention, in combination with the pretreatment of acquired images having made it possible to obtain the rectified images, consists of successively treating the voxels Vi in a predetermined order.
[0055] More precisely, if x, y and z designate the coordinates of a voxel Vi, and if u and v designate the coordinates of projection of that voxel Vi in the rectified frame of reference of the corresponding rectified image, the coordinates u and v are obtained by the following formulas:
[0056] where λ1, λ2and λ3are defined by the following formulas:
[0057] λ1=′11x+′12y+′13Z+′14
[0058] λ2=′22x+′23y+′24
[0059] λ2=′32x+′33y+′34
[0060] taking into account the fact that two of the coefficients of the modified projection matrix Qi are nil.
[0061] Also, if one seeks to determine the coordinates of projection of the following voxel, of coordinates (x, y, z+1), the coefficients λ1n, λ2nand λ3nassigned to that following voxel are defined by the formulas:
[0062] λ1n=λ1+′13
[0063] λ2n=λ2
[0064] λ3n=λ3
[0065] Consequently, the coordinates unand vnof projection of that following voxel are defined by the formulas:
[0066] Therefore since I13and λ3are constant, determination of the coordinate un necessitates making only one addition of a constant to the coordinate u known and previously stored.
[0067] In other words, the voxels will be treated group by group, all of the voxels of a same group having their pair of coordinates (x, y) constant and, within a group, the third coordinate of the voxel will be successively varied. In other words, this means successively treating the voxels situated on lines parallel to axis z. On a same treatment line calculation of the projection of the current voxel, knowing the projection of the previous voxel, will necessitate only one addition. Of course, supplementary operations are, nevertheless, necessary on each change of treatment line, that is, when either of the other two coordinates x and y of a voxel is modified.
[0068] Various modifications in steps and/or functions and/or structure of the disclosed embodiments of the invention may be made by one skilled in the art without departing from the scope and extent of the claims.
Claims
Certificate of Correction present
The claims shown below may not include correction changes. Use the corrected PDF for authoritative text.
What is claimed is:
1. Method of reconstruction of a three-dimensional image of an object from a set of digital two-dimensional projected images of the object obtained for different positions around the object, comprising the steps of:
a. acquiring the set of digital two-dimensional projected images and a reconstruction of the three-dimensional image from the projected two-dimensional acquired images and an iterative algorithm of algebraic image reconstruction;
b. calibrating the different position images for a volume containing the object and dividing into voxels, the space coordinates of which are identified in a chosen calibration frame of reference;
c. applying a pretreatment on each acquired image in order to establish a rectified image having a predetermined spatial orientation selected as a function of a chosen specific axis of the calibration frame of reference; and
d. applying the iterative algorithm of algebraic image reconstruction between each rectified image and the set of voxels by successively treating the voxels in a predetermined order linked to the choice of said specific axis, whereby a duration of reconstruction of the three-dimensional image is minimized.
2. Method according to claim 1, wherein the digital two-dimensional projected images are acquired from detectors placed in different projection planes corresponding to the different positions;
the calibration provides calibration parameters relative to the projection of voxels in the projection planes; in the pretreatment of each acquired two-dimensional image, the acquired two-dimensional image is projected in a rectified plane parallel to the specific axis chosen, a rectified frame of reference is defined in that rectified plane, one of the axes of which is parallel to the specific axis chosen, and rectification parameters are elaborated, making it possible to determine the coordinates of each pixel of the acquired image in the associated rectified frame of reference, in order to elaborate the digital rectified image for the acquired image, and the calibration parameters associated with each acquired image are modified, taking into account the rectification parameters, and applying the iterative algorithm of algebraic image reconstruction between each rectified image and the set of voxels, taking into account the modified calibration parameters by successively treating the voxels situated on lines parallel to said specific axis chosen.
3. Method according to claim 2, wherein an optical axis of each acquired image being determined on calibration, the rectified plane of an acquired image is determined as being the plane perpendicular to the projection of the optical axis of that acquired image in the plane perpendicular to the specific axis chosen.
4. Method according to claim 1 wherein the different projected images are obtained by a camera rotating around the object, calibration of the camera determines a mean estimated axis of rotation of the camera, and the mean estimated axis of rotation is chosen as specific axis of the calibration frame of reference.
5. Method according to claim 2 wherein the different projected images are obtained by a camera rotating around the object, calibration of the camera determines a mean estimated axis of rotation of the camera, and the mean estimated axis of rotation is chosen as specific axis of the calibration frame of reference.
6. Method according to claim 3 wherein the different projected images are obtained by a camera rotating around the object, calibration of the camera determines a mean estimated axis of rotation of the camera, and the mean estimated axis of rotation is chosen as specific axis of the calibration frame of reference.
7. Method according to claim 1 wherein the acquisition of each two-dimensional projected image entails a difference of two images of the object obtained before and after injection of a contrast product in the object, for a same position of the projected images.
8. Method according to claim 2 wherein the acquisition of each two-dimensional projected image entails a difference of two images of the object obtained before and after injection of a contrast product in the object, for a same position of the projected images.
9. Method according to claim 3 wherein the acquisition of each two-dimensional projected image entails a difference of two images of the object obtained before and after injection of a contrast product in the object, for a same position of the projected images.
10. Method according to claim 4 wherein the acquisition of each two-dimensional projected image entails a difference of two images of the object obtained before and after injection of a contrast product in the object, for a same position of the projected images.
11. Method according to claim 5 wherein the acquisition of each two-dimensional projected image entails a difference of two images of the object obtained before and after injection of a contrast product in the object, for a same position of the projected images.
12. Method according to claim 6 wherein the acquisition of each two-dimensional projected image entails a difference of two images of the object obtained before and after injection of a contrast product in the object, for a same position of the projected images.
Patent Citations (7)
Non-Patent Literature (8)
- Parker et al, “Three-Dimensional Reconstruction and Flow Measurements of Coronary Arteries Using Multi-view Digital Angiography” New Developments in Quantitative Coronary Angiography, J.C. Riebor, P.W. Surreys, Eds. pp 225-247, Kluwor Academic Publishers, 1988.Applicant
- D.J. Hawkes et al. “The Accurate 3-D Reconstruction of the Geometric Configuration of the Vascular Trees From X-Ray Recordings”, Physics And Engineering of Med/Caz Imaging, R. Guzzardi, Ed, Nijhaffi 1987.Applicant
- Garremi et al. “A Knowledge-Based Approach for 3-D Reconstruction and Labeling of Vascular Networks From Biplane Angiographic Projections”, IEEE Medical Imaging, vol. 10, No. 2, pp 122-131, Jun. 1991.Applicant
- Gordon et al, “Algebraic Reconstruction Techniques (ART) for Three-Dimensional Electron Microscopy and X-ray Photography”, Journal of Theo. Biol (1970) vol. 29, pp 971-981.Applicant
- Fundamentals of Digital Image Processing, Jain, Sep. 1988, Engineering/Science/Mathematics.Applicant
- Garcia, I et al, “Implementation and Experimental Evaluation of the Constrained Art Algorithm on a Multicomputer System,” Signal Processing, vol. 51, No. 1, May 1, 1996, pp 69-76.Applicant
- Herman, G.T. et al, “Algebraic Reconstruction Techniques Can be Made Computationally Efficient”, IEEE Trans. on Medical Imaging, vol. 12, No. 3 Sep. 1, 1993, pp 600-609.Applicant
- Payot et al, “An Adaptative and Constrained Model For 3D X-Ray Vascular Reconstruction”, Proceedings of the 1975 International Meeting on Fully Three Dimensional Image Reconstruction in Radiology and Nuclear Medicine, Jul. 4, 1995, France.Applicant