Method and apparatus for virtual digital subtraction angiography

[0001] The present invention relates generally to medical imaging systems and, more particularly, to a system that detects any change in relative position between a patient and the imaging device and corrects the image data for any such motion that occurs.

[0002] Modern diagnostic medicine has benefited significantly from radiology. Radiation, such as x-rays, may be used to generate images of internal body structures. In general, radiation is emanated towards a patient's body and absorbed in varying amounts by tissues in the body. An x-ray image is then created based on the relative differences of detected radiation passing through the patients' body.

[0003] Digital subtraction is a well known technique for visually enhancing differences between such images. For example, digital subtraction angiography (DSA) is used to visualize vasculature by comparing two or more images of the same blood vessels before and after injection of a contrast agent. Assuming that the only change between the pre-contrast image (or mask) and the contrast-containing image is related to the injection of the contrast agent, the difference image clearly outlines the vessels into which the contrast agent has flowed.

[0004] However, digital subtraction techniques assume a fixed relative position between the imaging device and the patient being imaged for any images which are being compared. If this relative position changes between the time that the initial image (the one to which all subsequent images are compared) is acquired and the time that any of the subsequent images are acquired, the difference image will not only convey changes in the anatomy of the patient, but also any artifacts or changes introduced by this change in relative position between the imaging device and the patient.

[0005] Therefore, it is desirable to provide a medical imaging system that addresses the patient motion artifact problem. It is envisioned that the system will directly measure the relative position between the imaging device and the patient, and then compensate the images for any motion that occurs between the time at which the initial image is acquired and the time at which any subsequent images are acquired by the imaging system.

[0006] In accordance with the present invention, a medical imaging system is provided that detects any change in relative position between the patient and the imaging device and compensates the image data for any patient motion that occurs. The medical imaging system includes: an imaging device for capturing two or more image data sets representative of a patient; a tracking subsystem for detecting patient position data that is indicative of the position of the patient and device position data that is indicative of the position of the imaging device; an image subtraction subsystem for performing a digital subtraction operation between at least two image data sets; and a motion correction subsystem configured to detect a change in the relative position between the patient and the imaging device and upon detecting a change in the relative position, compensate in at least one of the first image data set and the second image data set for the change in relative position prior to performing the digital subtraction operation. The digital subtraction image can detect the motion of a therapeutic device, motion of a therapy as it moves through the body, perfusion of a substance, contrast agents, chemical change of a substance, a drug as it attaches itself to anatomical material or interacts with diseased tissue, or any device or substance that has an image signature within one or multiple image modalities. The digital subtraction of any 3D volume such as those created by MR, CT, Isocentric C-arms, C-arms tracked to construct volumes, 3D ultrasound, etc. can now be viewed from any angle or with any cut plane. It is also important to point out that the tracking subsystem can be implemented via a number of different devices or techniques in order to correct for patient motion. Simple modeling of patient respiration or heart cycles can be used in conjunction with a tracking subsystem or solely to provide motion correction. The system could use actual signals as inputs to these models.

[0007] For a more complete understanding of the invention, reference may be had to the following specification and to the accompanying drawings.

[0008] FIG. 1 is a diagram of a surgical instrument navigation system in accordance with present invention;

[0009] FIG. 2 is a diagram of a true and a distorted image that may be captured by the surgical navigation system;

[0010] FIGS. 3A and 3B illustrates the projective transformation process employed by the surgical navigation system;

[0011] FIG. 4 is a flowchart depicting the operation of the enhanced surgical navigation system of the present invention;

[0012] FIG. 5 is a block diagram of an enhanced surgical instrument navigation system in accordance with the present invention;

[0013] FIGS. 6A-6C illustrates the different types of changes in the relative position between the patient and the imaging device; and

[0014] FIG. 7 is a flowchart depicting the operation of the motion correction subsystem in accordance with the present invention.

[0015] FIG. 1 is a diagram of an exemplary surgical instrument navigation system. The primary component of the surgical instrument navigation system is a fluoroscopic imaging device 100. The fluoroscopic imaging device 100 generally includes a C-arm 103 attached to a mobile base 102 or fixed room system. An x-ray source 104 is located at one end of the C-arm 103 and an x-ray receiving section 105 is located at the other end of the C-arm 103. In addition, the fluoroscopic imaging device 100 includes a calibration and tracking target 106 attached to the x-ray receiving section 105. As will be further described below, the calibration and tracking target 106 further includes one or more radiation sensors 107, a plurality of tracking targets 109, and a plurality of calibration markers 111. While the following description is provided by reference to an x-ray imaging device, it is readily understood that other types on imaging devices, such as a computed tomography imaging device, a magnetic resonance imaging device or an ultra-sound device, are within the scope of the present invention. It is also readily understood that two-dimensional projection images or three-dimensional volumetric images are within the scope of the present invention.

[0016] In operation, a patient 110 is positioned between the x-ray source 104 and the x-ray receiving section 105. In response to an operator's command input, x-rays emanating from source 104 pass through the patient area, including the patient 110 and the calibration and tracking target 106, and into the receiving section 105 of the imaging device. The receiving section 105 generates a two-dimensional image based on the intensities of the received x-rays. To do so, the receiving section 105 may be comprised an image intensifier that converts the x-rays to visible light and a charge coupled device (CCD) video camera that converts the visible light to digital images. Alternatively, the receiving section 105 may be a device that converts x-rays directly to digital images, thus potentially avoiding distortion introduced by first converting to visible light.

[0017] Furthermore, the C-arm 103 is capable of rotating relative to the patient 110, thereby allowing images of the patient 110 to be taken from multiple directions. For example, the physician may rotate the C-arm 103 in the direction of arrows 108 or about the long axis of the patient 110. Each of these directions of movement involves rotation about a mechanical axis of the C-arm 103. In this example, the long axis of the patient 110 is aligned with the mechanical axis of the C-arm 103. In sum, the imaging device 100 is generally operable to capture one or more sets of image data representative of the patient 110.

[0018] Resulting fluoroscopic images are then transmitted to an image processing device 120. In one embodiment, the image processing device 120 may be comprised of two computers. A control computer 122 which allows a physician to control the fundamental operation of the imaging device 100, such as setting imaging parameters, and a second computer 124 which may be used to perform more robust image processing functions. It is envisioned that either computer may provide facilities for displaying, saving, digitally manipulating, or printing a hard copy of the received images. It is further envisioned that images may be displayed to the physician through a heads-up display (not shown). It is readily understood that these computing functions may be integrated into a single computer or distributed across three or more computing devices.

[0019] An exemplary imaging device 100 is the Series 9800 Mobile Digital Imaging System manufactured by OEC Medical Systems, Inc. of Salt Lake City, Utah. It should be noted that calibration and tracking target 106 is typically not included in the Series 9800 Mobile Digital Imaging System, but otherwise this system is similar to the imaging system 100 described above. An alternative imaging device is the SIREMOBILE Iso-C System manufactured by Siemens Medical Systems, Inc. of Iselin, N.J.

[0020] Intrinsic calibration is the process of correcting image distortion in a received image and establishing the projective transformation for that image. Raw images generated by the receiving section 105 tend to suffer from undesirable distortion caused by a number of factors, including inherent image distortion in the image intensifier and external electromagnetic fields. An example of a true and a distorted image is shown in FIG. 2. Checkerboard 202 represents the true image of a checkerboard shaped object placed in the image tracking area. The image taken by receiving section 105, however, can suffer significant distortion, as illustrated by the distorted image 204.

[0021] Furthermore, the image formation process is governed by a geometric projective transformation which maps lines in the fluoroscope's field of view to points in the image (i.e., within the x-ray receiving section 105). This concept is illustrated in FIGS. 3A and 3B. Image 300 (and any image generated by the fluoroscope) is composed of discrete picture elements (pixels), an example of which is labeled as 302. Every pixel within the image 300 has a corresponding three-dimensional line in the fluoroscope's field of view. For example, the line corresponding to pixel 302 is labeled as 304. The complete mapping between image pixels 302 and corresponding lines 304 governs projection of objects within the field of view into the image. The intensity value at pixel 302 is determined by the densities of the object elements (i.e., portions of a patient's anatomy, operating room table, etc.) intersected by the line 304. For the purpose of computer assisted navigational guidance, it is necessary to estimate the projective transformation which maps lines in the field of view to pixels in the image, and vice versa.

[0022] Intrinsic calibration involves placing calibration markers in the path of the x-ray, where a calibration marker is an object opaque or semi-opaque to x-rays. Calibration markers 111 are rigidly arranged in predetermined patterns in one or more planes in the path of the x-rays and are visible in the recorded images. Because the true relative position of the calibration markers 111 in the recorded images are known, the image processing device 120 is able to calculate an amount of distortion at each pixel 302 in the image (where a pixel is a single point in the image). Accordingly, the image processing device 120 can digitally compensate for the distortion in the image and generate a distortion-free, or at least a distortion improved image. Alternatively, distortion may be left in the image, and subsequent operations on the image, such as superimposing an iconic representation of a surgical instrument on the image (described in more detail below), may be distorted to match the image distortion determined by the calibration markers.

[0023] Since the position of the calibration markers 111 are known with respect to the tracking targets 109 and ultimately with respect to a tracking sensor, the calibration markers 111 can also be used to estimate the geometric perspective transformation. A more detailed explanation of methods for performing intrinsic calibration is described in the following references B. Schuele et al., Correction of Image Intensifier Distortion for Three-Dimensional Reconstruction, presented at SPIE Medical Imaging 1995, San Diego, Calif., 1995 and G. Champleboux et al., Accurate Calibration of Cameras and Range Imaging Sensors: the NPBS Method, Proceedings of the 1992 IEEE International Conference on Robotics and Automation, Nice, France, May 1992, and U.S. Pat. No. 6,118,845, issued Sep. 12, 2000 by the present assignee, the contents of which are hereby incorporated by reference.

[0024] The surgical instrument navigation system further includes a tracking subsystem. The tracking subsystem is generally comprised of a non-contact position location sensor, numerous tracking targets embedded in different system components and accompanying computational software implemented in the image processing device 120. In a preferred embodiment, the tracking subsystem employs a real-time infrared tracking sensor 130. Although an infrared-based tracking subsystem (either passive or active) is presently preferred, it is envisioned that other well known types of positional location devices may be used to implement the tracking subsystem. For example, positional location devices based on mechanical arms, robotics, radio wave, magnetic fields, fiber optic, or sonic emissions are also within the scope of the present invention.

[0025] The tracking sensor 130 detects the presence and location of a surgical instrument 140. To do so, the specially constructed surgical instrument 140 is embedded with tracking targets 141, such as infrared reflectors or emitters. Because the relative spatial locations of the tracking targets 141 on the surgical instrument 140 are known a priori, the tracking subsystem is able to determine the location of the surgical instrument 140 in three-dimensional space using well known mathematical transformations.

[0026] During an operation, a dynamic reference frame marker 150 is attached in a fixed position relative to the portion of the patient 110 to be operated on. For example, when inserting a screw into the spine of the patient 110, the dynamic reference frame marker 150 may be physically attached to a portion of the spine of the patient 110. The reference frame marker 150 is similarly embedded with tracking targets 151, such as infrared reflectors or emitters. In this way, the tracking sensor 130 is also able to determine the location of reference frame marker 150 in three-dimensional space.

[0027] Consequently, the surgical instrument 140 can be accurately located in three dimensional space relative to the reference frame marker 150 and thus can be located relative to the patient's anatomy. The determination of the three-dimensional position of an object, such as the reference frame marker, relative to a patient is known in the art, and is discussed, for example, in the following references, each of which are hereby incorporated by reference: PCT Publication WO 96/11624 to Bucholz et al., published Apr. 25, 1996; U.S. Pat. No. 5,384,454 to Bucholz; U.S. Pat. No. 5,851,183 to Bucholz; and U.S. Pat. No. 5,871,445 to Bucholz.

[0028] Lastly, the calibration and tracking target 106 also includes tracking targets 109, such as infrared reflectors or emitters. In a similar manner, the tracking sensor 130 detects the presence and location of the tracking targets 109. Since the calibration and tracking target 106 is in a fixed position relative to the x-ray receiving section 105, the image processing device 120 can further determine the three-dimensional position of the x-ray receiving section 105 relative to the surgical instrument 140 and/or the dynamic reference frame 150 (and thus the patient).

[0029] In one embodiment, the tracking sensor 130 determines the position of the x-ray receiving section 105 at each point in time that it captures image data. In an alternative embodiment, position data for the x-ray receiving section 105 is reported by a mechanical localizer associated with the imaging device 100. The mechanical localizer is operable to report position data for the x-ray receiving section 105 relative to a baseline position. In operation, the tracking sensor 130 may be used to determine the baseline position of the x-ray receiving section 105 relative to the either the surgical instrument 140 and/or the dynamic reference frame 150. Alternatively, you may simply touch the surgical instrument 140 to various reference points on the x-ray receiving section 105. Thereafter, as position of the x-ray receiving section 105 changes over time, its position is reported by the mechanical localizer. In other words, by first determining the baseline location, subsequent movement and location of imaging device 100 can be determined by monitoring the drive or movement mechanism surgical localizer of the imaging device 100. The image processing device 120 can then determine the position of the x-ray receiving device 105 relative to either the surgical instrument 140 and/or the dynamic reference frame 150.

[0030] In operation, the enhanced surgical navigation system assists physicians performing surgery by displaying real-time or pre-acquired images, such as fluoroscopic x-ray images, of the patient 110 on a display associated with image processing device 120 that is visible to the surgeon. Representations of surgical instruments 140 are overlaid on pre-acquired fluoroscopic images of the patient 110 based on the position of the instruments 140 as determined by the tracking sensor 130. In this manner, the surgeon is able to see the location of the instrument 140 relative to the patient's anatomy, without the need to acquire real-time fluoroscopic images, thereby greatly reducing radiation exposure to the patient and to the surgical team.

[0031] FIG. 4 is a flowchart depicting the operation of the surgical navigation system. The physician begins by acquiring one or more fluoroscopic x-ray images of the patient 110 using the imaging device 100 as shown at 402. Radiation sensors 107 embedded in the calibration and tracking target 106 may be used to detect the presence of radiation. The image processing device 120 uses input from the radiation sensors 107 to determine the beginning and end of a radiation cycle. Alternatively, the operator may manually indicate the beginning and end of a radiation cycle or a signal may be sent from the imaging device.

[0032] For a fluoroscopic x-ray image to be useable for navigational guidance, the imaging device 100 must be stationary with respect to the patient 110 during image acquisition. If the C-arm 103 or the patient 110 is moving during image acquisition, the position of the imaging device will not be accurately determined relative to the patient's reference frame. Thus, it is important that the recorded position of imaging device 100 reflects its true position at the time of image acquisition.

[0033] During the image acquisition process, the image processing device 120 continuously examines the relative position between the patient 110 and the imaging device 100 as shown at 404. If the imaging device 100 moves during the image acquisition process, or if the imaging device 100 moves after image acquisition but before its position is recorded, calibration measurements will be erroneous, thereby resulting in incorrect graphical overlays. To prevent this type of erroneous image, image data may be discarded if the patient 110 moves relative to the imaging device 100 during the image acquisition process.

[0034] At the end of the radiation cycle, the image processing device 120 retrieves the acquired image data as well as positional data for the imaging device 100 and the dynamic reference frame 150. The image processing device 120 subsequently calibrates the acquired image to learn its projective transformation and optionally to correct distortion in the image at 406. Calibrated image data along with corresponding positional data is then stored. These processing steps are repeated for each image that is acquired.

[0035] During surgery, the tracking sensor 130 detects the position of surgical instrument 140 relative to the dynamic reference frame 150, and hence relative to the patient 110 at 408. The image processing device 120 dynamically computes, in real-time, the projection of instrument 140 into each fluoroscopic image as the instrument 140 is moved by the physician. A graphical representation of instrument 140 may then be overlaid on the fluoroscopic images at 410. The graphical representation of instrument 140 is an iconic representation of where the actual surgical instrument 140 would appear within the acquired fluoroscopic x-ray image as if the imaging device 100 was continuously acquiring new images from the same view as the original image. There is no theoretical limit to the number of fluoroscopic images on which the graphical representations of instrument may be simultaneously overlaid.

[0036] In accordance with the present invention, the surgical instrument navigation system described above has been enhanced to detect any change in relative position between the patient 110 and the imaging device 100, and compensate image data for any such patient motion. Referring to FIG. 5, the enhanced surgical navigation system 500 includes an imaging device 502 and a tracking subsystem 504. As described above, the imaging device 502 is operable to capture one or more image data sets representative of a patient; whereas the tracking subsystem 504 is operable to determine patient position data 506 indicative of the position of the patient as well as device position data 508 indicative of the position of the imaging device.

[0037] The enhanced surgical instrument navigation system 500 further includes a motion correction subsystem 510 and an image subtraction subsystem 520. The image subtraction subsystem 520 is adapted to receive two image data sets 503 from the imaging device 502. The image subtraction subsystem 520 is then operable to perform a digital subtraction operation between the two image data sets, thereby generating resulting image data 522 indicative of the differences between the two image data sets. Further explanation of exemplary digital subtraction techniques is described in Christensen's Physics of Diagnostic Radiology by Thomas S. Curry, III, James E. Dowdey, and Robert C. Murry, Jr., 4^th Ed. 1990, the contents of which are hereby incorporated by reference. It is readily understood that various well known digital subtraction techniques are within the scope of the present invention. It is further understood that other comparison techniques for assessing image data acquired at different points in time are also within the scope of the present invention. Any sort of image analysis or comparison can be applied to such a technique as segmentation, volume rendering, or shape analysis to a brain structure such as the hippocampus or tumor growth. One such technique for shape analysis is described in U.S. patent application Ser. No. 09/326,657 Method and Apparatus for Automatic Shape Characterization and U.S. Pat. No. 6,226,418 which are specifications are here included.

[0038] Prior to performing a digital subtraction operation, the motion correction subsystem 510 may be used to detect any patient motion that occurs between the times at which the image data is acquired. Referring to FIG. 7, the motion correction subsystem 510 is adapted to receive the patient position data 506 and device position data 508 corresponding to each of the two image data sets 503 as shown at 702. The motion correction subsystem 510 is then operable to detect any change in the relative position between the patient and the imaging device as shown at 704.

[0039] In one embodiment, the detection of patient motion merely serves as a triggering event for the operation of the enhanced surgical navigation system 500. When no change occurs in the relative position of the patient, a digital subtraction operation may simply be performed by the image subtraction subsystem 520. Alternatively, the digital subtraction operation may be performed when the measured change in relative position is less than a predetermined threshold value that is indicative of a maximum acceptable change in the relative position between the patient and the imaging device. On the other hand, when a unacceptable amount of patient motion is detected, the motion correction subsystem 510 may initiate an alternate operation, such as providing an operator alarm.

[0040] In another embodiment, the motion correction subsystem 510 may measure the change in relative position between the patient and the imaging device over time. When the measured change in relative position is less than the predetermined threshold value, the motion correction subsystem 510 initiates the digital subtraction operation at 722. On the other hand, when the measured change in relative position is equal to or greater than the predetermined threshold value, the motion correction subsystem 510 will apply a suitable correction to at least one of the two image data sets.

[0041] Since the suitable correction depends upon the nature of the relative motion, the motion correction subsystem 501 is operable to determine the nature of the relative motion at 708. For instance, when the relative motion (or change in relative position) is a translation that is parallel to the imaging plane 602 of the imaging device, a suitable translation correction would be performed to one of the two images as shown at 712. FIG. 6A illustrates a translation that is parallel to the imaging plane 602 of the imaging device (e.g., the face of the image intensifier of a fluoroscope). When the relative motion is a translation that is perpendicular to the imaging plane 602, a change occurs in the scale or size of the imaged anatomy as shown at 716. In this case, a suitable scale correction would be performed to one of the two images. FIG. 6B illustrates a translation that is perpendicular to the imaging plane 602 of the imaging device (as represented by the top surface of operating table). Suitable image correction techniques are well known in the art.

[0042] When the relative motion is a rotation of the imaging device about the patient's anatomy, the ability to perform a correction depends upon the magnitude and direction of the rotation. FIG. 6C illustrates motion that is rotational to the imaging plane 602 of the imaging device. Rotations of the imaging device relative to the anatomy in a plane parallel to the image plane can be corrected by applying an appropriate in plane rotation to one of the two images. However, in the case of two-dimensional projection imaging, it is impossible to correct for arbitrary rotations without knowing the three-dimensional shape of the underlying anatomy. In some of these cases, the rotational correction may be approximated using known translation techniques. Therefore, it is envisioned that the motion correction subsystem 510 may further include a mechanism for determining when a rotational correction can be performed by a translation and then applying an appropriate rotational translation as shown at 720. It is readily understood that in the case of three-dimensional volumetric images, the limitation on correcting for rotations does not apply, and a complete correction can be applied to the image data.

[0043] Once an appropriate correction has been applied to either of the two image data sets, the motion correction subsystem 510 initiates a digital subtraction operation at 722. It is to be understood that only the relevant steps of the methodology are shown in FIG. 7, but that other software-implemented instructions may be needed to control and manage the overall operation of the subsystem.

[0044] In two-dimensional projection imaging, determining how to apply a correction to an image given a measurement of the relative motion between patient and the imaging device requires an additional piece of information. In particular, the intrinsic calibration parameters of the imaging device must be known to the motion correction subsystem 510.

[0045] In three-dimensional digital subtraction, it is possible to compute volume changes of the contrast media as a function of time. Once the motion correction is applied, this would be computed as the volume of the difference image above a given intensity threshold. This difference volume could be further constrained to lie within a segmented region (e.g., within a vertebral body, outside a vertebral body). Computation of other shape attributes (surface area, moments of inertia, etc.) are also possible.

[0046] It is further envisioned that the motion correction subsystem 510 may be configured to eliminate motion artifacts caused by changes in the position of the tracked surgical instrument 140. Recall that any changes in the imager's field of view will result in an enhanced region of the difference image. Typically, the motion of the surgical instrument 140 is not clinically relevant and, therefore, it would be desirable to eliminate these enhanced regions from any resulting difference image. Using the measured positions of a given instrument 140 relative to the imaging device 100, together with data for the three-dimensional shape of the instrument (e.g., from a CAD model), it is possible to compute the resulting enhanced region in the difference image. Once this region is known, it is then possible to eliminate this instrument-induced motion artifact from the image. It is to be understood that this function can be applied to either two-dimensional projection images or three-dimensional volumetric images.

[0047] While the above description is provided with reference to a surgical navigation system, it is readily understood that the broader aspects of the present invention are generally applicable to medical imaging systems where motion between the patient and the imaging device is suspected.

[0048] Moreover, it is readily understood that the present invention is applicable to different types of medical applications. One medical application of interest is vertebroplasty, in which bone cement is injected into a vertebral body of the spine via a needle that passes through the spinal pedicle. Leakage of this cement into surrounding structures can result in serious complications. Therefore, visualizing the spread of this cement over time is critical so that the clinician can ensure that the cement does not leave the vertebral body. One way to achieve this goal is by using single or multi-planar fluoroscopic imaging to view the anatomical structures as the above-mentioned cement is introduced. Unfortunately, the cement is often poorly visualized in the fluoroscopic images. Digital subtraction radiology provides a mechanism for highlighting the location of the cement by comparing a pre-cement image to one or more images acquired while the cement is being injected. Unfortunately, due to the nature of the clinical procedure, there is a moderate likelihood that the spinal anatomy will move relative to the imaging device as the cement is being injected. By applying the methods and systems of the present invention, it is possible to compensate for this motion, thereby making digital subtraction techniques feasible for this application.

[0049] Another medical application may involve a comparison between vector fields computed in an area to represent brain shifts or other soft tissue deformations. Other possible medical applications may include but are not limited to monitoring the introduction of embolics into vascular aneurysms; introduction of cement into other bony anatomy; bone density analysis; etc. Analysis of bone morphogenic protein dispersion through the bone for spinal or orthopedic applications may be completed by looking at the slightest changes in bone density or other carrier with an image signature to monitor effectiveness or used to indicate or compute new dosages need to be administered. It is readily apparent from some of these examples that the time period between acquired image data may range from a few seconds to days or weeks.

[0050] While the invention has been described in its presently preferred form, it will be understood that the invention is capable of modification without departing from the spirit of the invention as set forth in the appended claims.