Method and apparatus for collection of cardiac geometry based on optical or magnetic tracking

[0001] a. Field of the Invention

[0002] The invention relates to imaging systems for use in the performance of medical diagnostic, therapeutic, mapping and ablative procedures. More particularly, the invention relates to an imaging system including optical or magnetic tracking for generating accurate 2D or 3D time varying images.

[0003] b. Background Art

[0004] A variety of imaging systems can be used to assist a clinician or physician in the performance of various medical diagnostic and therapeutic procedures related to different parts of the human anatomy, such as, the heart. Such imaging systems include, for example, those based on a variety of technologies, such as, wands (e.g. ultrasound), (fluoroscopy (e.g., x-rays), computed tomography (CT), magnetic resonance (MR), and intracardiac echocardiography (ICE). For ultrasound wands, an ultrasound emitter may be disposed in the wand, or in the case of catheters (e.g. ICE (intra-cardiac echo) or TEE (transesophageal echo)), the ultrasound emitter may be disposed in the catheter.

[0005] In the case of a wand, the wand may be placed on the skin of a patient and typically, 2D image slices may be collected. Alternatively, the wand may also contain a rotating sensor, or the wand itself may be rotated on a mechanical rotation device for collecting 3D images. In both cases, images are continuously collected so that CINE (e.g. real-time animated views) images are possible, thus making the 2D and 3D images, 3D and 4D respectively, with the additional dimension being time.

[0006] Based on the data collected by the wand, it is also possible to automatically extract the shape of an internal body structure using automatic segmentation algorithms. By evolving the segmented static models over time, time varying surface models can also be generated.

[0007] A drawback of these methods is that the various images collected by the wand are not registered together. In other words, since the ultrasound wand may be moved or tilted, the images from different points of time are not aligned with each other. For example, if parallel images are desired and the ultrasound wand is tilted as images are being taken, the images essentially will not be parallel. Moreover, since the wand can be manually or even mechanically rotated, there can be various gaps in the disparate images.

[0008] The inventors herein have thus recognized a need for a system and method for precise and automatic correlation of such images, regardless of whether a wand is manually or mechanically rotated or tilted, thus freeing an operator to move a wand with limited constraints. The inventors herein have also recognized a need for a system and method for providing guidance for such imaging, and a system that is capable of extrapolating omitted image data.

[0009] A cardiac imaging system may include an imaging module including an ultrasound emitter and one or more position markers. A tracking and control system may include a position sensing system for sensing a location and orientation of the position marker relative to a predetermined reference. The tracking and control system may be operable with the imaging module to produce at least a three dimensional image of a patient's anatomy.

[0010] For the cardiac imaging system described above, in an embodiment, the imaging module may be a hand-held module disposable against the patient's skin. Alternatively, in an embodiment, the imaging module may be operatively connected to a catheter. In an embodiment, the imaging module may be a two dimensional imaging module, with the third dimension of the three dimensional image produced by the tracking and control system being time. The imaging module, in an embodiment, may be a three dimensional imaging module, with the tracking and control system producing a four dimensional image, and the fourth dimension being time. In an embodiment, the three dimensional image may be a CINE image. The position marker, in an embodiment, may emit an infrared signal sensed by the position sensing system.

[0011] For the cardiac imaging system described above, in an embodiment, the position marker may include one or more optical fiducial markers. In an embodiment, the optical fiducial marker(s) may be a spherical marker that reflects infrared light emitted by illuminators on the position sensing system. In an embodiment, the cardiac imaging system may include a plurality of the optical fiducial markers disposed in a predetermined arrangement. The predetermined arrangement, in an embodiment, may be a generally triangular pyramid-shaped arrangement. The predetermined arrangement, in an embodiment, may be a generally symmetrical arrangement. In an embodiment, the cardiac imaging system may include one or more further optical fiducial markers disposed against a moving part of the patient's anatomy being imaged, with the further optical fiducial marker(s) serving as a moving reference for the position sensing system.

[0012] For the cardiac imaging system described above, in an embodiment, the position marker may include one or more magnetic markers. In an embodiment, the magnetic marker(s) may be operably connected to or disposed inside the imaging module. In an embodiment, the cardiac imaging system may further include a magnetic emitter operably connected to the position sensing system, with the magnetic emitter emitting a magnetic field for sensing a location and orientation of the magnetic marker.

[0013] For the cardiac imaging system described above, in an embodiment, the imaging system may be operably connected to a robotic catheter system for automatic cardiac imaging. In an embodiment, the imaging system may include a visualization sub-system for displaying the image of the patient's anatomy. The imaging system, in an embodiment, may be operably connected to a navigation and mapping sub-system. In an embodiment, the imaging system may be used to track and/or navigate an interventional device. In an embodiment, the interventional device may be a catheter.

[0014] For the cardiac imaging system described above, in an embodiment, the tracking and control system may produce the three dimensional image by processing a plurality of images obtained by the imaging module using automatic segmentation algorithms. In an embodiment, the tracking and control system may automatically align an image obtained by the imaging module relative to the predetermined reference. The tracking and control system, in an embodiment, may automatically align a plurality of images obtained by the imaging module relative to the predetermined reference to generate the three dimensional image. In an embodiment, the tracking and control system may automatically align a plurality of images obtained by the imaging module relative to the predetermined reference to generate a three dimensional time-varying image, with the tracking and control system further ascertaining gaps in the three dimensional time-varying image. In an embodiment, the tracking and control system may automatically complete any ascertained gaps. The tracking and control system, in an embodiment, may prompt a user to obtain further images for completing any ascertained gaps, for example, when aggregating a plurality of 2D images into a 3D image.

[0015] In an embodiment, a method of cardiac imaging may include operatively attaching one or more position markers to an imaging module including an ultrasound emitter, and sensing a location and orientation of the position marker(s) relative to a predetermined reference. The method may further include producing at least a three dimensional image of a patient's anatomy by evaluating one or more image(s) obtained by the imaging module and aligning the image(s) obtained by the imaging module relative to the location and orientation of the position marker.

[0016] For the method described above, the method may further include disposing the imaging module against the patient's skin to produce the image obtained by the imaging module. In an embodiment, the method may further include operatively connecting the imaging module to a catheter. The imaging module, in an embodiment, may be a two dimensional imaging module, with the third dimension of the three dimensional image being time. In an embodiment, the imaging module may be a three dimensional imaging module, with the method further including producing a four dimensional image, and the fourth dimension being time. In an embodiment, the three dimensional image may be a CINE image. The position marker, in an embodiment, may emit an infrared signal.

[0017] In an embodiment, the position marker may include one or more optical fiducial markers. The optical fiducial marker(s), in an embodiment, may be a spherical marker that reflects infrared light emitted by illuminators on a position sensing system. In an embodiment, the method may further include providing a plurality of the optical fiducial markers in a predetermined arrangement. The predetermined arrangement, in an embodiment, may be a generally triangular pyramid-shaped arrangement. In an embodiment, the predetermined arrangement may be a generally symmetrical arrangement. The method, in an embodiment, may further include disposing one or more further optical fiducial markers against a moving part of the patient's anatomy being imaged, with the further optical fiducial marker(s) serving as a moving reference.

[0018] In an embodiment, the method may further include providing a position marker including one or more magnetic markers. The magnetic marker(s) may be operably connected to or disposed inside the imaging module. In an embodiment, the method may further include providing a magnetic emitter for emitting a magnetic field for sensing the location and orientation of the magnetic marker. The method may, in an embodiment, further include using the three dimensional image with a robotic catheter system. In an embodiment, the method may include displaying the image of the patient's anatomy. The method, in an embodiment, may include navigation and mapping.

[0019] In an embodiment, the method may further include tracking and navigating an interventional device. In an embodiment, the interventional device may be a catheter. In an embodiment, the method may further include producing the three dimensional image by processing a plurality of images obtained by the imaging module using automatic segmentation algorithms. The method, in an embodiment, may further include automatically aligning the image obtained by the imaging module relative to the predetermined reference. In an embodiment, the method may include automatically aligning a plurality of images obtained by the imaging module relative to the predetermined reference to generate the three dimensional image. In an embodiment, the method may include automatically aligning a plurality of images obtained by the imaging module relative to the predetermined reference to generate a three dimensional time-varying image, and further ascertaining gaps in the three dimensional time-varying image. The method, in an embodiment, may further include automatically completing any ascertained gaps. In an embodiment, the method may further include automatically prompting a user to obtain further images for completing any ascertained gaps.

[0020] The foregoing and other aspects, features, details, utilities, and advantages of the invention will be apparent from reading the following description and claims, and from reviewing the accompanying drawings.

[0029] Referring now to the drawings wherein like reference numerals are used to identify identical components in the various views, FIG. 1 illustrates an optical-based imaging system 10 in accordance with the invention. Imaging system 10 may be configured to image desired internal anatomic structures of a patient, may be used to track and/or navigate interventional devices, and may be usable in combination with a robotic catheter system (e.g. disclosed in commonly owned and copending applications titled “Robotic Catheter System,” “Robotic Catheter Manipulator Assembly,” “Robotic Catheter Device Cartridge,” “Robotic Catheter Rotatable Device Cartridge,” “Robotic Catheter Input Device,” “Robotic Catheter System Including Haptic Feedback,” and “Robotic Catheter System with Dynamic Response,” the respective disclosures of which are incorporated herein by reference in their entirety). Accordingly, imaging system 10 may also be considered an “imaging and navigation” system rather than simply an “imaging” system.

[0030] As shown in FIG. 1 and described in detail below, optical-based imaging system 10 may generally incorporate an optical-based imaging module 12 (including an ultrasound emitter (not shown)), a tracking and control system 14, and a visualization system 16 that provides a user with real-time or near-real-time positioning information, partial or complete images generated by system 10, and guidance for imaging module 12. Other systems, such as an EnSite® NavX system (not shown) may be incorporated with imaging system 10 for providing closed-loop feedback and further visualization, navigation, and mapping. Control system 14 incorporates ultrasound imaging and optical based tracking.

[0031] As discussed herein, the ultrasound emitter may alternatively be disposed in a catheter (e.g. ICE (intra-cardiac echo) or TEE (transesophageal echo)), with both embodiments using the imaging module and catheter usable with the aforementioned robotic catheter system. A plurality of strategically disposed optical fiducial markers (e.g. spherical, retro-reflective markers that reflect infrared light emitted by illuminators on a position sensing system (not shown)) 18, 20, 22 and 24 may be symmetrically placed on imaging module 12 for allowing the position and orientation of module 12 to be determined in 6D (e.g. x, y, z and rotational along each axis) by tracking and control system 14. In an exemplary embodiment, tracking and control system 14 may employ an optical tracking sub-system such as the Polaris Spectra and Polaris Vicra optical tracking systems available from Northern Digital, or another triangulation based optical tracking sub-system. In the embodiment illustrated, optical fiducial markers 18, 20, 22 and 24 may be disposed in a three-dimensional pyramid shape with equal triangles as shown in FIG. 2, and attached to a framed or another structure (not shown) to imaging module 12. A series of three or more cameras 48 may be generally positioned as shown for tracking the movement (e.g. position and orientation) of markers 18, 20, 22 and 24, which are disposed, for example, 10 cm apart.

[0032] Generally, the optical tracking sub-system of tracking and control system 14 may provide for location and orientation measurement of imaging module 12 by means of a position sensing system operatively provided with cameras 48 and incorporated with tracking and control system 14. The position sensing system may detect the position and orientation of optical fiducial markers 18, 20, 22 and 24 within a defined coordinate system in the vicinity of imaging system 10, and calculate the position and orientation of imaging module 12 based on the information received from the markers. In an embodiment, instead of optical fiducial markers 18, 20, 22 and 24, alternatively, infrared-emitting markers may also be used.

[0033] Referring to FIGS. 1-3, with the position and orientation of imaging module 12 determinable, a user (such as an EP) may place imaging module 12 against the skin of a patient as shown in FIGS. 1 and 2, and then prompt tracking and control system 14 to begin obtaining images at location 26 (see flow-chart of FIG. 3) of the patient's anatomy while tracking movement of module 12. Imaging module 12 may then obtain 2D or 3D image slices at location 28, which may be converted to CINE 3D and 4D images at location 30. The image, time and position (e.g. position and orientation) data may then be processed at location 32 by tracking and control system 14 to automatically extract the shape of the patient's anatomy using, for example, automatic segmentation algorithms, such as, active template/shape models, snakes, level sets and fast marching methods. Exemplary applications of such techniques are disclosed in commonly owned and copending U.S. application Ser. No. 11/967,412 titled “Method and Apparatus for Encoding Interventional Devices,” and U.S. application Ser. No. 11/967,788, titled “Method and Apparatus for Real-Time Hemodynamic Monitoring,” the respective disclosures of which are incorporated herein by reference. Based on the predefined coordinate system set as a reference for tracking and control system 14, system 14 may register (e.g. align) the various 2D or 3D images at location 34 obtained by imaging module 14 relative to the coordinate system. Tracking and control system 14 may also process the data to evolve the segmented static models over time, to thus generate accurate, for example, 1 mm or less location accuracy, time varying surface models at location 36.

[0034] Still referring to FIGS. 1-3, with the images generated by imaging module 12 registered as discussed above, tracking and control system 14 may then prompt the user at location 38 to obtain additional images on any omitted areas to thus generate a complete model of the patient's anatomy. Further, tracking and control system 14 may ascertain at location 40, based on predetermined parameters, any gaps between corresponding images to determine whether additional images are needed or if the gaps may be completed by extrapolation. If no gaps are present, at locations 26 and 40, the user may then re-prompt tracking and control system 14 to obtain other images on the same or a different patient. In the latter case if gaps are present, tracking and control system 14 may extrapolate at location 42 any gaps in the image data to provide at location 44 a complete model of the patient's anatomy. At locations 26 and 44, the user may then re-prompt tracking and control system 14 to obtain other images on the same or a different patient. Tracking and control system 14 may also guide the user by suggesting increased or decreased tilting of imaging module 12 in order to obtain any missing data, and further alert the user when the data has been successfully captured. Thus by ascertaining the 6D position of imaging module 12, the position of the collected image in Cartesian coordinates may be determined through the assumption that the image is rigidly related to module 12.

[0035] Referring to FIGS. 4-6, an embodiment of a magnetic-based imaging system 50 incorporating a magnetic-based imaging module 52 will be described in detail.

[0036] Referring to FIGS. 4 and 5, imaging module 52, similar to imaging module 12, may generally include an ultrasound emitter (not shown). As discussed above, similar to imaging module 12, the ultrasound emitter may alternatively be disposed in a catheter (e.g. ICE (intra-cardiac echo) or TEE (transesophageal echo)), with both embodiments using the imaging module and catheter usable with the aforementioned robotic catheter system. A 6D (e.g. x, y, z and rotational along each axis) magnetic sensor 58 (see FIG. 5) may be attached to or disposed inside the frame of imaging module 52 for allowing the position and orientation of module 52 to be determined by tracking and control system 54, with the subsequently generated images being displayed on visualization system 56. Magnetic sensor 58 may generally include at least two magnetic coils that are disposed transverse to each other. An emitter 60 may be disposed adjacent magnetic sensor 58 as shown in FIG. 5 for emitting magnetic fields and thereby sensing the position and orientation of imaging module 52. In a particular embodiment, tracking and control system 54 may employ a magnetic tracking sub-system such as the Aurora magnetic tracking system available from Northern Digital, or other similar devices available from Ascension Technology Corporation.

[0037] Generally, the magnetic tracking sub-system of tracking and control system 54 may provide for location and orientation measurement of imaging module 52 by means of magnetic sensor 58 and a position sensing system (not shown) incorporated with emitter 60 of tracking and control system 54. The position sensing system may detect the position and orientation of magnetic sensor 58 within a defined coordinate system in the vicinity of imaging system 50, and calculate the position and orientation of imaging module 52 based on the information received from the sensor.

[0038] Referring to FIGS. 4-6, with the position and orientation of imaging module 52 determinable, a user (such as an EP) may place imaging module 52 against the skin of a patient as shown in FIGS. 4 and 5, and then prompt tracking and control system 54 to begin obtaining images at location 66 (see flow-chart of FIG. 6) of the patient's anatomy while tracking movement of module 52. Imaging module 52 may then obtain 2D or 3D image slices at location 68, which may be converted to CINE 3D and 4D images at location 70. The image, time and position (e.g. position and orientation) data may then be processed at location 72 by tracking and control system 54 to automatically extract the shape of the patient's anatomy using, for example, automatic segmentation algorithms, in a similar manner as that for imaging system 10. Based on the predefined coordinate system set as a reference for tracking and control system 54, system 54 may register (e.g. align) the various 2D or 3D images at location 74 obtained by imaging module 52 relative to the coordinate system. Tracking and control system 54 may also process the data to evolve the segmented static models over time, to thus generate an accurate, for example, 1 mm or less location accuracy, time varying surface model at location 76.

[0039] Still referring to FIGS. 4-6, with the images generated by imaging module 52 registered as discussed above, tracking and control system 54 may then prompt the user at location 78 to obtain additional images on any omitted areas to thus generate a complete model of the patient's anatomy. Further, tracking and control system 54 may ascertain at location 80, based on predetermined parameters, any gaps between corresponding images to determine whether additional images are needed or if the gaps may be completed by extrapolation. If no gaps are present, at locations 66 and 80, the user may then re-prompt tracking and control system 54 to obtain other images on the same or a different patient. In the latter case if gaps are present, tracking and control system 54 may extrapolate at location 82 any gaps in the image data to provide at location 84 a complete model of the patient's anatomy. At locations 66 and 84, the user may then re-prompt tracking and control system 54 to obtain other images on the same or a different patient. Thus by knowing the 6D position of imaging module 52, the position of the collected image in Cartesian coordinates may be determined through the assumption that the image is rigidly related to module 52.

[0040] In another embodiment, whereas for the embodiments of FIGS. 1-6 may utilize a distant reference point (e.g. adjacent a patient's bed) for generating the 3D or 4D images, in the case of a beating heart for example, a patient's anatomy may constantly change relative to time. Referring to FIG. 1, a secondary reference sensor 46 may be placed, for example, adjacent a patient's heart area or internally (likewise for FIG. 4). By using this secondary reference sensor, any generated images may be aligned relative to each other and relative to moving sensor 46. This would allow the origin of the reference point to be constantly updated to ensure accurate imaging.

[0041] Imaging systems 10, 50 are beneficial in that based on the tracking by respective tracking and control systems 14, 54, 2D imaging modules 12, 52 (instead of optional 3D imaging modules) may be used to collect 3D data by manual rotation of the modules about an axis normal to the patient. Because of the tracking provided by systems 14, 54, imaging modules 12, 52 may be maneuvered (rotated or translated) in a predetermined pattern, without specific focus on the positional/orientation control of the modules, thus potentially negating the need for a precision mechanical rotation/movement device. For example, imaging modules 12, 52 may be positioned adjacent a patient's heart or other anatomy and moved virtually randomly in that area to obtain a series of images by tracking and control systems 14, 54 to generate a 3D volumetric image of the patient's anatomy. However if a predetermined area of a patient's anatomy is of importance, imaging modules 12, 52 may be moved in a predetermined pattern in those areas to generate a 3D volumetric image without unnecessary computations due to random movement of the modules.

[0042] With regard to 4D volumetric images, instead of a 3D volume, tracking and control systems 14, 54 may capture a plurality of images with respect to time and in the case of a heart for example, systems 14, 54 may place the images in the correct phase to create separate 4D volumetric images.

[0043] Referring to FIG. 7, images 62 collected via imaging module 12 (or 52) may be displayed on visualization system 16, and any gaps present in the images may be displayed as shown in FIG. 8. As discussed above, when a user is directed to move imaging module 12 (or 52) to fill any voids, the graphical display of the images and any other indicators, such as arrows etc., would facilitate directing a user to move imaging module 12 (or 52) in a certain direction/manner to efficiently fill the voids. Further, whereas the images of FIGS. 7 and 8 are shown as an orthogonal stack, under typical operation, the images would not be perfectly aligned due to manual operation of imaging module. Thus the images may be displayed in a stack or a fan pattern (with the fan pattern being generated by rotation of an imaging module around an axis orthogonal to a patient's skin, or by tilting of the imaging module along an axis parallel to the patient's skin).

[0044] As briefly discussed above, systems such as the EnSite® NavX system (commercially offered by St. Jude Medical, Inc., which is described in detail in commonly owned U.S. Pat. No. 7,263,397 entitled “Method and Apparatus for Catheter Navigation and Location and Mapping in the Heart,” issued to Hauck et al., which is hereby incorporated by reference in its entirety) or the Biosense Webster Carto™ System, may be incorporated with imaging systems 10, 50 for providing closed-loop feedback. For example, the volumetric images produced by imaging systems 10, 50 may be utilized in conjunction with the EnSite® NavX visualization, navigation, and mapping system that may be a part of imaging systems 10, 50 or a separate and distinct system that works in conjunction with imaging systems 10, 50. For imaging systems 10, 50, the captured ultrasound data may be used in conjunction with the EnSite® Fusion system (commercially offered by St. Jude Medical, Inc.) if a cardiac geometry model is generated from the ultrasound data. Alternatively, the actual image data may be superimposed on the EnSite® Fusion model as either a 2D cut through plane or as a volume rendered 3D image. If CINE image data is used, the 2D cut through plane and volume rendered 3D image would be 3D and 4D respectively.

[0045] Although several embodiments of this invention have been described above with a certain degree of particularity, those skilled in the art could make numerous alterations to the disclosed embodiments without departing from the scope of this invention. All directional references (e.g., upper, lower, upward, downward, left, right, leftward, rightward, top, bottom, above, below, vertical, horizontal, clockwise and counterclockwise) are only used for identification purposes to aid the reader's understanding of the present invention, and do not create limitations, particularly as to the position, orientation, or use of the invention. Joinder references (e.g., attached, coupled, connected, and the like) are to be construed broadly and may include intermediate members between a connection of elements and relative movement between elements. As such, joinder references do not necessarily infer that two elements are directly connected and in fixed relation to each other. It is intended that all matter contained in the above description or shown in the accompanying drawings shall be interpreted as illustrative only and not as limiting. Changes in detail or structure may be made without departing from the invention as defined in the appended claims.