US 8,831,310 B2Grant
Systems and methods for displaying guidance data based on updated deformable imaging data
Issue Date:2014-09-09
•20 Claims
•9 Drawing Sheets
Abstract
Presented herein are methods, systems, and computer-readable medium for presenting imaging data related to an anatomical site. These include obtaining a first set of imaging data related to the anatomical site and tracking units at the anatomical site and, thereafter, optionally, obtaining a second set of imaging data related to the anatomical site. A deformed version of the first set of imaging data is then determined based on the relative arrangements of one or more of the tracking units at the time when the first set of imaging data is obtained and when the second set of imaging data is obtained. Then the relative emplacements of the second set of imaging data set and of the deformed version of the first set of imaging data set are determined and used, along with the second set of imaging data set and the deformed version of the first set of imaging data, as a basis for displaying image guidance data.
Metadata
Assignee
- InnerOptic Technology, Inc.
Inventors
- Sharif Razzaque
- Andrei State
Application Information
Application Number:US 13/723,705
Filing Date:2012-12-21
Priority Date:2008-03-07
Art Unit:2878
Classifications
IPC:
G06K9/00G06T7/00A61B19/00G06K9/32
Patent Drawings (9 sheets)
Description
Cross-Reference to Related Application
[0001] This application is a divisional of U.S. application Ser. No. 12/399,899 entitled SYSTEMS AND METHODS FOR DISPLAYING GUIDANCE DATA BASED ON UPDATED DEFORMABLE IMAGING DATA, filed Mar. 6, 2009, which claims the benefit of U.S. Provisional Application No. 61/068,469, filed Mar. 7, 2008, each of which is incorporated herein by reference in their entirety.
Background
[0002] Surgeons often need to be able to look at both pre-operative data, such as computed tomography (“CT”) scans and magnetic resonance imaging (“MRI”) scans, as well as intra operative data, such as two dimensional (“2D”) ultrasound or three dimensional (“3D”) ultrasound while they are in the operating room. Normally, doctors view the CT scans and ultrasound on separate displays and must use their imaginations in order to correlate the information in the two images. This is a difficult spatial task for the surgeons to accomplish. Further, when a target anatomical site is located within soft tissue, the pre-operative data is out of date with respect its pre-operative form because of the movement, compression and reorientation of the soft tissue and, therefore, it is difficult or impossible for the surgeon to appropriately utilize the pre-operative data during the operation.
[0003] Previous systems have attempted to aid the surgeon using computer vision registration techniques. Example systems are described in, among other papers, Aylward et al., Analysis of the Parameter Space of a Metric for Registering 3D Vascular Images, in W. Niessen and M. Viergever (Eds.), MEDICAL IMAGE COMPUTING AND COMPUTER -ASSISTED INTERVENTION —MICCAI 2001, pp. 932-939; and Aylward et al, Intra-Operative 3D Ultrasound Augmentation, Proceedings of the IEEE International Symposium on Biomedical Imaging, Washington, D.C., July 2002. The problem with these systems however is the massive computational strain required by the registration techniques.
Summary
[0004] Presented herein are methods, systems, and computer-readable medium for presenting imaging data related to an anatomical site. These include obtaining a first set of imaging data related to the anatomical site and tracking units at the anatomical site and, thereafter, optionally, obtaining a second set of imaging data related to the anatomical site. A deformed version of the first set of imaging data is then determined based on the relative arrangements of one or more of the tracking units at the time when the first set of imaging data is obtained and when the second set of imaging data is obtained. Then the relative emplacements of the second set of imaging data and the deformed version of the first set of imaging data are determined and used, along with the second set of imaging data and the deformed version of the first set of imaging data, as a basis for displaying image guidance data.
[0005] Presented herein are methods, systems, and computer-readable medium for presenting imaging data related to an anatomical site, that include obtaining, at a first time, a first set of imaging data related to the anatomical site. Thereafter, tracking information for a movable imaging device controlled by a user is obtained at a second time, after the first time. Then desired emplacement information is determined for an image of the first set of imaging data based on the tracking information. Finally, image guidance data is determined for display based on the first set of imaging data and the desired emplacement information.
Brief Description of the Drawings
[0006] FIG. 1 depicts one embodiment of a system capable of updating deformable imaging data.
[0007] FIG. 2 illustrates an example of an anatomical site with tracking units.
[0008] FIG. 3 illustrates the deformation of an anatomical site with tracking units.
[0009] FIG. 4 depicts an embodiment of a tracking unit.
[0010] FIG. 5 illustrates another example of an anatomical site within deformable tissue with tracking units.
[0011] FIG. 6 depicts an example process for providing guidance data based on updated deformable tracking information.
[0012] FIGS. 7A-7D depict marking and viewing features in imaging data.
[0013] FIG. 8 depicts an embodiment of image guidance data in which first set of imaging data is presented with second set of imaging data.
Detailed Description
I. Overview
[0014] FIG. 1 depicts merely one exemplary embodiment of a system 100 capable of updating deformable imaging data. There are numerous other possible embodiments of system 100, for example, numerous of the depicted modules may be joined together to form a single module and may even be implemented in a single computer or machine. Further, the position sensing units 110 and 140 may be combined and track all relevant tracking units 145 and movable imaging units 155, as discussed in more detail below. Additionally, imaging unit 150 may be excluded and only imaging data from the image guidance unit 130 may be shown on display unit 120. These and other possible embodiments are discussed in more detail below. Numerous other embodiments will be apparent to those skilled in the art and are covered by the invention as claimed.
[0015] In the pictured embodiment, the system 100 comprises a first position sensing unit 110, a display unit 120, and the second position sensing unit 140 all coupled to an image guidance unit 130. In some embodiments, the first position sensing unit 110, the displaying unit 120, the second position sensing unit 140, and the image guidance unit 130 are all physically connected to stand 170. The image guidance unit 130 may be used to produce images 125 that are presented on display unit 120. As discussed more below, the images 125 shown on the display unit 120 by the image guidance unit 130 may be determined based on imaging data, such as a CT scan, MRI, open-magnet MRI, optical coherence tomography, positron emission tomography (“PET”) scans, fluoroscopy, ultrasound, or other preoperative or intraoperative anatomical imaging data and any 3D anatomical imaging data. The images 125 produced may also be based on intraoperative or real-time data obtained using a movable imaging unit 155, which is coupled to imaging unit 150. Real-time may imply instantaneous or near-instantaneous obtaining of data. Real-time may also imply that it is taken with the intention to be used immediately. Imaging unit 150 may be coupled to image guidance unit 130. In some embodiments, imaging unit 150 may be coupled to a second display unit 151. The second display unit 151 may present imaging data from imaging unit 150. The imaging data displayed on display unit 120 and displayed on second display unit 151 are not necessarily the same. In some embodiments, the imaging unit 150 is an ultrasound machine 150, the movable imaging device 155 is an ultrasound transducer 155 or ultrasound probe 155, and the second display unit 151 is a display associated with the ultrasound machine 150 that shows the imaging data from the ultrasound machine.
[0016] The second position sensing unit 140 is coupled to one or more tracking units 145. The second position sensing unit 140 and tracking units 145 may together comprise a magnetic tracking system, an optical tracking system, or any other appropriate tracking system. The second position sensing unit 140 and tracking units 145 may be used to track the deformation of tissue at a target anatomical site on user 160. User 160 may be in an operating room, lying on an operating table, such as operating table 180, or in any other appropriate place or position. In various embodiments, second position sensing unit 140 may be an Ascension Flock of Birds, Nest of Birds, driveBAY, medSAFE, trakSTAR, miniBIRD, MotionSTAR, or pciBIRD, and tracking units 145 may be magnetic tracking coils. In some embodiments, the second position sensing unit 140 may be an Aurora® Electromagnetic Measurement System using sensor coils for tracking units 145. In some embodiments, the first position sensing unit 110 may also be an optical 3D tracking system using fiducials as tracking units 145. Such optical 3D tracking systems may include the NDI Polaris Spectra, Vicra, Certus, PhaseSpace IMPULSE, Vicon MX, InterSense IS-900, NaturalPoint OptiTrack, Polhemus FastTrak, IsoTrak, or Claron MicronTracker2.
[0017] Tracking unit 145 as used herein is a broad term and includes without limitation all types of magnetic coils or other magnetic field sensing devices for use with magnetic trackers, fiducials or other optically detectable markers for use with optical trackers, such as those discussed above and below. Tracking units 145 could also include optical position sensing devices such as the HiBall tracking system and the first and second position sensing units 110 and 140 may be HiBall tracking systems. Tracking units 145 may also include a GPS device or signal-emitting device that would allow for tracking of the position and, optionally, orientation of the tracking unit. In some embodiments, a signal-emitting device might include a radio-frequency identifier (RFID). In such embodiments, the first and/or second position sensing unit 110 and 140 may take in the GPS coordinates of the tracking units 145 or may, for example, triangulate the radio frequency signal being emitted by the RFID associated with tracking units 145.
[0018] The first position sensing unit 110 may be used to track the position of movable imaging unit 155. Tracking the position of movable imaging unit 155 allows for the determination of the relative emplacement, where emplacement may refer to position and orientation or merely position, of imaging data received using the movable imaging unit 155 and imaging unit 150 with that data being sent to image guidance unit 130. For example, image guidance unit 130 may contain CT data which is being updated and deformed based on the relative emplacements of tracking units 145 as received by the second position sensing unit 140. In such an example embodiment, the image guidance unit 130 may take in the emplacements, such as positions and orientations, of the tracking units 145 and from that determine an updated model for CT data stored in imaging guidance unit 130. Further, imaging guidance unit 130 may produce images based on the current ultrasound imaging data coming from imaging unit 150 and also based on an updated model determined based on the emplacements of tracking units 145. The images produced 125 made be presented on display unit 120. An example image 125 is shown in FIG. 8 .
[0019] In some embodiments, a movable imaging unit 155 may not be connected directly to an imagining unit 150, but may instead be connected to imaging guidance unit 130. The movable imaging unit 155 may be useful for allowing a user to indicate what portions of a first set of imaging data should be displayed. For example, if the movable imaging unit 155 may be an ultrasound transducer or a tracked operative needle, for example, and may be used by a user to indicate what portions of a pre-operative CT scan to show on a display unit 120 as image 125. Further, in some embodiments, there could be a third set of pre-operative imaging data that could be displayed with the first set of imaging data. Yet further, in some embodiments, each of the first and third sets of imaging data could be deformed based on updated positions of the tracking units 145 and the updated, deformed versions of the two sets of imaging data could be shown together or otherwise provide image guidance images 125 for presentation on display 120.
[0020] First position sensing unit 110 may be an optical tracker, a magnetic tracker, or any other appropriate type of position sensing device. For example, in various embodiments, first position sensing unit 110 may be an Ascension Flock of Birds, Nest of Birds, driveBAY, medSAFE, trakSTAR, miniBIRD, MotionSTAR, or pciBIRD. In some embodiments, the first position sensing unit may be an Aurora® Electromagnetic Measurement System using sensor coils. In some embodiments, the first position sensing unit 110 may also be an optical 3D tracking system such as the NDI Polaris Spectra, Vicra, Certus, PhaseSpace IMPULSE, Vicon MX, InterSense IS-900, NaturalPoint OptiTrack, Polhemus FastTrak, IsoTrak, or Claron MicronTracker2. The first position sensing unit 110 senses the position of movable imaging unit 155. If first position sensing unit 110 is an optical tracker, then movable imaging unit 155 may have fiducials placed thereon to make visual position and/or orientation detection possible. If first position sensing unit 110 is a magnetic tracker, then movable imaging unit 155 they have placed thereon magnetic tracking units.
[0021] In some embodiments, the display unit 120 displays 3D images to a user. This can be accomplished by a stereoscopic display, a lenticular auto-stereoscopic display, or any other appropriate type of display. In some embodiments, a user may wear a head mounted display in order to receive 3D images from the image guidance unit 130. In such embodiments, a separate display, such as the pictured display unit 120, may be omitted.
[0022] In some undepicted embodiments, there is no first position sensing unit 110 and the emplacements of both the movable imaging unit 155 and tracking units 145 are determined using the second position sensing unit 140. Similarly, in some embodiments, the first position sensing unit 110 may track the emplacements of both the movable imaging unit 155 and tracking units 145 and the second position sensing unit 140 may not be present.
II. Anatomical Sites and Deformation
[0023] FIG. 2 illustrates an example of an anatomical site 210 with tracking units 145A, 145B, and 145C. An anatomical site 210 can be anywhere within or on the body 160, human or otherwise. In some embodiments, as shown in FIG. 4 , tracking units 145A, 145B, and 145C may be implantable needles containing a magnetic tracking coil. If the tracking units 145A, 145B, and 145C are implantable, then the tracking units may be placed in or near an anatomical site 210. In some embodiments, tracking units 145A, 145B, and 145C may be placed on the surface of an anatomical site 210. In yet other embodiments, tracking units 145A, 145B, and 145C of known dimensions may be placed partially inside and partially external to an anatomical site 210. In such embodiments, a portion of the tracking unit 145A, 145B, and 145C may be in or near the anatomical site 210 while another portion may be external to the body or the anatomical site and allow tracking external to the body. This embodiment is useful, for example, if it is desired that the second position sensing unit 140 be an optical tracker or if there are other reasons, such as the size of a magnetic tracking coil, for not implanting that portion of the tracking units 145A, 145B, and 145C.
[0024] FIG. 3 illustrates the deformation of an anatomical site 210 to 310 with tracking units 145A, 145B, and 145C. In FIG. 3 , three tracking units 145A, 145B, and 145C have been placed near an anatomical site 210. The letters on the vertices of anatomical site 210 illustrate that anatomical site 210 may be deformed into deformed anatomical site 310. The letters illustrate which vertices in anatomical site 210 correspond to which vertices in deformed anatomical site 310. The deformation of anatomical site 210 into anatomical site 310 can be due to patient movement, breathing, pressure, force, or any other effect that may deform deformable tissue comprising and/or surrounding anatomical sites 210 and 310. FIG. 3 illustrates that the tracking units 145A, 145B, and 145C move from locations on anatomical sites 210 to corresponding locations in deformed anatomical site 310.
[0025] In some embodiments, the first set of imaging data, such as a CT scan, MRI, open-magnet MRI, fluoroscopy, PET scan, 3D ultrasound, or any other type of imaging data may be received in a format that is usable to perform the deformation techniques described herein. In other embodiments, when the first set of imaging data is received, a 3D model of that data may be produced. There are many known techniques for producing 3D models such as finite element models, volumetric models, or polygonal models. These include manual, human-driven techniques, such as tracing the boundaries of organs and tumors (also known as contouring), and automatic techniques such as iso-surface extraction (marching cubes, watershed), or hybrid techniques such as m-rep based segmentation. When the updated emplacement, such as position and orientation, or mere position, of the tracking units 145A, 145B, and 145C is determined, a model of the anatomical sites 210 can be updated to estimate the deformed anatomical site 310. This updating may be accomplished using known techniques for the various underlying models. For example, in some computer graphics hardware systems, one can use 3D textures. Each tracking unit can be associated with a texture location within the 3D texture, where the 3D texture comprises the first set of imaging data. Once updated positions of the tracking units 145A, 145B, and 145C are known, then the 3D texture can be updated using linear interpolation. Image guidance unit 130 may contain general hardware, such as a CPU, or specialized hardware, such as a graphics card, that is capable of performing linear interpolation on 3D textures. Relatedly, if a particular slice of the second set of imaging data is desired for display as part of image guidance data, then the particular slice for display may be determined based on the corresponding slice of the updated 3D texture. See, e.g., Yinghui, C., Jing, W., and Xiaohui, L. 2006, Real-time deformation using modal analysis on graphics hardware, in Proceedings of the 4th international Conference on Computer Graphics and interactive Techniques in Australasia and Southeast Asia (Kuala Lumpur, Malaysia, Nov. 29-Dec. 2, 2006). GRAPHITE '06. ACM, New York, N.Y., 173-176. Another example embodiment of linear deformation is discussed below. Other embodiments, bi-cubic or higher-order interpolation may also be used. Additionally, in some embodiments, the tracking units 145A, 145B, and 145C provide position data and not orientation data. Deformation of the 3D model can be accomplished based on the position of the tracking units 145A, 145B, and 145C. In some embodiments, the tracking units 145A, 145B, and 145C will provide both position and orientation data. The additional information on orientation can be used to provide a different kind of deformation of the model.
[0026] In some embodiments, once a new 3D model for deformed anatomical site 310 is determined, the updated model can be used, for example, by image guidance unit 130 of FIG. 1 in order to produce image guidance data that is based on a combination of imaging data from imaging unit 150 and movable imaging unit 155. This image guidance data may be displayed as imaging data 125 on display unit 120.
[0027] FIG. 4 depicts an embodiment of a tracking unit 145. In some embodiments, the tracking unit 145 comprises a shaft 410, a magnetic coil 430, and a cable 420. The shaft 410 may be a hollow needle or any implantable unit. The shaft 410 may be hollow in order to accommodate insertion of the cable 420 and magnetic coil 430, or maybe solid, in which case magnetic coil 430 and cable 420 must be built into the shaft 410 or the shaft 410 must be constructed around the magnetic coil 430 and cable 420. As noted above, in some embodiments a tracking unit 145 may include an optical device, such as a fiducial (not pictured), in order to allow proper tracking. In yet other embodiments, a tracking unit 145 may include an implantable portion in addition to and separate from the tracking portion (not pictured).
[0028] FIG. 5 illustrates another example of an anatomical site 210 within deformable tissue 510 with implanted tracking units 145A, 145B, 145C, and 145D. FIG. 5 illustrates that the deformable tissue 510 deforms into deformable tissue 520. Similar to the embodiments shown in FIG. 3 , anatomical site 210 is deformed into deformed anatomical site 310. Tracking units 145A, 145B, 145C, and 145D are shown implanted near anatomical site 210. The tracking units 145A, 145B, 145C, and 145D remain near the anatomical site after deformable tissue 510 has been deformed into deformed deformable tissue 520, and anatomical site 210 has deformed into deformed anatomical site 310. As was the case in the embodiment shown in FIG. 3 , a 3D model of anatomical site 210 can be deformed and updated based on the relative emplacements of the tracking units 145A, 145B, 145C, and 145D at the time before and after deformation.
III. Process for Providing Guidance Data
[0029] FIG. 6 depicts one of many possible example processes 600 for providing guidance data based on updated deformable tracking information. In some embodiments, all or portions of process 600 may be performed by image guidance system 130 or by any other appropriate unit or module. In step 610, a first set of imaging data from anatomical site 210 and tracking units 145 is obtained. This first set of imaging data may be a CT scan, MRI, open-magnet MRI, fluoroscopy, PET scan, 3D ultrasound, or any other imaging data. Obtaining the relative emplacements of the tracking units 145 when taking the data for the anatomical site 210 provides the ability to determine how the relative emplacements of the tracking units 145 have changed, from the time that the first set of imaging data is taken, and until any time later at which the emplacements of the tracking units are known. Generally, the tracking units 145 will be visible in the first set of imaging data, but this is not necessary. The tracking units 145 must simply be close enough to an anatomical site of interest to provide information on deformation of the anatomical site.
[0030] In some embodiments, once a first set of imaging data of the anatomical site and tracking units 145 is obtained, a 3D model of the first set of imaging data is produced. In other embodiments, a 3D model of the first set of imaging data is produced at a later time or is not produced at all, and deformation of the first set of imaging data is accomplished without using a 3D model. The production of a 3D model from the first set of imaging data is discussed above.
[0031] At some time after the first of imaging data is obtained in step 610, a second set of imaging data of the anatomical site is obtained in step 620. The second set of imaging data may, like the first set of imaging data, be any of a variety of types of imaging data. For example, a second set of imaging data may be 2D ultrasound, 3D ultrasound, fluoroscopy, or any other type of imaging data. The tracking units 145 may, but need not, be visible in the second set of imaging data. For example, if the second set of imaging data is 2D ultrasound, then the ultrasound image obtained may include a plane or slice of the anatomical site 210, but tracking units 145 need not be visible in that particular slice or plane.
[0032] In step 630, a deformed version of the first set of imaging data is determined. In some embodiments, updated emplacements of the tracking units 145, at the time the second set of imaging data is obtained, are used to determine an updated or deformed model of the first set of imaging data. This is discussed above. Once the updated or deformed version of the first set of imaging data and the recently obtained second set of imaging data are both available, the relative emplacements of those two sets of imaging data are determined. This may be accomplished based on both the emplacements of the tracking units 145, in order to determine the emplacement of the deformed version of the first set of imaging data, and the emplacement of the second set of imaging data. The emplacements of the second set of imaging data may be determined based on, for example, the location of a movable imaging unit 155, as depicted in FIG. 1 . As another example embodiment, if a movable imaging unit 155 is a 2D ultrasound wand, then tracking the location of a movable imaging unit 155 allows determination of the position and orientation of the second set of imaging data. As noted above, the movable imaging unit 155 may be tracked using the first position sensing unit 110. In some embodiments, the determination of relative emplacements of the two imaging data sets may take the form of a 3D transformation or other mathematical relationship.
[0033] Once the relative positions of the second set of imaging data and the deformed version of the first set of imaging data are known, the image guidance data can be determined and displayed in step 650. In some embodiments, the image guidance data shows features within the deformed version of the first set of imaging data in combination with a second set of imaging data, such as that depicted in FIG. 8 . In other embodiments, the image guidance data may be an overlay or a combination of the second set of imaging data and of the deformed version of the first set of imaging data. Other examples of image guidance data that may be displayed in step 650 are those depicted in FIGS. 7A-7D , discussed below.
[0034] In some embodiments, each time that more imaging data is received from the data source that provided the second set of imaging data, process 600 may repeat starting at step 620. Process 600 may also be restarted from step 610, especially in scenarios, such as resection at the anatomical site, where warping of the first set of imaging data is no longer possible. In such a case, the first set of imaging data may be re-obtained in step 610.
[0035] As noted above, in some embodiments, the deformation is accomplished by linear deformation. Linear deformation may be accomplished in number of ways, including using graphics hardware. As one example embodiment of linear deformation, consider an original volume image I (as scanned, for example, by the CT scanner at time t) as an anatomical site of interest. At time t, there are n tracking units implanted in the tissue near and around the anatomical site. The tracking units' positions are pt1 . . . ptn. For each tracking unit, we compute the 3d texture coordinate, tc, that indicates the position of the tracking sensor, in image I's coordinate system. 3d texture coordinates may lie in the range u=[0 . . . 1], v=[0 . . . 1], w=[0 . . . 1]. The eight corners of the image I in the 3d texture's coordinate system may be (0,0,0), (0,0,1), (0,1,0), (0,1,1), (1,1,0), (1,1,1). The points may then be stored in a table-like data structure as follows:
[0036]
where ptk.y and tck.u refer, for example, to point ptk's y coordinate and tck's u coordinate, respectively.
| Points table: | |
| point #1, pt1.x, pt1.y, pt1.z, tc1.u, tc1.v, tc1.w, | |
| point #2, pt2.x, pt2.y, pt2.z, tc2.u, tc2.v, tc2.w, | |
| point #3, pt3.x, pt3.y, pt3.z, tc3.u, tc3.v, tc3.w, | |
| . . . | |
| point #n, ptn.x, ptn.y, Ptn.z, tcn.u, tcn.v, tcn.w, | |
where ptk.y and tck.u refer, for example, to point ptk's y coordinate and tck's u coordinate, respectively.
[0037] The volume may then be tessellated into tetrahedra. The corner point of each tetrahedron k may be one of n tracking units, at position ptk. The edges of the tessellation may be stored in a data structure as follows:
[0038]
where each edge connects two points, each of which is a reference to a point #. For example, point a1 may refer to point #1 and point b1 may refer to point #4.
| Edges table: | |
| edge #1, point a1, point b1, | |
| edge #2, point a2, point b2, | |
| . . . | |
| edge #m, point an, point bn, | |
where each edge connects two points, each of which is a reference to a point #. For example, point a1 may refer to point #1 and point b1 may refer to point #4.
[0039]
where edge a1 and a2 may refer, for example, to edges #5 and edge #2, respectively.
| Tetrahedra table: | |
| tetrahedron #1, edge a1, edge b1, edge c1, edge d1, edge e1, edge f1, | |
| tetrahedron #2, edge a2, edge b2, edge c2, edge d2, edge e2, edge f2, | |
| . . . , | |
where edge a1 and a2 may refer, for example, to edges #5 and edge #2, respectively.
[0040] Some of the image I may lie outside of the convex hull of the points pt1 . . . ptn. These portions of image I may be ignored or other algorithms may be used to determine their distortion.
[0041] At time j, where j>t, the tissue may have changed shape, and image I (which represents the anatomical site at time t) may no longer represent the. The positions of the tracking sensors at time j, are pj1 . . . pjn. One may consider a new image J that is a linearly warped copy of image I. One may not need to compute image J, however. Instead, at time j, an updated, deformed image for a 2d cross-sectional plane through image J may be determined as follows:
-
- [0042] Update the points table, and replace each ptk with pjk where k={1. . . n}, thereby updating the position of each tracking unit, at time j, as supplied, for example, by position sensing system 140.
- [0043] Iterate through the tetrahedra table. For each tetrahedron, iterate through each of its six edges. For each such edge, compute its intersection with the cross-sectional plane. For those edges that do intersect with the plane, we compute the intersection point P. We then compute the texture coordinate for P, by linearly interpolating between the texture coordinates at its endpoints (those texture coordinates are stored in the points table).
- [0044] For each tetrahedron, there will be 0, 1, 2, 3, or 4 edges that intersect with the plane (resulting in 0, 1, 2, 3, or 4 intersections points P, and their corresponding texture coordinates).
- [0045] When there are 0, 1, or 2 intersections, render nothing associated with the tetrahedron.
- [0046] When there are 3 or 4 intersections of the cross-sectional plane, render a textured polygon using traditional graphics hardware commands (e.g. OpenGL or DirectX). The texture coordinates index into the image I, but because the point P will have moved from the original location, the result will be a portion of image J.
- [0047] Repeat this for each tetrahedron. After all tetrahedra have been processed, the graphics hardware will have rendered the intersection of the warped image J, with the chosen cross-sectional plane.
[0048] In some embodiments, if the second set of imaging data is a planar fluoroscopic image, then the deformed version of the first set of imaging data may be projected onto the plane of the planar fluoroscopy and the two images may be combined in order to produce the image guidance data.
[0049] In some embodiments, the image guidance data determined in step 650 from the deformed version of the first set of imaging data could be used to approximate another imaging modality. For example, the first set of imaging data may be a CT scan and a user may wish to have an approximation of a biplane fluoroscopy performed without exposing a patient to the harmful radiation associated with such a fluoroscopy. The deformed version of the first set of imaging data may be projected onto what would be the two planes of the biplane fluoroscopy. This would approximate the biplane fluoroscopy using the updated tracking unit information without exposing the patient to the radiation associated with the biplane fluoroscopy. Further, this approximation could be updated at a rate that exceeds that of conventional biplane fluoroscopy as the tracking units move with the surrounding tissues to which they are affixed, without harming the patient or disturbing the ongoing operation.
IV. Creating Features in Imaging Data
[0050] FIGS. 7A-7D depict marking and viewing features in imaging data. Generally FIGS. 7A-7D show the manual creation of features within a first set of imaging data and updating the placement of the feature based on the deformed model of the first set of imaging data. FIG. 7A shows a feature selection unit 710 being used to highlight, in a particular plane or visual slice of the first set of imaging data 720, a feature 730 within the first of imaging data. In FIG. 7B , the user points to the feature 730 and selects the feature in order to signify selected feature 740. When the first set of imaging data 720 is being viewed later, the selected feature 740 will still appear in its original position, as depicted in FIG. 7C .
[0051] FIG. 7D illustrates that if a plane of the first set of imaging data 720 is displayed, and the selected feature 740 may still be displayed even if it is not within the visual slice or plane or display data of the first set of imaging data 720. The displacement of the feature 740 from the visual slice of the first set of imaging data 720 may be shown with a displacement marker 750. In some embodiments, the displacement of the selected feature 740 from the visual slice of the first set of imaging data 720 may be shown with other visual techniques, such as visual depth on a 3D display, shadowing, foreshortening, or any other known technique. In FIGS. 7C and 7D , even if the first set of imaging data is deformed based on updated emplacements of the tracking units 145 (not pictured), the selected feature 740 may be shown in a new location based on the deformation of the first set of imaging data 720. Marking these features may be useful so that a user or surgeon could identify and later find points of interest, such as locations of tumors or lesions.
V. Image Guidance Data
[0052] FIG. 8 depicts an embodiment of image guidance data in which a first set of imaging data is presented with a second set of imaging data. In some embodiments, the second set of imaging data is displayed in approximately the form that is received as described above. The first of imaging data may be deformed based on the updated emplacements of tracking units 145. The updated or deformed model corresponding to the first set of imaging data may be used to provide image guidance, such as the location of an important feature of the anatomical site. For example, in the ultrasound image of the liver depicted in FIG. 8 , the ultrasound image 810 (the second set of imaging data) is augmented with a feature 820 from the first set of imaging data, such as a CT scan. This feature may be the location of a tumor, necrosed tissue, or any other relevant feature. It may have been detected or selected by a user, as illustrated in FIGS. 7A-7D . Upon deformation of the underlying tissue, the model for the first set of imaging data may be deformed and therefore the feature 820 shown from the second set of imaging data would also be updated and deformed. Over time, as new second sets of imaging data 810 were being generated, the movement and updated emplacements of the tracking units 145 in an anatomical site would cause the deformation and movement of the feature of the first set of imaging data 820. The updated emplacement of the feature 820 would continue to be shown with the newly received second set of imaging data 810. In some embodiments, the deformation of the first set of imaging data approximates the deformation of the underlying tissue. Therefore, the approximate placement of the feature 820 would approximate the location of the underlying anatomical feature within the second set of imaging data 810.
[0053] The processes, computer readable medium, and systems described herein may be performed on various types of hardware, such as computer systems. Computer systems may include a bus or other communication mechanism for communicating information, and a processor coupled with the bus for processing information. A computer system may have a main memory, such as a random access memory or other dynamic storage device, coupled to the bus. The main memory may be used to store instructions and temporary variables. The computer system may also include a read-only memory or other static storage device coupled to the bus for storing static information and instructions. The computer system may also be coupled to a display, such as a CRT or LCD monitor. Input devices may also be coupled to the computer system. These input devices may include a mouse, a trackball, or cursor direction keys. Computer systems described herein may include the image guidance unit 130, first and second position sensing units 110 and 140, and imaging unit 150. Each computer system may be implemented using one or more physical computers or computer systems or portions thereof. The instructions executed by the computer system may also be read in from a computer-readable medium. The computer-readable medium may be a CD, DVD, optical or magnetic disk, laserdisc, carrier wave, or any other medium that is readable by the computer system. In some embodiments, hardwired circuitry may be used in place of or in combination with software instructions executed by the processor.
[0054] As will be apparent, the features and attributes of the specific embodiments disclosed above may be combined in different ways to form additional embodiments, all of which fall within the scope of the present disclosure.
[0055] Conditional language used herein, such as, among others, “can,” “could,” “might,” “may,” “e.g.,” and the like, unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements and/or states. Thus, such conditional language is not generally intended to imply that features, elements and/or states are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without author input or prompting, whether these features, elements and/or states are included or are to be performed in any particular embodiment.
[0056] Any process descriptions, elements, or blocks in the flow diagrams described herein and/or depicted in the attached figures should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process. Alternate implementations are included within the scope of the embodiments described herein in which elements or functions may be deleted, executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those skilled in the art.
[0057] All of the methods and processes described above may be embodied in, and fully automated via, software code modules executed by one or more general purpose computers or processors, such as those computer systems described above. The code modules may be stored in any type of computer-readable medium or other computer storage device. Some or all of the methods may alternatively be embodied in specialized computer hardware.
[0058] It should be emphasized that many variations and modifications may be made to the above-described embodiments, the elements of which are to be understood as being among other acceptable examples. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.
Claims
What is claimed is:
1. A method of presenting imaging data related to an anatomical site, comprising:
obtaining, at a first time, a first set of imaging data related to the anatomical site and tracking units at the anatomical site;
obtaining information about the tracking units at a second time after the first time;
determining a deformed version of the first set of imaging data based on a difference between relative arrangements of one or more of the tracking units at the first time and at the second time; and
determining image guidance data for display based on, the deformed version of the first set of imaging data.
2. The method of claim 1, wherein obtaining a first set of imaging data related to the anatomical site and tracking units comprises obtaining an arrangement of the tracking units.
3. The method of claim 1, wherein the method further comprises obtaining an arrangement of the tracking units at the first time and the second time; and wherein:
determining a deformed version of the first set of imaging data comprises performing a deformation on a volumetric model of the first data set based on the arrangements of the tracking units at the first time and at the second time.
4. The method of claim 1, wherein:
obtaining the first set of imaging data comprises obtaining a volumetric model of obtained imaging data including the emplacement of the tracking units; and
determining the deformed version of the first set of imaging data comprises performing a deformation of the volumetric model.
5. The method of claim 1, wherein:
obtaining the first set of imaging data comprises obtaining a finite element model of obtained imaging data including the emplacement of the tracking units; and
determining the deformed version of the first set of imaging data comprises performing a deformation of the finite element model.
6. The method of claim 1, wherein the method further comprises receiving information about a feature within the first data set; and wherein:
determining image guidance data for display comprises displaying the feature within the first data set.
7. The method of claim 1, wherein determining a deformed version of the first set of imaging data comprises:
performing a linear interpolation on a model of the first set of imaging data.
8. The method of claim 1, wherein determining a deformed version of the first set of imaging data comprises:
performing a bi-cubic interpolation on a model of the first set of imaging data.
9. The method of claim 1, further comprising:
obtaining a second set of imaging data related to the anatomical site at the second time; and
determining relative emplacements of the second set of imaging data set and the deformed version of the first set of imaging data set,
wherein the image guidance data is based at least on the second set of imaging data and the determined relative emplacements.
10. The method of claim 2, wherein obtaining an arrangement of the tracking units comprises determining the location and orientation of the tracking units.
11. The method of claim 9, wherein the second set of imaging data is associated with a movable device and the emplacement of the second set of imaging data is determined using the emplacement of the movable device.
12. The method of claim 11, wherein the movable device is a movable ultrasound device.
13. The method of claim 9, wherein the second set of imaging data is associated with a first movable device and the emplacement of the second set of imaging data is determined using the emplacement of a second movable device, wherein the first movable device is distinct from the second movable device.
14. A system comprising:
one or more computing devices in communication with tracking units at an anatomical site, wherein the one or more computing devices are configured to:
obtain a first set of imaging data related to the anatomical site and the tracking units at a first time;
obtain information about the tracking units at a second time after the first time;
determine a deformed version of the first set of imaging data based on a difference between relative arrangements of one or more of the tracking units at the first time and at the second time;
determine image guidance data for display based on the deformed version of the first set of imaging data.
15. The system of claim 14, wherein the one or more computing devices are further configured to obtain an arrangement of the tracking units at the first time and the second time; and wherein:
determining a deformed version of the first set of imaging data comprises performing a deformation on a volumetric model of the first data set based on the arrangements of the tracking units at the first time and at the second time.
16. The system of claim 14, wherein:
obtaining the first set of imaging data comprises obtaining a volumetric model of obtained imaging data including the emplacement of the tracking units; and
determining the deformed version of the first set of imaging data comprises performing a deformation of the volumetric model.
17. The system of claim 14, wherein:
obtaining the first set of imaging data comprises obtaining a finite element model of obtained imaging data including the emplacement of the tracking units; and
determining the deformed version of the first set of imaging data comprises performing a deformation of the finite element model.
18. The system of claim 14, wherein the one or more computing devices are further configured to receive information about a feature within the first data set; and wherein determining image guidance data for display comprises displaying the feature within the first data set.
19. The system of claim 14, wherein determining a deformed version of the first set of imaging data comprises performing at least one of a linear interpolation on a model of the first set of imaging data, and a bi-cubic interpolation on a model of the first set of imaging data.
20. A computer-readable, non-transitory storage medium having one or more computer-executable modules, the one or more computer-executable modules comprising:
a first module in communication with tracking units at an anatomical site, wherein the first module is configured to:
obtain, at a first time, a first set of imaging data related to the anatomical site and the tracking units;
obtain information about the tracking units at a second time after the first time;
determine a deformed version of the first set of imaging data based on a difference between relative arrangements of one or more of the tracking units at the first time and at the second time; and
determine image guidance data for display based on the deformed version of the first set of imaging data.
Patent Citations (319)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| USRE30397(E) | 1980-09-01 | King | Applicant |
| US4249539(A) | 1981-02-01 | Vilkomerson et al. | Applicant |
| US4294544(A) | 1981-10-01 | Altschuler et al. | Applicant |
| US4407294(A) | 1983-10-01 | Vilkomerso | Applicant |
| US4567896(A) | 1986-02-01 | Barnea et al. | Applicant |
| US4671292(A) | 1987-06-01 | Matzuk | Applicant |
| US4839836(A) | 1989-06-01 | Fonsalas | Applicant |
| US4862873(A) | 1989-09-01 | Yajima et al. | Applicant |
| US4884219(A) | 1989-11-01 | Waldren | Applicant |
| US4899756(A) | 1990-02-01 | Sonek | Applicant |
| US4911173(A) | 1990-03-01 | Terwillige | Applicant |
| US4945305(A) | 1990-07-01 | Blood | Applicant |
| US5078140(A) | 1992-01-01 | Kwoh | Applicant |
| US5095910(A) | 1992-03-01 | Powers | Applicant |
| US5109276(A) | 1992-04-01 | Nudelman et al. | Applicant |
| US5193120(A) | 1993-03-01 | Gamache et al. | Applicant |
| US5249581(A) | 1993-10-01 | Horbal et al. | Applicant |
| US5261404(A) | 1993-11-01 | Mick et al. | Applicant |
| US5307153(A) | 1994-04-01 | Maruyama et al. | Applicant |
| US5309913(A) | 1994-05-01 | Kormos et al. | Applicant |
| US5323002(A) | 1994-06-01 | Sampsell et al. | Applicant |
| US5371543(A) | 1994-12-01 | Anderson | Applicant |
| US5383454(A) | 1995-01-01 | Bucholz | Applicant |
| US5394875(A) | 1995-03-01 | Lewis et al. | Applicant |
| US5411026(A) | 1995-05-01 | Carol | Applicant |
| US5446798(A) | 1995-08-01 | Morita et al. | Applicant |
| US5447154(A) | 1995-09-01 | Cinquin et al. | Applicant |
| US5452024(A) | 1995-09-01 | Sampsell | Applicant |
| US5457493(A) | 1995-10-01 | Leddy et al. | Applicant |
| US5483961(A) | 1996-01-01 | Kelly et al. | Applicant |
| US5488431(A) | 1996-01-01 | Gove et al. | Applicant |
| US5489952(A) | 1996-02-01 | Gove et al. | Applicant |
| US5491510(A) | 1996-02-01 | Gove | Applicant |
| US5494039(A) | 1996-02-01 | Onik et al. | Applicant |
| US5503152(A) | 1996-04-01 | Oakley et al. | Applicant |
| US5517990(A) | 1996-05-01 | Kalfas et al. | Applicant |
| US5526051(A) | 1996-06-01 | Gove et al. | Applicant |
| US5526812(A) | 1996-06-01 | Dumoulin et al. | Applicant |
| US5529070(A) | 1996-06-01 | Augustine et al. | Applicant |
| US5531227(A) | 1996-07-01 | Schneider | Applicant |
| US5532997(A) | 1996-07-01 | Pauli | Applicant |
| US5541723(A) | 1996-07-01 | Tanaka | Applicant |
| US5558091(A) | 1996-09-01 | Acker et al. | Applicant |
| US5570135(A) | 1996-10-01 | Gove et al. | Applicant |
| US5579026(A) | 1996-11-01 | Tabata | Applicant |
| US5588948(A) | 1996-12-01 | Takahashi et al. | Applicant |
| US5608468(A) | 1997-03-01 | Gove et al. | Applicant |
| US5608849(A) | 1997-03-01 | King, Jr. | Applicant |
| US5611345(A) | 1997-03-01 | Hibbeln | Applicant |
| US5611353(A) | 1997-03-01 | Dance et al. | Applicant |
| US5612753(A) | 1997-03-01 | Poradish et al. | Applicant |
| US5625408(A) | 1997-04-01 | Matsugu et al. | Applicant |
| US5629794(A) | 1997-05-01 | Magel et al. | Applicant |
| US5630027(A) | 1997-05-01 | Venkateswar et al. | Applicant |
| US5647373(A) | 1997-07-01 | Paltieli et al. | Applicant |
| US5660185(A) | 1997-08-01 | Shmulewitz et al. | Applicant |
| US5662111(A) | 1997-09-01 | Cosman | Applicant |
| US5699444(A) | 1997-12-01 | Palm | Applicant |
| US5701898(A) | 1997-12-01 | Adam et al. | Applicant |
| US5726670(A) | 1998-03-01 | Tabata et al. | Applicant |
| US5728044(A) | 1998-03-01 | Shan | Applicant |
| US5758650(A) | 1998-06-01 | Miller et al. | Applicant |
| US5766135(A) | 1998-06-01 | Terwilliger | Applicant |
| US5784098(A) | 1998-07-01 | Shoji et al. | Applicant |
| US5792147(A) | 1998-08-01 | Evans et al. | Applicant |
| US5793701(A) | 1998-08-01 | Wright et al. | Applicant |
| US5797849(A) | 1998-08-01 | Vesely et al. | Applicant |
| US5807395(A) | 1998-09-01 | Mulier et al. | Applicant |
| US5810008(A) | 1998-09-01 | Dekel et al. | Applicant |
| US5817022(A) | 1998-10-01 | Vesely | Applicant |
| US5820554(A) | 1998-10-01 | Davis et al. | Applicant |
| US5829439(A) | 1998-11-01 | Yokosawa et al. | Applicant |
| US5829444(A) | 1998-11-01 | Ferre et al. | Applicant |
| US5851183(A) | 1998-12-01 | Bucholz | Applicant |
| US5870136(A) | 1999-02-01 | Fuchs et al. | Applicant |
| US5891034(A) | 1999-04-01 | Bucholz | Applicant |
| US5920395(A) | 1999-07-01 | Schulz | Applicant |
| US5961527(A) | 1999-10-01 | Whitmore, III et al. | Applicant |
| US6019724(A) | 2000-02-01 | Gronningsaeter et al. | Applicant |
| US6048312(A) | 2000-04-01 | Ishrak et al. | Applicant |
| US6064749(A) | 2000-05-01 | Hirota et al. | Applicant |
| US6095982(A) | 2000-08-01 | Richards-Kortum et al. | Applicant |
| US6108130(A) | 2000-08-01 | Raj | Applicant |
| US6122538(A) | 2000-09-01 | Sliwa, Jr. et al. | Applicant |
| US6122541(A) | 2000-09-01 | Cosman et al. | Applicant |
| US6167296(A) | 2000-12-01 | Shahidi | Applicant |
| US6216029(B1) | 2001-04-01 | Paltieli | Applicant |
| US6245017(B1) | 2001-06-01 | Hashimoto et al. | Applicant |
| US6246898(B1) | 2001-06-01 | Vesely et al. | Applicant |
| US6248101(B1) | 2001-06-01 | Whitmore, III et al. | Applicant |
| US6261234(B1) | 2001-07-01 | Lin | Applicant |
| US6341016(B1) | 2002-01-01 | Malione | Applicant |
| US6348058(B1) | 2002-02-01 | Melken et al. | Applicant |
| US6379302(B1) | 2002-04-01 | Kessman et al. | Applicant |
| US6385475(B1) | 2002-05-01 | Cinquin et al. | Applicant |
| US6442417(B1) | 2002-08-01 | Shahidi et al. | Applicant |
| US6456868(B2) | 2002-09-01 | Saito et al. | Applicant |
| US6470207(B1) | 2002-10-01 | Simon et al. | Applicant |
| US6477400(B1) | 2002-11-01 | Barrick | Applicant |
| US6478793(B1) | 2002-11-01 | Cosman et al. | Applicant |
| US6503195(B1) | 2003-01-01 | Keller et al. | Applicant |
| US6511418(B2) | 2003-01-01 | Shahidi et al. | Applicant |
| US6518939(B1) | 2003-02-01 | Kikuchi | Applicant |
| US6527443(B1) | 2003-03-01 | Vilsmeier | Applicant |
| US6529758(B2) | 2003-03-01 | Shahidi | Applicant |
| US6545706(B1) | 2003-04-01 | Edwards et al. | Applicant |
| US6546279(B1) | 2003-04-01 | Bova et al. | Applicant |
| US6551325(B2) | 2003-04-01 | Neubauer et al. | Applicant |
| US6570566(B1) | 2003-05-01 | Yoshigahara | Applicant |
| US6587711(B1) | 2003-07-01 | Alfano et al. | Applicant |
| US6591130(B2) | 2003-07-01 | Shahidi | Applicant |
| US6594517(B1) | 2003-07-01 | Nevo | Applicant |
| US6597818(B2) | 2003-07-01 | Kumar et al. | Applicant |
| US6604404(B2) | 2003-08-01 | Paltieli et al. | Applicant |
| US6616610(B2) | 2003-09-01 | Steininger et al. | Applicant |
| US6626832(B1) | 2003-09-01 | Paltieli et al. | Applicant |
| US6669635(B2) | 2003-12-01 | Kessman et al. | Applicant |
| US6689067(B2) | 2004-02-01 | Sauer et al. | Applicant |
| US6695786(B2) | 2004-02-01 | Wang et al. | Applicant |
| US6711429(B1) | 2004-03-01 | Gilboa et al. | Applicant |
| US6725082(B2) | 2004-04-01 | Sati et al. | Applicant |
| US6733458(B1) | 2004-05-01 | Steins et al. | Applicant |
| US6764449(B2) | 2004-07-01 | Lee et al. | Applicant |
| US6766184(B2) | 2004-07-01 | Utzinger et al. | Applicant |
| US6768496(B2) | 2004-07-01 | Bieger et al. | Applicant |
| US6775404(B1) | 2004-08-01 | Pagoulatos et al. | Applicant |
| US6782287(B2) | 2004-08-01 | Grzeszczuk et al. | Applicant |
| US6783524(B2) | 2004-08-01 | Anderson et al. | Applicant |
| US6827723(B2) | 2004-12-01 | Carson | Applicant |
| US6873867(B2) | 2005-03-01 | Vilsmeier | Applicant |
| US6875179(B2) | 2005-04-01 | Ferguson et al. | Applicant |
| US6881214(B2) | 2005-04-01 | Cosman et al. | Applicant |
| US6895268(B1) | 2005-05-01 | Rahn et al. | Applicant |
| US6915150(B2) | 2005-07-01 | Cinquin et al. | Applicant |
| US6917827(B2) | 2005-07-01 | Kienzle, III | Applicant |
| US6923817(B2) | 2005-08-01 | Carson et al. | Applicant |
| US6936048(B2) | 2005-08-01 | Hurst | Applicant |
| US6947783(B2) | 2005-09-01 | Immerz | Applicant |
| US6968224(B2) | 2005-11-01 | Kessman et al. | Applicant |
| US6978167(B2) | 2005-12-01 | Dekel et al. | Applicant |
| US7008373(B2) | 2006-03-01 | Stoianovici et al. | Applicant |
| US7033360(B2) | 2006-04-01 | Cinquin et al. | Applicant |
| US7072707(B2) | 2006-07-01 | Galloway, Jr. et al. | Applicant |
| US7110013(B2) | 2006-09-01 | Ebersole et al. | Applicant |
| US7209776(B2) | 2007-04-01 | Leitner | Applicant |
| US7248232(B1) | 2007-07-01 | Yamazaki et al. | Applicant |
| US7331932(B2) | 2008-02-01 | Leitner | Applicant |
| US7351205(B2) | 2008-04-01 | Szczech et al. | Applicant |
| US7379769(B2) | 2008-05-01 | Piron et al. | Applicant |
| US7385708(B2) | 2008-06-01 | Ackerman et al. | Applicant |
| US7392076(B2) | 2008-06-01 | Moctezuma de La Barrera | Applicant |
| US7398116(B2) | 2008-07-01 | Edwards | Applicant |
| US7466303(B2) | 2008-12-01 | Yi et al. | Applicant |
| US7480533(B2) | 2009-01-01 | Cosman et al. | Applicant |
| US7505809(B2) | 2009-03-01 | Strommer et al. | Applicant |
| US7588541(B2) | 2009-09-01 | Floyd et al. | Applicant |
| US7662128(B2) | 2010-02-01 | Salcudean et al. | Applicant |
| US7728868(B2) | 2010-06-01 | Razzaque et al. | Applicant |
| US7833168(B2) | 2010-11-01 | Taylor et al. | Applicant |
| US7833221(B2) | 2010-11-01 | Voegele et al. | Applicant |
| US7846103(B2) | 2010-12-01 | Cannon, Jr. et al. | Applicant |
| US7876942(B2) | 2011-01-01 | Gilboa | Applicant |
| US7901357(B2) | 2011-03-01 | Boctor et al. | Applicant |
| US7920909(B2) | 2011-04-01 | Lyon et al. | Applicant |
| US7962193(B2) | 2011-06-01 | Edwards et al. | Applicant |
| US7976469(B2) | 2011-07-01 | Bonde et al. | Applicant |
| US8023712(B2) | 2011-09-01 | Ikuma et al. | Applicant |
| US8038631(B1) | 2011-10-01 | Sanghvi et al. | Applicant |
| US8041413(B2) | 2011-10-01 | Barbagli et al. | Applicant |
| US8050736(B2) | 2011-11-01 | Piron et al. | Applicant |
| US8052636(B2) | 2011-11-01 | Moll et al. | Applicant |
| US8066644(B2) | 2011-11-01 | Sarkar et al. | Applicant |
| US8073528(B2) | 2011-12-01 | Zhao et al. | Applicant |
| US8086298(B2) | 2011-12-01 | Whitmore, III et al. | Applicant |
| US8137281(B2) | 2012-03-01 | Huang et al. | Applicant |
| US8147408(B2) | 2012-04-01 | Bunce et al. | Applicant |
| US8152724(B2) | 2012-04-01 | Ridley et al. | Applicant |
| US8216149(B2) | 2012-07-01 | Oonuki et al. | Applicant |
| US8221322(B2) | 2012-07-01 | Wang et al. | Applicant |
| US8228028(B2) | 2012-07-01 | Schneider | Applicant |
| US8257264(B2) | 2012-09-01 | Park et al. | Applicant |
| US8340379(B2) | 2012-12-01 | Razzaque et al. | Applicant |
| US8350902(B2) | 2013-01-01 | Razzaque et al. | Applicant |
| US2001/0007919(A1) | 2001-07-01 | Shahidi | Applicant |
| US2001/0016804(A1) | 2001-08-01 | Cunningham et al. | Applicant |
| US2001/0045979(A1) | 2001-11-01 | Matsumoto et al. | Applicant |
| US2002/0010384(A1) | 2002-01-01 | Shahidi et al. | Applicant |
| US2002/0049375(A1) | 2002-04-01 | Strommer et al. | Applicant |
| US2002/0077540(A1) | 2002-06-01 | Kienzie, III | Applicant |
| US2002/0077543(A1) | 2002-06-01 | Grzeszczuk et al. | Applicant |
| US2002/0135673(A1) | 2002-09-01 | Favalora et al. | Applicant |
| US2002/0138008(A1) | 2002-09-01 | Tsujita et al. | Applicant |
| US2002/0140814(A1) | 2002-10-01 | Cohen-Solal et al. | Applicant |
| US2002/0156375(A1) | 2002-10-01 | Kessman et al. | Examiner |
| US2002/0198451(A1) | 2002-12-01 | Carson | Applicant |
| US2003/0040743(A1) | 2003-02-01 | Cosman et al. | Applicant |
| US2003/0073901(A1) | 2003-04-01 | Simon et al. | Applicant |
| US2003/0163142(A1) | 2003-08-01 | Paltieli et al. | Applicant |
| US2003/0164172(A1) | 2003-09-01 | Chumas et al. | Applicant |
| US2003/0231789(A1) | 2003-12-01 | Willis et al. | Applicant |
| US2004/0034313(A1) | 2004-02-01 | Leitner | Applicant |
| US2004/0078036(A1) | 2004-04-01 | Keidar | Applicant |
| US2004/0095507(A1) | 2004-05-01 | Bishop et al. | Applicant |
| US2004/0147920(A1) | 2004-07-01 | Keidar | Applicant |
| US2004/0181144(A1) | 2004-09-01 | Cinquin et al. | Applicant |
| US2004/0215071(A1) | 2004-10-01 | Frank et al. | Applicant |
| US2004/0238732(A1) | 2004-12-01 | State et al. | Applicant |
| US2004/0243146(A1) | 2004-12-01 | Chesbrough et al. | Applicant |
| US2004/0243148(A1) | 2004-12-01 | Wasielewski | Applicant |
| US2004/0254454(A1) | 2004-12-01 | Kockro | Applicant |
| US2005/0085717(A1) | 2005-04-01 | Shahidi | Applicant |
| US2005/0085718(A1) | 2005-04-01 | Shahidi | Examiner |
| US2005/0090742(A1) | 2005-04-01 | Mine et al. | Applicant |
| US2005/0111733(A1) | 2005-05-01 | Fors et al. | Applicant |
| US2005/0159641(A1) | 2005-07-01 | Kanai | Applicant |
| US2005/0182316(A1) | 2005-08-01 | Burdette et al. | Applicant |
| US2005/0192564(A1) | 2005-09-01 | Cosman et al. | Applicant |
| US2005/0219552(A1) | 2005-10-01 | Ackerman et al. | Applicant |
| US2005/0222574(A1) | 2005-10-01 | Giordano et al. | Applicant |
| US2005/0251148(A1) | 2005-11-01 | Friedrich | Applicant |
| US2006/0004275(A1) | 2006-01-01 | Vija et al. | Applicant |
| US2006/0020204(A1) | 2006-01-01 | Serra et al. | Applicant |
| US2006/0036162(A1) | 2006-02-01 | Shahidi et al. | Applicant |
| US2006/0052792(A1) | 2006-03-01 | Boettiger et al. | Applicant |
| US2006/0100505(A1) | 2006-05-01 | Viswanathan | Applicant |
| US2006/0122495(A1) | 2006-06-01 | Kienzle, III | Applicant |
| US2006/0184040(A1) | 2006-08-01 | Keller et al. | Applicant |
| US2006/0193504(A1) | 2006-08-01 | Salgo et al. | Applicant |
| US2006/0229594(A1) | 2006-10-01 | Francischelli et al. | Applicant |
| US2006/0235290(A1) | 2006-10-01 | Gabriel et al. | Applicant |
| US2006/0235538(A1) | 2006-10-01 | Rochetin et al. | Applicant |
| US2006/0241450(A1) | 2006-10-01 | Da Silva et al. | Applicant |
| US2006/0253030(A1) | 2006-11-01 | Altmann et al. | Applicant |
| US2006/0253032(A1) | 2006-11-01 | Altmann et al. | Applicant |
| US2006/0271056(A1) | 2006-11-01 | Terrill-Grisoni et al. | Applicant |
| US2006/0282023(A1) | 2006-12-01 | Leitner | Applicant |
| US2006/0293643(A1) | 2006-12-01 | Wallace et al. | Applicant |
| US2007/0016035(A1) | 2007-01-01 | Hashimoto | Applicant |
| US2007/0032906(A1) | 2007-02-01 | Sutherland et al. | Applicant |
| US2007/0073155(A1) | 2007-03-01 | Park et al. | Applicant |
| US2007/0078346(A1) | 2007-04-01 | Park et al. | Applicant |
| US2007/0167699(A1) | 2007-07-01 | Lathuiliere et al. | Applicant |
| US2007/0167701(A1) | 2007-07-01 | Sherman | Applicant |
| US2007/0167705(A1) | 2007-07-01 | Chiang et al. | Applicant |
| US2007/0167801(A1) | 2007-07-01 | Webler et al. | Applicant |
| US2007/0225553(A1) | 2007-09-01 | Shahidi | Applicant |
| US2007/0239281(A1) | 2007-10-01 | Gotte et al. | Applicant |
| US2007/0244488(A1) | 2007-10-01 | Metzger et al. | Applicant |
| US2007/0255136(A1) | 2007-11-01 | Kristofferson et al. | Applicant |
| US2007/0270718(A1) | 2007-11-01 | Rochetin et al. | Applicant |
| US2007/0276234(A1) | 2007-11-01 | Shahidi | Applicant |
| US2008/0004481(A1) | 2008-01-01 | Bax et al. | Applicant |
| US2008/0004516(A1) | 2008-01-01 | DiSilvestro et al. | Applicant |
| US2008/0030578(A1) | 2008-02-01 | Razzaque et al. | Applicant |
| US2008/0039723(A1) | 2008-02-01 | Suri et al. | Applicant |
| US2008/0051910(A1) | 2008-02-01 | Kammerzell et al. | Applicant |
| US2008/0091106(A1) | 2008-04-01 | Kim et al. | Applicant |
| US2008/0114235(A1) | 2008-05-01 | Unal et al. | Applicant |
| US2008/0161824(A1) | 2008-07-01 | McMillen | Applicant |
| US2008/0200794(A1) | 2008-08-01 | Teichman et al. | Applicant |
| US2008/0208031(A1) | 2008-08-01 | Kurpad et al. | Applicant |
| US2008/0208081(A1) | 2008-08-01 | Murphy et al. | Applicant |
| US2008/0214932(A1) | 2008-09-01 | Mollard et al. | Applicant |
| US2008/0232679(A1) | 2008-09-01 | Hahn et al. | Applicant |
| US2008/0287805(A1) | 2008-11-01 | Li | Applicant |
| US2009/0024030(A1) | 2009-01-01 | Lachaine et al. | Applicant |
| US2009/0118724(A1) | 2009-05-01 | Zvuloni et al. | Applicant |
| US2009/0137907(A1) | 2009-05-01 | Takimoto et al. | Applicant |
| US2009/0226069(A1) | 2009-09-01 | Razzaque et al. | Applicant |
| US2009/0234369(A1) | 2009-09-01 | Bax et al. | Applicant |
| US2009/0312629(A1) | 2009-12-01 | Razzaque et al. | Applicant |
| US2010/0045783(A1) | 2010-02-01 | State et al. | Applicant |
| US2010/0198045(A1) | 2010-08-01 | Razzaque et al. | Applicant |
| US2010/0208963(A1) | 2010-08-01 | Kruecker et al. | Applicant |
| US2010/0268067(A1) | 2010-10-01 | Razzaque et al. | Applicant |
| US2010/0268072(A1) | 2010-10-01 | Hall et al. | Applicant |
| US2010/0268085(A1) | 2010-10-01 | Kruecker et al. | Applicant |
| US2010/0305448(A1) | 2010-12-01 | Dagonnau et al. | Applicant |
| US2010/0312121(A1) | 2010-12-01 | Guan | Applicant |
| US2011/0043612(A1) | 2011-02-01 | Keller et al. | Applicant |
| US2011/0046483(A1) | 2011-02-01 | Fuchs et al. | Applicant |
| US2011/0057930(A1) | 2011-03-01 | Keller et al. | Applicant |
| US2011/0082351(A1) | 2011-04-01 | Razzaque et al. | Applicant |
| US2011/0130641(A1) | 2011-06-01 | Razzaque et al. | Applicant |
| US2011/0137156(A1) | 2011-06-01 | Razzaque et al. | Applicant |
| US2011/0201976(A1) | 2011-08-01 | Sanghvi et al. | Applicant |
| US2011/0237947(A1) | 2011-09-01 | Boctor et al. | Applicant |
| US2011/0251483(A1) | 2011-10-01 | Razzaque et al. | Applicant |
| US2011/0282188(A1) | 2011-11-01 | Burnside et al. | Applicant |
| US2011/0288412(A1) | 2011-11-01 | Deckman et al. | Applicant |
| US2011/0295108(A1) | 2011-12-01 | Cox et al. | Applicant |
| US2011/0301451(A1) | 2011-12-01 | Rohling | Applicant |
| US2012/0035473(A1) | 2012-02-01 | Sanghvi et al. | Applicant |
| US2012/0059260(A1) | 2012-03-01 | Robinson | Applicant |
| US2012/0071759(A1) | 2012-03-01 | Hagy et al. | Applicant |
| US2012/0078094(A1) | 2012-03-01 | Nishina et al. | Applicant |
| US2012/0101370(A1) | 2012-04-01 | Razzaque et al. | Applicant |
| US2012/0108955(A1) | 2012-05-01 | Razzaque et al. | Applicant |
| US2012/0143029(A1) | 2012-06-01 | Silverstein et al. | Applicant |
| US2012/0143055(A1) | 2012-06-01 | Ng et al. | Applicant |
| US2012/0165679(A1) | 2012-06-01 | Orome et al. | Applicant |
| US2012/0259210(A1) | 2012-10-01 | Harhen et al. | Applicant |
| EP427358 | 1991-05-01 | Applicant | |
| EP1955284 | 2008-08-01 | Applicant | |
| JP2005-058584 | 2005-03-01 | Applicant | |
| JP2005-323669 | 2005-11-01 | Applicant | |
| JP2009-517177 | 2009-04-01 | Applicant | |
| WO3/032837 | 2003-04-01 | Applicant | |
| WOPCT/US2003/17987 | 2003-12-01 | Applicant | |
| WO2005/010711 | 2005-02-01 | Applicant | |
| WO2007/019216 | 2007-02-01 | Applicant | |
| WO2007/067323(A2) | 2007-06-01 | Applicant | |
| WO2007/067323 | 2007-09-01 | Applicant | |
| WO2008/017051 | 2008-02-01 | Applicant | |
| WO2009/094646 | 2009-07-01 | Applicant | |
| WO2010/057315 | 2010-05-01 | Applicant | |
| WO2010/096419 | 2010-08-01 | Applicant | |
| WO2009/063423 | 2010-10-01 | Applicant | |
| WO2011/014687 | 2011-02-01 | Applicant |
Non-Patent Literature (95)
- U.S. Appl. No. 11/828,826, filed Jul. 26, 2007, Keller et al.Applicant
- “3D Laparoscope Technology,” http://www.inneroptic.com/tech—3DL.htm, copyright 2007 InnerOptic Technology, Inc. printed Sep. 19, 2007, 2 pages.Applicant
- “Cancer Facts & Figures 2004,” www.cancer.org/downloads/STT/CAFF—finalPWSecured.pdf, copyright 2004 American Cancer Society, Inc., printed Sep. 19, 2007, 60 pages.Applicant
- “David Laserscanner ←Latest News ← Institute for Robotics and Process Control ← Te . . . ,” http://www/rob.cs.tu-bs.de/en/news/david, printed Sep. 19, 2007, 1 page.Applicant
- “Laser scanned 3d model Final” video, still image of video attached, http://www.youtube.com/watch?v+DaLgIgmoUf8, copyright 2007 YouTube, LLC, printed Sep. 19, 2007, 2 pages.Applicant
- “Olympus Endoscopic Ultrasound System,” www.olympusamerica.com/msg—section/download—brochures/135—b—gfum130.pdf, printed Sep. 20, 2007, 20 pages.Applicant
- “Point Grey Research Inc.—Imaging Products—Triclops SDK Samples,” http://www.ptgrey.com/products/triclopsSDK/samples.asp, copyright 2007 Point Grey Research Inc., printed Sep. 19, 2007, 1 page.Applicant
- “Robbins, Mike—Computer Vision Research—Stereo Depth Perception,” http://www.compumike.com/vision/stereodepth. php, copyright 2007 Michael F. Robbins, printed Sep. 19, 2007, 3 pages.Applicant
- “RUE, Registered Ultrasound-Endoscope,” copyright 2007 InnerOptic Technology, Inc., 2 pages.Applicant
- Advertisement, “Inspeck 3DC 3D Capturor,” Inspeck 3DC 3D Capturor (www.inspeck.com), 1998.Applicant
- Advertisement, “Virtual 3D High Speed Non-Contact Surface Perception,” Virtual 3-D Technologies Corporation (www.virtual3dtech.com)., Dec. 21, 1998.Applicant
- Advertisements, “Virtuoso,” Visual Interface, Inc. (www.visint.com), Dec. 21, 1998.Applicant
- Akka, “Automatic Software Control of Display Parameters for Stereoscopic Graphics Images,” SPIE vol. 1669: Stereoscopic Displays and Applications III, pp. 31-38 (1992).Applicant
- Ali et al., “Near Infrared Spectroscopy and Imaging to Probe Differences in Water Content in Normal and Cancer Human Prostate Tissues,” Technology in Cancer Research & Treatment; Oct. 2004; 3(5):491-497; Adenine Press.Applicant
- Aylward et al., Analysis of the Parameter Space of a Metric for Registering 3D Vascular Images, in W. Niessen and M. Viergever (Eds.): MICCAI 2001, LNCS 2208, pp. 932-939, 2001.Applicant
- Aylward et al., Registration and Analysis of Vascular Images, International Journal of Computer Vision 55(2/3), 123-138, 2003.Applicant
- Aylward, et al., Intra-Operative 3D Ultrasound Augmentation, Proceedings of the IEEE International Symposium on Biomedical Imaging, Washington, Jul. 2002.Applicant
- Azuma, “A Survey of Augmented Reality,” Presence: Teleoperators and Virtual Environments 6, 4:1-48 (Aug. 1997).Applicant
- Bajura, Michael et al.,, “Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient,” Computer Graphics, Proceedings of SIGGRAPH 1992, vol. 26(2), pp. 203-210, available from www.cs.unc.edu/˜fuchs/publications/MergVirtObjs92.pdf, printed Sep. 20, 2007, 8 pages.Applicant
- Benavides et al., “Multispectral digital colposcopy for in vivo detection of cervical cancer,” Optics Express; May 19, 2003; 11(1 0) Optical Society of America; USA.Applicant
- Beraldin, J.A. et al., “Optimized Position Sensors for Flying-Spot Active Triangulation Systems,” Proceedings of the Fourth International Conference on a 3-D Digital Imaging and Modeling (3DIM), Banff, Alberta, Canada, Oct. 6-10, 2003, pp. 334-341, NRC 47083, copyright 2003 National Research Council of Canada, http:/iit-iti.nrc-cnrc.gc.ca/iit-publications-iti/docs/NRC-47083.pdf, printed Sep. 19, 2007, 9 pages.Applicant
- Billinghurst, M. et al., Research Directions in Handheld AR; Int. J. of Virtual Reality 5(2),51-58 (2006).Applicant
- Bishop, Azuma et al., “Improving Static and Dynamic Registration in an Optical See-Through HMD,” Paper Presented at SIGGRAPH '94 Annual Conference in Orlando, FL (1994).Applicant
- Blais, F., “Review of 20 Years of Range Sensor Development,” Journal of Electronic Imaging, 13(1): 231-240, Jan. 2004, NRC 46531, copyright 2004 National Research Council of Canada, http://iit-iti.nrc-cnrc.gc.ca/iit-publications-iti/docs/NRC-46531.pdf, printed Sep. 19, 2007, 14 pages.Applicant
- Bouguet, Jean-Yves, “Camera Calibration Toolbox for Matlab,” www.vision.caltech.edu/bouguetj/calib—doc, printed Sep. 20, 2007, 5 pages.Applicant
- Buxton et al.; “Colposcopically directed punch biopsy: a potentially misleading investigation,” British Journal of Obstetrics and Gynecology; Dec. 1991; 98:1273-1276.Applicant
- Cancer Prevention & Early Detection Facts & Figures 2004; National Center for Tobacco-Free Kids; 2004; American Cancer Society; USA.Applicant
- Cantor et al., “Cost-Effectiveness Analysis of Diagnosis and Management of Cervical Squamous Intraepithelial Lesions,” Diagnostic Strategies for SILs; Feb. 1998; 91(2):270-277.Applicant
- Catalano et al. “Multiphase helical CT findings after percutaneous ablation procedures for hepatocellular carcinoma.” Abdom. Imaging, 25(6),2000, pp. 607-614.Applicant
- Chiriboga et al., “Infrared Spectroscopy of Human Tissue. IV. Detection of Dysplastic and Neoplastic Changes of Human Cervical Tissue Via Infrared Microscopy,” Cellular and Molecular Biology; 1998; 44(1): 219-229.Applicant
- Crawford, David E. et al., “Computer Modeling of Prostate Biopsy: Tumor Size and Location—Not Clinical Significance—Determine Cancer Detection,” Journal of Urology, Apr. 1998, vol. 159(4), pp. 1260-1264, 5 pages.Applicant
- Deering, Michael “High Resolution Virtual Reality.” Proceedings of SIGGRAPH '92, Computer Graphics, 26(2), 1992, pp. 195-202.Applicant
- Depiero et al., “3-D Computer Vision Using Structured Light: Design, Calibration and Implementation Issues,” The University of Tennessee, pp. 1-46, (1996).Applicant
- Dodd, G.D. et al. “Minimally invasive treatment of malignant hepatic tumors: at the threshold of a major breakthrough.” Radiographies 20(1),2000, pp. 9-27.Applicant
- Drascic et al., “Perceptual Issues in Augmented Reality,” SPIE vol. 2653: Stereoscopic Displays and Virtual Reality Systems III, pp. 123-134 (Feb. 1996).Applicant
- Fahey et al., “Meta-analysis of Pap Test Accuracy; American Journal of Epidemiology,” 1995 141(7):680-689; The John Hopkins University School of Hvqiene and Public Health; USA.Applicant
- Foxlin et al., “An Inertial Head-Orientation Tracker with Automatic Drift Compensation for Use with HMD's,” Proceedings of the 1994 Virtual Reality Software and Technology Conference, Aug. 23-26, 1994, Singapore, pp. 159-173 (1994).Applicant
- Fronheiser et al., Real-Time 3D Color Doppler for Guidance of Vibrating Interventional Devices, IEEE Ultrasonics Symposium, pp. 149-152 (2004).Applicant
- Fuchs, Henry et al. “Augmented Reality Visualization for Laparoscopic Surgery,” Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI) 1998, pp. 934-943, available from www.cs.unc.edu/˜fuchs/publications/AugRealVis—LaparoSurg98.pdf, printed Sep. 20, 2007, 10 pages.Applicant
- Fuchs, et al.: “Virtual Environments Technology to Aid Needle Biopsies of the Breast,” Health Care in the Information Age, Ch. 6, pp. 60-61, Presented in San Diego, Jan. 17-20, 1996, published by IOS Press and Ohmsha Feb. 1996.Applicant
- Garrett, William F. et al., “Real-Time Incremental Visualization of Dynamic Ultrasound Volumes Using Parallel BSP Trees,”Proceedings of IEEE Visualization 1996, pp. 235-240, available from www.cs.unc.edu/˜andrei/pubs/1996—VIS—dualBSP—Mac.pdf, printed Sep. 20, 2007, 7 pages.Applicant
- Georgakoudi et al., “Trimodal spectroscopy for the detection and characterization of cervical precancers in vivo,” American Journal of Obstetrics and Gynecology; Mar. 2002; 186(3):374-382; USA.Applicant
- Herline et al., Surface Registration for Use in Interactive, Image-Guided Liver Surgery, Computer Aided Surgery 5:11-17 (2000).Applicant
- Holloway, R.; Registration Error Analysis for Augmented Reality; Presence: Teleoperators and Virtual Environments 6(4), 413-432 (1997).Applicant
- Hornung et al., “Quantitative near-infrared spectroscopy of cervical dysplasia in vivo,” Human Reproduction; 1999; 14(11):2908-2916; European Society of Human Reproduction and Embryology.Applicant
- Howard, M.D., et al.: “An Electronic Device for Needle Placement during Sonographically Guided Percutaneous Intervention”, Radiology 2001; 218:905-911.Applicant
- InnerAim Brochure; 3D Visualization Software for Simpler, Safer, more Precise Aiming, Published no earlier than Apr. 1, 2010.Applicant
- InVision System Brochure; A “GPS” for Real-Time 3D Needle Visualization & Guidance, Published no earlier than Mar. 1, 2008.Applicant
- InVision User Manual; Professional Instructions for Use, Published no earlier than Dec. 1, 2008.Applicant
- Jacobs, Marco C. et al., “Managing Latency in Complex Augmented Reality Systems,” ACM SIGGRAPH Proceedings of the Symposium of Interactive 3D Graphics 1997, pp. 49-54, available from www.cs.unc.edu/˜us/Latency//ManagingRelativeLatency.html, printed Sep. 20, 2007, 12 pages.Applicant
- Kanbara et al., “A Stereoscopic Video See-through Augmented Reality System Based on Real-time Vision-Based Registration,” Nara Institute of Science and Technology, pp. 1-8 (2000).Applicant
- Lass, Amir, “Assessment of Ovarian Reserve,” Human Reproduction, 2004, vol. 19(3), pp. 467-469, available from http://humrep.oxfordjournals.orgcgi/reprint/19/3/467, printed Sep. 20, 2007, 3 pages.Applicant
- Lee et al., “Modeling Real Objects Using Video See-Through Augmented Reality,” Presence, 11(2):144-157 (Apr. 2002).Applicant
- Leven et al., DaVinci Canvas: A Telerobotic Surgical System with Integrated, Robot-Assisted, Laparoscopic Ultrasound Capability, in J. Duncan and G. Gerig (Eds.): MICCAI 2005, LNCS 3749, pp. 811-818, 2005.Applicant
- Levy, et al., An Internet-Connected, Patient Specific, Deformable Brain Atlas Integrated into a Surgical Navigation System, Journal of Digital Imaging, vol. 10, No. 3. Suppl. 1 (Aug. 1997): pp. 231-237.Applicant
- Livingston, Mark A. et al., “Magnetic Tracker Calibration for Improved Augmented Reality Registration,” Presence: Teleoperators and Virtual Environments, 1997, vol. 6(5), pp. 532-546, available from www.cs.unc.edu/˜andrei/pubs/1997—Presence—calibr.pdf, printed Sep. 20, 2007, 14 pages.Applicant
- Matsunaga et al., “The Effect of the Ratio Difference of Overlapped Areas of Stereoscopic Images on each Eye in a Teleoperalion,” Stereoscopic Displays and Virtual Reality Systems VII, Proceedings of SPIE, 3957:236-243 (2000).Applicant
- Meehan, Michael et al., “Effect of Latency on Presence in Stressful Virtual Environment,” Proceedings of IEEE Virtual Reality 2003, pp. 141-148, available from http://www.cs.unc.edu/˜eve/pubs.html, printed Sep. 20, 2007, 9 pages.Applicant
- Milgram et al., “Adaptation Effects in Stereo due to Online Changes in Camera Configuration,” SPIE vol. 1669-13, Stereoscopic Displays and Applications III, pp. 1-12 (1992).Applicant
- Mtchell et al., “Colposcopy for the Diagnosis of Squamous Intraepithelial lesions: A metaanalysis,” Obstetrics and Gynecology; Apr. 1998; 91(4):626-631.Applicant
- Nakamoto et al., 3D Ultrasound System Using a Magneto-optic Hybrid Tracker for Augmented Reality Visualization in Laparoscopic Liver Surgery, in T. Dohi and R. Kikinis (Eds.): MICCAI 2002, LNCS 2489, pp. 148-155, 2002.Applicant
- Nordstrom et al., “Identification of Cervical Intraepithelial Neoplasia (CIN) Using UV-Excited Fluorescence and Diffuse-Reflectance Tissue Spectroscopy,” Lasers in Surgery and Medicine; 2001; 29; pp. 118-127; Wiley-Liss, Inc.Applicant
- Ohbuchi et al. “An Incremental Volume Rendering Algorithm for Interactive 3D Ultrasound Imaging”, UNC-CH Computer Science Technical Report TR91-003, (1991).Applicant
- Ohbuchi et al., “Incremental Volume Reconstruction and Rendering for 3D Ultrasound Imaging,” Visualization in Biomedical Computing, SPIE Proceedings, pp. 312-323, (Oct. 13, 1992).Applicant
- Ohbuchi, “Incremental Acquisition and Visualization of 3D Ultrasound Images,” Ph.D. Dissertation, UNC-CH Computer Science Technical Report TR95-023, (1993).Applicant
- PCT, International Search Report and Written Opinion, re PCT Application No. PCT/US07/75122, mailing date Aug. 20, 2008.Applicant
- PCT, International Preliminary Report on Patentability, re PCT Application No. PCT/US07/75122, mailing date Mar. 3, 2009.Applicant
- PCT, International Search Report and Written Opinion, re PCT Application No. PCT/US2010/024378, mailing date Oct. 13, 2010.Applicant
- PCT, International Search Report and Written Opinion, re PCT Application No. PCT/US2010/043760, mailing date Mar. 3, 2011.Applicant
- PCT, The International Search Report and Written Opinion of the International Searching Authority, mailed Sep. 9, 2009, for case PCT/US2009/032028.Applicant
- Progue, Brian W. et al., “Analysis of acetic acid-induced whitening of high-grade squamous intraepitheliallesions,” Journal of Biomedical Optics; Oct. 2001; 6(4):397-403.Applicant
- Raij, A.B., et al., Comparing Interpersonal Interactions with a Virtual Human to Those with a Real Human; IEEE Transactions on Visualization and Computer Graphics 13(3), 443-457 (2007).Applicant
- Raz et al, Real-Time Magnetic Resonance Imaging—Guided Focal Laser Therapy in Patients with Low-Risk Prostate Cancer, European Urology 58, pp. 173-177. Mar. 12, 2010.Applicant
- Robinett et al., “A Computational Model for the Stereoscopic Optics of a Head-Mounted Display,” SPIE vol. 1457, Stereoscopic Displays and Applications II, pp. 140-160 (1991).Applicant
- Rolland et al., Towards Quantifying Depth and Size Perception in Virtual Environments, Presence: Teleoperators and Virtual Environments, Winter 1995, vol. 4, Issue 1, pp. 1-21 and 24-49.Applicant
- Rosenthal, Michael et al., “Augmented Reality Guidance for Needle Biopsies: An Initial Randomized, Controlled Trial in Phantoms,” Proceedings of Medical Image Analysis, Sep. 2002, vol. 6(3), pp. 313-320, available from www.cs.unc.edu/˜fuchs/publications/AugRealGuida—NeedleBiop02.pdf, printed Sep. 20, 2007, 8 pages.Applicant
- Rosenthal, Michael et al., “Augmented Reality Guidance for Needle Biopsies: A Randomized, Controlled Trial in Phantoms,” Proceedings of MICCAI 2001, eds. W. Niessen and M. Viergever, Lecture Notes in Computer Science, 2001, vol. 2208, pp. 240-248, available from www.cs.unc.edu/˜us/AugmentedRealityAssistance.pdf, printed Sep. 20, 2007, 9 pages.Applicant
- Splechtna, Fuhrmann A. et al., Comprehensive calibration and registration procedures for augmented reality; Proc. Eurographics Workshop on Virtual Environments 2001,219-228 (2001).Applicant
- State et al., “Case Study: Observing a Volume Rendered Fetus within a Pregnant Patient,” Proceedings of IEEE Visualization 1994, pp. 364-368, available from www.cs.unc.edu/˜fuchs/publications/cs-ObservVolRendFetus94.pdf, printed Sep. 20, 2007, 5 pages.Applicant
- State et al., “Interactive Volume Visualization on a Heterogenous Message-Passing Multicomputer,” Proceedings of 1995 Symposium on Interactive 3D Graphics, 1995, pp. 69-74, 208, available from www.cs.unc.edu/˜andrei/pubs/1995—I3D—vol2—Mac.pdf, printed Sep. 20, 2007.Applicant
- State et al., “Simulation-Based Design and Rapid Prototyping of a Parallax-Free, Orthoscopic Video See-Through Head-Mounted Display,” Proceedings of International Symposium on Mixed and Augmented Reality (ISMAR) 2005, available from www.cs.unc.edu/˜andrei/pubs/2005—ISMAR—VSTHMD—design.pdf, printed Sep. 20, 2007, 4 pages.Applicant
- State et al., “Stereo Imagery from the UNC Augmented Reality System for Breast Biopsy Guidance” Proc. Medicine Meets Virtual Reality (MMVR) 2003 (Newport Beach, CA, Jan. 22-25, 2003).Applicant
- State et al., “Superior Augmented Reality Registration by Integrating Landmark Tracking and Magnetic Tracking,” ACM SIGGRAPH Computer Graphics, Proceedings of SIGGRAPH 1996, pp. 429-438, available from www.cs.princeton.edu/courses/archive/fall01/cs597d/papers/state96.pdf, printed Sep. 20, 20007, 10 pages.Applicant
- State et al., “Technologies for Augmented Reality Systems: Realizing Ultrasound-Guided Needle Biopsies,” Computer Graphics, Proceedings of SIGGAPH 1996, pp. 429-438, available from www.cs.princeton.edu/courses/archive/fall01/cs597d/papers/state96.pdf, printed Sep. 20, 2007.Applicant
- State, Andrei “Exact Eye Contact with Virtual Humans.” Proc. IEEE International Workshop on Human Computer Interaction 2007 (Rio de Janeiro, Brazil, Oct. 20, 2007), pp. 138-145.Applicant
- Takagi et al., “Development of a Stereo Video See-through HMD for AR Systems,” IEEE, pp. 68-77 (2000).Applicant
- Ultraguide 1000 System, Ultraguide, www.ultraguideinc.com, 1998.Applicant
- Van Staveren et al., “Light Scattering in Intralipid-10% in the wavelength range of 400-1100 nm,” Applied Optics; Nov. 1991; 30(31):4507-4514.Applicant
- Viola et al., “Alignment by Maximization of Mutual Information,” International Journal of Computer Vision, vol. 24, No. 2, pp. 1-29 (1997).Applicant
- Viola, Paul A., Alignment by Maximization of Mutual Information, Ph.D. Dissertation, MIT-Artificial Intelligence Laboratory Technical Report No. 1548 (Jun. 1995).Applicant
- Ware et al., “Dynamic Adjustment of Stereo Display Parameters,” IEEE Transactions on Systems, Many and Cybernetics, 28(1):1-19 (1998).Applicant
- Watson et al., “Using Texture Maps to Correct for Optical Distortion in Head-Mounted Displays,” Proceedings of the Virtual Reality Annual Symposium '95, IEEE, pp. 1-7 (1995).Applicant
- Welch, Hybrid Self-Tracker: An Inertial/Optical Hybrid Three-Dimensional Tracking System, University of North Carolina Chapel Hill Department of Computer Science, TR 95-048 (1995).Applicant
- Yinghui et al., Real-Time Deformation Using Modal Analysis on Graphics Hardware, GRAPHITE 2006, Kuala Lumpur, Malaysia, Nov. 29-Dec. 2, 2006.Applicant
- Zitnick et al., “Multi-Base Stereo Using Surface Extraction,” Visual Interface Inc., (Nov. 24, 1996).Applicant