Projecting patient image data from radioscopic imaging methods and/or tomographic imaging methods onto video images

[0001] The present invention relates to a device for projecting patient image data from radioscopic imaging methods and/or tomographic imaging methods onto video images. The device comprises an image display device, at least one video camera and a computer-assisted navigation system which can detect the spatial position of the image display device and/or the video camera and the spatial positions of a part of a patient's body via tracking means attached to it.

[0002] Such devices serve to visually assist a physician during diagnosis or while treating a patient. The option should be provided of combining images from the patient's interior with conventional video images, in order to facilitate treatment and diagnosis.

[0003] A video-based surgical target system is known from U.S. Pat. No. 5,765,561, for which among other things the use of a tracked video camera is proposed, whose images are superimposed onto assigned images from radioscopic imaging methods and/or tomographic imaging methods on the statically arranged screen of a navigation system. A particular disadvantage of such an embodiment is that the physician wishing to view the images always has to look away from the patient, to the screen of the navigation system. He can then no longer pay exact attention to the position in which the camera is, i.e. the exterior position from which he is obtaining the superimposed images. The system is therefore somewhat awkward and does not enable direct inspection within the part of the patient's body.

[0004] The present invention provides a device for projecting patient image data from radioscopic imaging methods and/or tomographic imaging methods onto video images, which overcomes the disadvantages of the prior art cited above. In particular, it should be made possible for the treatment staff to obtain the desired image data directly and in direct view of the patient.

[0005] In accordance with the invention, an image display device and the video camera are assigned to each other and realized as a portable unit. Using this embodiment in accordance with the invention, the physician carrying out the treatment then has the option of holding the image display device, even with the assigned camera, in front of the part of the patient's body, and therefore of reading the desired anatomical information while looking directly at the patient. It is possible in this respect to adjust the video image or the image from the radioscopic imaging method and/or tomographic imaging method to be more or less transparent, as desired, in the image shown on the image display device, in order to highlight specifically the desired information most clearly within the display as a whole. Using the device in accordance with the invention, another step is taken towards creating a “glass patient”, simply by holding the image display device in front of the relevant part of the patient's body. Since the camera assignment to the image display device is tracked via the tracking means, the position of these two elements is always known, and the navigation system has information on how and from which direction the part of the patient's body is being viewed. In particular when performing minimally invasive surgery in which the area of surgery is not exposed, the device in accordance with the invention thus offers the physician carrying out the treatment the option of again informing himself of the exact position of the parts of the patient's anatomy during the operation. In this way, tracked or navigated instruments can of course also been shown on the image display device. The physician no longer has to look away from the patient in order to look into the interior of the patient's body.

[0006] In accordance with a preferred embodiment of the invention, the video camera is arranged on the rear side of the image display device. This results in a compact apparatus; the camera can be completely removed from the operator's field of view. It should be noted in principle that the camera can be attached at any point on the rear side of the image display device, or can be integrated into it, for example even in the area of the edge of the image display device.

[0007] The video camera is preferably spatially assigned to the image display device in a way which is predetermined and/or known to the system, and the tracking means is arranged on the image display device. Spatially assigning the video camera and the image display device in a way which is predetermined and/or known to the system opens up the option of tracking just one of these two apparatus by means of the tracking means, since it is also known exactly where the other apparatus is situated. The designer is then also at liberty to arrange the tracking means either on the image display device itself or on the video camera, according to how said tracking means can best be detected by the navigation system. In optically based navigation systems, the tracking means can also be an arrangement of reflecting or actively emitting markers, although it is of course also possible within the context of the present invention to perform navigation via a magnetic tracking system in which coils are tracked in a generated magnetic field.

[0008] In order to design the image display device to be easy to handle, it can be designed as a portable LCD flat screen.

[0009] Data transmission between the individual apparatus of the device in accordance with the invention is realized in different ways. On the one hand, it is possible to communicate the image data from the radioscopic imaging method and/or tomographic imaging method to be projected onto the video images to the image display device via radio interfaces of the navigation system, while on the other hand communicating the data via data cables is also conceivable. One embodiment ought to prove particularly advantageous within the context of handling the device in accordance with the invention, in which the image display device is provided with its own energy supply (battery or power pack) and transmission is by radio, since this achieves the most extensive freedom of handling.

[0010] In accordance with a preferred embodiment, the video camera should exhibit a small aperture and a low depth of field, so that only a small area of the detected image is in the focal plane. Using such a camera, it is possible to determine the distance between the image and the focal plane. If the spatial position of the camera is known from the navigation system, and the distance from the image plane is also known, then the computer system connected to the navigation system can calculate the spatial position of the video image in real time. This enables the video image and the image data from a radioscopic imaging method and/or tomographic imaging method carried out beforehand to be optimally assigned.

[0011] In accordance with another advantageous embodiment of the device in accordance with the invention, a data transmission means is arranged on the video camera or on the image display device, wherein information on the image shown on the image display device is transmitted to the navigation system by means of said data transmission means. The image data to be projected, communicated from the navigation system to the image display device, are then adapted in size and position on the image display device such that they are in registration with the video images, i.e. a 1:1 representation of the two images in overlap is created. To this end, a contour-matching unit can be provided in the navigation system, said contour-matching unit superimposing the video images and the projected images, in particular via outer contour matching of the part of the patient's body.

[0012] It should also be noted here that it may be conceivable and advantageous within the context of the invention to attach to the portable unit many and various control apparatus and command input apparatus for altering the combined image shown. For example, the focal plane both of the radioscopic imaging method or tomographic imaging method employed and of the video image shown can be altered using such control elements. As already mentioned above, it is also possible to provide control elements which alter the respective degree of transparency of one of the images with respect to the other.

[0013] An illuminating device for the part of the patient's body is advantageously provided on the camera or in its vicinity on the image display device, especially for example as a ring light (LEDs, lamps, fluorescent tubes).

[0018] FIG. 1 schematically shows a device in accordance with the invention, using which the interior of a patient's joint may for example be inspected. The leg 1 of the patient is to be examined at the joint 5. To this end, the flat screen 10 designed in accordance with the invention is held directly in front of the joint, and an image then appears on the screen 10 containing both image information from the video camera image of the camera 14 (FIG. 2) as well as image information on the interior anatomical structure, said information being communicated for example by a navigation apparatus, by radio. To this end, a tomographic image (MR or CT image) is produced beforehand of the patient or of the patient's leg 1.

[0019] As is indicated more clearly in FIG. 2, the camera 14 is situated, firmly attached, on the rear side of the screen 10.

[0020] This type of image representation is made possible by the spatial position of the camera 14 or of the image screen 10 being tracked around the leg 1 of the patient by means of a tracking means. In the present case, this tracking means is an optical tracking means, namely an arrangement 12 of markers whose position is detected by means of the schematically represented navigation system 20. To this end, the navigation system 20 includes for example two spaced-out infrared cameras 24, 26 and an infrared light emitter 22. The three-dimensional spatial position of the arrangement 12 of markers, and therefore also of the screen 10 and the firmly installed camera 14, can be determined from the two images from the infrared cameras 24, 26. Furthermore, it is possible to determine the spatial position of the leg 1 using a corresponding arrangement 3 of markers.

[0021] If the position of the screen 10 and the camera 14 is then known, the position of the current image plane can be detected as follows. The camera 14 has, namely, a small aperture and a low depth of field, such that the image only appears in focus in a particular focal plane. This allows the position of said image plane to be determined and an image from the CT or MR imaging method to be transmitted on the screen 10, which is also exactly in said image plane. The important thing here is that the image ratio of the video image is shown at a scale of 1:1 with respect to the tomographic image. If the joint of a patient is then viewed using the device in accordance with the invention, one sees for example the contours of the patient's leg at the upper and lower edge of the screen. Using this information, the image from the tomographic imaging method can then be adapted such that it lies exactly over the video image, to ensure a realistic impression. The person viewing the screen 10 can then view on the screen both the video image, i.e. a real image, and the “interior” image from the tomographic imaging method, and in any desired depth and with any adjusted transparency of the two images. These adjustments can be made via control devices (not shown), for example flip switches on the front side of the screen.

[0022] For the first time, therefore, the device in accordance with the invention gives the observer the option of looking into the interior treatment area in real time while looking at the patient, and of planning or adapting the treatment accordingly. He no longer has to look away from the patient to a firmly installed navigation screen, which is particularly advantageous when the patient's responses to particular medical measures are to be tested. Since the “interior” images from the radioscopic imaging method and/or tomographic imaging method are formed such that they represent virtual 3-D views of the interior parts of the body, the image as a whole can be made very vivid.

[0023] Although a radioscopic and/or tomographic image is therefore not taken in situ, a virtual “glass patient” is thus created.

[0024] FIG. 3 shows the sequence of a method such as is performed using the device in accordance with the invention; the activity carried out by the computer of the navigation system is illuminated here in particular. The upper left side of the flow diagram relates to the processes in video image detection, while on the right-hand side image detection for the interior of the body is explained, using the example of MR/CT image detection. In video image detection, the focal plane is firstly determined, as has already been explained above. Then the distance from the focal plane to the camera is determined, whereupon the camera is registered by means of its tracking means in the navigation system. The determined spatial position of the camera with respect to the navigation system is then known.

[0025] With respect to MR/CT image detection, an MR/CT image is firstly detected beforehand, whereupon a three-dimensional image is reconstructed from the data obtained. Then, the object—for example, a part of the patient's body—is spatially registered, i.e. the spatial relation of the position of the object to the MR/CT images is determined. This can only be realized by spatially detecting an arrangement of markings situated on the object.

[0026] When the video image detection and MR/CT image processing steps cited above have been performed, then video/MR/CT-image matching is performed, i.e. the two images are harmonized under computer guidance, with respect to image position and image size. Then, the MR/CT/3-D images, projected onto the video images, can be superimposed on the image display device, and can assist the physician performing the treatment in diagnose and treatment.