Three-dimensional image display device

[0001] The present invention relates to a three-dimensional image display apparatus; and, more particularly, the invention relates to a three-dimensional image display apparatus for constructing and displaying a three-dimensional image, using the volume rendering method to shade a plurality of tomograms obtained by a medical imaging apparatus, such as an X-ray CT apparatus or an MRI apparatus.

[0002] In accordance with conventional techniques, it is well known that, by using an X-ray CT apparatus, an MRI apparatus, or an ultrasonic apparatus, a tomogram of an object to be examined can be obtained. Also, the volume rendering method, the surface rendering method, the depth method and the like are known methods of constructing a three-dimensional image (3D image) by shading, on a projection plane, a volume image that has been constructed by aligning a plurality of such tomograms.

[0003] Incidentally, the volume rendering method is mentioned in the document “Marc Levoy; “Display of Surfaces from Volume Data”, IEEE Computer Graphics & Applications, pp29–37, 1988.”

[0004] However, it is substantially impossible to emphasize the display of a particular portion of a 3D image that is constructed by the conventional volume rendering method so as to easily distinguish it from other portions. For example, it has been impossible to emphasize display of a particular lesion area so as to easily find it in a diagnostic examination, or to emphasize display of a stent graft in order to explain to a patient how it has been inserted in a blood vessel during surgery. Even if the particular lesion area or the stent graft is provided with colors, they are not displayed distinctively from other portions.

[0005] The object of the present invention is to provide a three-dimensional image display apparatus that can construct and display a three-dimensional image where a particular portion is emphasized in a display by using the volume rendering method.

[0006] To achieve the above-stated object, a three-dimensional image display apparatus according to the present invention projects a volume image that is constructed by aligning a plurality of tomograms on a predetermined projection plane using the volume rendering method, and displays a three-dimensional image using the volume rendering method where light beams are radiated from a virtual light source that is set outside the volume image in an algorithm for shading, and pixel values of the projection plane are calculated based on the reflectivity and the density gradient of the light beams at the respective pixels of the volume image. The three-dimensional image display apparatus includes luminescence ratio allocating means for allocating a greater luminescence ratio to the light beams applied to the pixels corresponding to a particular object to be emphasized in the display over other objects in the volume image, and calculating means for adding the luminescence ratio allocated by the luminescence ratio allocating means as a parameter of the shading calculation in the volume rendering method and for calculating pixel values on the projection plane, wherein the particular object to which the greater luminescence ratio is allocated is emphasized in display.

[0007] Further, the luminescence ratio allocating means has light emission patterns matching lesions in each portion of the body to be examined, in which specific luminescence ratios are allocated to density values of pixels representing the lesions, and the luminescence ratio corresponding to a lesion of the portion is selected by selecting the lesion of the portion to be emphasized in the display.

[0008] That is, when a particular lesion area is chosen to be emphasized in the display, only a light emission pattern corresponding to the lesion is selected, and, if there is a lesion, the lesion area is displayed luminously (becomes bright).

[0009] Further, a three-dimensional image display apparatus projects a volume image constructed by aligning a plurality of tomograms with shading on a predetermined projection plane and thus displays a three-dimensional image, using the volume rendering method, where, in an algorithm for shading, light beams are radiated from a virtual light source set outside the volume image and pixel values on the projection plane are calculated based on the reflectivity and the density gradient of the light beam at the pixels of the projection plane. The three-dimensional image display apparatus includes color luminescence ratio allocating means for allocating a color luminescence ratio to pixels showing a particular object to be emphasized in the display among all the objects to be displayed, and calculating means for adding the color luminescence ratio allocated by the color luminescence ratio allocating means as a parameter for shading calculation in the volume rendering method and thus calculating the pixel values on the projection plane, whereby the particular object to which the color luminescence ratio is allocated is emphasized in the display.

[0010] That is, when a particular lesion area is chosen to be emphasized in the display, only the color luminescence ratio corresponding to the lesion is selected, and, if there is such a lesion, that lesion area is displayed with a color.

[0020] Various embodiments of the three-dimensional image display apparatus according to the invention will be described with reference to the drawings.

[0021] First, a system of constructing a 3D image by the volume rendering method according to the invention will be described. Generally, in the volume rendering method, each pixel of a volume image constructed by aligning a plurality of tomograms, as shown in FIG. 1, is referred to as a voxel having a cubic shape. Therefore, a volume image is an image constructed by three-dimensionally aligning the voxels. The data of a voxel is the measured CT value calculated on the basis of the X-ray absorption at the portion of a human body corresponding to that voxel when an X-ray CT apparatus is used, and it is based on the measured proton density when an MRI apparatus is used.

[0022] Parameters for shading, in the volume rendering method, include the opacity of light set according to the CT values and the gradient of contour lines of the CT values (hereinafter referred to as a density gradient).

[0023] In the surface rendering method, a suitable threshold CT value condition is set, and those pixels meeting this condition are extracted. On the other hand, in the volume rendering method, pixels with CT values which indicate a suitably determined degree of opacity are extracted. By use of either rendering method a desired object, such as bone and skin, can be exclusively extracted, and the pixel values of the object can be used as a parameter for shading on the projection plane.

[0024] That is, the virtual light beams radiated by the virtual light source, as shown in FIG. 1, which pass through the respective voxels of the volume image, are attenuated according to the opacity determined by the CT values at the respective voxels. Consequentially, the size of reflected lights at the respective voxels can be measured. In the shading algorithm according to the volume rendering method, a voxel becomes brighter as the size of a reflected light becomes larger. Further, it is assumed that a virtual surface exists where the CT value of a voxel greatly changes, and the voxel is made brighter as the virtual surface faces more to the front, that is, as the concentration gradient is larger.

[0025] The sum of brightness values of voxels along the line from the viewpoint toward the depth direction until the virtual light beam is attenuated to 0 is regarded as the value representing the brightness of the respective pixels on the projection plane (hereinafter referred to as pixel value), and the pixel values of all pixels on the projection plane are calculated.

[0026] According to this invention, a pixel value on the projection plane is calculated in accordance with the formula 1 so as to emphatically display the particular portion:

• • [0027] Pixel value on the projection plane $\begin{array}{cc}\begin{array}{c}\begin{array}{c}\mathrm{Pixel}\mathrm{value}\mathrm{on}\mathrm{the}\\ \mathrm{projection}\mathrm{plane}\end{array}=\mathrm{fixed}\mathrm{value}\times \sum _{j=0}^k\xi \left(Vj\right)·{\Delta }^{\delta \left(Vj\right)}·\\ \left\{1-\sum _{j=0}^{j-1}[\xi \left(Vj\right)+\eta \left(Vj\right)]\right\}\end{array}& \left(\mathrm{formula}1\right)\end{array}$

[0028] where V: CT value, density value;

[0029] ξ: reflectivity;

[0030] η: luminescence ratio (negative absorption ratio);

[0031] Δ: concentration gradient; and

[0032] δ(V): actual number substantially in the range from 0.1 to 6.0, a function of V.

[0033] Intensity of incident light beam is assumed to be 1.

[0034] As shown by the formula 1, the pixel value on the projection plane is the information acquired by summing the brightness value of each voxel (j=0 to k) along the path of a virtual light beam. Here, k is the number of voxels that the virtual light beam passes through while being reflected and attenuated at each voxel until the beam intensity becomes 0 (that is, the value within { } of the formula 1 becomes 0).

[0035] According to the invention, the luminescence ratio η(Vj) having a negative value is added to two parameters within [ ] in the formula 1. That is, although the virtual light beam is generally attenuated at each pass through a voxel and is not amplified, according to the invention, a luminescence ratio proportional to the intensity of the incident light beam is allocated to the voxels of the portion chosen to be emphasized in the display, and, consequentially, the intensity of the incident light beams is amplified at the voxels of the portion chosen to be emphasized in the display.

[0036] FIG. 2 shows one example of the reflectivity pattern A, where the reflectivity ξ is allocated according to the CT value of the voxels, and the reflectivity pattern B, where the luminescence ratio η is allocated thereto. Incidentally, the luminescence ratio η has a negative value and increases the value of { } in the formula 1 (that is, the intensity of the incident light beam). Further, in FIG. 2, C represents a pattern of the absorption ratio, and the luminescence ratios as defined above can be made similar to the negative absorption ratio of the virtual light beam.

[0037] FIG. 3 shows an example of the display monitor of a CRT display for displaying a 3D image and the like that has been constructed on the basis of formula 1.

[0038] As shown in the figure, at a reflectivity allocating area 1, ◯ marks are moved with a mouse to allocate a desired reflectivity to CT values. Similarly, at a luminescence ratio allocating area 2, □ marks are moved with the mouse to allocate a desired luminescence ratio to CT values. The pixel values on the projection plane are calculated from the formula 1, based on the thus-allocated reflectivity and luminescence ratio, and a 3D image 3 is constructed and displayed.

[0039] Also, at the luminescence ratio allocating area 2, a lesion area of the liver, cartilage, bone, and the like can be selected by the mouse. When a desired lesion area is selected from among such lesion areas, a luminescence ratio pattern for emphasizing the display of the lesion area is also set automatically.

[0040] Incidentally, to shift the point of view of the 3D image 3 within a cube, the white circles 4 on the corners of the cube are dragged by the mouse and the cube is rotated suitably with respect to the center point of the cube. Consequentially, the direction of the sight line of the 3D image 3 is determined, and a 3D image is newly constructed and displayed. The thus-constructed 3D image is stored in a magnetic disk when needed. Further, in FIG. 3, the reference number 5 represents a tomogram (CT image).

[0041] FIG. 4(a) shows a 3D image 10 of a head constructed by setting a luminescence ratio to CT values corresponding to the brain, and FIG. 4(b) shows a 3D image 12 of head constructed without setting a luminescence ratio (luminescence ratios of all CT values are 0). The brain (particularly the eyeball portion 11), as shown in FIG. 4(a) is brightly displayed, and the brain (particularly the eyeball portion 13), as shown in FIG. 4(b), is darkly displayed.

[0042] Further, FIG. 5(a) shows a 3D image 14 of a blood vessel constructed by setting a luminescence ratio to CT values corresponding to a stent graft 15, and FIG. 4(b) shows a 3D image 16 of a blood vessel constructed without setting a luminescence ratio. The stent graft 15 shown in FIG. 5(a) is brightly displayed, and the stent graft 17 shown in FIG. 5(b) is darkly displayed.

[0043] FIG. 6 is a flow chart showing the procedure employed in displaying a 3D image according to the invention.

[0044] [Step 50]

[0045] Reflectivity, luminescence ratios, and initial values are set.

[0046] [Step 52]

[0047] Initial values in the direction of the sight line (the direction of the virtual light beam) are set.

[0048] [Step 54]

[0049] Pixel values on the projection plane are calculated based on the formula 1, and a 3D image is constructed and displayed. To display the 3D image on the CRT display, values in the projection plane memory are converted with reference to a table for adjusting values to fit the display tone and then they are written in a display memory, and the 3D image is thus displayed.

[0050] [Step 56, 58, 60]

[0051] It is judged whether the sight line direction, the reflectivity, or the luminescence ratio is changed or not. If any of them is changed, the procedure jumps to the step 54 and the 3D image is again constructed and displayed.

[0052] [Step 62]

[0053] It is judged whether the 3D image displayed at this point is stored or not. If there is an instruction to store it, the procedure jumps to step 64. If not, the procedure goes to step 66.

[0054] [Step 64]

[0055] The 3D image displayed at this point is stored in a magnetic disk.

[0056] [Step 66]

[0057] It is judged whether there is an instruction to end the 3D image display or the like. If there is no instruction to end processing, the procedure jumps to step 56. If there is an instruction to end, the processing operation is ended.

[0058] Further, as shown in FIG. 7, the portion which is made to emit light may be a particular region of interest. This region of interest is determined by the distance between the projection plane and the portion emitting light, as shown in FIG. 7(a). The region of interest is set when the operator encloses the desired region in the displayed image using a mouse, as shown in FIG. 7(b), and the setting in the depth direction is made when the operator suitably sets the distance between the desired light-emitting portion and the projection plane.

[0059] Further, although the above description is directed mainly to monochrome display, it goes without saying that the invention can be applied to color display as well. In this case, the color display can be realized by converting the luminescence ratio in the monochrome image into a color luminescence ratio in the color image.

[0060] More specifically, as shown in FIG. 8, color components of red (R), green (G), and blue (B) can be separately emitted. The horizontal axis represents CT values. For example, when the brain is chosen to emit red light, the light reflectivity (opacity) of the components G and B in the range of CT values showing the brain is controlled to be low by mouse operation, and thus the luminescence ratio of the component R becomes larger. Consequentially, the brain emits red light. Incidentally, this colored light emission is just an example, and it goes without saying that colors that do not depend on a coloring system determined by RGB are also included.

[0061] FIG. 9 is a block diagram showing an example of the hardware configuration of the three-dimensional image display apparatus according to the invention. This three-dimensional image display apparatus constructs and displays a 3D image based on the above-mentioned formula 1 from a plurality of tomograms of a portion of the object obtained by an X-ray CT apparatus. For example, the apparatus includes a central processing unit (CPU) 20 for controlling operation of each component; a main memory 22 where control programs of the apparatus are stored; a magnetic disk 24 where data of a plurality of tomograms and programs are stored; a display memory 26 for temporarily storing image data for display; a CRT monitor 28 serving as a display device for displaying an image based on the image data received from the display memory 26; a mouse 30 serving as a position inputting device; a mouse controller 32 for detecting the state of the mouse 30 and for outputting to the CPU 20 signals indicating the position of the mouse pointer on the CRT monitor 18 and the state of the mouse 30; a keyboard 34 for inputting instructions to conduct various operations; and a common bus 26 for connecting each of the above components.

[0062] Incidentally, in this embodiment, the pixel values on the projection plane are calculated, based on the above-mentioned formula 1, to construct a 3D image; however, the invention is not limited thereto, and the pixel values on the projection plane also may be calculated by the following formula 2 and formula 3: $\begin{array}{cc}\begin{array}{c}\begin{array}{c}\mathrm{Pixel}\mathrm{value}\mathrm{on}\mathrm{the}\\ \mathrm{projection}\mathrm{plane}\end{array}=\mathrm{fixed}\mathrm{value}\times \sum _{j=0}^k[\xi \left(Vj\right)+C·\eta \left(Vj\right)]·\\ {\Delta }^{\delta \left(\mathrm{Vj}\right)}·\left\{1-\sum _{j=0}^{j-1}[\xi \left(Vj\right)+\eta \left(Vj\right)]\right\}\end{array}& \left(\mathrm{formula}2\right)\end{array}$
where C is a constant; $\begin{array}{cc}\begin{array}{c}\begin{array}{c}\mathrm{Pixel}\mathrm{value}\mathrm{on}\mathrm{the}\\ \mathrm{projection}\mathrm{plane}\end{array}=\mathrm{fixed}\mathrm{value}\times \sum _{j=0}^k\xi \left(Vj\right)·{\Delta }^{\delta \left(\mathrm{Vj}\right)}·\\ \sum _{j=0}^{j-1}\left\{1-[\xi \left(Vj\right)+\eta \left(Vj\right)]\right\}\end{array}& \left(\mathrm{formula}3\right)\end{array}$
These formulas are added to increase the reflectivity and for image enhancement. Further, according to the formula 3, the incident intensity is multiplied and then attenuated. Since the attenuation of the incident intensity is slower than that in the formula 1, a 3D image of portion that is located further inside the body can be constructed.

[0063] Further, according to the above-described invention, the luminescence ratio and the color to be emitted are respectively allocated; however, it is also included in the invention that they are combined and a particular portion or stent graft is emphasized in the display as a result of such combinations.

[0064] As described above, according to the invention a three-dimensional image can be displayed where a particular portion is emphasized in the display by use of the volume rendering method, whereby the invention has the effect that a lesion area can be easily found in a health examination, and the like.