Surface Net Smoothing for Surface Representation From Binary Sampled Data

The field of the invention pertains to computer graphics, volume graphics, and scientific visualization and more specifically to the generation of smooth surface representations for objects stored in a 3D array of binary-sampled data values.

High quality object rendering in computer graphics with shading and surface reflections requires an accurate model of object surfaces and their surface normals. However, when objects are stored in a sampled volume, as in Volume Graphics, such a description of the surface does not exist and the surface must be estimated from the sampled data. There are a number of ways to estimate surfaces from sampled data as described, for example, by A. Kaufman, in "Volume Visualization", IEEE Computer Society Press, 1991. However, according to this article, it is stated that compared with the results achieved in surface shading, volumetric shading is still rudimentary, exhibiting problems such as accuracy of the normal values, ambiguity, discretization artifacts, and aliasing. The conclusion from this article is that volume visualization still lacks a simple yet general paradigm that provides high fidelity shading for natural appearance and optimal recognition.

Volumetric data can be stored as grayscale data, where the sample values take on a range of intensities, or as binary data, where one value is given to points inside an object and a different value is given to points outside the object. Surface estimation is especially hard for binary sampled data because the intensity values change abruptly at object surfaces. These abrupt intensity changes result in high frequency components in the intensity data at object surfaces. High spatial frequencies mean that no sampling rate will be large enough to guarantee that the object and its surface normals can be accurately reconstructed. The result is jagged edges and aliasing artifacts in shaded volume rendered images.

Existing rendering methods that estimate surface normals from binary sampled data apply local low-pass filters to the data that remove high spatial frequencies and smooth out jagged binary surfaces. However, there are two problems with this approach. First, in addition to smoothing out the object surface, a low pass filter smoothes out important features such as cracks and thin protrusions. Second, a local filter is often not sufficient to achieve a smooth surface. This is the case, for example, when the surface is at a shallow angle to the sampling grid of the volume or when volumetric data is sampled more frequently within planar slices than between slice planes. In these cases, the object surface appears to be terraced, with planar surface regions separated by sudden elevation changes. Local filters can round out the elevation changes but they do not remove the terraces unless the filter size is as large as the width of the terrace. In medical data, where terraces can be tens of pixels wide, a local filter large enough to remove terraces would also unacceptably smooth out important detail from the rendered image.

An alternative to estimating surface normals from volume data during rendering is to reconstruct the surface as a polygonal model and render it using polygon rendering. The most common method for constructing a polygonal model from volumetric data is to use the Marching Cubes algorithm, described in "Marching Cubes: a High Resolution 3D Surface Construction Algorithm" in Proceedings for Siggraph'87, by W. Lorensen and H. Cline. While this algorithm is designed for grayscale data, it can be used to construct surfaces from binary data by low-pass filtering the data and then applying Marching Cubes to the filtered data. This technique is often used to create surface models from binary-segmented medical image data.

However, there are two problems with this approach. First, as stated above, the low-pass filtering will not remove terracing artifacts without removing significant detail from the data. Hence, the terracing artifacts show up in the polygonal model or the surface is unduly smoothed. Second, Marching Cubes produces models that contain a very large number of polygons. In order to further smooth out the surface and reduce the number of triangles in the model, the polygonal model can be smoothed and decimated using a variety of triangle reduction methods such as that described in "Decimation of Triangle Meshes", in Proceedings Siggraph'92, by W. Schroeder, W. Lorensen, and S. Linthicum. However, these methods are applied to the triangle model without considering the original binary data and can result in the elimination of important structural detail from the model.

A smooth representation of object surfaces is required in rendering so that objects appear smooth and free from artifacts. This is true both when the object is represented by a triangulated surface model and rendered with polygon rendering, and when the object is stored in a sampled volume and rendered with volume rendering. Unfortunately, when the object originates as a binary-sampled volume, achieving a smooth representation of the object surface is difficult. Binary-sampled volume data does not contain an explicit surface representation and hence the surface must be estimated from the data. However, existing methods for estimating the surface or surface normals from binary data, including local filtering methods and the Marching Cubes algorithm discussed above are subject to aliasing and terracing artifacts or excessive loss of detail.

Unlike existing techniques, the subject system creates a smooth surface representation from binary sampled data while faithfully representing details on the surface of the binary object. This is accomplished by fitting an elastic surface net over the surface and relaxing the net to smooth out its shape while constraining the surface net to follow details in the surface of the binary object.

More specifically, the first step in initializing an elastic surface net is to locate surface cubes in the volume, where a cube is a cubic region of space between eight neighboring volume elements, four each from two adjacent planes in the data volume. A surface cube is a cube for which at least one of the eight surrounding neighbors lies inside the cube and at least one of the neighbors lies outside of the object. Once surface cubes have been determined, the surface net is initialized by placing nodes at the center of each surface cube and linking nodes that lie in adjacent left, right, top, bottom, front, or back neighboring cubes in the data volume. This creates a so-called surface net of links between the nodes, initially at the centers of the cubes.

After the linked surface net has been created, the positions of individual node elements are sequentially moved to reduce the energy in the surface net. This is called relaxing the link net. In one instance, the energy is defined to be the sum of all of the link lengths and the surface net energy is reduced by moving elements to shorten the lengths of the links between nodes.

The movement of any node in the surface net is restricted so that it cannot move outside of the original surface cube containing that node, and it is this constraint which preserves the original structure during the smoothing process. This constrained relaxation is performed iteratively until the total energy in the surface net is minimized or until some user defined stopping criterion is met.

After the surface net has been smoothed in this manner, the result is a smooth representation of the binary object surface that is faithful to the original binary segmentation. The surface net representation can be converted into a form useful for polygon rendering by triangulating the surface net. Because of the existence of links between neighboring nodes, this triangulation simply involves adding selective additional links between diagonal elements in the surface net.

Alternatively, the surface net representation can be converted into a form useful for volume rendering. Such a form is a discrete map of distance values, as described in "Using Distance Maps for Accurate Surface Representation in Sampled Volumes", in Proceedings 1998 Symposium on Volume Visualization, by S. Gibson, where the closest distance to the surface is stored at each sample point in the distance map and the distance map is used to estimate surface normals during rendering.

These and other features of the Subject Invention will be better understood in relation to the Detailed Description in conjunction with the Drawings, of which:

FIGS. 1A, 1B, 1C and 1D are respectively photographs of image renderings before and after the application of the subject smoothing techniques, showing the elimination of aliasing and terracing artifacts for a sphere and the bones in a knee joint without sacrifice of necessary detail;

FIG. 2 is a diagrammatic representation of a generalized system for generating smooth surfaces from binary sampled data;

FIG. 3 is a flow chart of the surface net process for creating a smooth surface from binary sampled data and for generating a sampled volumetric distance map for volume rendering or for generating a polygonal model for polygon rendering;

FIG. 4A is a diagrammatic representation of a 3D voxel lattice of sampled data points and a single surface cube from the lattice;

FIG. 4B is a diagrammatic representation of a surface cube of the voxel lattice showing an internal node;

FIG. 5 is a diagrammatic representation of the initial configuration of a 2D surface net that has been fit around a 2D binary-sampled object;

FIGS. 6A-6C are diagrammatic representations of the progressive relaxation of a 2D surface net around a portion of a 2D binary sampled object through node movement.

FIG. 7 is a diagrammatic representation of a relaxed surface net around a 2D object, illustrating the application of constraints to node positions;

FIGS. 8A and 8B are diagrammatic representations of the relaxation of a portion of a 3D surface net from an initial surface net to a relaxed surface net;

FIGS. 9A and 9B are diagrammatic representations of the triangulation of a surface net through the provision of new links as shown in FIG. 9B for the triangle surface net of FIG. 9A; and,

FIG. 10 is a diagrammatic representation of the calculation of a distance map from a triangulated surface net.

Referring now to FIGS. 1A-1D, what can be seen is that the aliasing terracing artifacts associated with prior methods of volume rendering objects stored as binary-sampled data is severe, whereas the smoothing associated with the subject technique not only smoothes the surfaces, but also, as can be seen in FIG. 1D, preserves the original surface detail. As will be seen, this is accomplished by first providing a surface net and then relaxing the net, but only insofar as certain constraints put in place to preserve detailed surface structure are not violated. In one embodiment the constraint is that a node within a surface cube cannot be moved outside the cube during the relaxation process.

Referring now to FIG. 2, the subject invention comprises a method for using a computer 10 to convert binary volumetric data 12 into a smooth surface representation that can then be displayed using either polygonal rendering or volume rendering on a display device 14.

How this is accomplished in one embodiment is as follows: Referring to the block diagram of FIG. 3 the surface net approach comprises: a procedure 16 for detecting surface cubes in the sampled binary data, where surface cubes are such that the at least one of the eight bounding volume elements is inside the object and at least one of the 8 bounding volume elements is outside of the object; a procedure 18 which places surface nodes at the centers of the detected surface cubes and creates links between adjacent surface nodes; a procedure 20 for relaxing the surface net, where at 22 net node positions are adjusted to reduce the energy in the net by the series of steps depicted at 24, iteratively reducing the link lengths under the constraint that net nodes must remain in their original surface cubes until, as illustrated at 26, the energy is minimized or some other stopping criteria is reached. If at 28 the rendering method is volume rendering as illulstrated at 30, then a procedure 32 triangulates the surface net and calculates the distance from each point in the distance map to the closest point on the triangulated surface and a procedure 34 volume renders the object using the distance map to estimate surface normals. If, as illustrated at 28, the rendering method is polygon rendering 36, then a procedure 38 triangulates the surface net and decimates the triangulated model and a procedure 40 renders the polygonal model.

Referring to FIG. 4A, in one representative example, the volumetric object consists of volume elements 42 arranged in a 3D lattice, of which 2 layers are shown at 44. A cube 46 is the space bounded by 8 neighboring volume elements, 4 each from two adjacent layers in the lattice. As shown in FIG. 4B, a surface cube 48 is a cube that contains at least one element 50 that is inside of the object and at least one element 52 that is outside of the object. A surface node 54 is initially placed at the center of each surface cube in the 3D lattice.

Referring to FIG. 5, a 2D instance of the surface net shows the nodes 60 and 62 of the surface net, connected by a link 64. The links separate those nodes which are interior nodes of the object from those which are exterior nodes. Here link 64 separates interior node 68 from exterior node 70.

Referring to FIG. 6A, a 2D instance of a portion of a surface net shows the interior points 80 exterior points 82, the initial surface net 84 and motions of net nodes 86 and 88 towards lower energy positions 90 and 92.

Referring to FIG. 5B, the surface net 94 has been relaxed if from its position in FIG. 6A to a lower energy state and a smoother shape. Referring to FIG. 6C, the surface net 96 has been further relaxed to a smooth shape.

Referring to FIG. 7, a 2D instance of the surface net shows portions 100, 102 and 104 of the surface net, where the constraint that nodes in the surface net can not move outside of their original surface cube has limited the smoothing of the object surface.

Referring to FIG. 8A, a 3D surface net is initialized so that nodes 110 lie at the centers of surface cubes 112 and links 114 join nodes in adjacent surface cubes. In FIG. 8B surface net 116 is relaxed to reduce the energy in the net by reducing the link lengths, producing a smoother surface net.

Referring to FIG. 9A, the surface net is a 3D structures consisting of surface nodes 120 and links 122. Referring to FIG. 9B, the 3D surface is triangulated by joining selective diagonal elements 124 in the surface net to create a mesh of triangular elements.

Referring to FIG. 10, values for elements 130 and 132 in the discrete distance map, are set to the minimum distances 134 and 136 respectively from the element to triangular elements 138 and 140 respectively in surface 142.

There are several possible methods for defining the energy in the surface net and for relaxing the net to a low energy state. One such method is to define the energy as the sum of the lengths of the links between nodes in the surface net: ##EQU1## where (x.sub.1, y.sub.1, z.sub.1) and (x.sub.2, y.sub.2, z.sub.2) are the two nodes joined by the link, and .dbd..dbd. v .dbd..dbd. indicates the magnitude of the vector, v. In this case, the energy is reduced by reducing the lengths of all of the links in the net. One way to achieve at least a local minimum of this energy is to iteratively reduce the link lengths by sequentially moving each node to a position midway between its neighboring nodes. For example, move a node n to the point: ##EQU2## where N.sub.n is the number of neighboring links for node n.

Other methods include minimizing the curvature of the surface or a combination of minimizing surface curvature and the link length.

A program listing in C is presented hereinafter to illustrate the utilization of the subject system for use in generating a distance map for volume rendering from a binary object representation using a surface net to generate a smooth representation of the object surface.

Having above indicated several embodiments of the Subject Invention, it will occur to those skilled in the art that modifications and alternatives can be practices within the spirit of the invention. It is accordingly intended to define the scope of the invention only as indicated in the following claims.