Fuzzy Method and Device for the Recognition of Geometric Shapes in Images

1. Field of the Invention

The present invention relates to a fuzzy method and device for the recognition of geometric shapes in images. The fields of application of this method and of this device range from automatic guidance and navigation systems to image understanding systems.

2. Discussion of the Related Art

The primary goal of vision systems is to recognize objects in an image. However, before the object can be recognized it is necessary to recognize the characteristic lines that compose it and provide a symbolic representation of the object. One way to achieve these goals is to use the Hough transform, which is capable of performing a mapping from structures of the feature space to structures of an appropriate parameter space.

Straight lines can be considered as an example of this Hough transform also because they are undoubtedly the lines that most frequently occur in the description of the shape of objects. It is thus evident that providing a method capable of identifying and describing these lines, which so often compose an object, is extremely important.

The Hough transform, introduced in 1962, was in fact developed to solve this problem and generalized only at a later time so as to consider other types of parameter lines.

Consider a point that has the coordinates (x.sub.i, y.sub.i) and belongs to a generic straight line defined by the equation y=ax+b. It is evident that an infinite number of lines pass through this point; however, all such straight lines must satisfy the equation y.sub.i =ax+b for different values of a and b.

Therefore, by writing this last equation in the form b=y.sub.i -ax.sub.i and considering the plane ab (parameter plane) it is possible to obtain the equation of a single straight line at a point having preset coordinates.

The same line of reasoning can of course be followed for another point that has the coordinates (x.sub.j, y.sub.j); a new straight line associated with this point will be obtained in the parameter space. The intersection of the two straight lines generated in the parameter plane allows one to obtain the parametric coordinates (a,b) of the straight line that connects the two points of the feature plane. FIGS. 1A and 1B illustrate this graphically.

The parametric equation for straight lines proposed above has some severe drawbacks: even for limited portions of the feature space, the domain of values of both parameters a and b can be the entire set of real numbers (for example, in vertical straight lines a=.infin.).

In order to obviate this drawback, the following representation is usually adopted instead:

where p is the distance of the straight line from the origin and .theta. is the angle formed by the axis x and by the normal to the straight line, as shown in FIG. 2.

By using this representation, each point of the feature space is matched by a sinusoid in the parameter space. Clearly the parameter space is limited, since p is within the range ›-R.sqroot.2, R.sqroot.2! for images with a linear dimension R and .theta. can be limited between 0.degree. and 180.degree., since 180.degree. translations in .theta. and an optional reversal of sign in p allow to return to the range ›0.degree., 180.degree.!.

Accordingly, if one has a set of n points ›(X1, X2), . . . ,(Xn,Yn)! that belong to a given straight line in the feature plane and one wishes to determine the parameters that describe this straight line, it is necessary to transform the points (Xi, Yi) into sinusoidal curves in the plane p-.theta., which are described by p=Xi cos.theta.+Yi sin.theta., and to note that these curves all intersect at the parametric values of the straight line being sought, as shown in FIGS. 3A and 3B. In other words, by transforming co-linear pixels in the feature plane one obtains several sinusoids that intersect in a single point that has the coordinates (.theta..sub.0,p.sub.0) and corresponds to the straight line on which said points lie. Accordingly, the problem of finding collinear points in the feature can be converted into the problem of finding concurrent curves.

The computational advantage of the Hough transform arises from dividing the parameter space into cells, that is to say, from introducing an acceptable error in p and .theta. in the quantization of the parameters in a grid. Quantization can be reduced to the region 0.ltoreq..theta..ltoreq..pi. and -R.ltoreq.p.ltoreq.R, where R is the maximum distance that a point of the feature plane can have from the center of the image (in practice, half of a diagonal). The quantized region can thus be reduced to a two-dimensional array of accumulators. For each point (Xi, Yi) in the feature plane, the corresponding curve is mapped onto the array, increasing the accumulator in each cell that lies along the curve. Accordingly, each cell of the parameter space stores the number of curves that have passed through it, minus the noise and the discretization that is carried out.

After transforming all the points of the edge, the problem is to find the local maximums in the array of accumulators. These local maximums must be higher than a certain threshold in order to extract only lines that are long enough, and most of all in order to eliminate noise (false maximums). It is therefore necessary to find a threshold that varies in each situation in the most intelligent possible manner according to the image to be analyzed.

After extracting the local maximums from the array of accumulators it is possible to draw the lines that they have generated. In other words, the inverse transform is applied, reconstructing the corresponding straight line for each maximum point (.theta., p) by using the following equation: ##EQU1## It should be noted that the Hough transform can also be used for other geometric shapes, such as curves, circles, etcetera.

However, the transformation performed with the Hough transform is sensitive to the noise that is present in the image and depends on the performance of the edge detection system that is used. Another problem in the use of the Hough transform is the recovery of points lost due to discretizations and noise.

A problem of vision systems in general is processing speed. Known systems have a very high level of processing complexity and have processing speeds that are not satisfactory for the required applications, (for example, theoretically real-time processing). Another problem of known vision systems is the complexity of the hardware.

Accordingly, an aim of the present invention is to provide a fuzzy method and device for the recognition of geometric shapes in images that gives greater robustness to the standard Hough transform. Within the scope of the above aim, an object of the present invention is to provide a device that is less complex than known systems. Another object of the present invention is to provide a method and a device that allow faster processing than known systems, theoretically in real time.

This aim, these objects, and others which will become apparent hereinafter are achieved by a fuzzy method for the recognition of geometric shapes in images, comprising the following steps: detection of at least one edge of a curve of an image; recognition of the gradient of said edge; correction of said gradient by virtue of first fuzzy means; and determination of the number of points that belong to said curve by virtue of second fuzzy means.

According to an illustrative embodiment of present invention, the method is performed by a fuzzy device that recognizes geometric shapes in images, comprising an edge detector, adapted to detect at least one edge of a curve of an image and to recognize the gradient of said edge; a fuzzy gradient corrector, adapted to correct said gradient; and a fuzzy vote attributor, adapted to determine the number of points that belong to said curve.

The characteristics and advantages of the present invention will become apparent from the description of a preferred but not exclusive embodiment, illustrated only by way of non-limitative example in the accompanying wherein:

FIGS. 1A and 1B are graphical representations of a normal parameterization of a straight line;

FIG. 2 is a graphical representation of a normal parameterization of a straight line with polar coordinates;

FIGS. 3A and 3B are plots of a quantization of the parameters p and .theta. of FIG. 2;

FIG. 4 is a block diagram of a device according to the present invention;

FIG. 5 is a block diagram of a device according to an embodiment of the present invention, adapted for recognizing geometric shapes formed by line segments;

FIG. 6 is a block diagram of the interfaces of a device according to an embodiment of the present invention;

FIG. 7 is a block diagram of a fuzzy gradient corrector;

FIG. 8 is a block diagram of a fuzzy vote attributor of a device according to an embodiment of the present invention

FIGS. 9A and 9B are graphical representations of a mask used to correct the gradient;

FIGS. 10A and 10B are diagrams of the fuzzy sets for the antecedents of the rules used by the fuzzy means of the fuzzy gradient corrector;

FIG. 11 shows the calculation of a rule by fuzzy means;

FIG. 12 is a view of an example of points that belong to a straight line;

FIG. 13 is a view of a fuzzy transformation for a straight line in the parameter space, performed by the fuzzy means of the fuzzy vote attributor .

With reference to FIG. 4, a device according to the present invention comprises an A/D converter 1 adapted to sample the image signal IN and convert it into a sequence of digital data. The image signal is scanned by consecutive lines (raster scanning). The AID converter 1 is not necessary if the image signal IN is already in digital form.

The output of the A/D converter 1 is connected to a first interface 2 comprising memory elements that are adapted to create image portions, defined as sub-images.

The output of the first interface 2, that is to say, the pixels of the sub-images, is input to an edge detector 3 that is adapted to detect the edges that are present in said sub-images. The size of said sub-images depends on the type of edge detector that is used. An important aspect of the present invention is that the output of the edge detector 3 is determined by information on the gradient GRAD at the identified edge. This allows fewer calculations than the standard Hough transform.

The data on the gradient GRAD is sent to a second interface 4, which also comprises a number of memory elements, the number depending on the particular implementation of the system. The second interface is adapted to select a portion of the GRAD gradient image that is associated with the input image IN.

Said image portion is the input of a fuzzy gradient corrector 5. Said corrector 5 is adapted to make fuzzy inferences starting from the structure of the data in the neighborhood of the edge point to be classified, and is capable of correcting a stream of data affected by the intrinsic inaccuracy of the edge detector 3 and by noise. The fuzzy gradient corrector 5 produces as output the parameters PARAM related to the parametric representation of the curve being sought, after establishing a reference system within the feature plane.

The parameters PARAM are then sent to a fuzzy vote attributor 6 adapted to determine the number of points that belong to a given curve. The output of the fuzzy vote attributor 6 comprises the parameters OUTPARAM that characterize the curve and the number of points N belonging thereto.

Finally, all the blocks that compose the device are synchronized by a clock 7 that is adapted to generate a timing signal CLOCK.

A simple embodiment of a device according to the present invention is one adapted to identify straight lines, line segments, and geometric shapes formed by line segments, and is shown in FIG. 5. In this case, the parameters PARAM related to the parametric representation of the straight line are the radius R and the phase F, if a polar coordinate system is used, and are generated by the fuzzy gradient corrector 5.

The fuzzy vote attributor 6 generates the output of the device, that includes the line segment (determined by R.sub.out and F.sub.out) and the length N thereof.

With reference to this implementation of the device according to the present invention, the internal architecture of each block will now be elaborated.

The A/D converter 1 will not be explained in detail, since it is assumed to be known.

The first interface 2 comprises a plurality of line and pixel memory elements. In the particular case of a 3.times.3 work window required by the edge detector 3, the architecture of the interface 2 is shown in FIG. 6.

Assume that the 3.times.3 window is designated by columns A, B, and C, and that the rows of the window are designated by I, II, and III. In a raster scan, the pixels are sent to the input I of the interface 2 in the following sequence: AI, BI, CI, AII, BII, CII, AIII, BIII, and CIII. The first pixel AI is stored in the first line memory 11. Then said pixel is transferred to the second line memory 12, while the subsequent pixel BI is stored in the first line memory 11. When the third pixel CI is sent, the first three pixels AI, BI, and CI are transferred into the respective first delay memories 13, 14, and 15. This sequence is then repeated with the second three pixels AII, BII, and CII, while the first three pixels are transferred into the respective second delay memories 16, 17, and 18. When the third three pixels AIII, BIII, and CIII are sent, the outputs of the interface 2 are as shown in FIG. 6.

The window ›AI..CIII! is sent to the edge detector 3. The edge detector 3 divides an image into uniform regions according to any principle and separates these regions with lines that form their borders. The edge detector reliably identifies these borders, known as edges. As a first hypothesis, one may consider an image as a continuous function with two variables that establishes a relationship between each point in the plane and a third value, which is generally light intensity.

Seeking the edges of regions therefore entails identifying the areas where this function varies more rapidly, that is to say, where the value of the derivative is higher. Indeed because of this fact, one of the categories of segmentation methods is known in the literature as "differential operators". It can in fact be easily demonstrated that in one dimension, the first derivative is a simple and fast method for pointing out sudden transitions of a function.

In order to locate the exact point that separates two regions, one proposed method consists of making said point coincide with the zero value of the second derivative. However, it should be noted that the differentiation operation is not robust with respect to noise. This fact means that a small variation in light intensity leads to a large value of the first derivative and correspondingly to a larger effect on the second derivative. Therefore, a threshold on the value of the first derivative is introduced in order to select only significant transitions.

The above described situation for single-variable functions can obviously be applied to the two-dimensional case without significant variations: in the two-dimensional case, the gradient and the second derivative of the intensity function, calculated in the direction of the gradient, are considered. However, images are discrete functions, not continuous functions, and therefore it is necessary to perform appropriate approximations of the differential operators; these approximations are produced by means of discrete convolutions with appropriate masks. These masks differ in terms of dimension and weight, but they are all based on an approximation of the incremental ratio: ##EQU2## The partial derivative (in the direction x or y) calculated at one point, however, is obtained from a weighted average of a set of incremental ratios in order to limit the noise that is assumed to be present, especially in the frequency spectrum part of the signal being processed. A mask (window) measuring 3.times.3 has been used in the present case, and this has resulted in good approximations of the gradient without excessively affecting the computation time. The following operators are used (they are Sobel operators, but others may be used): ##STR1##

Regardless of the operator that is used, the modulus of the gradient is calculated by means of the formula:

This formula, too, is computationally intensive and is therefore approximated as follows:

Actually, since the use of a single threshold has been shown to generate jagged edges, consideration was given to using a heuristically-defined threshold in the algorithm. Ultimately, a solution of hysteresis thresholding was adopted:

if the weight of an edge point is greater than the upper threshold, the point is considered valid;

if instead the weight of an edge point is smaller than the upper threshold and greater than the lower threshold, the point is considered valid only if at least one of the neighboring points fits the previous case; otherwise the edge point is discarded.

When the pixel is classified as an edge pixel, the gradient GRAD is calculated as: ##EQU3##

Alternatively, it is possible to use the Canny algorithm, which filters the image with a Gaussian derivative operator and detects the edges that are present by detecting the gradient maximums. It is important to filter the image with a low-pass filter, since differentiation is highly noise-sensitive.

The Canny recognition algorithm, too, in addition to the edge detection step, includes a post-processing procedure which consists of hysteresis thresholding.

Both the Sobel and the Canny recognition methods have positive and negative aspects, and therefore one method or the other is chosen according to the characteristics of the images to be analyzed. The Sobel algorithm is very fast, due to its considerable structural simplicity, but its behavior is not very reliable in the case of particularly complex and noisy images. In contrast, the Canny algorithm combines lower processing speed with good reliability as to the existence of the detected edge and accurate localization (meaning that the points of the image that the detection operator determines are very close to the real edge).

The value of the gradient GRAD thus determined by means of one of these two algorithms, or in any other manner, is sent to the second interface 4, which selects a portion of the gradient image associated with the input image.

The structure of the second interface 4 is identical to the structure of the first interface 2 and will not be explained in detail, since there may be a different number of memory elements depending on the size of the mask. It is assumed to be 3.times.3 by way of example.

The portion of the gradient image is sent to the fuzzy gradient corrector 5 shown in FIG. 7. The corrector 5 comprises a differential block 21 that receives as input the 3.times.3 window of the gradient image from the second interface 4. The differential block 21 is adapted to calculate the differences between the gradient at the central point of the window being used and the neighboring pixels inside said window. The 3.times.3 mask as shown in FIG. 9 has been used for this purpose.

The reference symbols P1, P2, . . . , P8 designate the pixels in the positions shown in FIGS. 9A and 9B, the values of the gradient at each one of these points are obtained. These values are used to obtain the differences D1, D2, . . . , D8 between the central pixel and the gradient in the n-th position.

These differences Dn are sent to fuzzy means 22, that makes inferences on these differences, so as to determine the correct value of the gradient (or phase).

The differences Dn are used to obtain the antecedent of the fuzzy rules being applied. The Negative High, Negative Low, Positive High, and Positive Low membership functions are defined as shown in FIGS. 10A and 10B. Accordingly, there are two fuzzy sets for the positive differences and two sets for the negative differences. The input variables for each rule are the differences Dn between the central pixel and one of the neighboring pixels.

FIG. 11 shows how the output of a rule is determined with the MAX-DOT method. The rule can be the following:

IF D1 IS PL AND D2 IS PL THEN OUT IS HIGH

The MAX-DOT method has been chosen to determine the output fuzzy set due to computational reasons. When a fixed reference system is assigned for all the fuzzy sets that are consequents of the rules, the output of the inference process is calculated by means of the formula: ##EQU4## where A is the area of the output fuzzy set and C is the centroid of the output fuzzy set.

The above formula can be simplified if one considers that each rule produces a triangular output fuzzy set having a constant base b.sub.i and an altitude h.sub.i that is equal to the activation level: ##EQU5## where h is the activation level of the i-th rule and C.sub.i is the centroid of the i-th output fuzzy set.

In this manner, in order to calculate the output of the fuzzy process it is sufficient to assign the centroid C.sub.i that corresponds to the output fuzzy set of the rule, avoiding the calculation of the area.

The rules that are applied are of the following kind:

IF D1 IS POSITIVE LOW AND D2 IS POSITIVE LOW THEN OUT IS HIGH

IF D1 IS POSITIVE HIGH AND D2 IS POSITIVE HIGH THEN OUT IS LOW

IF D1 IS POSITIVE LOW AND D2 IS POSITIVE HIGH THEN OUT IS MEDIUM

The output of the inference process in the fuzzy means 22 is now used to correct the gradient. In the provided algorithm, the decision as to the value of the gradient F in the central pixel is reached by using a formula of the following kind: ##EQU6## where OUT is the final output of the fuzzy rules, P is the gradient value of the central pixel, and P.sub.Dn1 and P.sub.Dn2 are values of the gradient of two pixels that are aligned with the central pixel and are more representative of the trend of the gradient within the mask.

Let us consider an example to clarify the operation of the algorithm. Assume a pixel configuration as in the mask shown below: ##STR2##

The Dn terms that are chosen are D1 and D8, whose values are respectively 3 and 2. By applying the fuzzy rules, one obtains a final output value OUT=0.9 which, when applied to the correction formula, produces: ##EQU7## This yields a corrected value of the gradient of the central pixel F=45.75.degree., rounded to 46.degree..

This procedure has obtained a more accurate image of the objects in the parameter plane, eliminating the relative maximums that were essentially due to the errors in the gradient.

The fuzzy gradient corrector 5 is furthermore provided with an arithmetic unit 23 adapted to calculate the radius R related to the point in a polar coordinate system by applying the previously described formula:

where F is the gradient of the line being sought.

Two counters are connected to the arithmetic unit 23: a first horizontal counter 24 for generating the X coordinate, and a second vertical counter 25 for generating the Y coordinate. Both counters 24 and 25, as well as the differential block 21 and the arithmetic unit 23, are driven by a clock signal which increases the value of X of the counter 24 at each pulse and increases the value of Y of the counter 25 every 1/n clock pulses, where n is the number of columns of the image being analyzed. The division 1/n occurs by means of a divider 26 that is connected to the vertical counter 25. This establishes the position of the point within the image and determines the row and column to which the pixel of an edge belongs.

The arithmetic unit 23 then calculates the radius R related to the point in a polar coordinate system.

The value of the corrected gradient F and the radius R related to the processed pixel are sent to the fuzzy vote attributor 6 shown in FIG. 8.

The fuzzy vote attributor 6 assigns an edge point to the curve to which it belongs. The parameter space comprises a memory element: the parameters that identify the curve comprise the address of a memory location that contains the number of points of the curve. Every time the parameters identify a memory location, the content of that memory location is increased by 1. At the same time, a fuzzy system determines weights to be added to other memory locations determined by an address expander.

The input parameters of the fuzzy vote attributor 6 are the corrected gradient F and the radius R. These parameters are sent to an address generator 31. Said generator is connected to a memory 32 and is capable of selecting a memory location. By means of a first adder block 33 connected to the memory 32, the content of this memory location is increased by 1 (Count).

The output of the address generator 31 is furthermore connected to an address expander 34. With respect to the address provided by the address generator 31, the address expander 34 identifies other memory locations that contain the number of the points of straight lines that are close to the one to which the point being considered belongs.

For each one of these memory locations, second fuzzy means 35 determine a weight W, comprised between 0 and 1, to be added (Count 1) by means of a second adder block 36 to the content of the memory locations identified by the address expander 34. The weight W indicates the degree of confidence assigned to the fact that the edge point might also belong to straight lines other than the one identified by the first analysis.

The output of the memory 32 is the content N of the memory locations identified by the parameters R and F. Accordingly, the output of the system comprises the parameters that identify the straight line (R and F) and by the number of points (N) that compose it.

The fuzzy analysis used in this block will be explained in greater detail. Consider the straight line and the corresponding points of FIG. 12. In a case like this, the standard Hough transform would treat points P3 and P5 in the same manner, since both would vote for a cell other than the one of the straight line represented by the dashes.

Actually, these two points must be evaluated in a completely different manner. Indeed, it is evident that the distance of the point P3 from the straight line represented by the dashes is very probably due to noise.

Since a situation of this type occurs many times in the images to be processed, it is possible to provide a fuzzy process that allows to return, when necessary, the points to the positions to which they really belong.

An attempt will be made to analyze the Hough transform from a fuzzy point of view. Consider an image that contains several points aligned along a straight line. Their transformation in the parameter plane will produce lines that intersect each other in a single point. If instead the original points lie only approximately along a straight line, their transforms will intersect only approximately in a single point. Actually, they will pass through a region around the theoretical point. The parametric lines will therefore intersect in a "fuzzy point", in that they will pass through an area around the theoretical point in a manner that can be indicated by a membership function.

In particular, it can be stated that a fuzzy point in the parameter plane represents a fuzzy straight line in the feature plane, whereas a fuzzy point in the feature plane is a fuzzy line in the parameter plane. These remarks can be demonstrated by theory. Assume that each point in the feature plane is represented by a (circular) fuzzy area centered on the original point. It is evident that the membership function relating to this area must be isotropic and have a Gaussian behavior, decreasing as it moves away from the center.

Consider, for example, a point P which is at a distance d from (x,y). This point belongs to the transform of (x,y) according to a membership function whose value is m(d). The further from (x,y), the lower the contribution.

Consider now the case of straight line recognition. The neighborhood of the point can be parameterized according to the following equations:

Accordingly, each point that belongs to the neighborhood being considered traces, in the parameter space, a curve of the following kind:

It is possible to obtain the envelope of all these curves at different values of .alpha.. To achieve this, eliminate a between the previous equation and its derivative:

This produces the following expression:

Replacement yields the following:

In this manner, the points of a circular neighborhood are transformed in the parameter space by translating by .+-.r the line produced by the transformation of its central point, as shown in FIG. 13.

It is therefore possible to project a single point as in the standard Hough transform, subsequently performing a convolution along the axis p with a membership function whose value is m(r), or taking m(r) times the contribution of each point of the circle, yielding the same result.

The invention thus conceived is susceptible of numerous modifications and variations, all of which are within the scope of the inventive concept of the present invention. Thus, for example, the pixel mask of the interfaces can also be larger, according to the requirements. Furthermore, the fuzzy means of the fuzzy vote attributor and of the gradient corrector can be provided as an on-line or off-line fuzzy processor, in which case it may be a memory element (look-up table) that contains the result of the fuzzy calculation. Although the example analyzed herein in detail is adapted for identifying straight lines, the inventive concept can be applied to other geometric shapes (curves).

Finally, all the details may be replaced with other technically equivalent ones. In practice, the materials employed, as well as the shapes and dimentions, may be according to the requirements without thereby abandoning the scope of the protection of the appended claims.

Having thus described at least one illustrative embodiment of the invention, various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be within the spirit and scope of the invention. Accordingly, the foregoing description is by way of example only and is not intended as limiting. The invention is limited only as defined in the following claims and the equivalents thereto.