Apparatus and Method for Improving Accuracy of an Automatic Travelling Apparatus

The present invention relates to an automatic travelling apparatus capable of identifying a permissible travel path for the purpose of permitting a vehicle to automatically run on a road.

Recently, there has been developed an automatic travelling apparatus which is capable of identifying a permissible travel path, setting a target course thereon and steering a vehicle to run along the target course.

The automatic travelling apparatus disclosed in Japanese laid open patent publication No. 199610-88 continuously generates an image of an area ahead of a vehicle and in the vehicle's running direction using an image pick-up device attached to the vehicle. The apparatus samples the image thus generated and extracts therefrom continuous line segments such as road edges. The apparatus then determines a permissible travel path ahead of the vehicle based on the extracted continuous line segments and sets a target course in the determined permissible travel path. Next, the apparatus estimates a steering correction necessary to allow the vehicle to follow the target course based on the currently detected running condition of the vehicle and, based on the steering correction, steers the vehicle to follow the target course.

Because the above-mentioned automatic travelling apparatus performs the steering control of the vehicle at repeated sampling intervals, the steering control will be more accurate as the sampling period i.e., steering control cycle, is made shorter.

However, where the processing of generated image data is performed by a microcomputer or like means, the processing may take a rather long time because an image of an area ahead of the vehicle may be complicated and may include a large amount of information to be processed. Thus, it is often difficult to reduce the control cycle time of an image processing system.

The time required for processing each image and for determining a permissible travel path is fairly long in comparison with the processing time required for steering the vehicle to follow the target course set in the permissible travel path. For example, where image processing may take 500 milliseconds to complete, the processing time required for steering control may take only 10 milliseconds.

If the image processing time is extended, the vehicle may travel enough to produce a deviation from its actual position with respect to the determined permissible travel path, causing a considerable decrease in accuracy of the steering control. It will be noted that, as the running speed of the vehicle increases, the amount of deviation will also increase.

In view of the foregoing, the present invention is directed to an automatic travelling apparatus which is capable of recalculating the position of a vehicle relative to the permissible travel path, which is given for the current control cycle, based on the actual running condition of the vehicle. Thus, the apparatus is capable of resetting a corresponding target course in the permissible travel path in response to the recalculated vehicle position. In this fashion, deviations of the actual position of the vehicle with respect to the permissible travel path may be minimized even if the image processing time is extended.

FIG. 1 is a block diagram showing a structure of an automatic travelling apparatus embodying the present invention.

FIG. 2 shows a line segment of a road obtained on the basis of processing an image taken by a video camera.

FIG. 3 shows an image obtained by projective transformation of the image shown in FIG. 2.

FIG. 4 shows an example of a target course set in a permissible area within a road width.

FIGS. 5(a) and 5(b) show target courses set on a road which a vehicle is to travel at a low speed (a) and a high speed (b) respectively.

FIG. 6 shows a relationship between a target course and a presumed course.

FIG. 7 shows a relationship between a vehicle's steering angle and its turning radius.

FIG. 8 shows a travelling state of a vehicle in a permissible travelling area.

FIG. 9(a) shows a position and a running direction of a vehicle in a permissible travelling area at a certain time.

FIG. 9(b) shows a state obtained by coordinate transformation of a permissible travelling area in relation to a position and a running direction of a vehicle in a permissible travelling area.

FIG. 10 shows a line segment in the X-Y coordinates.

FIG. 11 shows a point on the p-8 point coordinates obtained by the Hough conversion of the line segment shown in FIG. 10.

FIG. 12 shows timings of processings in one control cycle.

FIG. 13 shows a flow chart of image processing.

FIG. 14 shows a flow chart of updating the vehicle position during image sampling.

Referring now to the drawings, a preferred embodiment of the present invention will be described in detail as follows:

It will be observed from FIG. 1 that an automatic travelling apparatus according to the present invention comprises: an image pick-up device 1 such as a video camera attached to a vehicle for continuously generating an image of ground ahead of the vehicle; an image processor 2 for processing the images generated by the image pick-up device 1 to extract therefrom segments of continuous lines such as road edges and the like; permissible travel path determining means 3 for determining, on the basis of the obtained continuous line segments, a permissible travel path such as a road in the direction in which the vehicle is to travel; target course setting means 4 for setting a target course in the permissible travel path thus determined; control means 5 for determining the instantaneous running condition of the vehicle on the basis of an output signal from a speed sensor 6 representing the vehicle's running speed "v", an output signal from a yaw rate sensor 7 representing the yaw rate ".UPSILON." and an output signal from a steering angle sensor 8 representing the tire angle ".delta." which varies with the steering of the vehicle, and for estimating, on the basis of the instantaneous running condition, a steering correction necessary to permit the vehicle to follow the target course; and steering control means 9 (and a steering drive 10) for steering the vehicle with reference to the steering correction.

Actually, a microcomputer aided control is used in place of the image processor 2, the permissible travel path determining means 3, the target course setting means 4, and the control means 5; and the steering control means 9 can be included in the microcomputer aided control, if occasions demand.

The extraction of continuous line segments such as road edges from the generated image in the image processing means 2 can be made as follows:

First, each image supplied from the image pick-up device 1 is subjected to a differentiation process for detection of the road edges. Then, an automatic threshold setting circuit in the image processing means 2 sets an optimum threshold value in consideration of the degree of shade of the road edge image information just processed. Then, the road edge image will be subjected to binary transformation.

Alternately, first, the images may be subjected to binary transformation, and then the binary data may be subjected to differentiation. In place of binary transformation polydigitization may be performed to express some shade details of the image.

Digitalized image information will be subjected to Hough conversion to convert the X-Y linear coordinates to .rho.-.theta. point coordinates, thus transforming the isolated points into continuous line segments of the road edges as shown in FIG. 2.

The symbol .theta. stands for an angle formed between the X-axis and a perpendicular from the origin of the X-Y coordinate system to a line segment. The symbol .rho. stands for the length of the normal line. For instance, the line L in the X-Y coordinate system depicted in FIG. 10 is expressed as the point 01 in the .rho.-.theta. point coordinate system depicted in FIG. 11.

In this fashion, edge tracing may be performed on the basis of binary-coded image information to obtain a continuous road edge. The Hough conversion, edge tracing and other appropriate processings may be performed simultaneously. Then synthetic judgement may be made on the results of these processings to obtain precise road edge information. More accurate road edge information may be taken out when the above-mentioned image processings are made with developing an input image area as the vehicle travels.

The image taken by a video camera represents a perspective view. The perspective road edge image as shown in FIG. 2 can be converted to a non-perspective road edge image as shown in FIG. 3 according to known projective conversion processes.

The permissible travel path determining means 3 has projective conversion characteristics set in accordance with the perspective characteristics of an associated video camera.

The permissible travel path determining means 3 can determine, based on the non-perspective road image obtained by projective conversion, for example, an area between the continuous road edges E1 and E2 shown in FIG. 4. This area comprises a permissible travel path RA in the X-Y coordinate system wherein the Y-axis corresponds to the direction in which the image is pictured by the image pick-up device 1, i.e., the direction in which the vehicle travels.

In FIG. 4, a current or instantaneous position of the vehicle 11 is indicated at a point "P", and the video camera of the image pick-up device 1 is mounted at a predetermined position on the vehicle such that the point "P" will appear at the center lower point of the display screen as the origin of the X-Y coordinate system.

After a permissible travel path is determined by the permissible travel path determining means 3, the target course setting means 4 will select a course most appropriate for running in the permissible travel path, and will set the so selected course as a target course to follow.

Preferably, the course will be determined based on the road contour and the running speed of the vehicle. However, the course may be basically determined based on the width of the road as follows:

If the target course setting means 4 finds that the width of the road exceeds a predetermined value, and that vehicles must keep to the left, a target course OC will be set a given constant distance "w" (for instance, 1.5 meters) from the left edge of the road, as shown in FIG. 4.

If the width of the road is below the predetermined value, a target course will be set along the center line (not shown) of the road.

The coordinates of the target course are stored in the memories of the target course setting means 4 and are continuously recalculated as the vehicle is running. The divisions of the X-Y coordinates for the permissible travel path and the target course are selected in accordance with the magnification of the video camera of the image pick-up device 1.

In FIG. 4, the trace of the vehicle from "P" to "O" represents the course actually followed by the vehicle under the control of the control means 5 until the vehicle has come to the target course OC at the point 0.

According to the present invention, it is also possible to set a target course based on the running condition of the vehicle as follows:

If the target course setting means 4 finds that the running speed measured by the speed sensor 6 is below a predetermined speed, the target course OC will be set in conformity with the road contour as seen from FIG. 4.

When the running speed of the vehicle is higher than a predetermined speed, and when the vehicle is running along a curved road as shown in FIG. 5(b), a target course OC of reduced curvature is set so as to reduce the lateral force which is applied to the vehicle.

After setting a target course in the road, the control means 5 will calculate a steering correction necessary to permit the vehicle to follow the target course as follows:

The steering correction relates to a steering angle of the vehicle, and the control means 5 presumes a course which the vehicle will run along based on the currently detected running condition of the vehicle. The control means 5 then calculates a deviation between the presumed running course and the target course, and determines a steering correction necessary for bringing the vehicle to follow the target course. Finally, the steering control means 5 adjusts the steering control of the vehicle according to the steering correction.

In practice, for example, the position the vehicle will attain is represented as a point in the X-Y coordinate system where the Y-axis represents the running direction of the vehicle, and the lateral deviation of the represented point from the point of target position is measured to finally determine therefrom a corresponding amount of steering angle to be corrected.

Now it is assumed that a vehicle 11 at Point "P" is controlled to get on the target course OC.

First, the distance L (m) (L=v.times.T) on the Y-axis which the vehicle can run in T seconds will be determined on the basis of the vehicle's running speed v (m/s) which is determined by the speed sensor 6. Then, the lateral deviation xl from Point "C" (on which the vehicle would be in T seconds if it travelled straight along the Y-axis by the distance L) to the target course OC will be estimated.

Second, the course AC which the vehicle is supposed to follow will be estimated from the yaw rate T (rad/sec), and then the lateral deviation xm from Point "C" to the presumed course AC will be estimated by the following equation: ##EQU1## where R stands for a radius of the presumed course AC.

When R>>v X tm, we obtain ##EQU2##

From Equations (1) and (2)

A positive sign of yaw rate .UPSILON. represents that the presumed course AC turns to the left whereas a negative sign indicates the presumed course turning to the right.

The yaw rate .DELTA..UPSILON. to which the yaw rate of the vehicle is to be corrected will be determined from the following equation:

Then, based on the tire angle a detected at Point "P" by the steering angle sensor 8, the steering correction .delta.' for permitting the vehicle to get on the target course OC can be determined as follows:

Referring to FIG. 7, when R>>l, the following equation can be obtained;

From equations (2) and (5) we can derive

where l stands for a wheelbase. According to equation (6) the tire angle .DELTA..delta. for correction in accordance with the yaw rate .DELTA..UPSILON. to be corrected can be given by the following equation:

Substituting l=(1+Kv.sup.2) into the equation (7), we can obtain

where "K" is a constant which is determined both from the tire characteristics and the vehicle characteristics.

Therefore, a steering correction .delta.' for permitting the vehicle to get on the target course can be obtained by the following equation:

In response to the steering amount .delta.' given from the control means 5 the steering control means 9 issues a drive command to the steering drive 10 which in turn steers the vehicle toward the target course OC.

The above-mentioned processing operations will be repeated at specified intervals such that the steering control of the vehicle may be continuously performed and will permit the vehicle to follow the target course OC set in the permissible travel path. It may be noted that the permissible travel path is recalculated during each control cycle.

The automatic travelling apparatus according to the present invention further includes means for repeatedly recalculating the vehicle position relative to the permissible travel path during each current control cycle. The recalculations are based on the instantaneous running condition of the vehicle. Further, means are provided for resetting the target course in response to the recalculated vehicle position. Finally, a steering correction for permitting the vehicle to follow the reset target course will be determined.

The above-mentioned means is practically included in the control means 5 shown in FIG. 1.

If a permissible travel path RA is established for one control cycle in the X-Y coordinates and a vehicle 11 is positioned at Point P (Xo, Yo), i.e., at the origin of the X-Y coordinates at the beginning of said control cycle as shown in FIG. 8, the vehicle position Pn (Xn, Yn) (n=1, 2, 3, . . . ) in X-Y coordinates, which varies with time as the vehicle runs, is estimated from the running speed "v" and the yaw rate ".UPSILON." (i.e. angular velocity increment in yawing direction) which are detected respectively by a running speed sensor 6 and a yaw rate sensor 7 at an interval of an unit time .DELTA.t preset shorter than the image processing time, and thereby said position may be recalculated successively in relation to the permissible travel path RA. In FIG. 8, "OC" designates a target course set at an initial stage of the control cycle. ##EQU3## where .DELTA.L is a distance which the vehicle travels for a unit time and can be determined by the following equation:

.DELTA..theta.(=.theta.n-.theta.n-1) is an change of the vehicle running direction for the unit time, which is determined by the detected value of yaw rate .UPSILON. as follows:

The target course for the renewed position Pn (Xn, Yn) of the vehicle is reset with respect to the permissible travel path RA, and a steering correction is also estimated with respect to the reset target course so as to steer the vehicle toward the target course.

As shorn in FIGS. 9(a) and 9(b) , a new X'-Y' coordinate system in which the Y'-axis stands for the axis of running direction of the vehicle 11 at the recalculated position Pn is formed, and then coordinate transformation of every points of the permissible travel path RA' and the target course OC' is carried out in such a way that the renewed point Pn (Xn, Yn) in the X-Y coordinate system may be placed to the origin Po' (X', Y').

Namely, each point shown in FIG. 9(a) is shifted respectively by (-.DELTA.L.multidot. cos .DELTA..theta., -.DELTA.L.multidot. sin .DELTA..theta.) and the permissible travel path pattern RA and the target course pattern OC are turned respectively by -.DELTA..theta., with the result that the axis of the vehicle running direction coincides with the Y'-axis of the X'-Y' coordinate system and vehicle 11 is positioned at the origin of said coordinate system.

Then, a steering correction for permitting the vehicle 11 to follow the target course OC' is determined with reference to the X'-Y' coordinate system in the same manner as mentioned above, and the steering control will be performed with reference to said recalculated steering amount.

The steering correction for permitting the vehicle to follow the target course OC initially set in the permissible travel path RA was estimated from the variable data as aforementioned.

Accordingly, when a vehicle 11 is steered from an initial position Po with the initially estimated steering correction for a relatively long period of time, (i.e., over a relatively long control cycle) the vehicle may progressively deviate from the target course OC due to the error included in the steering correction estimation.

The present invention provides a means for minimizing such deviation. For example, in one control cycle a target course OC' can be reset in a permissible travel path RA' based on the vehicle's present position. Thus, the steering control of the vehicle 11 can be carried out with a higher accuracy according to the steering correction newly estimated for the recalculated target course. Since the interval between renewals of the steering correction is relatively short, the error of estimation may have no practical influence to the steering control.

According to the present invention, it is also possible to optimally change the target course with a change of the vehicle running speed in one control cycle in case the target course is set in a permissible travel path depending on the vehicle's running speed as shown in FIGS. 5(a) and 5(b).

FIG. 12 is a timing chart showing each processing timing in one control cycle.

FIG. 13 shows a flow of image processing, and FIG. 14 shows a flow of renewing the vehicle position during the image processing.

As is apparent from the foregoing description, the automatic travelling apparatus according to the present invention offers such an advantage that in each steering control cycle comprising the following steps--sampling an image of an area ahead of a running vehicle (said image being taken by the image pick-up means attached to the vehicle); processing sampled data and extracting therefrom continuous line segments; determining a permissible travel path ahead of the vehicle based on the continuous line segments extracted; setting a target course in the permissible travel path thus determined; detecting the instantaneous running condition of the vehicle; estimating, on the basis of the instantaneous running condition, a steering correction to permit the vehicle to follow the target course; and steering the vehicle with reference to the steering amount, --the automatic travelling apparatus of the present invention can recalculate, on the basis of the instantaneous running condition of the vehicle during a period of image sampling, the preceding position of the vehicle in the determined permissible travel path and reset a target course in the permissible travel path in relation to the recalculated position. Thus, the image sampling period, i.e., the steering control cycle can be shortened to increase the steering control accuracy of that apparatus and the steering control can be carried out at constant intervals irrespective of possible variations of image processing period.