Vehicle periphery monitoring system

[0001] The present application claims the benefit of Japanese Patent Application No. 2004-109979 filed on Apr. 2, 2004, the disclosure of which is incorporated herein by reference.

[0002] 1. Field of the Invention

[0003] The present invention relates to a vehicle periphery monitoring system, and more particularly to a vehicle periphery monitoring system having an obstacle detecting unit for detecting the presence of an obstacle in the periphery of a vehicle and a controller for controlling an indicating unit to indicate the presence of an obstacle when the presence of such an obstacle is detected by the obstacle detecting unit.

[0004] 2. Description of the Related Art

[0005] Heretofore, there have been known various systems for detecting an obstacle that is present in the periphery of a vehicle and indicating the detected obstacle to the driver of the vehicle. One known system employs sonic waves to detect an obstacle as disclosed in Japanese laid-open patent publication No. 2002-59798, and another conventional system detects an obstacle based on a contrast difference in a captured image as disclosed in Japanese laid-open patent publication No. H09-287915.

[0006] The former known system, which employs sonic waves to detect an obstacle, needs to keep a certain detection distance in which to detect obstacles in order to provide a certain detection range. However, since sonic waves that are radiated into the detection range are vertically spread, if the detection distance is equal to or greater than a predetermined distance, the system tends to also detect undangerous obstacles on the ground, such as pebbles and wheel blocks, and indicate those detected undangerous obstacles to the driver of the vehicle. Therefore, the system is liable to keep the driver excessively on full alert while in operation, i.e., to fail to give the driver an adequate sense of safety. The latter known system, which detects an obstacle based on a contrast difference in a captured image, requires a CPU having a high processing capability due to the need for processing the entire area of the captured image.

[0007] It is therefore an object of the present invention to provide a vehicle periphery monitoring system for giving the driver of a vehicle an adequate sense of safety by indicating whether a detected obstacle is a dangerous obstacle or a safe obstacle, and which can be realized without the need for a CPU having a high processing capability.

[0008] According to the present invention, an obstacle detecting unit detects the presence of an obstacle in a periphery of a vehicle. If the detected distance from the vehicle to the obstacle detected by the obstacle detecting unit is equal to or greater than a preset distance, e.g., a maximum distance in which all detected obstacles need to be indicated as dangerous obstacles, then a control unit performs an image processing process on a viewpoint-changed image which is produced by changing the viewpoint of the image captured by the imaging unit, for thereby acquiring the height of the detected obstacle. The control unit controls an indicating unit to either indicate the detected obstacle as a safe obstacle or inhibit the detected obstacle from being indicated if the acquired height of the detected obstacle is less than a preset height, e.g., the height of the bumper of the vehicle, and controls the indicating unit to indicate the detected obstacle as a dangerous obstacle if the acquired height of the detected obstacle is equal to or greater than the preset height.

[0009] For example, if a person is detected at a distance greater than the preset distance from the vehicle, then the control unit indicates the detected person as a dangerous obstacle. If a pebble or a wheel block is detected at a distance greater than the preset distance from the vehicle, then the control unit either indicates the detected pebble or wheel block as a safe obstacle or inhibits the detected pebble or wheel block from being indicated. Therefore, the vehicle periphery monitoring system is able to indicate to the driver of the vehicle whether the detected obstacle is a dangerous obstacle or a safe obstacle. The driver is not kept excessively on full alert, i.e., is given an adequate sense of safety while the vehicle periphery monitoring system is operation. Since the vehicle periphery monitoring system acquires the height of the obstacle by performing the image processing process on the viewpoint-changed image, the vehicle periphery monitoring system is not required to perform the image processing process on the entire area of the captured image, and hence does not require a CPU having a high processing capability.

[0010] The indicating unit may comprise a display unit for displaying an image captured by an imaging unit, and the control unit may control the display unit to display the detected obstacle as a safe obstacle overlappingly in the captured image displayed by the display unit if the acquired height of the detected obstacle is less than the preset height. Therefore, the obstacle detected as the safe obstacle is visually indicated to the driver.

[0011] Alternatively, the indicating unit may comprise a display unit for displaying a bird's-eye image converted from the image captured by the imaging unit, and the control unit may control the display unit to display the detected obstacle as a safe obstacle overlappingly in the bird's-eye image displayed by the display unit if the acquired height of the detected obstacle is less than the preset height. Inasmuch as the obstacle is displayed overlappingly in the bird's-eye image, the driver is able to have an accurate sense of distance with respect to the obstacle.

[0012] The control unit may control the display unit to display the detected obstacle in combination with distance indicator lines indicative of respective distances from the vehicle and distance indicator markings associated with the distance indicator lines, overlapping in the bird's-eye image if the acquired height of the detected obstacle is less than the preset height. Consequently, the driver can visually perceive the distance from the vehicle to the obstacle with the aid of the distance indicator lines and the distance indicator markings.

[0013] The control unit may perform the image processing process on a portion of the viewpoint-changed image which is produced by changing the viewpoint of the image captured by the imaging unit, for thereby acquiring the height of the detected obstacle. As the image processing process is performed on a portion or limited area of the viewpoint-changed image, but not on the entire viewpoint-changed image, the processing load imposed on the vehicle periphery monitoring system for acquiring the height of the obstacle is relatively small.

[0014] The above and other objects, features, and advantages of the present invention will become apparent from the following description when taken in conjunction with the accompanying drawings which illustrate a preferred embodiment of the present invention by way of example.

[0021] FIG. 1 schematically shows in side elevation a vehicle periphery monitoring system 1 according to an embodiment of the present invention.

[0022] As shown in FIG. 1, the vehicle periphery monitoring system 1 is mounted on a vehicle 2 such as an automobile and has a controller 3, a vehicle-mounted camera 4, a sonic sonar 5, and a display unit 6. The controller 3 serves as a control unit, the vehicle-mounted camera 4 as an imaging unit, the sonic sonar 5 as an obstacle detecting unit, and the display unit 6 as an indicating unit which may server as a display unit.

[0023] The controller 3 comprises a CPU, a RAM, a ROM, and I/O buses, and executes a control program to control the overall operation of the vehicle periphery monitoring system 1. The vehicle-mounted camera 4 is installed on an upper portion of the rear end of the vehicle 2. The vehicle-mounted camera 4 captures an, image of an area behind the vehicle 2, and outputs the captured image to the controller 3. The sonic sonar 5 is installed on a lower portion of the rear end of the vehicle 2. The sonic sonar 5 radiates sonic waves rearwardly of the vehicle 2 into a vertical range having a central horizontal axis P extending centrally therein. The sonic sonar 5 detects whether there is an obstacle in the vertical range or not based on whether there is an echo received from the vertical range or not, and outputs a detected signal to the controller 3. If the sonic sonar 5 detects an obstacle, then the sonic sonar 5 also calculates a detected distance depending on the period of time that is spent after it has radiated the sonic waves until it receives the echo, and outputs the detected distance to the controller 3. The display unit 6 is disposed in such a position in the passenger compartment of the vehicle 2 that the display unit 6 can easily be visually perceived by the driver of the vehicle 2. The display unit 6 has a display function to display an image and a command accepting function to accept commands input by the driver through a touch panel.

[0024] The vertical range into which the sonic sonar 5 radiates sonic waves is defined between an upper edge P1 and a lower edge P2. The area imaged by the vehicle-mounted camera 4 is also spread vertically and defined between edges Q1, Q2. The vertical range into which the sonic sonar 5 radiates sonic waves and the area imaged by the vehicle-mounted camera 4 overlap each other, so that the vehicle-mounted camera 4 can capture an image of the range for the sonic sonar 5 to detect an obstacle or obstacles therein.

[0025] The vehicle periphery monitoring system 1 may be part of a car navigation system incorporated in the vehicle 2, and the display unit 6 may be the display monitor of the car navigation system.

[0026] Operation of the vehicle periphery monitoring system 1 will be described below with reference to FIGS. 2 through 5A, 5B. FIGS. 2 and 3 are flowcharts of an operation sequence of the vehicle periphery monitoring system 1 and a subroutine of the operation sequence, respectively.

[0027] When the vehicle periphery monitoring system 1 is activated, the controller 3 monitors whether the sonic sonar 5 has detected the presence of an obstacle or not in step S1. If the sonic sonar 5 has detected the presence of an obstacle (“YES” in step S1), then the controller 3 determines whether or not the detected distance up to the obstacle is equal to or greater than a preset distance in step S2. The preset distance is defined as a maximum distance in which all detected obstacles need to be indicated as dangerous obstacles, and is set to 50 cm., for example.

[0028] If the sonic sonar 5 has not detected the presence of an obstacle (“NO” in step S1), then the controller 3 repeatedly monitors whether the sonic sonar 5 has detected the presence of an obstacle or not in step S1. As shown in FIG. 1, an obstacle Z2 positioned outside of the vertical range of the sonar 5 is not detected.

[0029] If the detected distance up to the obstacle is smaller than the preset distance (“NO” in step S2), then the controller 3 controls the display unit 6 to display the detected obstacle as a dangerous obstacle overlappingly in an image that is captured by the vehicle-mounted camera 4 in step S3. Therefore, when an obstacle is detected and the detected distance up to the detected obstacle is less than the preset distance, since the detected obstacle is displayed as a dangerous obstacle overlapping in the captured image by the display unit 6, the driver of the vehicle 2 is able to confirm that the dangerous object is present in an area within the preset distance from the vehicle 2. Specifically, in FIG. 1, the driver is able to confirm the presence of an obstacle Z1 within the preset distance.

[0030] If the detected distance up to the obstacle is equal to or greater than the preset distance (“YES” in step S2), then the controller 3 performs an image processing process on a viewpoint-changed image that is generated by changing the viewpoint of the image captured by the vehicle-mounted camera 4, for thereby calculating the height of the obstacle in step S4. The image processing process in step S4 will be described in detail below with reference to FIGS. 3 and 4.

[0031] When the image processing process is started, the controller 3 generates two viewpoint-changed images by changing the viewpoint of two images that are captured at different times by the vehicle-mounted camera 4 to an overhead viewpoint. Specifically, as shown in FIG. 4, the controller 3 generates a viewpoint-changed image “a” by changing the viewpoint of an image that is captured by the vehicle-mounted camera 4 when the vehicle 2 is positioned at a location “A”. After the vehicle 2 moves from the location “A” to a location “B”, the controller 3 generates a viewpoint-changed image “b” by changing the viewpoint of an image that is captured by the vehicle-mounted camera 4 when the vehicle 2 is positioned at the location “B”. The viewpoint-changed image A contains two feature points “a1”, “a2” corresponding to two edges, i.e., a lower left edge and an upper right edge, of the obstacle, and the viewpoint-changed image B contains two feature points “b1”, “b2” corresponding to the two edges of the obstacle.

[0032] The controller 3 performs the image processing process on the viewpoint-changed image B from a location “L−La”, where L represents the detected distance up to the obstacle detected by the sonic sonar 5 and La a detection error, in a direction away from the vehicle 2, for thereby searching for the position of the feature point “b1” closer to the sonic sonar 5 in step S11. If the searching is successful (“YES” in step S12), then the controller 3 calculates the distance up to the located position as a detected distance “L′” according to the image processing process in step S13.

[0033] Then, the controller 3 calculates a searching-completion location “L3” according to the equation (1) shown below where “H” represents the height of the vehicle-mounted camera 4 and “X′” the tentative height of the obstacle in step S14. Stated otherwise, the controller 3 limits the imaged area to be processed by the image processing process. The tentative height “X′” of the obstacle is an allowable maximum height that does not obstruct the travel of the vehicle 2, and may be the height of the bumper of the vehicle 2, for example.
L3=(X′/(H−X′))×L′+La  (1)

[0034] Thereafter, the controller 3 performs the image processing process on the viewpoint-changed image B from the feature point “b1” closer to the sonic sonar 5, which has previously been identified, in the away from the vehicle 2 toward the location “L3”, for thereby searching for the position of the feature point “b2” remoter from the sonic sonar 5 in step S15. If the searching is successful (“YES” in step S16), then the controller 3 calculates the distance “L2” between the position of the feature point “b1” closer to the sonic sonar 5 and the position of the feature point “b2” remoter from the sonic sonar 5 in step S17. The controller 3 calculates the height “X” of the obstacle according to the following equation (2) in step S18:
X=(L2/(L′+L2))×H  (2)

[0035] If the controller 3 fails to search for the position of the feature point “b2” remoter from the sonic sonar 5 (“NO” in step S16), then the controller 3 judges the detected obstacle as a dangerous obstacle in step S19.

[0036] After the height “X” of the obstacle is calculated, control goes back to the main routine shown in FIG. 2. The controller 3 determines whether or not the calculated height “X” is equal to or greater than a preset height in step S5. The preset height is the same as the tentative height “X′” of the obstacle, i.e., an allowable maximum height that does not obstruct the travel of the vehicle 2, and may be the height of the bumper of the vehicle 2, for example.

[0037] If the height “X” of the obstacle is equal to or greater than the preset height (“YES” in step S5), then the controller 3 controls the display unit 6 to display the detected obstacle as a dangerous obstacle overlappingly in the captured image in step S3. Therefore, when an obstacle is detected and the detected distance up to the detected obstacle is equal to or greater than the preset distance and the height “X” of the detected obstacle is equal to or greater than the preset height, since the detected obstacle is displayed as a dangerous obstacle overlapping in the captured image by the display unit 6, the driver of the vehicle 2 is able to confirm that the dangerous object is present in the area within the preset distance from the vehicle 2. Specifically, in FIG. 1, the driver is able to confirm the presence of an obstacle Z3 within the preset distance.

[0038] If the height “X” of the obstacle is smaller than the preset height (“NO” in step S5), then the controller 3 controls the display unit 6 to display the detected obstacle as a safe obstacle overlappingly in the captured image in step S6. Therefore, when an obstacle is detected and the detected distance up to the detected obstacle is equal to or greater than the preset distance and the height “X” of the detected obstacle is smaller than the preset height, since the detected obstacle is displayed as a safe obstacle overlapping in the captured image by the display unit 6, the driver of the vehicle 2 is able to confirm that the safe obstacle is present in the area within the preset distance from the vehicle 2. Specifically, in FIG. 1, the driver is able to confirm the presence of an obstacle Z4 within the preset distance. In the present embodiment, wheel blocks “M1”, “M2” shown in FIG. 5B are detected and displayed as safe obstacles by way of example. Other safe objects such as pebbles and curbs are also similarly detected and displayed.

[0039] In the above embodiment, obstacles are displayed overlappingly in the captured image. However, obstacles may be displayed overlappingly in a bird's-eye image that is converted from a captured image, as shown in FIG. 5B. Specifically, as shown in FIG. 5B, distance indicator lines “Sa”, “Sb”, “Sc” indicative of respective distances from the rear end of the vehicle 2 and distance indicator markings “Ta”, “Tb”, “Tc” representing metric indications, for example, which are associated with the distance indicator lines “Sa”, “Sb”, “Sc” may also be displayed overlappingly in the bird's-eye image. In the above embodiment, if the height “X” of the obstacle is smaller than the preset height, the detected obstacle is displayed as a safe obstacle overlappingly in the captured image. Alternatively, however, the detected obstacle as a safe obstacle may not be displayed, i.e., may be inhibited from being displayed, in the captured image.

[0040] According to the present embodiment, as described above, if the detected distance up to a detected obstacle is equal to or greater than the preset distance, then the vehicle periphery monitoring system 1 performs the image processing process on the viewpoint-changed image to acquire the height “X” of the obstacle, and displays the detected obstacle as a safe obstacle if the acquired height “X” is less than the preset height, and displays the detected obstacle as a dangerous obstacle if the acquired height “X” is equal to or greater than the preset height. Therefore, the vehicle periphery monitoring system 1 is able to indicate to the driver of the vehicle 2 whether the detected obstacle is a dangerous obstacle or a safe obstacle. The driver is not kept excessively on full alert, i.e., is given an adequate sense of safety while the vehicle periphery monitoring system 1 is operation. Since the vehicle periphery monitoring system 1 acquires the height “X” of the obstacle by performing the image processing process on the viewpoint-changed image, the vehicle periphery monitoring system 1 is not required to perform the image processing process on the entire area of the captured image, and hence does not require a CPU having a high processing capability.

[0041] Inasmuch as an obstacle detected as a safe obstacle is displayed overlappingly in the captured image or the bird's-eye image, the obstacle detected as the safe obstacle is visually indicated to the driver. The distance indicator lines “Sa”, “Sb”, “Sc” and the distance indicator markings “Ta”, “Tb”, “Tc” associated with the respective distance indicator lines “Sa”, “Sb”, “Sc”, which are displayed overlappingly in the bird's-eye image, also give the driver a visual indication of the distance up to the object from the rear end of the vehicle 2.

[0042] The vehicle periphery monitoring system 1 may indicate the presence of an obstacle to the driver audibly through sounds, rather than visually through a displayed image, or may indicate the presence of an obstacle to the driver both visually through a displayed image and audibly through sounds. If the vehicle periphery monitoring system 1 indicates the presence of an obstacle to the driver audibly through sounds, then the vehicle periphery monitoring system 1 use different sound levels, different frequencies, or different sound patterns to indicate whether a detected obstacle is a dangerous obstacle or a safe obstacle.

[0043] The vehicle periphery monitoring system 1 may employ an ultrasonic sonar or a radar sonic sonar, rather than the sonic sonar described above.

[0044] Although a certain preferred embodiment of the present invention has been shown and described in detail, it should be understood that various changes and modifications may be made therein without departing from the scope of the appended claims.