Display apparatus for displaying virtual image to driver

[0001] The present application is based on and claims the benefit of priority of Japanese Patent Application No. 2007-241134 filed on Sep. 18, 2007, the disclosure of which is incorporated herein by reference.

[0002] The present disclosure generally relates to a display apparatus for clearly notifying a user of an obstacle such as a human, a vehicle or the like by using a headup display apparatus or the like.

[0003] Conventionally, an obstacle around a vehicle is notified for a driver of the vehicle by using a display system. For example, a night vision system uses a far-infrared ray camera disposed on the vehicle to detect a position and/or a movement of a detecting object that emits an infrared ray, and recognizes that the obstacle is existing on a road ahead of the vehicle, or crossing a road. Further, the night vision system determines whether or not the object is a walker based on a size and shape of the object. Then, an image of a vehicle front field is displayed on a display unit for notifying the driver of the vehicle of the walker on an expected travel route of the vehicle or of the walker crossing the street.

[0004] However, the above system displays the image of the detecting object on the display unit that is disposed on an instrument panel or the like, thereby making it difficult for the driver to watch the display unit due to a requirement of a move of a look of the driver away from a windshield to the display unit.

[0005] Therefore, a display system that projects an image light to be reflected on the windshield to generate in a vehicle front space a virtual image in a view of the driver for information notification is proposed. For example, when a walker 101 that causes unsafeness of the subject vehicle is seen through a windshield 17 as shown in FIG. 7, the proposed system such as the one described on page 73 of a non-patent document of Automotive Technology 2006 Vol. 60 No. 2 can display a frame image 105 as a virtual image for surrounding the walker 101 for the purpose of notifying the driver of a position of the unsafe obstacle.

[0006] However, due to the vibration of the vehicle during driving, the frame image 105 is also vibrated, thereby making the positions of the images 105a, 105b, 105c at times T, T+1, T+2 respectively different from each other in the view of the driver as shown in FIG. 8. The respectively different position of the image 105 annoys the driver of the vehicle in a great deal.

[0007] In view of the above and other problems, the present disclosure provides a display apparatus that displays a virtual image without letting a driver of a vehicle to recognizing a vibration of the image.

[0008] The display apparatus of the present disclosure uses an image display unit to display the virtual image in front of the vehicle, and the virtual image is controlled to circle around an obstacle on a path or an orbit in a view of the driver. Therefore, the position of the virtual image at a certain point of time is different from the position of the virtual image at a different point of time, thereby making it difficult for the driver to recognize the vibration of the virtual image. That is, the display unit of the present invention does not let the driver to recognize the vibration of the vehicle in the course of displaying the virtual image. Further, by controlling the position of the virtual image to move along an obit that surrounds the obstacle, the existence of the obstacle is clearly notified for the driver. Furthermore, the obstacle is marked by the virtual image that circles around the obstacle only after determining that the obstacle is unsafe for the driver. Therefore, the display apparatus prevents the driver from being annoyed by impartially displaying unnecessary virtual images for all of the detected objects that are not necessarily unsafe for the driver.

[0018] An embodiment of the present invention is explained with reference to FIGS. 1, 4, 5.

[0019] 1. Configuration of a Display Unit

[0020] A display unit 1 is a device disposed on a vehicle 2, and includes an image output circuit 3, a liquid crystal panel 5, a camera 7, an obstacle detection circuit 9, a danger detection circuit 11, a vehicle speed sensor 13 and a speaker 14 as shown in FIG. 1. The image output circuit 3 outputs image data on the liquid crystal panel 5. The liquid crystal panel 5 is disposed in an upper part of an instrumental panel 15, and displays an image output from the image output circuit 3. The light to display an image output from the liquid crystal panel 5 reflects on a windshield 17, and enters a viewpoint 19 of a driver. As a result, in a view of the driver, a virtual image 21 of an image displayed on the liquid crystal panel 5 is displayed in a space in front of the vehicle. The image output circuit 3 and the liquid crystal panel 5 can control a display position of the virtual image 21 so that the virtual image 21 moves sequentially over an orbit 27 surrounding an obstacle (e.g., another vehicle 23 in FIG. 4, a walker 25 in FIG. 5) in the circumference of the vehicle (e.g., in particular, in the front field of the vehicle) in a view of the driver as shown in FIGS. 4 and 5. In addition, a position, a size, and a shape of the orbit 27 vary depending on determination results of the danger detection circuit 11. The orbit 27 will be explained later in detail in the description.

[0021] The camera 7 is an imaging device to photograph a picture of a front field (designated as a “front picture” in the following) of the vehicle 2. The camera 7 is disposed inside of the vehicle 2 at a position that is close to the viewpoint 19 of the driver. The obstacle detection circuit 9 determines whether there is the obstacle (e.g., a walker or other vehicles) in the front picture photographed by the camera 7, and calculates an obstacle direction (i.e., a direction to the obstacle from the viewpoint 19), an obstacle distance (i.e., the distance to the obstacle from the vehicle 2) and the size of the obstacle when there is the obstacle in the front picture. The calculation can be performed by a well-known method from the shape, the movement and the brightness of the obstacle. The danger detection circuit 11 is a device to determine whether or not the obstacle recognized by the obstacle detection circuit 9 is unsafe. The method of how the unsafeness is determined is mentioned later. The vehicle speed sensor 13 detects a vehicle speed of the vehicle 2, and the result is output to the danger detection circuit 11. The speaker 14 is installed inside of the vehicle 2, and, depending on a determination result of the danger detection circuit 11, a beep sound is output therefrom.

[0022] 2. The Processing Performed by the Display Unit

[0023] The processing performed by the display unit 1 is explained with reference to a flow chart in FIG. 2 and illustrations in FIGS. 3 to 6.

[0024] The processing in FIG. 2 is started when an engine of the vehicle 2 is started. In step S100, the front picture of the vehicle 2 is photographed by the camera 7.

[0025] In step S110, the obstacle detection circuit 9 extracts an obstacle from the front picture acquired in step S100. The extraction of the obstacle can be performed by using a well-known image recognition technology. For example, an element in the front picture can be extracted as an obstacle when the element is similar to the shape, size or other features of various obstacles (e.g., a walker, a vehicle or the like) memorized in advance.

[0026] Then, in step S120, whether an obstacle has been extracted from the front picture by the obstacle detection circuit 9 in step S110 is determined. When the obstacle has been extracted, the process proceeds to step S130. When no obstacle is extracted, the process returns to step S100.

[0027] In step S130, the obstacle direction (a direction to the obstacle seen from the viewpoint 19) of an extracted obstacle, the obstacle distance (the distance to the obstacle from the vehicle 2) and the size of the obstacle are calculated by the obstacle detection circuit 9.

[0028] Based on the vehicle speed of the vehicle 2 derived from the vehicle speed sensor 13 and the obstacle distance, a time-to-reach (TTR) for the vehicle 2 to reach the obstacle is calculated by the danger detection circuit 11 in step S140.

[0029] Then, in step S150, whether or not the calculated time-to-reach (TTR) is within a certain range (e.g., equal to or smaller than 10 seconds in this case) is determined by the danger detection circuit 11. In addition, whether or not the obstacle direction is within a certain scope (e.g., on a road) is determined in step S150.

[0030] When the time-to-reach (TTR) is determined to be equal to or smaller than 10 seconds and the obstacle is determined to be in the certain scope, the obstacle is determined as being unsafe, and the process proceeds to step S160. On the other hand, when the time-to-reach (TTR) is determined to be greater than 10 seconds or the obstacle is not in the certain scope, the process returns to step S100.

[0031] Then, in step S160, whether or not the time-to-reach (TTR) calculated in step S140 is within a predetermined range (e.g., 5 to 10 seconds in this case) is determined by the danger detection circuit 11. When TTR is within the predetermined range, the process proceeds to step S170. When TTR is not within the predetermined range, the process proceeds to step S220.

[0032] In step S170, a radius R of the orbit 27 of the virtual image 21 is determined by the danger detection circuit 11 based on an equation 1.
R=K×(S/D)  (Equation 1)

[0033] In the above equation 1, K is a constant, S is a size of the obstacle in a unit of meter, and D is the obstacle distance in a unit of meter.

[0034] In step S180, it is determined by the danger detection circuit 11 whether the obstacle is a human or another vehicle. This determination can be conducted by using a well-known image recognition technology. For example, the shape, size, and other features are memorized in advance respectively for the human and vehicle, and the obstacle is determined as one of the human or the vehicle by the comparison of the shape, size, and other features. When the obstacle is determined as human, the process proceeds to step S190. When the obstacle is determined as the vehicle, the process proceeds to step S200.

[0035] In step S190, the shape of the orbit 27 is determined as a vertically elongated ellipse by the image output circuit 3. In step S200, the shape of the orbit 27 is determined as a horizontally elongated ellipse by the image output circuit 3.

[0036] In step S210, the image data is output to the liquid crystal panel 5 from the image output circuit 3. The virtual image displayed by the image data, that is, the virtual image 21 in a circular shape, moves on the orbit 27 arranged in a surrounding manner relative to the obstacle in a view of the driver as shown in FIG. 4 and FIG. 5 (e.g., the vehicle 23 in FIG. 4, the walker 25 in FIG. 5). The position, the radius R, and the shape (a vertically elongated ellipse/a horizontally elongated ellipse) of the orbit 27 in the image data are determined in the following manner.

[0037] The direction of the obstacle is calculated by the obstacle detection circuit 9, as stated above, in step S130. The obstacle direction of an obstacle 33 is represented as a combination of X direction (a horizontal direction) and Y direction (a vertical direction) as shown in FIG. 3D. In a memory unit of the display unit 1 (not illustrated), a conversion table that converts the obstacle direction of the obstacle 33 calculated by the obstacle detection circuit 9 to display coordinates in the liquid crystal panel 5, as shown in FIG. 3C, is memorized. When a pixel 31 in the liquid crystal panel 5 is lit as shown in FIG. 3A according to the display coordinates converted by the conversion table, a virtual image 21 (in FIG. 3B) generated by the pixel 31 is positioned at a position of the obstacle 33 as shown in FIGS. 3B and 3D in a view of the driver.

[0038] The image output circuit 3 calculates, based on the obstacle direction calculated by the obstacle detection circuit 9 and the conversion table stated above, the display coordinates corresponding to the obstacle 33 in the liquid crystal panel 5, and the display coordinates are used as a center of the orbit 27. Further, the radius of the orbit 27 is set by using the radius R calculated in step S170. That is, the radius of the orbit 27 along the longer axis is set to the radius R, and the radius along the shorter axis is set to a value that is calculated by multiplying a fixed coefficient of, for example, 0.8 to the radius R.

[0039] Furthermore, the shape of the orbit 27 (a vertical ellipse/a horizontal ellipse) is set to the shape that is determined in step S190 or step S200. On the other hand, when the time-to-reach (TTR) is determined to be smaller than 5 seconds in step S160, the process proceeds to step S220, and a beep sound is output from the speaker 14.

[0040] 3. The Advantageous Effects of the Display Unit

[0041] The display unit 1 provides the following effects.

[0042] (i) The virtual image displayed by the display unit 1 is, as shown in FIGS. 4 and 5, the virtual image 21 partially occupying the orbit 27 and sequentially moving around the obstacle. In this case, the vibration of the virtual image 21 is hardly recognized by the driver because of the continuous movement of the virtual image 21, that is, because a position of the virtual image 21 at a certain point in time is different from a position of the virtual image 21 at a next point in time. For example, suppose that the virtual image 21 stands at the position as shown in FIG. 4 and FIG. 5 at a time t1, and the virtual image 21 circles around the orbit 27 to return to the same position at a time t2. Even if the position of the virtual image 21 at the time t2 is different from the position at the time t1 due to the vibration of the vehicle 2, the difference is hardly recognized by the driver because of the travel of the virtual image 21 away from the above position during the time between t1 and t2. In other words, the display unit 1 can display the virtual image without allowing the driver to recognize the vibration.

[0043] (ii) Because the display unit 1 displays an obstacle (e.g., the vehicle 23, the walker 25) as a virtual image, the display unit 1 allows a driver to easily recognize the obstacle in an intelligible manner.

[0044] (iii) The display unit 1 sets the radius of the orbit 27 to be small when the obstacle is small in size and when the obstacle is distant from the vehicle. Therefore, the display of the virtual image on the orbit 27 is provided in a fitting manner that naturally represents a sense of size and distance.

[0045] (iv) The display unit 1 uses different orbit shapes for the obstacle when the obstacle is human or when the obstacle is vehicle. That is, in other words, though the shape of the orbit 27 is a horizontally elongated oval in correspondence to the shape of the vehicle 23 as shown in FIG. 4, the shape of the orbit 27 is a vertically elongated oval in correspondence to the shape of the human 25 as shown in FIG. 5. As a result, even when the driver hardly distinguish in his/her direct view whether the obstacle is the vehicle 23 or the human 25, the obstacle can be instantaneously recognized as the vehicle 23 or the human 25 based on the shape of the orbit 27.

[0046] (v) When the display unit 1 determines whether or not an obstacle is dangerous, and a virtual image to move around an obstacle is displayed only when it is determined to be dangerous (step S150 in FIG. 2). Therefore, a virtual image is not displayed for the obstacle which is not dangerous, thereby preventing annoying feeling of the driver.

[0047] (vi) The display unit 1 only outputs the beep warning sound without displaying the virtual image circling around the obstacle when the time-to-reach (TTR) is very short, because there is no time for the driver to confirm the virtual image in that case. Therefore, the unsafe condition can be conveyed to the driver in an appropriate manner.

[0048] Although the present invention has been fully described in connection with the preferred embodiment thereof with reference to the accompanying drawings, it is to be noted that various changes and modifications will become apparent to those skilled in the art.

[0049] For example, the speed of the vehicle may be received from an engine ECU, or from other devices, instead of the speed sensor 13.

[0050] Further, the unsafeness of the obstacle may be determined by using a prediction of the movement of the obstacle in a later time based on the movement of the obstacle at the time of the determination.

[0051] Furthermore, though the obstacle is recognized by using the sensing machinery such as the camera 7 on the vehicle 2 in the above embodiment, positional information of the obstacle may be obtained by a method externally from outside of the vehicle 2, that is, from an infrastructure such as a vehicle-to-vehicle communication or the like, and the virtual image may be displayed based on the positional information from outside.

[0052] Furthermore, the extraction and the recognition of the obstacle may be performed by a laser radar, a millimeter wave radar, a stereo camera, or by a combination of these sensors, instead of using the camera 7.

[0053] Furthermore, the TTR (i.e., 10 seconds and 5 seconds) used in steps S150 and S160 in FIG. 2 may be changed to other values. That is, the TTR may be determined based on the time to stop according to the speed of the vehicle 2, or may be determined based on the time to travel a certain distance (e.g., a distance ‘too far to watch’ for the driver) such as 200 meters.

[0054] Furthermore, as shown in FIG. 6, the orbit 27 may always be a perfect circle that circles around the obstacle (i.e., the walker 25 in FIG. 6). In this case, the processing in the display unit 1 can be simplified because it is not necessary to change the shape of the orbit 27 depending on the obstacle. The radius of the perfect circle may be set to the radius R.

[0055] Such changes and modifications are to be understood as being within the scope of the present invention as defined by the appended claims.