Method and device for determining light conditions in front of a moving object, in particular a motor vehicle

[0002] The present invention relates to a method for monitoring and estimating the lighting conditions in front of a moving object in particular in front of a motor vehicle,

[0003] In German Published Patent Application No. 195 23 262, a sensor system and an evaluation system are provided to automatically switch lighting units in vehicles. The sensor system has at least one non-directional global sensor that captures the general lighting conditions around the vehicle and at least one directional sensor that captures the lighting conditions in front of the vehicle in the direction of vehicle travel. Based on the signals from the global sensor, the evaluation system can detect a change in the general lighting conditions around the vehicle, for example in twilight, and can turn the lighting units on or off. Based on the signals from the directional sensor, the evaluation system can detect a tunnel or underpass located in front of the vehicle in the direction of vehicle travel and turn the lighting units on in good time before entering the tunnel or underpass.

[0004] The disadvantage of this arrangement is the fact that the separate ambient and directional sensors tend to make the system more complex. Photocells to which the incident light is supplied with the help of optical concentrators are used as the light-sensitive elements. This makes the arrangement dependent on shutters and exposure times.

[0005] A method in which a vehicle is provided with a side-mounted optical scanning system to provide non-contact scanning of the side road area is described in European No. 0736 414. An evaluation unit is connected downstream from the scanning system. The optical scanning unit includes multiple infrared transmitting elements, arranged side-by-side, and a corresponding CCD array for scanning the road surface to detect the edge of a traffic lane. The evaluation unit is configured to determine runtime and contrast and to detect contours based on the data supplied by the scanning system.

[0006] A significant disadvantage that arises when using CCD sensors or even linear CMOS sensors to measure brightness is the fact that they can capture only a limited range of brightness dynamics. As a result, they are provided with electronic or mechanical exposure time controllers and shutters. This makes it impossible to use the signal measured at the sensor output for absolute brightness measurement because the exposure time and shutter position, respectively, should always be taken into account.

[0007] The object of the present invention is to considerably simplify the determination of lighting conditions in front of a moving object, in particular in front of a vehicle, so that suitable measures can be taken for the manual or automatic operation of the lights in front of the moving object according to the situation.

[0008] The method according to the present invention has an advantage in that it can considerably reduce the mechanical and electrical complexity in determining lighting conditions in the direction of movement in front of a moving object such as, in particular, a vehicle. This is generally achieved with a considerable reduction in the cost of the overall system. The sectoral brightness measurement that this affords vastly increases flexibility.

[0009] In the method according to the present invention for determining the lighting conditions in front of a moving object, in particular in front of a vehicle, the brightness measurement is, in principle, carried out with a two-dimensional image sensor, and one or more information values relating to the lighting conditions is generated with the help of a sectoral and/or global evaluation of the brightness values measured by the image sensor in the respective sectors used.

[0010] In one particularly advantageous embodiment of the method according to the present invention, a CMOS image sensor is used as the image sensor for the two-dimensional brightness measurement. It is suitable to use an image sensor with a non-linear conversion characteristic to make the best possible use of the existing, frequently quite narrow, current and voltage ranges involved in the high dynamics of light incidence.

[0011] In a further particularly advantageous embodiment of the method according to the present invention, a logarithmic image sensor is used for measuring brightness.

[0012] In a further suitable embodiment of the method according to the present invention, the brightness is measured by averaging pixel brightness in measurement windows.

[0013] In a further particularly suitable embodiment of the present invention, the brightness measurement is combined with an automatic traffic lane tracking system to selectively measure ambient brightness in the path of the traffic lane.

[0014] A further advantageous embodiment provides an arrangement for detecting objects, making it possible to eliminate, for example, large trucks traveling a short distance in front of the vehicle in the sectoral brightness measurement.

[0015] The arrangement for carrying out the method according to the present invention principally includes a two-dimensional image sensor, onto which the area in front of the vehicle can be or is mapped. A CMOS image sensor, which, in a particularly suitable embodiment, can be a logarithmic image sensor, is advantageously provided as the two-dimensional image sensor. In another embodiment, an image sensor that generally has an advantageous, non-linear conversion characteristic can also be provided.

[0016] The method according to the present invention or the arrangement according to the present invention is preferably used, in particular, to give the driver of a vehicle advance warning and alert him to the need to turn on the vehicle lights; alternatively, the method and/or the arrangement is used to automatically activate the vehicle lights in advance, in particular to turn them on and possibly also off again. This makes it possible to respond to tunnel entrances and exits as well as underpasses according to the situation.

[0017] The method according to the present invention or the arrangement according to the present invention is also preferably used, in particular, to alert the driver of a vehicle to the need to turn the vehicle lights on and off based on the ambient brightness measured by the two-dimensional image sensor; alternatively, the method and/or the arrangement is used to automatically activate the vehicle lights, in particular to turn them on and possibly also off again, also based on the ambient brightness measured by the two-dimensional image sensor. This makes it possible to respond to global brightness variations caused by daylight, nightfall, cloud cover, twilight, etc.

[0018] FIG. 1 shows a schematic representation of a vehicle having an image sensor that senses the area in front of the vehicle in the direction of vehicle travel.

[0019] FIG. 2 shows a schematic sketch of the scene in front of a vehicle before the vehicle enters a tunnel.

[0020] FIG. 3 shows a schematic block diagram of the components contained in the overall system.

[0021] FIG. 1 shows a schematic representation of a vehicle 1 having an image sensor 2 that senses area 20 in front of the vehicle in the direction of vehicle travel. The schematic representation illustrated in FIG. 1 shows a cross-section along the longitudinal vehicle axis and a preferred location for attaching image sensor 2. The latter is advantageously provided as high as possible behind windshield 11 in a way that provides a good view of the traffic scene in area 20 in front of vehicle 1 in the direction of vehicle travel, while image sensor 2 is also located within the active area of the windshield wipers. The traffic scene in area 20 in front of vehicle 1 in the direction of vehicle travel is preferably mapped onto the light-sensitive surface of planar, i.e. two-dimensional, image sensor 2 by a lens arrangement 21.

[0022] FIG. 2 shows a schematic sketch of area 20 located in front of vehicle 1, representing a particular scene prior to entering a tunnel 6. The traffic lane in front of the vehicle is identified by reference number 3, and its edges, i.e. limits, by reference numbers 4 and 5. In the right half of the entrance, the lane opens into tunnel 6. During the day, only a low ambient brightness is measured in the tunnel entrance area. According to one particular feature of the present invention, area 20 in front of the vehicle, in particular the traffic lane path in this area, is divided into special measurement fields 7 which are placed in different locations. These include, for example, a large global measurement field 71 right in front of the vehicle, a measurement window 72 largely directed onto traffic lane 3 at a medium distance, and a measurement window 73 directed at a greater distance and used to sense the entrance to tunnel 6.

[0023] FIG. 3 shows a schematic block diagram of the main components contained in the overall system. The overall system can be used in two different ways. The one basic application involves alerting the vehicle driver to the need to turn the vehicle lights on and off, respectively. The other fundamental application involves automatically activating the vehicle lights, in particular turning them on and possibly also off again.

[0024] Image sensor 2 supplies signals relating to the visual appearance of the traffic scene in front of the vehicle. These signals are supplied to a suitable evaluation unit 8. The latter generates the parameters needed for alerting the driver and controlling the lights, respectively, based on these signals. These parameters may be traffic lane path, ambient brightness, brightness variation along road surface 3, or even objects of a specific type in the road path, particularly directly in front of vehicle 1. Suitable algorithms and calculation methods, like those used in image processing, are used to calculate these parameters. One of the signals calculated and generated in this manner is a signal to turn the vehicle lights on and off, respectively, and is provided to a lighting control unit 9. This lighting control unit 9 switches the lighting unit, represented by a headlight 10, according to the signal supplied. Another of the signals calculated and generated in this manner is a signal for alerting the driver. This signal can be generated as an alternative or in addition to the automatic switching signal. This signal produces a suitable driver warning indication or driver information on a man-machine interface, such as, in particular, an instrument panel 11 with a lighting status indicator 12. This alerts the driver to the need to turn his lights on or off. In addition to the image information for image evaluation to measure brightness and also possibly to detect the traffic lane, evaluation unit 8 can expediently also receive information signals via other sensors that are generally identified by reference number 13. These signals can provide information, for example, about the vehicle speed.

[0025] According to the present invention, a two-dimensional, i.e. planar, image sensor is used for image sensor 2. A two-dimensional, i.e. planar, image sensor of this type includes a multiplicity of individual brightness sensors arranged side-by-side on a plane. Certain points in the vehicle surroundings, particularly in area 20 in front of the vehicle, are mapped onto specific receptors—the individual brightness sensors of image sensor 2—producing a visual impression of the vehicle surroundings. The ambient brightness can be measured on the basis of the ambient image generated in this manner. The total brightness of the monitored scene is determined by correspondingly processing the brightness signals of the individual receptors located within one of measurement windows 71, 72 or 73 of the difference scenes. The brightness can be measured in an especially suitable manner by averaging the pixel brightness, i.e. averaging the corresponding individual signals of the receptors located in the measurement window.

[0026] Through comparison with a threshold value, evaluation unit 8 determines whether or not the ambient brightness justifies illuminating the traffic lane by vehicle headlights 10. If headlights 10 are not turned on, the driver can receive the corresponding suitable signal, alerting him to the need to turn them on. If, on the other hand, they are turned on, the driver can receive a signal at a certain ambient brightness alerting him to the fact that the vehicle lights are no longer needed and he can turn headlights 10 off again. To avoid frequent contradictory indications, the threshold for determining whether the lights are turned on and off, respectively, should be described by a hysteresis.

[0027] Unlike individual brightness sensors, such as an individual photoelement, planar image sensor 2 provided according to the present invention permits a differentiated and sectoral brightness determination over the entire scene being monitored. This makes it possible to measure brightness in certain regions of the image by evaluating the signal of specific pixels. If the image sensor is positioned as shown in FIG. 1, it senses the area in front of vehicle 1. The course of travel that vehicle 1 will take can be predicted, particularly if image sensor 2 is combined with a traffic lane detection system according to one advantageous embodiment of the present invention. The illustration shown in FIG. 2 depicts the selective brightness measurement in areas along measured traffic lane 3. In the advantageous embodiment, the brightness values are measured in certain areas, image windows 71, 72, and 73, which are placed along traffic lane 3. It is not necessary for the measurement windows to be rectangular in shape. The brightness variation is advantageously measured by averaging the pixel brightness values within the individual measurement windows.

[0028] For example, it is possible to measure an excessively low brightness in measurement window 73, which lies directly in front of the entrance to tunnel 6, as shown in FIG. 2. It is therefore possible to request the driver to turn on his vehicle lights before entering tunnel 6. The vehicle lights can also be turned on automatically if the vehicle is equipped with a suitable mechanism. The lights should not be turned off too early, but rather only when adequate brightness can be measured in the areas of the scene directly in front of the vehicle. Detection can be improved by additional information from sensors 13, indicating, for example, the vehicle speed. Large measurement window 71, which extends over the entire scene, can be advantageously used to detect global brightness states, such as those caused, for example, by daylight, nightfall, twilight, cloud cover or even trees along the side of the road.

[0029] According to the present invention, the brightness measurement is therefore carried out with two-dimensional, i.e. planar, image sensor 2, and one or more information values relating to the lighting conditions is generated with the help of a sectoral and/or global evaluation of the brightness values measured by image sensor 2 in the respective sectors used, for example in measurement windows 71, 72, 73, e.g. using the individual receptors in the measurement windows. The sectoral brightness measurement obtained in this manner provides a great deal of flexibility.

[0030] The type of sectoral brightness measurement according to the present invention can be further improved by providing object detection. Image evaluation methods that have one or two simultaneously operating cameras can be used to detect objects located in front of the vehicle. For example, it is possible to determine that a large truck or another passenger car is traveling in front of or approaching the driver's own vehicle. Because the other vehicle may be very dark or very light in color, it could interfere with a global or even sectoral brightness measurement, particularly if the object is located a short distance in front of the driver's own vehicle and more or less completely fills area 20, thus occupying nearly the entire image area of image sensor 2. Object detection also makes it possible to correct the brightness measurement so that the focus is only on the area outside the detected object, thus avoiding measurement mistakes.

[0031] A CMOS image sensor is used as image sensor 2. It is very advantageous to use a logarithmic image sensor. A logarithmic variation of the light intensity/current or voltage conversion characteristic ensures that the contrast resolution remains constant over the entire brightness area. This simplifies the image processing operations that need to be carried out in evaluation unit 8. A logarithmic image sensor of this type has been developed, for example, by the Institut für Mikroelektronik Stuttgart (IMS), Allmandring 30a, D-70569 Stuttgart, and is available under the name HDRC2-EC (High Dynamic Range CMOS Evaluation Camera).

[0032] Clearly other image sensors with non-linear conversion characteristics, having an exponential or even another variation, can also be used. The important thing is to make the best possible use of what is frequently a very narrow range when converting the light intensity to current and voltage signals, respectively. In other words, the gradient of the conversion characteristic should decrease as the intensity of the light striking the image sensor increases, thus causing the magnitude of the current and voltage signals to increase more and more slowly as the light intensity rises. This means that, compared to a linear characteristic, a larger brightness area can be mapped onto a current or voltage range of a uniform size. As a result the brightness dynamics of the image sensor increases.

[0033] The method and/or the arrangement according to the present invention are used to give the driver of a vehicle advance warning and to alert him to the need to turn on the vehicle lights, or to automatically activate the vehicle lights in advance, in particular to turn them on and possibly also off again. This makes it possible, according to the present invention, to respond to a tunnel or underpass appearing in front of the vehicle according to the situation.

[0034] Alternatively, and possibly in addition to this, the method and/or the arrangement according to the present invention are used to alert the driver of a vehicle to the need to turn the vehicle lights on and off based on the ambient brightness measured by two-dimensional image sensor 2, or to automatically activate the vehicle lights, in particular to turn them on and possibly also off again, also based on the ambient brightness measured by two-dimensional image sensor 2. This makes it possible, according to the present invention, to respond to the variable ambient brightness caused by different factors according to the situation.

[0035] The advantage of the present invention lies in its ability to automatically measure brightness distribution in the direction of vehicle travel. This makes it possible to measure ambient brightness as well as brightness in certain areas of the vehicle surroundings. Based on these information values, the driver can be alerted to the need to activate the lights, or it may also be possible to activate the lights automatically. According to the present invention, the brightness is advantageously measured according to specific sectors. In particular, the brightness variation along the traffic lane can be measured in advance by combining the arrangement with an arrangement for detecting a traffic lane path and/or for detecting an object. This makes it possible to take into account tunnel entrances and underpasses appearing in the path of the vehicle at an early point in time.