DRIVING ASSISTANCE SYSTEM AND METHOD FOR DRIVER

[0001] The present application claims priority to Korean Patent Application No. 10-2020-0136846, filed Oct. 21, 2020, the entire contents of which is incorporated herein for all purposes by this reference.

[0002] The present disclosure relates to a technology capable of monitoring whether a driver's glare occurs by sunlight and the like and assisting the driver's driving.

[0003] In general, there is a trend to extend a windshield on a vehicle front so that visibility is sufficiently secured and a bright interior atmosphere is created. In particular, the front windshield is gradually becoming larger in order to secure the front view directly related to safety.

[0004] However, as the front windshield of the vehicle becomes larger, the driver's field of vision is widened, but the amount of sunlight incident through the front windshield and the amount of light that obstructs the driver's view are relatively increased, which act as factors impeding safe driving. In particular, when driving while facing the sun, the driver may temporarily lose the front view due to glare in severe cases.

[0005] Conventionally, in order to prevent the driver's glare by sunlight, the glass surface of the vehicle was tinted or the sun visor was manually operated.

[0006] However, tinting the entire front windshield can cause a secondary accident by obscuring the driver's view. In the case of a manual sun visor, there is an inconvenience in that the sun visor has to be manually operated according to the movement of sunlight with respect to the top/left/right of the field of view in a moving vehicle.

[0007] The matters described as the background art above are only for improving the understanding of the background of the present disclosure, and should not be taken as acknowledging that they correspond to the prior art already known to those of ordinary skill in the art.

[0008] The present disclosure has been proposed to solve this problem, and a purpose of the present disclosure is to provide a system and method for assisting a vehicle driving by preventing the driver's glare caused by a light source located in front of the vehicle.

[0009] In order to achieve the above purpose, the driving assistance system for a driving according to the present disclosure includes: a sensor that senses a state in front of a vehicle corresponding to a driver's field of vision through a vehicle windshield; a detector that photographs the driver's face and detects an illumination distribution of the face or a state of the driver based on a photographed image of the driver; and a processor that determines whether or not the driver's glare occurs based on the state in front of the vehicle sensed by the sensor and the illumination distribution of the face or the state of the driver detected by the detector.

[0010] The sensor may be connected to a front camera that is installed to face the front of the vehicle and photographs an image of the front of the vehicle and may sense a light source from the front image photographed by the front camera.

[0011] The sensor may divide the front image into a plurality of ranges and sense a location of the light source based on an illuminance of each divided range.

[0012] The detector may be connected to a recognition camera that is installed to face the driver in an interior of the vehicle and photographs the driver's facial image.

[0013] The detector may recognize the driver's eyes from the facial image photographed by the recognition camera and detect the state of the driver based on the recognized driver's eyes.

[0014] The detector may recognize the driver's eyeball area from the facial image photographed by the recognition camera and detect an illuminance of a divided eyeball area.

[0015] The processor may determine that the driver's glare occurs when the detector detects the illumination distribution of the face in a state where the illuminance of the eyeball area is equal to or greater than a predetermined illumination, or when the detector detects the state of the driver where an opening of the driver's eyes is equal to or less than a predetermined opening amount.

[0016] The system may further include a front camera that is installed on the vehicle and photographs the front of the vehicle; and a controller that displays the front image photographed by the front camera on a display device when the processor determines the driver's glare occurs.

[0017] The controller may vary an illuminance distribution of the front image photographed by the front camera and display the image with the varied illuminance distribution on the display device.

[0018] In order to achieve the above purpose, the driving assistance method for a driver according to the present disclosure includes the steps of: sensing a state in front of a vehicle corresponding to a driver's field of vision through a vehicle windshield, and photographing the driver's face; detecting an illumination distribution of a face or a state of the driver based on the photographed image of the driver; and determining whether or not the driver's glare occurs based on the photographed front image of the vehicle and the detected illuminance distribution of the face or the detected state of the driver.

[0019] The step of sensing the state in front of the vehicle and photographing the driver's face may further include the step of sensing a light source from a photographed front image after the step of photographing an image of the front of the vehicle with a front camera installed to face the front of the vehicle, sensing the state in front of the vehicle and photographing the driver's face.

[0020] In the step of sensing the light source, the front image may be divided into a plurality of ranges, and a location of the light source may be sensed based on an illuminance of each divided range.

[0021] In the step of sensing the state in front of the vehicle and photographing the driver's face, the driver's facial image may be photographed with a recognition camera installed to face the driver in an interior of the vehicle.

[0022] In the step of detecting the illuminance distribution of the face or the state of the driver, the driver's eyes may be recognized from the facial image photographed by the recognition camera, and the state of the driver may be detected based on the recognized driver's eyes.

[0023] In the step of detecting the illuminance distribution of the face or the state of the driver, the driver's eyeball area may be recognized from the facial image photographed by the recognition camera, and an illuminance of a divided eyeball area may be detected.

[0024] In the step of determining whether the driver's glare occurs, it may be determined that the driver's glare occurs when the illuminance distribution of the face is detected in a state where an illuminance of the eyeball area is equal to or greater than a predetermined illuminance, or when the state of the driver where an opening of the driver's eyes is equal to or less than a predetermined opening amount is detected.

[0025] The method may further include the step of displaying a front image photographed by a front camera photographing the front of the vehicle on a display device when it is determined that the driver's glare occurs, after the step of determining whether or not the driver's glare occurs.

[0026] In the step of displaying of the front image, an illuminance distribution of the front image photographed by the front camera may be varied and the image with the varied illuminance distribution may be displayed on the display device.

[0027] According to the driving assistance system and method for a driver of the present disclosure, there is an effect of determining whether or not the driver's glare occurs in the view from which the driver looks forward of the vehicle.

[0028] In addition, when the driver's glare is determined, a front image is displayed to the driver, and a phenomenon in which backlight is generated in the front image can be prevented.

[0035] Specific structural or functional descriptions of the embodiments of the present disclosure disclosed in the present specification or application are only exemplified for the purpose of describing the embodiments according to the present disclosure, and the embodiments according to the present disclosure are implemented in various forms and should not be construed as being limited to the embodiments described in the present specification or application.

[0036] Since the embodiment according to the present disclosure may have various changes and forms, specific embodiments will be illustrated in the drawings and described in detail in the present specification or application. However, this is not intended to limit the embodiment according to the concept of the present disclosure with respect to a specific disclosed form, and should be understood to include all changes, equivalents or substitutes included in the spirit and scope of the present disclosure.

[0037] Terms such as first and/or second may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one element from another element, for example, without departing from the scope of the present disclosure, a first element may be termed a second element, and similarly the second component may also be referred to as the first component.

[0038] When a component is referred to as being “connected” or “contacted” to another component, it may be directly connected or contacted to another component, but it is understood that another component may exist in between. On the other hand, when it is said that a certain element is “directly connected” or “directly contacted” to another element, it should be understood that another element does not exist in the middle. Other expressions describing the relationship between elements, such as “between” and “immediately between” or “adjacent to” and “directly adjacent to”, etc., should be interpreted similarly.

[0039] The terms used herein are used only to describe specific embodiments, and are not intended to limit the present disclosure. The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate that an embodied feature, number, step, operation, component, part, or combination thereof exists, and it should be understood that it does not preclude in advance the possibility of the existence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof.

[0040] Unless defined otherwise, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Terms such as those defined in commonly used dictionaries should be interpreted as meanings consistent with the context of the related art, and unless explicitly defined in the present specification, they are not to be interpreted in an ideal or excessively formal meaning.

[0041] Hereinafter, the present disclosure will be described in detail by describing preferred embodiments of the present disclosure with reference to the accompanying drawings. Like reference numerals in each figure indicate like elements.

[0042] FIG. 1 is a block diagram of a driving assistance system for a driver according to an embodiment of the present disclosure.

[0043] Referring to FIG. 1, a driving assistance system for a driver according to an embodiment of the present disclosure includes: a sensor 10 that senses a state in front of a vehicle corresponding to a driver's field of vision through a vehicle windshield; a detector 20 that photographs the driver's face and detects an illuminance distribution of the face or a state of the driver based on the photographed image of the driver; and a determinator module 30 that determines whether or not the driver's glare occurs based on the state in front of the vehicle sensed by the sensor 10 and the illuminance distribution of the face or the state of the driver detected by the detector 20.

[0044] The sensor 10, the detector 20, the determinator module 30, and the controller 40 according to an exemplary embodiment of the present disclosure may be implemented by an algorithm configured to control the operations of various components of a vehicle, a non-volatile memory (not shown) configured to store data relating to software instructions for reproducing the algorithm, and a processor (not shown) configured to perform the operations described below using the data stored in the memory.

[0045] Here, the memory and the processor may be implemented as separate chips. Alternatively, the memory and processor may be implemented as a single chip integrated with each other. A processor may take the form of one or more processors.

[0046] In addition, the sensor 10, the detector 20, the determinator module 30, and the controller 40 according to an exemplary embodiment of the present disclosure may communicate with each other through a communication method such as CAN, LIN, MOST and the like, but the present disclosure is not limited thereto.

[0047] The sensor 10 senses a state in front of the vehicle, where the front of the vehicle may be a range corresponding to the field of view that the driver sees through the vehicle windshield. In particular, the sensor 10 may detect a state such as illuminance or light distribution introduced from the front of the vehicle.

[0048] The detector 20 may photograph the driver's face and detect an illuminance distribution of the face from the photographed facial image. In particular, the detector 20 may detect the illuminance around the driver's face and eyes by detecting the illuminance distribution reflected on the driver's face.

[0049] In addition, the detector 20 may detect the driver's facial expression or the state of eyes open from the driver's face. Specifically, the detector 20 may detect whether the driver's expression is a frown expression or the driver's eyes are slightly opened or closed, and the like.

[0050] The determinator module 30 may determine whether or not the driver's glaring occurs based on the state in front of the vehicle sensed by the sensor 10 and the illuminance distribution of the face or the state of the driver detected by the detector 20.

[0051] Specifically, the determinator module 30 may determine that the driver's glare occurs when the illuminance flowing through the windshield is strong which is sensed at the front of the vehicle by the sensor 10 and when the facial expression of the driver detected by the detector 20 is a frown expression or the driver's eyes are slightly opened or closed, or the illuminance around the driver's eyes is strong.

[0052] Therefore, according to the present disclosure, there is an effect of determining whether or not the driver's glare occurs in the driver's field of view looking ahead of the vehicle.

[0053] FIG. 2 illustrates a front image of a vehicle photographed by a front camera 11 according to an embodiment of the present disclosure.

[0054] Referring further to FIG. 2, the sensor 10 is connected to the front camera 11 that is installed to face the front of the vehicle and photographs an image of the front of the vehicle, and the sensor 10 can sense a light source from the front image photographed by the front camera 11.

[0055] Here, the light source may be the sun, and may be a light irradiated from the front or a reflector that reflects ambient light.

[0056] In one embodiment, the front camera 11 may be a multifunction front recognition camera 21 which is a multi function camera (MFC). The multifunction front recognition camera 21 detects the shape information on the objects located in front of the vehicle, such as vehicles, lanes, road boundaries, signs, two-wheeled vehicles, pedestrians, the lamp of an opposite vehicle and the like. In particular, in a case where a stereo camera is used, distance information may be additionally obtained.

[0057] Specifically, the multifunctional front recognition camera 21 is composed of a device that blocks indoor reflected light and shields electromagnetic waves, a camera module composed of an image sensor and a lens, a digital signal processing processor that converts and analyzes the collected image information, a microcomputer and the like.

[0058] In another embodiment, the front camera 11 may be a camera facing the front of vehicle, and may be a camera for photographing the front of the vehicle to form a Surround View Monitor (SVM).

[0059] In particular, the front camera 11 is located inside the vehicle, and the front camera 11 can photograph the outside of the vehicle through the windshield in the interior of the vehicle.

[0060] FIG. 3 illustrates a process of sensing a location of a light source according to an embodiment of the present disclosure, and FIG. 4 illustrates a histogram of illuminance around the light source according to an embodiment of the present disclosure.

[0061] Referring further to FIGS. 3 to 4, the sensor 10 may divide the front image into a plurality of ranges (pixels), and sense the location of the light source based on the illuminance of each of the divided ranges.

[0062] The sensor 10 may determine whether the average value of the light value (LV) of the front image is equal to or greater than a predetermined average illuminance. Also, the sensor 10 may divide the front image into a plurality of ranges (pixels) and sense the illuminance of the plurality of ranges.

[0063] Also, when the sensor 10 determines that the average value of the illuminance of the front image is equal to or greater than the predetermined average illuminance, the front image may be pre-processed. In particular, a plurality of ranges for the front image is colored as hue (H), saturation (S), and value (V) of a Hue Saturation Value (HSV) model, or preprocessing may be performed to express an image with brightness information using a gray model. In another embodiment, the sensor 10 may process the front image as an RGB model.

[0064] Also, the sensor 10 may generate a binary map where a pixel constituting the front image is processed as white when the image pixel intensity is equal to or greater than a predetermined brightness, and the pixel is processed as black when the image pixel intensity is less than the predetermined brightness. The sensor 10 may sense that the light source is located in a portion treated with white in the generated binary map.

[0065] Additionally, the sensor 10 may extract a maximum set having the largest size among the pixel sets composed of white in the binary map through a labeling algorithm. The sensor 10 may sense that the light source is located in the maximum set having the largest size and extract a histogram around the maximum set having the largest size.

[0066] In addition, the sensor 10 may project the front image on the binary map through a back projection process and sense the center point of the light source therefrom. In particular, it is possible to extract a histogram for the maximum set having the largest size through the back projection on the binary map, and sense the location of the light source from the distribution of the extracted histogram.

[0067] In one embodiment, when the light source is the sun, the histogram distribution around the sun is as shown in FIG. 4.

[0068] FIG. 5 illustrates a facial image of a driver and a histogram of illuminance around the face and eyes according to an embodiment of the present disclosure.

[0069] The detector 20 may be connected to the recognition camera 21 that is installed to face the driver in the interior of the vehicle and photographs the driver's facial image.

[0070] In one embodiment, the recognition camera 21 may be a Driver State Warning (DSW) camera. The recognition camera 21 may photograph the driver's facial image while facing the driver's face, and the detector 20 may detect the state of the driver from the photographed driver's facial image.

[0071] In one embodiment, the detector 20 may recognize the driver's eyes from the facial image photographed by the recognition camera 21, and detect the state of the driver based on the recognized driver's eyes.

[0072] The detector 20 may recognize the driver's eyes from the facial image photographed by the recognition camera 21. In one embodiment, the detector 20 may extract a feature point corresponding to the eye from the facial image, and in another embodiment, the detector 20 may histogram the illuminance distribution for each pixel to recognize the driver's eyes. The driver's facial image and the histogram of the illuminance distribution around the eyes may be extracted as shown in FIG. 5, and the detector 20 may recognize the eyes through the histogram of the illuminance distribution of the facial image.

[0073] In particular, the detector 20 may detect the driver's facial expression or the driver's eye opening state based on the driver's eyes recognized from the facial image photographed by the recognition camera 21. For example, the detector 20 may detect whether the recognized driver's eyes are in a grimaced state, or whether the driver's eyes continue to remain closed or whether the driver's eyes continue to remain open slightly, and the like.

[0074] That is, the detector 20 may detect whether or not the driver's field of view is not secured as the driver's eyes are irradiated with strong light.

[0075] In another embodiment, the detector 20 may recognize the driver's eyeball area from the facial image photographed by the recognition camera 21 and detect the illuminance of the divided eyeball area.

[0076] Specifically, the detector 20 may detect the illuminance of the eyeball area around the driver's eyes by dividing the facial image into the facial area and the eyeball area and detecting the illuminance of each area.

[0077] In one embodiment, the detector 20 may detect a difference between the average illuminance of the facial area and the average illuminance of the eyeball area. If the difference between the average illuminance for the facial area and the average illuminance for the eyeball area is equal to or greater than a predetermined illuminance difference, the determinator module 30 may determine there is the driver's glare state. Specifically, the detector 20 may extract the average of the Y data histogram.

[0078] The determinator module 30 may determine that the driver's glare occurs when the detector 20 detects the illuminance distribution of the face as the illuminance of the eyeball area equal to or greater than a predetermined illuminance, or when the detector 20 detects the state of the driver as an opening of the driver's eyes equal to or less than a predetermined opening amount.

[0079] Also, the determinator module 30 may determine the driver's glare occurs when the average value of the illuminance of the front image sensed by the sensor 10 is greater than a predetermined range (normal illuminance condition).

[0080] The system may further include the front camera 11 that is installed on the vehicle and photographs the front of the vehicle; and the controller 40 that displays the front image photographed by the front camera 11 on a display device 50 (e.g., a screen) when the determinator module 30 determines that the driver's glare occurs.

[0081] Specifically, the front camera 11 may photograph the front image in a WIDE mode (wide area visible up to the left/right road). In addition, the controller 40 may crop and display the front image so that the lower bumper of the vehicle and the road below are not visible so that the front image is close to the driver's field of view.

[0082] In particular, the controller 40 may vary the illuminance distribution for the front image photographed by the front camera 11, and display the image with the varied illuminance distribution on the display device 50.

[0083] In one embodiment, the controller 40 may vary the illuminance distribution of the front image by performing preprocessing such as histogram equalization on the front image photographed by the front camera 11.

[0084] That is, the controller 40 may vary the illuminance distribution of the front image so as to attenuate the difference in the illuminance distribution in order to prevent a phenomenon in which backlight is generated in the front image by the light source sensed by the sensor 10.

[0085] FIG. 6 is a flowchart of a driving assistance method for a driver according to an embodiment of the present disclosure.

[0086] Reference further to FIG. 6, the driving assistance method for a driver according to an embodiment of the present disclosure includes the steps of sensing a state in front of the vehicle corresponding to the driver's field of view through the vehicle windshield, and photographing the driver's face (S100); detecting the illumination distribution of the face or the state of the driver based on the photographed image of the driver (S200); and determining whether or not the driver's glare occurs based on the photographed front image of the vehicle and the detected illuminance distribution of the face or the detected state of the driver (S400).

[0087] The step of sensing the state in front of the vehicle and photographing the driver's face (S100) may further include the step of sensing a light source from the photographed front image (S300), after the step of photographing the image of the front of the vehicle with the front camera 11 installed to face the front of the vehicle, sensing the state in front of the vehicle and photographing the driver's face.

[0088] In the step of sensing the light source (S300), the front image may be divided into a plurality of ranges, and the location of the light source may be sensed based on the illuminance of each divided range.

[0089] In the step of sensing the state in front of the vehicle and photographing the driver's face (S100), the driver's facial image may be photographed with the recognition camera 21 installed to face the driver in the interior of the vehicle.

[0090] In the step of detecting the illuminance distribution of the face or the state of the driver (S200), the driver's eyes are recognized from the facial image photographed by the recognition camera 21, and the state of the driver may be detected based on the recognized driver's eyes.

[0091] In step of detecting the illuminance distribution of the face or the state of the driver (S200), the driver's eyeball area may be recognized from the facial image photographed by the recognition camera 21, and the illuminance of the divided eyeball area may be detected.

[0092] In the step of determining whether or not the driver's glare occurs (S400), it is determined that the driver's glare occurs when the illuminance distribution of the face is detected as the illumination of the eyeball area equal to or greater than a predetermined illuminance or the state of the driver is detected as the opening of the driver's eyes equal to or less than a predetermined opening amount.

[0093] After the step of determining whether or not the driver's glare occurs (S400), the method may further include the step of displaying the front image photographed by the front camera 11 photographing the front of the vehicle on the display device 50 when it is determined that the driver's glare occurs (S500).

[0094] In the step of displaying the front image (S500), the illuminance distribution of the front image photographed by the front camera 11 may be varied, and the image with the varied illuminance distribution may be displayed on the display device 50.

[0095] Although shown and described with respect to specific embodiments of the present disclosure, it will be apparent to those of ordinary skill in the art that the present disclosure can be variously improved and changed without departing from the spirit of the present disclosure provided by the following claims.