VEHICLE COLLISION AVOIDANCE SYSTEM WITH ENHANCED PEDESTRIAN AVOIDANCE

[0001] The present application is related to U.S. provisional applications, Ser. No. 62/129,285, filed Mar. 6, 2015, and Ser. No. 62/051,446, filed Sep. 17, 2014, which are hereby incorporated herein by reference in their entireties.

[0002] The present invention relates generally to a collision avoidance system for a vehicle and, more particularly, to a collision avoidance system that detects pedestrians in or approaching.

[0003] Use of imaging sensors in vehicle imaging systems is common and known. Examples of such known systems are described in U.S. Pat. Nos. 5,949,331; 5,670,935 and/or 5,550,677, which are hereby incorporated herein by reference in their entireties.

[0004] The present invention provides a collision avoidance system or vision system or imaging system for a vehicle that utilizes one or more sensors, such as one or more cameras (preferably one or more CMOS cameras) to sense regions exterior (such as forward) of the vehicle and/or to capture image data representative of images exterior of the vehicle, and provides a pedestrian collision warning system that is operable to generate an alert or warning to a driver of the vehicle and/or to control the brake system of the vehicle responsive to a determination that the vehicle may collide with a pedestrian approaching the path of travel of the vehicle ahead of (or behind) the vehicle. The system may determine a baseline time to collision (TTC) based on vehicle speed and pedestrian speed and distance between the vehicle and pedestrian, and the system adjusts the TTC responsive to various parameters, including vehicle parameters (pertaining to traction or braking ability of the vehicle at that time), environmental parameters, location parameters (such as the location of the vehicle being at or near where a pedestrian is more likely to be found), condition/time/place parameters (such as the location of the vehicle being at or near a location and at a particular time where a pedestrian is more likely to be found at that location) and/or driver parameters (attentiveness of driver, distractions and/or the like). For example, when the vehicle is at a location near a bus stop when the bus is at the bus stop (thus a high likelihood that pedestrians will be present), the system may increase the sensitivity and provide an earlier warning to the driver of the vehicle or may control the vehicle (such as apply the vehicle brakes) at an earlier time, when it is determined that a pedestrian may be moving in or towards the path of travel of the vehicle.

[0005] These and other objects, advantages, purposes and features of the present invention will become apparent upon review of the following specification in conjunction with the drawings.

[0017] A vehicle vision system and/or driver assist system and/or object detection system and/or alert system operates to capture images exterior of the vehicle and may process the captured image data to display images and to detect objects at or near the vehicle and in the predicted path of the vehicle, such as to assist a driver of the vehicle in maneuvering the vehicle in a rearward direction. The vision system includes an image processor or image processing system that is operable to receive image data from one or more cameras and provide an output to a display device for displaying images representative of the captured image data. Optionally, the vision system may provide a top down or bird's eye or surround view display and may provide a displayed image that is representative of the subject vehicle, and optionally with the displayed image being customized to at least partially correspond to the actual subject vehicle.

[0018] Referring now to the drawings and the illustrative embodiments depicted therein, a vehicle 10 includes an imaging system or vision system 12 that includes at least one exterior facing imaging sensor or camera, such as a rearward facing imaging sensor or camera 14a (and the system may optionally include multiple exterior facing imaging sensors or cameras, such as a forwardly facing camera 14b at the front (or at the windshield) of the vehicle, and a sidewardly/rearwardly facing camera 14c, 14d at respective sides of the vehicle), which captures images exterior of the vehicle, with the camera having a lens for focusing images at or onto an imaging array or imaging plane or imager of the camera (FIG. 1). The vision system 12 includes a control or electronic control unit (ECU) or processor 18 that is operable to process image data captured by the cameras and may provide displayed images at a display device 16 for viewing by the driver of the vehicle (although shown in FIG. 1 as being part of or incorporated in or at an interior rearview mirror assembly 20 of the vehicle, the control and/or the display device may be disposed elsewhere at or in the vehicle). The data transfer or signal communication from the camera to the ECU may comprise any suitable data or communication link, such as a vehicle network bus or the like of the equipped vehicle.

[0019] ASPECSS (Assessment methodologies for forward looking Integrated Pedestrian and further extension to Cyclist Safety Systems) is a project to develop harmonized test and assessment procedures for forward looking integrated pedestrian safety systems. See, for example, ‘ASPECSS-D1.1-FINAL-Scenariosweighting-BASt-2013-02-17-PUBLIC’, which is hereby incorporated herein by reference in its entirety.

[0020] As disclosed in the ASPECSS document (incorporated above), it may be justified to adjust the size of a safety zone depending on the pedestrian's walking speed. Therefore, the quantity safe lateral time-gap SLT is introduced. The conversion of safe lateral distance (SLD) to safe lateral time (SLT) is:

$\mathrm{SLT}=\frac{\mathrm{SLD}}{v_{\mathrm{Ped}}};$

where V_Ped is the speed component of a pedestrian lateral to the way an ego-vehicle is heading. This is linear vector algebra. ASPECSS shows that more distant pedestrians have to be reflected when they are approaching faster and less distant when they are approaching slower.

[0021] ASPECSS describes a safety zone which expands in a cone shape in front of the vehicle. The faster a potential endangered pedestrian is, the more time he or she may have to walk in front of the approaching vehicle.

[0022] FIG. 2 shows a diagram of the basic TTC calculation. The calculation assumes a contact point in the center of the vehicle, a constant vehicle speed and a constant speed of the pedestrian. In FIG. 3, the remaining distance to the collision point is calculated out of vehicle speed and TTC. The corridor (B) in which a pedestrian (or other vulnerable road user or VRU) with a constant approaching speed may be hit in is shown in FIGS. 4 and 5. Thus, if the pedestrian will be hit (at point B in FIG. 4 or to either side of B and within the determined path of travel corridor), the system may warn the driver and/or the pedestrian and/or may apply the vehicle brakes. Also shown in FIG. 4 is the case where the pedestrian walks faster so he or she passes the vehicle corridor before the vehicle arrives at his or her location (see A in FIG. 4). In case the pedestrian walks slower he or she will not reach the vehicle corridor before the vehicle arrives at his or her location (see C in FIG. 4).

[0023] The diagrams in FIG. 6 shows different scenarios which apply depending on whether the pedestrian is walking faster or slower (or if the vehicle is traveling faster or slower), one starting from a more distant position than the other for arriving at the collision point at the same moment. The examples show that, with the same detection cone angle, a faster pedestrian starting from a higher distant position gets detected later, and by that the reaction time for him or her is shorter. FIG. 7 shows in which way the detection cone angle should be adapted depending on the vehicle speed for capturing a fast approaching pedestrian in time for safe braking of the vehicle. For example, and in accordance with the present invention, the vehicle's corridor may be effectively widened (reducing the determined time to collision) for a faster moving pedestrian (or responsive to other parameters as discussed below), such that the distance at which the system may act for both the slower moving pedestrian and the faster moving pedestrian is about the same. Thus, since the system already measures the lateral position of the pedestrian relative to the vehicle, the speed of the pedestrian can be determined and used to widen the corridor (shorten the determined TTC) for the pedestrian position at which the system determines that the vehicle and pedestrian may be on course for a collision. FIGS. 8 and 9 show that not the center but the far edges of the vehicle are relevant when it comes to barely hitting or missing a pedestrian walking in the way of travel of an approaching vehicle.

[0024] Since semi-automated abrupt accelerating may be disturbing to a driver for passing an approaching pedestrian before he or she may be able to enter the path of travel of the approaching vehicle, only deceleration may be acceptable as an automated measure. The case A in FIG. 9 shows that, for allowing an inattentive pedestrian to exit the path of travel of the own approaching vehicle, a small delay may be sufficient. FIG. 10 shows a distance over time diagram (which implies the speed), from which it is apparent that an early soft braking (upon sufficient detection of a hazard) may be sufficient for gaining sufficient time which allows an inattentive pedestrian to escape or avoid collision. This soft braking early may be much safer for pedestrians and more comfortable to vehicle occupants as compared to waiting to the last moment and then applying or engaging full braking to avoid or rescue an endangered pedestrian. As can be seen in FIG. 11, since the time to engage measures is comparably high, the vehicle may flash its lights or may acoustically signal to the pedestrian. When this does not help, it may engage forward collision warning (FCW) and/or automated emergency braking (AEB) of the vehicle.

[0025] As another aspect of the invention, for implementation in active pedestrian collision avoidance or collision warning systems, it is preferred to engage actions stepwise depending on the remaining time to collision (TTC). Typically, in a first stage actuation or warning levels become elevated. Audible, visual and/or haptic measures may be actuated to draw the driver's attention to a potential collision endangered pedestrian (assuming the pedestrian continues approaching the driving path of the vehicle). The systems are often not developed enough to avoid a collision by steering (in combination with braking), by that these are meant to brake only. In another higher actuation or warning levels when the TTC is shorter, the system may prefill the brake pressure and may actively lower the torque demand of the engine. The warnings may be switched to become more obtrusive such as like sounding a beep and warning lights may flicker. At a TTC when a collision seems to become unavoidable in case of not braking the vehicle may start full braking of the vehicle.

[0026] While the last stages of actuation or warning levels may be reached very seldom, the lower levels may be reached quite often. To limit or avoid disturbing the driver too often with false positive or obviously easy to avoid conflict warnings but to brake sufficiently when it is necessary for pedestrians'safety, the OEMs aspire to optimize the parameters which lead to the early warning or warning levels. This is done by parameters.

[0027] There is a base TTC at which a system may actuate early warnings. Most of the parameters lead to earlier warnings which equates to a higher TTC value. For simplifying the concept some OEMs quantize the parameters in levels which lead to a fixed value of time which has to be added to the base TTC (from a baseline time). Any count of levels may be used. The OEM typically use three levels, such as a level for ‘wiper status.’ A level of 3 (such as engaged in heavy rain) leads to an addition of 0.6 seconds to the TTC, while a level of 1 (such as ‘Interval,’ engaged in slight rain) leads to an addition of 0.2 seconds to the TTC.

[0028] Known other parameters include:

• • [0029] Headlight condition
  • [0030] Steering Wheel angle
  • [0031] Yaw rate
  • [0032] Count of pedestrians
  • [0033] Pedestrian size (such for distinguishing children from adults—children may be more at risk)

[0034] The system of the present invention uses additional vehicle imminent parameters in determining an adjustment of the TTC (where the system may reduce the alert time if the system determines excessive tire wear or excessive brake temperature or wear or the like, where a time to stop the vehicle may be increased), such as:

• • [0035] Tire wear
  • [0036] Tire pressure
  • [0037] Tire age
  • [0038] Vehicle load (or common weight)
  • [0039] Brake temperature
  • [0040] Brake wear (effectiveness)

[0041] Additionally, the system of the present invention may also take environmental and/or temporal parameters (where the system may reduce the alert time if conditions are such that a time to stop the vehicle may be increased) into account such as:

• • [0042] Ambient Temperature
  • [0043] Weather conditions
  • [0044] Time of day
  • [0045] Date or season

[0046] A more sophisticated system may be able to detect ground or road or vehicle tire-road interface conditions. This may done by assessing the tire slip (where the system may reduce the alert time if conditions are such that a time to stop the vehicle may be increased). Such information may be generated by the ABS and TCS (traction control system) of the vehicle. Alternatively, or additionally, the system may assume a road condition by the weather forecast or may receive the road condition from a data server, specific to the position the vehicle is at that moment.

[0047] The geographical location or position may come from a navigation system with GPS. Additionally or alternatively the system may have parameters according a position dependent context information. The vehicle may use the inherent present information or may receive information from a context server. For example, at times when the navigation system indicates the system is approaching a school, the context parameter may add a higher value to the base TTC than when driving on a highway (where no pedestrian are to be expected normally).

[0048] Additional contexts at which increased or decreased parameter levels may be engaged may pertain to the current geographical location of the vehicle (where the system may reduce the alert time if the geographical location of the vehicle is at or near a location where more pedestrians are expected to be), and may include, for example:

• • [0049] In city
  • [0050] Out of city
  • [0051] In living area
  • [0052] On highway
  • [0053] Off road
  • [0054] Close to a bus stop
  • [0055] Close to a train station
  • [0056] Within a pedestrian walking zone
  • [0057] On a play street

[0058] Some contexts may be engaged in combination, such as condition, time and place in combination (where the timing of an event that occurs at a particular location at a particular time may be considered when the vehicle is determined to be at or near a particular location at or near the time for an event at that location, such that the alert time may be reduced if the system determines that the vehicle is at or near such a location at such a time) such as:

• • [0059] Close to a stadium when a game just ended (position, game schedule and time)
  • [0060] Close to a bus stop with the bus scheduled at that time (position, bus schedule and time)
  • [0061] Approaching a traffic jam that is at a full stop (people may exit their cars) (road type and traffic condition)
  • [0062] Fire alert in a building close by (exceptional event and position)

[0063] There may be off line data involved such as the map information or the bus schedule as well as online data such as the fire alert event. Sophisticated systems may keep the bus schedule updated online, by that a bus which is a minute delayed can be reflected in the TTC parameters correctly when the bus is actually present (not when it was scheduled instead).

[0064] As another aspect of the present invention, the system may also take the condition of the driver and or driver distractions into account as a parameter (where the system may reduce the alert time if it is determined that the driver may be distracted or inattentive) such as:

• • [0065] Attention and or drowsiness level of the driver (detected by an eye tracker and or by monitoring gas pedal and steering wheel actuations)
  • [0066] Volume of music in the vehicle
  • [0067] Driver is on a cellphone (hands free/not hands free)
  • [0068] Driver is drunk or otherwise impaired
  • [0069] Driver is beginner
  • [0070] Driver is an elderly driver
  • [0071] Driver is alone in the vehicle vs. accompanied by one or more passengers in the vehicle

[0072] Optionally, the same procedure may be used accordingly and simultaneously for setting (increasing) the parameters of “Present position of the Pedestrian.”

[0073] This procedure may also be used for TTC parameters of cyclists, motorcyclists, rickshaws, horse riders (vulnerable road users or VRU) or other vehicles or animals or other (potentially moving) obstacles such as dropped cargo (rolling around), rolling bushes or toys (e.g., balls, RC or autonomous crafts or drones); it means all AEB features can take advantage of the adjustment of the adaption of the thresholds for warnings or braking maneuvers.

[0074] Thus, the system of the present invention is operable to adjust or weight the processing of data associated with the vehicle traveling along a road to optimize the system's ability to warn against or avoid collision with a pedestrian. The system may increase the sensitivity of the alert (to effectively widen the vehicle path corridor in FIG. 7) to provide the alert (or to control the vehicle brake system or the like) earlier (by reducing a time to alert) responsive to parameter that are indicative of an increased likelihood that a pedestrian may be present in or near the path of travel of the vehicle and/or that are indicative of poor driving conditions (such as rain or snow or poor traction) that may require additional time and distance to stop the vehicle when the brakes are applied. The system is thus responsive to vehicle parameters (pertaining to traction or braking ability of the vehicle at that time), environmental parameters, location parameters (such as the location of the vehicle being at or near where a pedestrian is more likely to be found), condition/time/place parameters (such as the location of the vehicle being at or near a location and at a particular time where a pedestrian is more likely to be found at that location) and/or driver parameters (attentiveness of driver, distractions and/or the like), and adjusts the vehicle control or warning time and/or intensity responsive to such parameters. For example, when the vehicle is at a location near a bus stop when the bus is at the bus stop (thus a high likelihood that pedestrians will be present), the system may increase the sensitivity (and reduce the time to alert) and provide an earlier warning to the driver of the vehicle or may control the vehicle (such as apply the vehicle brakes) at an earlier time, when it is determined that a pedestrian may be moving in or towards the path of travel of the vehicle.

[0075] Thus, the system of the present invention uses vehicle inherent parameters to influence the TTC warning time (at which the driver will be alerted to a potential collision with a pedestrian). The system may also or otherwise use environmental parameters and may generate context information from several input conditions, which influence the various parameters and the TTC warning time. The system may utilize one or more cameras of the vehicle to assist in determining the presence of pedestrians and may be responsive to an output of a GPS system of the vehicle (that indicates the current geographical location of the vehicle) and/or may be responsive to an external service provider or communication system (that may provide data pertaining to bus schedules or real time bus locations and/or school crossing information and/or weather details and/or the like). The system may be responsive to the various parameters (as provided or determined or as adjusted in response to other inputs or data) to determine a time at which the system may warn the driver of the vehicle of a potential hazard (collision with pedestrian) as the vehicle is driven along a road.

[0076] Thus, the system may initially determine a potential hazard or collision with a pedestrian and generate an alert to the driver of the vehicle that the hazardous condition has been determined. If the pedestrian continues on his or her path and the driver of the vehicle does not alter the vehicle's path or speed, the system may then control the vehicle and/or generate a pedestrian alert to alert the pedestrian of the potentially hazardous condition. For example, responsive to an initial determination that a detected pedestrian is moving towards the path of travel of the vehicle, the system may generate a pedestrian alert (such as actuating the vehicle's horn or flashing the vehicle's headlights) to alert the pedestrian of a potential hazard. If the pedestrian does not alter course, the system may (if a determination is made that the vehicle may collide with the pedestrian) apply the vehicle brakes to slow down or stop the vehicle before arriving at the location where the pedestrian crosses the vehicle's path. This may be done after the processor determines a time to collision based on a determined distance to the pedestrian and determined speed of the pedestrian and speed of the vehicle, and after the collision avoidance system generates an alert to the driver of the vehicle at a threshold time before the determined collision with the pedestrian.

[0077] The collision avoidance system may be operable to apply the brakes of the vehicle to avoid collision with a determined pedestrian. Optionally, the system may adjust the degree of braking responsive to the predicted location of the pedestrian at the time that the vehicle arrives at the pedestrian's path. For example, the system may gently or lightly apply the brakes to slow the vehicle's speed responsive to a determination that the pedestrian will be exiting the path of travel of the vehicle towards the end of the determined time to collision (i.e., the pedestrian is fully or almost fully across the vehicle path by the time the vehicle arrives at the pedestrian's path). Optionally, the collision avoidance system may apply the brakes of the vehicle to stop the vehicle responsive to a determination that the pedestrian will be entering the path of travel of the vehicle towards the end of the determined time to collision (i.e., the pedestrian will likely be in the path of travel of the vehicle at the time that the vehicle arrives at the pedestrian's path). Optionally, the collision avoidance system may generate a pedestrian alert to the pedestrian responsive to a determination that the pedestrian will be entering the path of travel of the vehicle towards the end of the determined time to collision (i.e., at or before the time at which the vehicle arrives at the pedestrian's path).

[0078] The camera or sensor may comprise any suitable camera or sensor. Optionally, the camera may comprise a “smart camera” that includes the imaging sensor array and associated circuitry and image processing circuitry and electrical connectors and the like as part of a camera module, such as by utilizing aspects of the vision systems described in International Publication Nos. WO 2013/081984 and/or WO 2013/081985, which are hereby incorporated herein by reference in their entireties.

[0079] The system includes an image processor operable to process image data captured by the camera or cameras, such as for detecting objects or other vehicles or pedestrians or the like in the field of view of one or more of the cameras. For example, the image processor may comprise an EyeQ2 or EyeQ3 image processing chip available from Mobileye Vision Technologies Ltd. of Jerusalem, Israel, and may include object detection software (such as the types described in U.S. Pat. Nos. 7,855,755; 7,720,580 and/or 7,038,577, which are hereby incorporated herein by reference in their entireties), and may analyze image data to detect vehicles and/or other objects. Responsive to such image processing, and when an object or other vehicle is detected, the system may generate an alert to the driver of the vehicle and/or may generate an overlay at the displayed image to highlight or enhance display of the detected object or vehicle, in order to enhance the driver's awareness of the detected object or vehicle or hazardous condition during a driving maneuver of the equipped vehicle.

[0080] The vehicle may include any type of sensor or sensors, such as imaging sensors or radar sensors or lidar sensors or ladar sensors or ultrasonic sensors or the like. The imaging sensor or camera may capture image data for image processing and may comprise any suitable camera or sensing device, such as, for example, a two dimensional array of a plurality of photosensor elements arranged in at least 640 columns and 480 rows (at least a 640×480 imaging array, such as a megapixel imaging array or the like), with a respective lens focusing images onto respective portions of the array. The photosensor array may comprise a plurality of photosensor elements arranged in a photosensor array having rows and columns. Preferably, the imaging array has at least 300,000 photosensor elements or pixels, more preferably at least 500,000 photosensor elements or pixels and more preferably at least 1 million photosensor elements or pixels. The imaging array may capture color image data, such as via spectral filtering at the array, such as via an RGB (red, green and blue) filter or via a red/red complement filter or such as via an RCC (red, clear, clear) filter or the like. The logic and control circuit of the imaging sensor may function in any known manner, and the image processing and algorithmic processing may comprise any suitable means for processing the images and/or image data.

[0081] For example, the vision system and/or processing and/or camera and/or circuitry may utilize aspects described in U.S. Pat. Nos. 8,694,224; 7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331; 6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202; 6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452; 6,822,563; 6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935; 6,636,258; 7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229; 7,301,466; 7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287; 5,929,786 and/or 5,786,772, which are all hereby incorporated herein by reference in their entireties. The system may communicate with other communication systems via any suitable means, such as by utilizing aspects of the systems described in International Publication Nos. WO/2010/144900; WO 2013/043661 and/or WO 2013/081985, and/or U.S. Publication No. US-2012-0062743, which are hereby incorporated herein by reference in their entireties.

[0082] Optionally, the vision system may include a display for displaying images captured by one or more of the imaging sensors for viewing by the driver of the vehicle while the driver is normally operating the vehicle. Optionally, for example, the vision system may include a video display device disposed at or in the interior rearview mirror assembly of the vehicle, such as by utilizing aspects of the video mirror display systems described in U.S. Pat. No. 6,690,268 and/or U.S. Publication No. US-2012-0162427, which are hereby incorporated herein by reference in their entireties. The video mirror display may comprise any suitable devices and systems and optionally may utilize aspects of the compass display systems described in U.S. Pat. Nos. 7,370,983; 7,329,013; 7,308,341; 7,289,037; 7,249,860; 7,004,593; 4,546,551; 5,699,044; 4,953,305; 5,576,687; 5,632,092; 5,677,851; 5,708,410; 5,737,226; 5,802,727; 5,878,370; 6,087,953; 6,173,508; 6,222,460; 6,513,252 and/or 6,642,851, and/or European patent application, published Oct. 11, 2000 under Publication No. EP 0 1043566, and/or U.S. Publication No. US-2006-0061008, which are all hereby incorporated herein by reference in their entireties. Optionally, the video mirror display screen or device may be operable to display images captured by a rearward viewing camera of the vehicle during a reversing maneuver of the vehicle (such as responsive to the vehicle gear actuator being placed in a reverse gear position or the like) to assist the driver in backing up the vehicle, and optionally may be operable to display the compass heading or directional heading character or icon when the vehicle is not undertaking a reversing maneuver, such as when the vehicle is being driven in a forward direction along a road (such as by utilizing aspects of the display system described in International Publication No. WO 2012/051500, which is hereby incorporated herein by reference in its entirety).

[0083] Optionally, the vision system (utilizing the forward facing camera and a rearward facing camera and other cameras disposed at the vehicle with exterior fields of view) may be part of or may provide a display of a top-down view or birds-eye view system of the vehicle or a surround view at the vehicle, such as by utilizing aspects of the vision systems described in International Publication Nos. WO 2010/099416; WO 2011/028686; WO 2012/075250; WO 2013/019795; WO 2012/075250; WO 2012/145822; WO 2013/081985; WO 2013/086249 and/or WO 2013/109869, and/or U.S. Publication No. US-2012-0162427, which are hereby incorporated herein by reference in their entireties.

[0084] Changes and modifications in the specifically described embodiments can be carried out without departing from the principles of the invention, which is intended to be limited only by the scope of the appended claims, as interpreted according to the principles of patent law including the doctrine of equivalents.