US 6,859,705 B2Grant
Method for operating a pre-crash sensing system with object classifier in a vehicle having a countermeasure system
Issue Date:2005-02-22
•19 Claims
•6 Drawing Sheets
Abstract
A control system (10) for an automotive vehicle (50) coupled to a countermeasure system having a countermeasure includes an object sensor system (18) generating an object signal, an object distance signal, an object azimuth position signal, and object relative velocity signal. The control system (10) further includes an object classifier coupled to the object sensor system (18) generating an object classification signal in response to the object signal and a controller coupled to the object sensor object classifier for activating the countermeasure (42) in response to the object distance, object azimuth position, relative velocity and the object classification signal.
Metadata
Assignee
- Ford Global Technologies, LLC
Inventors
- Manoharprasad K. Rao
- Kwaku O. Prakah-Asante
- Gary Steven Strumolo
- Samuel Edward Ebenstein
Application Information
Application Number:US 09/683,884
Filing Date:2002-02-27
Priority Date:2001-09-21
Art Unit:3661
Patent Drawings (6 sheets)
Description
Cross Reference to Related Applications
[00002] This application claims the benefit of Provisional Application Ser. No. 60/324,137 filed Sep. 21, 2001.
[00003] The present invention is related to U.S. Applications 09/683,774, filed Feb. 13, 2002, entitled “Method For Operating A Pre-Crash Sensing System In A Vehicle Having A Countermeasure System” and Ser. No. 09/683,782, filed Feb. 13, 2002, entitled “Method For Operating A Pre-Cash Sensing System In A Vehicle Having A Countermeasure System Using Stereo Cameras” filed simultaneously herewith and hereby incorporated by reference.
Background of Invention
[00004] 1. Technical Field
[00005] The present invention relates to pre-crash sensing systems for automotive vehicles, and more particularly, to pre-crash sensing systems having countermeasures operated in response to pre-crash detection.
[00006] 2. Background
[00007] Auto manufacturers are investigating radar, lidar, and vision-based pre-crash sensing systems to improve occupant safety. Current vehicles typically employ accelerometers that measure decelerations acting on the vehicle body in the event of a crash. In response to accelerometers, airbags or other safety devices are employed.
[00008] In certain crash situations it would be desirable to provide information before forces actually act upon the vehicle when a collision is unavoidable.
[00009] Remote sensing systems using radar, lidar or vision based technologies for adaptive cruise control, collision avoidance and collision warning applications are known. These systems have characteristic requirements for false alarms. Generally, the remote sensing system reliability requirements for pre-crash sensing for automotive safety related systems are more stringent than those for comfort and convenience features, such as, adaptive cruise control. The reliability requirements even for safety related features vary significantly, depending upon the safety countermeasure under consideration. For example, tolerance towards undesirable activations may be higher for activating motorized belt pre-tensioners than for functions such as vehicle suspension height adjustments. Non-reversible safety countermeasures, including airbags, require extremely reliable sensing systems for pre-crash activation. However, the size of objects is typically not taken into consideration in the activation of such countermeasure devices.
[00010] It would therefore be desirable to provide a pre-crash sensing system that provides accurate determinations as to the presence of a potential collision target. It would also be desirable to provide a system that takes into consideration the size of the object detected.
Summary of Invention
[00011] The present invention provides an improved pre-crash sensing system that reduces false activations and activates a countermeasure in response to the size of the object detected.
[00012] In one aspect of the invention, a control system for an automotive vehicle coupled to a countermeasure system having a countermeasure includes an object sensor generating an object signal, an object distance signal, object azimuth position signal and object relative velocity signal. The control system further includes an object classifier coupled to the object sensor generating an object classification signal in response to the object signal and a controller coupled to the object sensor object classifier for activating the countermeasure in response to the object distance, object azimuth position, relative velocity and the object classification signal.
[00013] In a further aspect of the invention, a method of operating a pre-crash sensing system for an automotive vehicle having a countermeasure system comprises establishing a detection zone relative to the vehicle, detecting an object within the detection zone, determining an object distance, determining the object azimuth position and relative velocity, determining an object classification, and activating the countermeasure system in response to the distance, azimuth position, relative velocity and classification.
[00014] One advantage of the invention is that the size and orientation of the sensed object may be taken into consideration. This is extremely useful if the object is another automotive vehicle such as a sport utility, car or truck. By knowing the size of the vehicle, different countermeasures and different countermeasure activation modes may be chosen.
[00015] Another advantage of the invention is that unintentional or inadvertent activation of countermeasure devices is minimized.
[00016] Other advantages and features of the present invention will become apparent when viewed in light of the detailed description of the preferred embodiment when taken in conjunction with the attached drawings and appended claims.
Brief Description of Drawings
[00017] FIG. 1 is a block diagrammatic view of a pre-crash sensing system according to the present invention.
[00018] FIG. 2 is a block diagrammatic view of the controller of FIG. 1.
[00019] FIG. 3 is a top view of an automotive vehicle having a stereo pair of cameras 28, 30 mounted behind the rear view mirror.
[00020] FIG. 4 is a side view of an automotive vehicle indicating the vision sensor line of sight in front of the vehicle.
[00021] FIG. 5 is a flow chart of a method for operating the pre-crash sensing system according to the present invention.
Detailed Description
[00022] In the following figures the same reference numerals will be used to identify the same components. While the present invention is illustrated with respect to several types of remote object sensors, various types and combinations of remote object sensors may be used as will be further described below.
[00023] Referring now to FIG. 1 , a pre-crash system 10 has a controller 12. Controller 12 is preferably a microprocessor-based controller that is coupled to a memory 14 and a timer 16. Memory 14 and timer 16 are illustrated as separate components from that of controller 12. However, those skilled in the art will recognize that memory 14 and timer 16 may be incorporated into controller 12.
[00024] Memory 14 may comprise various types of memory including read only memory, random access memory, electrically erasable programmable read only memory, and keep alive memory. Memory 14 is used to store various thresholds and parameters as will be further described below.
[00025] Timer 16 is a timer such as a clock timer of a central processing unit within controller 12. Timer 16 is capable of timing the duration of various events as well as counting up or counting down. For example based on time, the acceleration of the vehicle can be determined from a velocity.
[00026] A remote object sensor 18 is coupled to controller 12. Remote object sensor 18 generates an object signal in the presence of an object within its field of view. Remote object sensor 18 may be comprised of one or a number of types of sensors including a radar 22, a lidar 24, and a vision system 26. Vision system 26 may be comprised of one or more cameras, CCD, or CMOS type devices. As illustrated, a first camera 28 and a second camera 30 may form vision system 26. Both radar 22 and lidar 24 are capable of sensing the presence and the distance of an object from the vehicle. When used as a stereo pair, cameras 28 and 30 acting together are also capable of detecting the distance of an object from the vehicle. Alternatively, as will be further described below, radar 22 or lidar 24 may be used to detect an object within a detection zone and vision system 26 may be used to confirm the presence of the object within the detection zone and to provide the size of the object to controller 12. In another embodiment of the invention vision system consisting of cameras 1 and 2 alone may use established triangulation techniques to determine the presence of an object and the distance from the vehicle as well as the object's size that may include area, height or width, or combinations thereof. Preferably, the cameras are high-speed cameras operating in excess of 100 Hz. A suitable example is a CMOS-based high dynamic range camera cable of operating under widely differing lighting and contrast conditions.
[00027] A receiver 31 may also be included within object sensor 18. The receiver may, however, be a stand alone device. Receiver 31 is also coupled to controller 12.
[00028] A vehicle dynamics detector 32 is also coupled to controller 12. The vehicle dynamics detector 32 generates a signal or signals indicative of the dynamic conditions of the vehicle. The vehicle dynamics detector 32 may comprise various numbers or combinations of sensors but preferably include a speed sensor 34, a yaw rate sensor 36, and a steering wheel angle sensor 38.
[00029] Speed sensor 34 may be one of a variety of speed sensors known to those skilled in the art. For example, a suitable speed sensor may include a sensor at every wheel that is averaged by controller 12. Preferably, controller translates the wheel speeds into the speed of the vehicle. Suitable type of speed sensors 34 may include, for example, toothed wheel sensors such as those employed on anti-lock brake systems.
[00030] Yaw rate sensor 36 preferably provides the yaw rate of the vehicle about the center of gravity of the vehicle. The yaw rate measures the rotational tendency of the vehicle about an axis normal to the surface of the road. Although yaw rate sensor is preferably located at the center of gravity, those skilled in the art will recognize that the yaw rate sensor may be located in various locations of the vehicle and translated back to the center of gravity either through calculations at the yaw rate sensor 36 or through calculations within controller 12 in a known manner.
[00031] Steering wheel angle sensor 38 provides a steering wheel angle signal to controller 12. The steering wheel angle signal corresponds to the steering wheel angle of the hand wheel of the automotive vehicle.
[00032] A global positioning system (GPS) 40 may also be coupled to controller 12. GPS 40 system generates a position of the host vehicle in response to satellites. Controller 12 may use this information in determining the dynamics of the host vehicle.
[00033] A transponder 41 may also be coupled to controller 12. Transponder 41 may generate information from controller 12 and transmit it to other vehicles upon the reception of a predetermined frequency signal from another vehicle. Also, transponder 41 may always be activated and broadcasting vehicle information to other vehicles. Transponder 41 and receiver 31 may be located in a common location and integrally formed therewith.
[00034] Controller 12 is used to control the activation of a countermeasure system 42. Each countermeasure may have an individual actuator associated therewith. In that case, controller 12 may direct the individual countermeasure actuator to activate the countermeasure. Various types of countermeasure systems will be evident to those skilled in the art. Examples of a countermeasure within countermeasure system include occupant belt pretensioning, bumper height changing including nose dipping, braking, the pre-arming of internal airbags, the deployment of exterior or internal airbags, pedal control, steering column position, head restraint and knee bolster control. Preferably, controller 12 is programmed to activate the appropriate countermeasure in response to the inputs from the various sensors. As will be described below, the controller may choose the countermeasure based on the type and orientation of the target vehicle.
[00035] Referring now to FIG. 2 , a portion of controller 12 is illustrated in further detail. Controller 12 has an object classifier 44 therein. Object classifier 44 may be implemented in hardware or software. Although object classifier 44 is illustrated as part of controller 12, object classifier 44 may be part of vision system 26 or object sensor 18. Object classifier 44 may be used to provide an object orientation and an object classification to a pre-crash decision module 46. Pre-crash decision module 46 receives information from vehicle dynamics detector 32, object classifier 44, and object sensor 18. Pre-crash decision module 46, based on the information, controls a deployment activation module based upon a rule-based computation such as a fuzzy logic control system. Deployment activation module 48 is illustrated also as a part of controller 12. However, deployment activation module may be a separate module or included within countermeasure system 42.
[00036] Object classifier 44 may compute various information based on the images received. For example, the shape and feature-based metrics may be used for potential collision assessment and countermeasure activation decisions. Vehicle features may include but are not limited to ground clearance, tire profiles, tire size, tire separation distance, the number of tires, height and width of the object, a cross-sectional contour of the vehicle including engine compartment, passenger compartment, and trunk or truck bed area, presence of bumpers, bumper height, front and rear license plates, front and rear lighting fixtures, front and rear lights, front grill, front and rear windshield wipers, exterior mounted spare tire, sideview mirrors, B and C pillar lines, towing gear, wheel well profiles, steering wheel profiles, human passenger profiles, relative positioning of the objects, rear axle and exhaust systems. Typically, the target vehicle information will be maintained over time until an accurate classification can be determined. Object classifier 44 may also be fuzzy logic-based.
[00037] Pre-crash decision module 46 combines the information such as object distance, azimuth position, relative velocity, relative acceleration, object classification and orientation, and other host vehicle information from the vehicle dynamics detector such as speed, yaw rate, and steering wheel position to assess a threat. Pre-crash decision module 46 may also be implemented in fuzzy logic. The pre-crash decision module governs the sensor inputs and based on rules deploys safety systems only to the extent that it is required according to the sensed condition, vehicle dynamics, and compatibility with the other vehicle. Preferably, reversible safety systems are deployed before a crash and non-reversible safety systems during a crash.
[00038] Pre-crash decision module 46 may also use near-zone sensing from sensors such as a radar/lidar sensor, transponder, and global positioning system to improve the reliability and robustness of the pre-crash sensing decisions. The pre-crash decision module 46 may also be included within object sensor 18 or may be a stand-alone processor or part of another vehicle system.
[00039] Referring now to FIG. 3 , a stereo pair of cameras 28, 30 are used on vehicle 50. The camera system provides information on the size, distance, and orientation of the object within a detection zone 52. The camera system alone can also be used to detect the presence of an object in the detection zone, obtain its distance, azimuth position, relative velocity, size and orientation information. For pre-crash sensing applications, it is permissible to have both radar and vision based systems to ensure good performance under all weather conditions and also to provide redundancy for improved reliability.
[00040] Referring now to FIG. 4 , automotive vehicle 50 is illustrated having a vision system 26 mounted at the back of a rear view mirror 62. A typical line of sight of the vision system, which defines the near side of the vehicle longitudinal detection zone in FIG. 3 is shown.
[00041] In contrast to vision systems, the radar sensor systems are typically mounted in front of the vehicles, behind the front grill or behind the front bumper fascia and have fields of coverage, which are unrestricted by the host vehicle's front-end.
[00042] Referring now to FIG. 5 , a method according to the present invention starts at step 70. In step 72, the detection zone 52 in front of the vehicle is monitored with the object detector. In the present example, the detection zone 52 is monitored with both radar and vision systems. If an object has not been detected in the detection zone, step 72 is again executed. If an object has been detected, step 76 is executed. In step 76, the host vehicle dynamics such as velocity, acceleration, yaw rate, and steering wheel angle are determined. In this example, the host vehicle is vehicle 50 described above. In step 78, other information such as the position using a GPS or receiver 31 is obtained.
[00043] In step 80 the relative velocity (VR), the distance D of the target vehicle from the host vehicle and the target vehicle azimuth position are determined. In step 82 the object size is determined from the object sensor. This may include the overall size of the vehicle as well as other sizes such as width, wheel base, wheel size and the other size related characteristics mentioned above. By knowing both the object width and object height, the object area may also be determined. When viewing object height, the difference between a small sports car, a full size sedan, a sport utility or light truck, and a heavy duty truck may be distinguished.
[00044] In step 84, the object is classified. The object may be classified into various classes depending on the object size and other characteristics as set forth in step 82. Also, the size of the object may be classified for orientation. In step 86 the vehicle orientation is determined. Thus, from the object sensor, the side, front or rear of the target vehicle may be determined.
[00045] The orientation may ultimately lead to a different decision as to which of the countermeasures may be activated and may also define the mode of activation of the selected countermeasures. By looking at the object area or the object height and width as a function of time, a front view of a sport utility vehicle or car may be distinguished from the side view.
[00046] In step 88, object size and orientation information is compared to object classification based deployment criteria for the elements of the countermeasure system. If the classification based deployment criteria are not satisfied then step 72 is again executed.
[00047] In step 90, when the classification based activation criteria are met, appropriate elements of the countermeasure system are activated in response to the various inputs described above. Such activation may result in choosing one countermeasure over another countermeasure due to the sensed conditions including classification or orientation. Of course, two countermeasures may be activated in response to the sensed conditions simultaneously. In step 92, the method ends after activation.
[00048] While particular embodiments of the invention have been shown and described, numerous variations and alternate embodiments will occur to those skilled in the art. Accordingly, it is intended that the invention be limited only in terms of the appended claims.
Claims
1. A pre-crash sensing system for an automotive vehicle coupled to a countermeasure system having a countermeasure, said pre-crash sensing system comprising:
an object sensor generating an object signal, an object distance signal, object azimuth position signal and object relative velocity signal;
an object classifier coupled to the object sensor generating an object classification signal corresponding to a type of a second vehicle and an orientation signal corresponding to an orientation of the second vehicle in response to the object signal; and
a controller coupled to said object sensor and said object classifier, said for controller activating said countermeasure in response to said object distance, object azimuth position, relative velocity, said orientation signal and said object classification signal.
2. A system as recited in claim 1 wherein said a vision system generates an object signal, object distance signal, object azimuth position signal, and object relative velocity signal.
3. A system as recited in claim 1 wherein said object classifier generates said object classification signal in response to at least one selected from a target vehicle ground clearance, tire profile, tire size, tire separation, number of tires, object size, cross-section contour, presence of a bumper, presence of front and rear license plates, front and rear lighting, front grill, operating front and rear windshield wipers, exterior mounted spare tire, side view mirrors, wheel well profile, steering wheel profile, human passenger profiles, rear axle and exhaust system.
4. A system as recited in claim 3 wherein said object size comprises height.
5. A system as recited in claim 3 wherein said object size comprises object area and object height.
6. A system as recited in claim 3 wherein said object size comprises width.
7. A system as recited in claim 1 wherein said object sensor comprises a vision system, a radar system, a lidar system or combinations of these sensor systems.
8. A system as recited in claim 1 further comprising a vehicle speed sensor generating a speed signal corresponding to the longitudinal speed of the vehicle; wherein said controller activates either a first countermeasure or a second countermeasure in response to the longitudinal speed signal.
9. A pre-crash sensing system coupled to a countermeasure system having a first countermeasure and a second countermeasure, said pre-crash sensing system comprising:
an object sensor generating an object signal, an object distance signal, an object azimuth position signal, and object relative velocity signal;
an object classifier coupled to the object sensor generating an object classification signal corresponding to a type of a second vehicle and an orientation signal corresponding to an orientation of the second vehicle in response to the object signal; and
a controller coupled to said object sensor and said object classifier for said controller activating said first countermeasure or said second countermeasure or said first and second countermeasures in response to said object distance, object azimuth position, relative velocity, said orientation signal and said object classification signal.
10. A system as recited in claim 9 wherein said object signal comprises an object size signal, wherein object size comprises height.
11. A system as recited in claim 9 wherein said object signal comprises an object size signal, wherein object size comprises object area and height.
12. A system as recited in claim 9 wherein said controller object classifier classifies said object and determines an object orientation in response to said object distance, said object size and said object height.
13. A method for operating a pre-crash sensing system for an automotive vehicle having a countermeasure system, said method comprising:
establishing a detection zone relative to the vehicle;
detecting an object within the detection zone;
determining an object distance, object azimuth position and relative velocity;
determining an object classification corresponding to a type of a second vehicle and an orientation signal corresponding to an orientation of the second vehicle; and
activating the countermeasure system in response to the object distance, object azimuth position, object relative velocity, orientation and classification.
14. A method as recited in claim 13 further comprising determining an object size and wherein activating the countermeasure comprises activating the countermeasure in response to object distance, azimuth position, relative velocity, classification, and object size.
15. A method as recited in claim 14 wherein determining object size comprises determining an object height; wherein activating the countermeasure system in response to the object size comprises activating the countermeasure system in response to the object height.
16. A method as recited in claim 14 wherein determining an object size comprises determining an object cross-sectional area; wherein activating the countermeasure system in response to the object size comprises activating the countermeasure system in response to the object cross-sectional area.
17. A method as recited in claim 14 wherein determining an object size comprises determining an object cross-sectional area and object height; wherein activating the countermeasure system in response to the object size comprises activating the countermeasure system in response to the object cross-sectional and object height.
18. A method as recited in claim 14 wherein prior to the step of activating, choosing the first countermeasure or the second countermeasure or the first and the second countermeasures in response to said object size.
19. A method as recited in claim 13 wherein detecting an object within the detection zone comprises detecting the object within the detection zone with a radar or lidar sensor system and confirming the presence with a vision system.
Patent Citations (26)
| Patent | Date | Inventor | Cited By |
|---|---|---|---|
| US5185701(A) | 1993-02-01 | Blackburn et al. | Examiner |
| US5343206(A) | 1994-08-01 | Ansaldi et al. | Examiner |
| US5377108(A) | 1994-12-01 | Nishio | Applicant |
| US5448484(A) | 1995-09-01 | Bullock et al. | Applicant |
| US5465308(A) | 1995-11-01 | Hutcheson et al. | Applicant |
| US5529138(A) | 1996-06-01 | Shaw et al. | Examiner |
| US5541590(A) | 1996-07-01 | Nishio | Applicant |
| US5559695(A) | 1996-09-01 | Daily | Applicant |
| US5583771(A) | 1996-12-01 | Lynch et al. | Applicant |
| US5613039(A) | 1997-03-01 | Wang et al. | Applicant |
| US5621807(A) | 1997-04-01 | Eibert et al. | Examiner |
| US5761326(A) | 1998-06-01 | Brady et al. | Applicant |
| US5835007(A) | 1998-11-01 | Kosiak | Examiner |
| US5870493(A) | 1999-02-01 | Vogl et al. | Applicant |
| US5959569(A) | 1999-09-01 | Khodabhai | Examiner |
| US5983161(A) | 1999-11-01 | Lemelson et al. | Applicant |
| US5995652(A) | 1999-11-01 | Chiu et al. | Applicant |
| US6018308(A) | 2000-01-01 | Shirai | Examiner |
| US6036225(A) | 2000-03-01 | Foo et al. | Examiner |
| US6087928(A) | 2000-07-01 | Kleinberg et al. | Examiner |
| US6092059(A) | 2000-07-01 | Straforini et al. | Applicant |
| US6147637(A) | 2000-11-01 | Morikawa et al. | Examiner |
| US6209909(B1) | 2001-04-01 | Breed | Examiner |
| US6226389(B1) | 2001-05-01 | Lemelson et al. | Applicant |
| US6311123(B1) | 2001-10-01 | Nakamura et al. | Examiner |
| WO9623290 | 1996-08-01 | Applicant |
Non-Patent Literature (9)
- Lippman, Richard. “Pattern Classification Using Neural Networks”; IEEE Communication, vol. 27, No. 11, pp. 45-65, Nov. 1989.Applicant
- Wan, Yue, “A New Edge Detector for Obstacle Detection with a Linear Stereo Vision System”, Proceedings of the Intelligent Vehicles 1995 Symposium, Sep. 25-26, 1995, Detroit, USA, sponsored by IEE Industrial Electronics Society, pp. 130-135, 1995.Applicant
- Kruger, W. “Real-Time Estimation and Tracking of Optical Flow Vectors for Obstacle Detection”, Proceedings of the Intelligent Vehicles 1995 Symposium, Sep. 25-26, 1995, Detroit, USA, sponsored by IEE Industrial Electronics Soeiety, pp. 304-309, 1995.Applicant
- Lipton, A.J., etc., “Moving Target Classification and Tracking from Real-Time Video”, Proceedings of Image Understanding Workshop, 1998.Applicant
- Weiss, I., “Model-Based Recognition of 3D Object from One View”, Proceedings of Image Understanding Workshop, 1998.Applicant
- Kamat, V. etc., “An Efficient Implementation of the Hough Transform for Detecting Vehicle License Plates Using DSP's”, Proceedings of IEEE Real-Time Technology and Applications, Los Angeles, 1995.Applicant
- Kamat, V. & Ganesan, S. “An Algorithm for Vehicle Ientification Using Digital Signal Processors”, Intl. Conf. On Signal Processing Applications and Technology, vol. 1, 1993, pp. 875-888.Applicant
- Schneiderman, H., “A Statisical Approach to 3D Object Detection Applied to Faces and Cars”, CMU-RI-TR-00-06, 2000.Applicant
- Zhao, L. and Thorpe, C., Stereo-and Neural Network-Based Pedestrian Detection, Proc. ITSC'99, Tokyo, Japan, 1999.Applicant