Attention control for operators of technical equipment

[0002] The present invention relates to a method and to a device which is suitable for carrying out a method and device for controlling the attention of an operator of technical equipment, having a display device for displaying action-relevant information in the form of images or symbols, the duration of the display of the specific image or symbol lying below a conscious and above an unconscious perception threshold of the operator

[0003] Sensors which monitor the environment are increasingly used in technical equipment for increasing safety and operating convenience. Due to the rapid development in processor technology and in the miniaturization of integrated components, cameras are meanwhile also available at a reasonable cost.

[0004] For these reasons, camera systems are increasingly used in motor vehicles as well. In recent times, one has been working on the use of image sensors which are more sensitive at night than the human eye for improving the sight for the vehicle driver. European Patent Document EP 0 686 865 discloses a night vision system working in the far infrared range. In this connection, the image generated by the system is imaged into the lower part of the windshield. In other attempts, the image data of near infrared systems were imaged via a head-up display on a plane screen in front of the windshield. However, these systems have the negative effect that, on one hand, the vehicle driver must partially turn his/her view away from the actual road scene for monitoring the additional image information and, on the other hand, that the imaged scene partially covers the road scene. It has also turned out to be difficult to impress this additional image information of the infrared system on the driver because it does not correspond to his/her habitual wealth of sensations.

[0005] To avoid these disadvantages, it has been proposed not to offer the driver an additional graphical representation of the environment but to overlay an LED line underneath the objects to be set off within a road scene. However, such an overlay creates contrast problems and is extremely disturbing due to the constant presence of additional information within the main direction of view.

[0006] Investigations into the physiology of perception show that the attention of the human being can be guided by short, hardly perceivable changes in contrast. Commonly-owned German Patent DE 199 52 506, published Jun. 7, 2001, describes a system is known in which action-relevant information is offered, via a unit for the display thereof, to an operator of technical equipment below a conscious but above an unconscious perception threshold for a short time. In this manner, it is possible to trigger the attention of the operator without overtaxing him/her by constantly displaying unnecessary information. This is known as “visual priming.” While it is not believed necessary to the further understanding of the present invention, the entire disclosure of German Patent DE 199 52 506 is hereby incorporated by reference herein.

[0007] An object of the present invention is to find a method and a device which is suitable for carrying out a method for controlling the attention of an operator of technical equipment.

[0008] The present invention provides a method for controlling the attention of an operator of technical equipment, having a display device for displaying action-relevant information in the form of images or symbols, the duration of the display of the specific image or symbol lying below a conscious and above an unconscious perception threshold of the operator. The specific image or symbol is displayed at those locations of the field of vision of the operator at which objects are located to which his/her attention is to be drawn.

[0009] The duration of the display is adjustable that it can be displayed below a conscious perception threshold and above an unconscious perception threshold of the operator. In this manner, the optical information is presented for such a short period that it cannot be consciously realized, but nevertheless is absorbed by humans. This phenomenon, known as “visual priming” is described, for example, by A. J. Marcel in “Conscious and unconscious perception: Experiments in visual masking and word recognition,” Cognitive Psychology 15 (1983), pp. 197-237. A well-known example of this phenomenon is the substitution of specific frames in a film sequence with frames having a completely different content, which cannot be consciously perceived by an observer of the film, but which nevertheless affect the observer's perceptions and/or his later behavior.

[0010] This effect is used by the present invention to provide a subliminal warning, subliminal information, or a subliminal impact on the operator of a machine. This makes it possible, for example, to indicate a warning without using a distracting signal, such as lighting an indicator light or sounding an alarm, as is done in conventional warning systems. Though experiments described in German Patent DE 199 52 506, it was shown that the subliminal presentation of action-relevant information on a display in the operator's field of vision effectively sensitized the operator to the presented information. The subliminal presentation thus considerably shortened the reaction time of the operator when the situation for which the operator was unconsciously prepared actually occurred. Moreover, it was found that the operator was not disturbed or distracted by the subliminally presented information. It was therefore concluded that, once an appropriate situation is recognized, action-relevant information can be displayed for a short time without regard to possible negative effects on the operator's actions.

[0011] According to the present invention, the specific image or symbol is displayed at least once for a short duration, wherein the duration of the display lies below a conscious perception threshold and above an unconscious perception threshold of the operator. The unconscious perception threshold is defined as the shortest display duration at which the displayed information is unconsciously absorbed, i.e. the displayed information has an effect on the operator's later behavior. The conscious perception threshold is defined as the shortest display duration at which the operator becomes consciously aware of the displayed information.

[0012] The perception thresholds may depend on the size and brightness of the presented image or symbol as well as the relative location of the image or symbol in the field of view of the operator (e.g. in the center or near the edge of the operator's field of view). It has been shown that even a momentary presentation of information in the peripheral field of an operator's view can affect the operator's behavior.

[0013] In one test arrangement described in German Patent DE 199 52 506, a test vehicle having a controllable CRT display on the dashboard in place of the otherwise typical panel of individual instruments, was modified so as to be able to measure the driver's reaction time to road signs and traffic lights. The reaction times of an adequately large number of subjects were measured both with and without subliminal optical sensitization, To sensitize the subjects, a symbol of a road sign or a traffic light was displayed for a duration of approximately 50 milliseconds on the CRT display before the actual road sign or traffic light appeared in the subject's field of vision. This subliminal sensitization resulted in a significant reduction in the reaction time of the subject of 50 milliseconds as compared to the reaction time without sensitization.

[0014] In a second test arrangement described in German Patent DB 199 52 506, a similar test vehicle with a CRT display as instrument panel was allowed to follow a vehicle driving ahead of the test vehicle at 20 to 40 km per hour. The subject was told to actuate the brake pedal as soon as the brake lights of the lead vehicle lit up. Randomly, in half of the cases, a picture of two lit brake lights was flashed in the instrument panel before the brake lights of the lead vehicle actually lit up. The duration of the displayed image was chosen to be so short that the driver did not consciously perceive it. The cases with subliminal sensitization resulted in a significant reduction in the reaction time of 150 milliseconds as compared to the cases without sensitization.

[0015] In this context, the object of the present invention may be achieved by a display device for displaying action-relevant information in form of images or symbols, the duration of the display of the specific image or symbol lying below a conscious and above an unconscious perception threshold of the operator, and the specific image or symbol being displayed at those locations of the field of vision of the operator at which objects are located to which his/her attention is to be drawn.

[0016] The advantage of the present invention includes that the operator does not need to turn his/her view away from the scene to be monitored and in that he/she is not unnecessarily disturbed by the representation of information.

[0017] Advantageous embodiments and refinements of the present invention include that the objects to which attention is to be drawn are at least partially obtained by evaluating infrared sensor data. The duration of the display of the action-relevant information may be varied, and may increase proportionally with the increase of a potential danger. From this certain danger potential, the action-relevant data may displayed for a period which lies above a conscious perception threshold.

[0018] The action-relevant information may for example be displayed in the form of light spots, arrows or other pointer symbols pointing to those objects to which the attention of an operator is to be drawn, in the form of borders or underlinings which are displayed around or under those objects to which the attention of an operator is to be drawn, or image segments containing image information obtained via an image-forming sensory system on those objects to which the attention of an operator is to be drawn.

[0019] The environmental area which is in the operator's field of vision may be determined via a device and compared to the field of vision of the sensor device used within the framework of the attention control, and the position at which the action-relevant information is to be displayed to the operator may be determined on the basis of this comparison.

[0020] The environmental area which is in the operator's field of vision may be determined with knowledge of the invariable geometries and of the current position of variably configurable geometries within the space surrounding the operator. Inside a motor vehicle, the current sitting position and/or the adjustment of the mirror adjuster may be determined for determining the variably configurable geometries. The environmental area which is in the operator's field of vision also may be determined, for example, via a sensor for determining the head position and/or the eye alignment of the operator.

[0021] The present invention also provides a device for controlling the attention of an operator of technical equipment, having a display device for displaying action-relevant information in the form of images or symbols comprising

[0022] sensors for sensing at least a subarea of the area surrounding an operator of a technical system,

[0023] a data processing system, and

[0024] a display unit for displaying action-relevant information, the duration of the display of the specific image or symbol lying below a conscious and above an unconscious perception threshold of the operator, wherein a means exists which allows the specific image or symbol to be displayed at those locations of the field of vision of the operator at which objects are located to which his/her attention is to be drawn.

[0025] The present invention is suitable for controlling the attention of operators of the most varied technical equipment where persons have to monitor the operational environment and to react to occurring changes.

[0026] In the following, the present invention will be exemplarily explained within the framework of its use for controlling the attention of vehicle drivers. In this context, it is particular advantageous to use the method according to the present invention within a system for improving night vision. In this connection, the objects to which attention is to be drawn are at least partially obtained by evaluating infrared sensor data. Thus, it is possible to draw the driver's attention to pedestrians which are detected in this manner also in darkness even before danger to them can be reliably established. In order to not overload the vehicle driver with information, he/she should deliberately be clearly alerted only in the case of danger situations. This clear alert can be brought about by the system, for example, in that the display duration of the images or symbols lies above the conscious perception threshold. In this manner, it is also conceivable to convey different danger potentials to the driver by a simple variation in the duration of the display.

[0027] The action-relevant information is advantageously displayed in the form of light spots which are superimposed on the images of the actual objects in the field of vision of the driver. In this connection, it is also conceivable for the light spots to be colored or provided with corresponding textures according to the respective danger potential. In a further advantageous embodiment, the action-relevant information is displayed in the form of arrows or other pointer symbols pointing to those objects to which the driver's attention is to be drawn. It is certainly profitable as well to set off important objects in the form of borders or underlinings. However, it is also possible for action-relevant information to be displayed in an inventive manner in the form of image segments. These image segments can be directly derived from the image information acquired by the image-forming sensory system, it being possible in this context to use the entire image information or only contours. However, it is also conceivable for the image information to be symbolically superimposed prior to display so that always the same symbol is displayed, for example, when a pedestrian is detected. This would help further reduce the overstimulation of the vehicle driver.

[0028] To bring the display of the action-relevant image information into cover with the driver's sight of the environment as best as possible, it is conceivable to prevent an additional outlay of processing and a necessary sensory system by assuming, for example, a standardized sitting position and height of a person. However, the method according to the present invention can be profitably developed if the environmental area which is in the operator's field of vision is determined via a device and compared to the field of vision of the sensor device used within the framework of the attention control. The position at which the action-relevant information is to be displayed to the operator can then be determined on the basis of this comparison. For carrying out such a process step, ideally, the environmental area which is in the operator's field of vision is to be determined with knowledge of the invariable geometries and of the current position of variably configurable geometries within the space surrounding the operator. To this end, for example inside a motor vehicle, the current sitting position and/or the adjustment of the mirror adjuster can be detected for determining the variably configurable geometries. However, it is also conceivable to determine the environmental area which is in the operator's field of vision via a sensor for determining the head position and/or the eye alignment of the operator.

[0029] In accordance with the explained process steps, a device for carrying out the method according to the present invention is composed at least of sensors for sensing at least a subarea of the area surrounding an operator of a technical system, of a data processing system, and of a display unit for displaying action-relevant information. In an advantageous embodiment, the device further includes a unit which is suitable for determining variably configurable geometries within the space surrounding the operator and/or for determining the head position and/or the eye alignment of the operator.

[0030] The data processing system advantageously contains a storage unit in which at least data with respect to the invariable geometries of the space surrounding the operator is stored.

[0031] Particularly suitable as the display unit within the device according to the present invention is a head-up display via which the action-relevant information is imaged into the field of vision of an operator. In this context, it is conceivable for the head-up display to be designed in a simple manner in the form of an LED line to image pure line information (such as ‘underlinings’) into the field of vision of an operator.

[0032] The present invention will be further described with respect to the following drawings, in which:

[0033] FIG. 1 shows an embodiment of the device of the present invention in an automobile;

[0034] FIG. 2 shows a view of a driver in the automobile with an enhanced display;

[0035] FIG. 3 shows a same view as in FIG. 2, but with only the objects to be enhanced by the method of the present invention being shown; and

[0036] FIG. 4 shows the same view as in FIG. 2 but with the objects to be enhanced without an overlay, but rather through arrows, frames or underlining.

[0037] FIG. 1 shows a partial side view of the device of the present invention, including a projector 18 for projecting an image 16 for a certain time lying below a conscious and above an unconscious perception threshold of the operator onto a windshield 22. The image 16 can be sent via a mirror 20 onto the windshield. The projector may be located on, in or near a dashboard 26 of an automobile. At least one sensor 27, located for example at the front of the automobile, can send signals to a data processing system 28 having a data storage unit 29. Data storage unit 29 can store information that does not vary, such as passenger cabin configuration data. The images 16 may be formed as a function of data from sensor 27. A sensor 25 can sense data regarding the operator, such as head position, and data regarding the vehicle cabin, such as seat position.

[0038] FIG. 2 shows a view of vehicle dashboard 26 and windshield 22 as seen by the operator. The street 210 and automobiles 201, 202, 203 may be enhanced according to the method of the present invention by the projector 18, for example by overlay.

[0039] FIG. 3 shows for example the image as in FIG. 2 but solely showing an overlay to be projected on windshield 22 for a period of time lying below a conscious and above an unconscious perception threshold of the operator. Only automobiles 201, 202 and 203 are enhanced in this version.

[0040] FIG. 4 shows an alternative to the overlays of FIG. 3, with arrows 320 enhancing automobile 301, frame 321 enhancing automobile 302 and underlining 322 enhancing automobile 303. Street 310 may be enhanced by overlay, for example, or not enhanced. Arrows 320, frame 321 and underlining 322 are displayed for a period of time lying below a conscious and above an unconscious perception threshold of the operator