System for assisting the operator of a technical appliance

[00002] Field of the Invention

[00003] The invention concerns a system for assisting an operator of a technical appliance, with an image display device, of which the image is provided within the normal viewing field of the operator, for indication of operationally relevant information in the form of images or symbols.

[00004] Modern warning systems are being designed to warn the automobile operators of dangerous situations, of which the recognition has hitherto been the sole responsibility of the driver. For example, a “curve warner” provides the operator with early warning of the encountering of curves, which are too tight for the actual driving speed. Presently, this warning occurs using acoustic enunciators, for example “sharp left curve ahead,” and/or by offering optical warning signals in the dashboard or reflected into the windshield.

[00005] This type of warning system is, as a rule, so designed, that the warning of a danger only occurs when it is to be presumed that the driver has not himself recognized this danger, since otherwise the acceptance and the effect of the warning information would be ignored or lost. In many warning systems, it is however difficult or even impossible to determine or fix a suitable time point for generating the warning. On the one hand, the warning must occur early enough so that the driver can be appraised of the dangerous situation, should he not have recognized it himself. On the other hand, the warning should not occur so early, that a driver, who has already recognized the danger and intends to take appropriate measures, is unnecessarily disturbed or distracted. These two requirements can frequently not be accommodated at the same time.

[00006] The same dilemma occurs in warning or information systems for other technical apparatus other than automotive vehicles. For example, the operator of a machine or industrial apparatus is normally optically or acoustically warned or informed when particular situations occur which might possibly necessitate a rapid intervention. Too early or, as the case may be, too many warning signals could lead thereto that the attentiveness of the operator is reduced with time and important information is disregarded. On the other hand, the warning should not occur so late, that the operator might possibly not be able to react.

[00007] The invention is accomplished with providing the optical information for such a short period of time, that it is not consciously recognized, however is registered by the person. This phenomenon, referred to as “visual priming,” is described for example by A. J. Marcel in Conscious and unconscious perception: Experiments on visual masking and word recognition, Cognitive Psychology 15 (1983), Pages 197-237. The best known example of this phenomenon is the exchange of individual frames of a film sequence with frames having a completely different content, which the observer does not consciously comprehend during watching of the film, but which however do affect his perceptions or, as the case may be, his behavior.

[00008] This effect is used in the invention for sub-threshold warning, informing, or influencing of the operator of a machine. This makes possible for example a warning without distracting signals such as the illumination of lights or warning noises with conventional warning systems.

[00009] During experiments for carrying out the invention, it has been demonstrated that the sub-threshold offering of operationally relevant information in an image display, of which the image is in the normal visual field of the operator, effectively sensitizes the operator to the offered information. Thereby, the reaction time of the operator is substantially reduced when the situation, to which he was unconsciously prepared, actually occurs. The operator is however not disturbed or distracted by the information offered on the sub-threshold level. Thus, the operationally relevant information can immediately be displayed for a short period of time after the appropriate situation has been recognized, that is, without consideration as to whether the operator has initiated any manipulations.

[00010] According to the invention, an image or symbol, which reflects the operationally relevant information, is displayed at once for a short period of time, wherein the duration of the display is preferably below that of conscious recognition and above that of subconscious recognition of the operator. Therein, the threshold of subconscious recognition is defined as the display duration with which the information is just barely accepted unconsciously, that is, has an effect on the later behavior of the operator, and the conscious recognition threshold is the duration of display at which the operator becomes conscious of the information.

[00011] These two recognition thresholds can, by the way, be influenced by the size and magnitude or intensity of the displayed image and, besides this, by whether the image or symbol is offered in the center of the field of vision or at the edge. It has been shown that a short time presentation in the periphery of the visual field influences the cognition or recognition or, as the case may be, the behavior of the operator.

[00012] Basis on the condition that the operationally relevant information could be offered even in the periphery of the field of vision of the operator, the invention is suitable also for technical appliances, such as for example, an automobile.

[00013] When adapting the invention to an automobile, one employs normally one or more devices for automatic recognition of situations, which may possibly require a reaction of the automobile driver, and a device for controlling the image display device, in order to display the appropriate operationally relevant information as soon as the appropriate situation is recognized.

[00014] For recognition of dangerous situations, there can be employed for example devices for recognition of an inappropriate speed of the vehicle prior to a curve, for recognition of traffic signs, for recognition of road conditions, for recognition of traffic signals, for recognition of approach situations, for recognition of crossing situations, for recognition of pedestrians and/or for recognition of low stimulation or signal intensity and tedious driving situations.

[00015] The invention is suitable not only for sensitization with respect to dangerous situations, but rather also to sensitize automobile drivers or operators of other technical apparatus to various other important conditions without imposing or disturbing, i.e. conditions which might possibly require a reaction. Devices for recognition of various possible important situations during the operation of a vehicle, which can advantageously be combined or utilized with the present invention, are for example a navigation device and/or a device for monitoring of operational values of the vehicle, for example the status of the fuel.

[00016] The recognition signals of the above-mentioned devices can in many ways be combined with each other, in order to prepare the information and in present it in an optimal manner to the driver. For example one symbol may be a symbol, which alerts the driver below the threshold of recognition to the possible need of a fuel stop, and can be given exactly then when the navigation system determines that a filling station is being approached. The briefly displayed images or symbols can be pictures of corresponding situations, pictograms, traffic symbols, arrows or written words.

[00017] In a preferred embodiment, the image display device is a so-called heads-up display. In an automobile, one realizes a heads-up display thereby that the image information is reflected into the visual field of the driver via the windshield. The reflected image information can in a first stage or level be the conventional operating parameters such as RPM, speed, blinker, etc. and the operationally relevant information according to the invention can supplementally be reflected or reflected during an appropriate short time fade-out of the main image.

[00018] The invention is also suitable for retrofitting to vehicles, which have for example conventional individual instruments on the instrument panel. In such a case, it can be useful when all the short time images are reflected via the windshield into the visual field of the driver.

[00019] For adapting a motorcycle or other machine or vehicle, which has no suitable reflecting device such as a windshield, a heads-up display can be designed in the manner of, for example, glasses or goggles wearable by the operator.

[00020] In place of a heads-up display, the image display device can be a pixel based controllable display screen for direct viewing by the operator, for example an LCD-display on the instrument panel, upon which important operational parameters of the vehicle are represented. As already described, the briefly offered images, even when peripherally displayed, leave behind a measurable impression with the driver, so that it is not detrimental when he normally does not look directly at the image screen or, as the case may be, in complex driving situations frequently changes his direction of viewing.

[00021] For safety, the short time images can also be offered multiple times sequentially at spaced-apart time intervals. This is possible, since in accordance with the invention it is not necessary to wait to see whether the driver reacts by himself. Rather, all necessary warning or other information can be given immediately, so that substantially more time latitude exists than with conventional warning systems.

[00022] It is useful when the duration, with which the respective image or symbol is displayed for a short period of time, can be adjusted or controlled by the operator. Thereby, the operator can adapt the system precisely to his personal threshold of recognition. In order to facilitate this, the system can offer an appropriate adjustment procedure, through which the operator is guided.

[00026] A system for assisting a vehicle operator includes for example a device for recognition of an inappropriate speed of the vehicle prior to a curve, a device for recognition of traffic signs, a device for recognition of the road conditions, a device for recognition of traffic signals, a device for recognition of approach situations, a device for recognition of crossing situations, a device for recognition for pedestrians and/or a device for recognition of difficult to recognize and tedious driving situations.

[00027] Besides this, there is a freely or variably programmable display device in the vehicle, for example an LCD-display in the instrument panel, on which the operating parameters such as for example, speed, RPM, etc. are displayed. Alternatively, the display device can be a heads-up display.

[00028] As shown in FIG. 1, upon recognition of a particular environment or situation (S1) the recognized situation is evaluated (S2) with respect to its danger potential or, as the case may be, its importance. Subsequently, there occurs a selection of a suitable excitation or stimulation based on type and duration (S3), wherein the appropriate images or symbols and the appropriate display duration can be stored, for example in a storage chip or component. Subsequently, there occurs the stimulus (S4), in which the image or symbol is briefly displayed on the display device. These process steps can be carried out or, as the case may be, controlled by a suitably programmed microprocessor.

[00029] The respective display durations are determined in advance for example by testing. The display duration is selected to be so short, that the appropriate image or symbol is just below the threshold level of conscious recognition of the driver.

[heading-00030] Test Device 1

[00031] A test vehicle with a variously controllable display screen on the instrument panel in place of the conventional individual instruments was so modified, that the reaction of the driver to traffic signs and traffic lights could be measured. The reaction time of a sufficiently large number of test persons was measured once with and once without subconscious optical sensitization. For sensitization of the test persons, symbols of the traffic signals or, as the case may be, traffic lights were displayed approximately 50 ms on the display screen, before the traffic signs or, as the case may be, traffic lights actually appeared in the visual field of the test person. With this subconscious sensitization, a significant reduction of the reaction time of 50 ms in comparison to the reaction time without sensitization resulted.

[heading-00032] Test Device 2

[00033] A similar test vehicle with an image display screen displaying a combination of instruments was allowed to follow a preceding vehicle traveling at 20-40 km/h. The test person was required to operate the brake pedal as soon as the brake light of the lead vehicle was illuminated. Coincidentally, in half the cases, the image of two illuminated backlights (vehicle stoplights) was blended into the combination instrument, before the vehicle brake lights actually illuminated. The duration of the blending in of the display of the image was selected to be so short, that the driver did not consciously recognize the images.

[00034] FIG. 2 shows the instrument panel of the test vehicle from the 15 view of the driver at precisely the instant, in which the symbolic image 1 of two backlights appear in the combination instrument 2 outside the tachometer 3. The preceding or lead vehicle is not shown in FIG. 2. In the cases with subconscious sensitization, a significant reduction of the reaction time of 150 ms occurred in comparison to the cases without sensitization.

[00035] The described test systems were designed especially for testing the effectiveness of the invention and do not represent a commercially suitable warning system. The person working in this art would, however, recognize that the use of many warning systems for vehicle drivers, which are presently available, can be substantially improved in accordance with the present invention. For example, if one associated the curve warner with the present invention in such a manner that the driver is presented with the clearly recognizable optical warning signal (for example the traffic symbol for “curve”) mirrored in windshield or displayed on the instrument panel with a duration below the threshold of recognition. As a warning signal, it would also be possible to use the image of a typical curve with a radius or curvature as detected by the curve warner.

[00036] If one takes into the consideration the fact that 90% of all accidents of vehicles could be prevented if the driver had better control over the vehicle in the last 500 ms prior to the accident, then the invention makes it possible, to increase the attention of the driver in the decisive phases, without however unintentionally disturbing or distracting him.

[00037] In comparison to conventional warning systems, the information offered below the threshold of recognition can always be offered, even independent of whether or not the operator has actually recognized a dangerous situation. For example, in situations, in which the vehicle is urgently to be brought to a halt, for example at a red light or a stop sign, it is useful, to offer the warning information independent of whether or not the operator has actually taken action or not.