Method of pilot support in landing helicopters in visual flight under brownout or whiteout conditions

[0001] This application claims the priority of German patent document 10 2004 051 625.1, filed Oct. 23, 2004, the disclosure of which is expressly incorporated by reference herein.

[0002] The invention relates to a method for supporting pilots in landing helicopters in visual flight under brownout or whiteout conditions.

[0003] When helicopters approach for landing on a dusty or snow-covered landing field, whirling dust or snow suddenly and greatly restricts visibility in the final phase. This phenomenon, referred to as “brownout” or “whiteout”, presents a significant risk in VFR (Visual Flight Rules) landings. The pilot can become disoriented with respect to the aircraft's attitude and altitude relative to the ground, as the whirling dust or snow gives him the impression that the helicopter is banking or nosing down. This situation frequently leads to accidents, and is the single most frequent cause of all helicopter crashes.

[0004] The article entitled “Brown-out survival—technology or training?” in the periodical “Defence Helicopter,” Shepard, pp. 10-12, February/March 2004 issue, gives an instructive introduction to the problem of brownouts or whiteouts and suggests first approaches to technical solutions, which will be discussed briefly below.

[0005] According to the current state of the art, the pilot has instruments for instrument flight conditions available in correspondingly equipped helicopters. If brownout or whiteout conditions occur, these instruments can be used to land the aircraft under instrument flight rules, using, for example, the artificial horizon and the radar altimeter. A problem, however, is that in the critical phase just before landing, a complete switch is necessary from manual orientation based on the natural external view to orientation by means of abstract instrument information. This requires continued training of the pilots for this critical situation and still leads to many accidents.

[0006] Other proposals are known, which attempt to provide the pilot with auxiliary information provided by GPS-based sensors. Although this avoids the complexity of full Instrument Flight Rules instrumentation, it does not change the basic problem of the need to switch from sight orientation to instrument orientation.

[0007] It has also been proposed to use radar to detect the landing site through dust or snow. This technique fails, however, because of the currently available radars, the inadequate resolution and the minimum detection distance. The costs and weight of radar are also considerable. The ROSAR system proposed in German patent document DE 43 28 573 C2, for example, could in principle penetrate dust or snow under brownout or whiteout conditions and aid the pilot through visual representation of the radar echo at a given instant, but such a radar system is very costly and heavy, so that no ROSAR system is currently available on the market.

[0008] One object of the invention is therefore to provide a helicopter pilot with reliable flight support for VFR landings under brownout or whiteout conditions.

[0009] This and other objects and advantages are achieved by the method and apparatus according to the invention, in which the helicopter is equipped with a forward-looking high-resolution 3D sensor and a position reference system. The 3D sensor preferably has a resolution of at least 0.5° and a data rate greater than approximately 20,000 pixels/second. The 3D sensor continuously records the landing site during approach. The cumulative 3D data together with the attitude and position data of the position reference system of the helicopter are used to generate a natural virtual view of the landing site in a perspective corresponding to the actual attitude and position of the helicopter. This virtual external view is provided to the pilot by means of a display device (screen, head-up display, etc.). A control routine ensures that 3D data generated under brownout or whiteout conditions are not used in the calculation of the virtual external view. For this purpose, in a first possible embodiment, the incoming data are analyzed to detect brownout or whiteout conditions directly.

[0010] Current high-resolution 3D sensors are suitable only as VFR sensors; that is, they cannot penetrate a brownout or whiteout. At the instant when the brownout or whiteout occurs, it is automatically detected by means of the sensor data through image analysis using the 3D image of the dust cloud. During the brownout or whiteout, the pilot continues to be provided with the virtual view based on the 3D data obtained prior to the brownout or whiteout, such that the perspective is still continuously adapted to the actual helicopter position and attitude using the position reference data.

[0011] To ensure that 3D data generated during brownout or whiteout conditions are not used in the calculation of the virtual external view, as an alternative to the above-described direct detection of brownout or whiteout conditions, it may be provided that only 3D data generated during the approach above a specific predefined minimum flight altitude are used. The minimum altitude is selected such that brownout or whiteout conditions can still be reliably excluded above this minimum flight altitude—typically approximately 10-15 m. The altitude can be measured particularly by means of a radar altimeter. Below the minimum flight altitude, the pilot continues to be presented with the virtual external view based on the 3D data obtained above the minimum flight altitude, such that the perspective is continually adapted to the actual helicopter position and attitude using the position reference data.

[0012] The virtual external view should be adapted as quickly as possible to the flight progress of the helicopter. A refresh rate of at least 20 Hz is sufficient for this purpose.

[0013] Thus, the pilot has a high-resolution virtual external view of natural appearance available during the entire landing phase. Since the data of the 3D sensor system are accumulated during the approach and the points in space being detected continually change because of the flight movement, it is possible to achieve video-quality resolution. For example, an approach over 10 seconds at a data rate greater than 20,000 pixels/second yields more than 200,000 pixels. Since the perspective corresponds to the actual helicopter position and attitude at all times, the pilot can use this virtual view to estimate the attitude of the helicopter relative to the landing site at any time. When brownout or whiteout conditions occur, he can complete the landing using this virtual view without the need for a critical switch. Since brownout or whiteout conditions occur only at approximately 3-5 m above ground, it is sufficient to generate the virtual view from data obtained before this limited visibility occurs because it is generally unlikely that obstacles appear at the landing site during these last 3 seconds. A (currently unavailable) high-resolution sensor system that penetrates dust or snow is therefore not required.

[0014] The 3D sensor used is a range imaging sensor, particularly an imaging laser radar, such as described, for example in German patent document DE 39 42 770 C2 or German patent document DE 43 20 485 A1. The helicopter laser radar HELLAS by EADS Deutschland GmbH, Ottobrunn, Germany which provides 40,000 pixels/second at a range of up to 1 km, which is sufficient by far for this application, may be used as the 3D sensor, for instance. As an alternative, however, the 3D data may be generated in a manner known per se. For example, a stereo camera may be used for this purpose.

[0015] For the required position reference system many suitable devices are available on the market. One example is the inertial navigation system AH 2100 made by Honeywell. In a particularly preferred embodiment, the position reference system also uses the 3D data of the 3D sensor itself in addition to other sensor data to calculate the attitude and position data.

[0016] The required conversion of the 3D data into a photorealistic view from the perspective of the helicopter pilot is readily possible and cost-effective with the use of standard modern 3D visualization technology. In particular, OpenGL and DirectX may be used as the 3D interface for rendering 3D graphics.

[0017] To achieve a particularly realistic representation of the rendered scene, it is advantageous to take the following measures:

[0018] In addition to the 3D data, the color and gray scale distribution of the landing site is recorded by a color video camera. This makes it possible to generate the virtual external view so that it approximates the actual color and gray scale distribution of the landing site.

[0019] To enhance scene perception, sunlight with shadow formation may be simulated in the rendered scene based on the solar altitude, which is known from the position and the time of day.

[0020] Based on available cost-effective and low-weight sensors, the invention provides an overall reliable pilot aid for VFR landings under brownout or whiteout conditions.

[0021] Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.

[0023] The 3D sensor 1 is mounted forward-looking on the helicopter, such that the planned landing site is in view during the approach. The 3D sensor supplies its data to the image processor 2, which continuously receives the position and attitude of the helicopter from the position reference system 3. The image processor uses the 3D sensor data accumulated over several seconds in conjunction with the position and attitude data to initially generate a high-resolution 3D representation of the landing site. The image processor 2 then uses this data to continuously generate the virtual view from the pilot's perspective, which is displayed by means of the display device 4, which can be a standard multifunction display or a head-up display. The image processor 2 is capable of detecting brownout or whiteout conditions without a major analysis effort, by image analysis using well known image processing techniques, and 3D data from the high resolution 3D sensor 1. The 3D data generated under brownout or whiteout conditions are not used to generate the virtual view of the landing site.

[0024] The foregoing disclosure has been set forth merely to illustrate the invention and is not intended to be limiting. Since modifications of the disclosed embodiments incorporating the spirit and substance of the invention may occur to persons skilled in the art, the invention should be construed to include everything within the scope of the appended claims and equivalents thereof.