Selection of Objects in a Three-Dimensional Virtual Scene

[0001] Exemplary embodiments of the invention relate to a representation device for representing a three-dimensional virtual scenario and for interacting with the three-dimensional virtual scenario, a workstation device for representing and operating a three-dimensional virtual scenario, the use of a workstation device for representing and monitoring air spaces, as an air traffic control workstation and for monitoring unmanned aircraft, and to a method for selecting virtual objects in a three-dimensional virtual scenario.

[0002] Stereoscopic visualization techniques are used to create the impression of a three-dimensional scenario in a viewer of a stereoscopic display. The viewer experiences the three-dimensional impression in that the viewer's eyes perceive different images.

[0003] Interaction with three-dimensional virtual objects in a three-dimensional virtual scenario requires the input of three translational degrees of freedom because, compared to a two-dimensional scenario, the three-dimensional scenario also has a dimension of depth. The input of three degrees of freedom requires special interaction devices and/or methods since conventional interaction devices, such as a so-called computer mouse or a so-called trackball, provide only two translational degrees of freedom.

[0004] Exemplary embodiments of the invention are directed to a representation device for representing a three-dimensional virtual scenario and for interacting with this three-dimensional virtual scenario, which representation device offers an alternative interaction possibility with the three-dimensional virtual scenario.

[0005] Many of the features described for the representation device and the workstation device may also be implemented as method steps and vice versa.

[0006] In accordance with a first aspect of the invention, a representation device for representing a three-dimensional virtual scenario and for interacting with the three-dimensional scenario is provided that has at least a first representation region for representing the three-dimensional scenario and has an input unit. The input unit is designed for controlling a movement of a marking element in the three-dimensional scenario, which movement has two degrees of freedom, so that each virtual object is selectable in the three-dimensional virtual scenario.

[0007] The representation device may of course have more than one representation region, for instance two, three, four, or an even greater number of representation regions.

[0008] The representation regions may be a display element designed for stereoscopic visualization. The representation regions may thus be displays or projection surfaces suitable for being used for a stereoscopic visualization technique.

[0009] The representation device as described in the foregoing and in the following permits interaction with the virtual objects in a three-dimensional virtual scenario while using an input unit that offers two translational degrees of freedom.

[0010] In contrast to a rotational degree of freedom, a translational degree of freedom means that a body moves in one direction when performing a translational movement.

[0011] Movement of a marking element with two translational degrees of freedom shall be construed to mean the movement of a marking element along a surface area. In other words, a marking element may move on a surface area along a first direction and along a second direction, a movement vector for the first direction and a movement vector for the second direction being perpendicular to one another. Naturally a movement in a composite direction is also possible, i.e. a composite movement direction has one component from the first movement vector and one component from the second movement vector.

[0012] In other words, this means that marking elements may move along two established movement directions. This means that the marking element has two translational degrees of freedom.

[0013] Thus the representation device as described in the foregoing and in the following permits a simple and rapidly learned interaction with a three-dimensional virtual scenario.

[0014] The term interaction with a three-dimensional virtual scenario may be construed to mean the selection and manipulation of virtual objects that are disposed in the three-dimensional virtual scenario.

[0015] In accordance with one embodiment of the invention, the movement of the marking element having two translational degrees of freedom takes place in a virtual surface area.

[0016] The virtual surface area is determined by the two permissible movement directions of the marking element.

[0017] The virtual object in the three-dimensional virtual scenario may be selected in that a connecting line is calculated from one eye of the operator of the representation device using the marking element. The movement of the marking element on the virtual surface area changes the position of the connecting line. Likewise, the position of the connecting line changes when the viewer changes the position of his eyes. A virtual object in the three-dimensional virtual scenario is selected in that the connecting line is extended into the three-dimensional virtual scenario and a virtual object is selected when the connecting line intersects the virtual coordinates of the virtual location of the virtual object.

[0018] In particular a virtual object may be selected in that a virtual object is looked at, i.e., in that the marking element is moved such that the connecting line intersects the virtual object, and by actuating a selection element on an input element, for instance by pressing a key on a so-called computer mouse, the selection of the virtual object is confirmed.

[0019] In accordance with another embodiment of the invention, the virtual surface area is embodied as a plane.

[0020] It should be noted that the virtual surface area is merely a notional surface area in the three-dimensional virtual scenario and is defined by the two permissible movement directions of the marking element.

[0021] In accordance with another embodiment of the invention, the virtual surface area is embodied as a circular arc.

[0022] In the virtual three-dimensional scenario, the virtual surface area thus represents the physical equivalent of a circular arc, that is, a hollow cylinder arc.

[0023] Even if the virtual surface area obtains a depth component from the rounded shape, it is possible that the marking element may move from one point on the virtual surface area to any other desired point on the virtual surface area using the specification of two movement directions.

[0024] The virtual surface area may be embodied in any desired shape, such as for instance even in the shape of a hemisphere or sphere.

[0025] The three-dimensional scenario may, for example, be projected such that the viewer is surrounded by virtual objects, in other words, such that the viewer is disposed in the middle of the three-dimensional virtual scenario. In this case, the virtual surface area may surround the viewer as a sphere and the marking element may be embodied to be moved over the spherical surface.

[0026] In accordance with another embodiment of the invention, the arrangement of the virtual surface area may be changed in the three-dimensional virtual scenario.

[0027] A change in the arrangement of the virtual surface area in the three-dimensional virtual scenario does not change the fact that the marking element is moved only along the virtual surface area. Since the virtual surface area that is disposed in the three-dimensional virtual scenario may have a depth component, the change to the arrangement of the virtual surface area in the three-dimensional virtual scenario may lead to the marking element having a correspondingly changed movement with respect to the three-dimensional virtual scenario.

[0028] If the virtual surface area is for instance a plane, an angle of inclination that the virtual surface area has in the three-dimensional virtual scenario may be changed. Likewise, the position of the virtual surface area in the three-dimensional virtual scenario may be changed.

[0029] In accordance with another aspect of the invention, a workstation device for representing and operating a three-dimensional virtual scenario having a representation device for representing a three-dimensional virtual scenario and for interacting with the three-dimensional virtual scenario is provided as described above and in the following.

[0030] The workstation device may, for example, be used by one or a plurality of users to control unmanned aircraft or to monitor any desired scenarios.

[0031] The workstation device as described in the foregoing and in the following may of course also have a plurality of representation devices, but may also have one or a plurality of conventional displays for representing additional two-dimensionally represented information.

[0032] Moreover, the workstation device may have input elements that may be used alternatively or in addition to the interaction, described in the foregoing and in the following, with the three-dimensional virtual scenario.

[0033] The workstation device may have a so-called computer mouse, a keyboard, or use-typical interaction devices, for instance those for an air traffic control workstation.

[0034] Likewise, the displays or representation units may be conventional displays or touch-sensitive displays or representation units (so-called touchscreens).

[0035] In accordance with another aspect of the invention, a workstation device as described in the foregoing and in the following is provided for monitoring air spaces.

[0036] In accordance with another aspect of the invention, a workstation device as described in the foregoing and in the following is provided as an air traffic control workstation.

[0037] Thus it is possible for an operator of the workstation device to interact in a simple manner with the represented virtual objects, i.e. the aircraft in the region to be monitored and represented, i.e. to select these aircraft and to transmit instructions to the selected aircraft.

[0038] In accordance with another aspect of the invention, the use of a workstation device as described in the foregoing and in the following is provided for monitoring and controlling unmanned aircraft.

[0039] Thus it is possible for an operator of the workstation device to interact in a simple manner with the represented virtual objects, i.e. the unmanned aircraft in the represented three-dimensional virtual scenario, i.e. to select these aircraft and to transmit instructions to the selected aircraft.

[0040] Likewise, the workstation device may also be used for controlling components such as, for example, a camera or other sensors that are components of an unmanned aircraft.

[0041] In accordance with another aspect of the invention, a method is provided for selecting virtual objects in a three-dimensional virtual scenario. The method has the following steps: reproducing a marking element in the three-dimensional virtual scenario, moving the marking element with two degrees of freedom along a virtual surface area, selecting a virtual object in the three-dimensional virtual scenario by positioning the marking element in a certain region on the virtual surface area.

[0042] In accordance with one embodiment of the invention, the method further has the following step for selecting virtual objects in a three-dimensional virtual scenario: Determining a connecting line from an eye of the viewer to the marking element, wherein determining the connecting line occurs prior to selecting the virtual object and the object is selected in that the marking element is moved on the virtual surface area such that the connecting line intersects a virtual location of the virtual object in the three-dimensional virtual scenario.

[0043] In accordance with another aspect of the invention, a computer program element for controlling a representation device for representing a three-dimensional virtual scenario and for interacting therewith as described in the foregoing and in the following is provided that is embodied to perform the method for selecting virtual objects in a three-dimensional virtual scenario as described in the foregoing and in the following when the computer program element is executed on a processor of a computing unit.

[0044] The computer program element may instruct a processor of a computing unit to perform the method for selecting virtual objects in a three-dimensional virtual scenario.

[0045] In accordance with another aspect of the invention, a computer-readable medium is provided with the computer program element as described in the foregoing and in the following.

[0046] A computer-readable medium may be a volatile or non-volatile storage medium, for instance a disk drive, a CD, and DVD, a diskette, a storage card, or any other desired computer-readable medium or storage medium.

[0047] Exemplary embodiments of the invention shall be described in the following with reference to the figures.

[0055] FIG. 1 depicts a representation device 100 for representing a three-dimensional virtual scenario 300 having a representation region 111. A virtual object 301 and a virtual surface area 601 are disposed in the three-dimensional virtual scenario 300.

[0056] The virtual surface area 601 is embodied such that a marking element may be moved along on it. The marking element may move in two directions within the three-dimensional virtual scenario 300.

[0057] FIG. 2 depicts a side view of the representation device 100 having a representation region 111 for representing a three-dimensional virtual scenario 300.

[0058] The virtual surface area 601 and the virtual object 301 are disposed in the three-dimensional virtual scenario.

[0059] The virtual surface area 601 may be arranged as desired in the three-dimensional virtual scenario. This means that, for example, an inclination and a position of the virtual surface area may be adjusted.

[0060] If the position of the virtual surface area 601 is changed, this may lead to the virtual location of a virtual object 301 changing with respect to the virtual surface area 601. In FIG. 2, the virtual surface area 601 is arranged such that the virtual object 301 is disposed between the representation region 111 and the virtual surface area 601.

[0061] However, the virtual surface area 601 may also be arranged such that it is disposed between one or a plurality of virtual objects 301 and the representation region 111.

[0062] In principle the position of the virtual surface area in the three-dimensional virtual scenario does not effect any change in the method for selecting a virtual object.

[0063] If a virtual object to be selected is disposed between the virtual surface area and the viewer's eye, the virtual object is selected in that the connecting line from the viewer's eye to the marking element runs through the virtual coordinates of the location of the virtual object.

[0064] If the virtual object to be selected is disposed between the virtual surface area and the representation unit, the virtual object is selected in that the connecting line is extended into the three-dimensional virtual scenario and there runs through the coordinates for the location of the virtual object.

[0065] FIG. 3 depicts a representation device 100 having a representation region 111 for a three-dimensional virtual scene 300, wherein a virtual object 301 and a virtual surface area 601 are depicted in the three-dimensional virtual scene.

[0066] The virtual surface area is embodied in the shape of a circular arc or in the shape of a hollow cylindrical arc, the spatial equivalent thereof. A marking element may move along the surface of the virtual surface area.

[0067] FIG. 4 depicts a representation device 100 for a three-dimensional virtual scenario having a representation unit 110 with a first representation region 111 and a second representation region 112. Virtual three-dimensional objects 301 are reproduced in the representation space 130.

[0068] Arranged in the three-dimensional virtual scene is a virtual surface area 601 on which a marking element 602 may be moved. The marking element 602 moves only on the virtual surface area 601, so that the marking element 602 has two translational degrees of freedom in its movement. In other words, the marking element 602 is designed to perform a two-dimensional movement. Thus the marking element may be controlled, for example, by means of a conventional computer mouse.

[0069] A virtual object in the three-dimensional scenario is selected in that the position of at least one of a viewer's eyes 503 is detected and a connecting line 504 from the determined position of the eye 503 into the three-dimensional scenario in the representation space 130 is calculated using the marking element 602.

[0070] The position of the eye may be detected for instance in that the user wears eyeglasses with reflectors and a camera system detects the position of the reflectors. However, other methods may also be used for detecting the position of the eye.

[0071] The connecting line may of course also be calculated starting from a position averaged for both eyes of the viewer. In addition, the position of the user's eyes may be calculated with or without eyeglasses having appropriate reflectors. It should be noted that, with respect to the invention, any mechanisms and methods for determining the position of the eyes may be used.

[0072] The virtual object 301 in the three-dimensional scenario is selected in that the connecting line 504 is extended into the representation space 130 and the virtual object whose virtual coordinates are intersected by the connecting line 504 is selected. A virtual object 301 is then selected for instance by means of a selection indicator 603.

[0073] Naturally the virtual surface area 601 on which the marking element 602 moves may also be arranged in the virtual scenario in the representation space 130 such that from the point of view of the user virtual objects 301 are disposed in front of and/or behind the virtual surface area 601.

[0074] As soon as the marking element 602 is moved on the virtual surface area 601 such that the connecting line 504 intersects the coordinates of a virtual object 301, the marking element 602 may be represented in the three-dimensional scenario such that, with additional depth information or a change in the depth information, it adopts the virtual three-dimensional coordinates of the selected object. From the point of view of the user, this change is then represented such that, as soon as a virtual object 301 is selected, the marking element 602 makes a spatial movement toward the user or away from the user.

[0075] This permits interaction with virtual objects in three-dimensional scenarios by means of two-dimensional interaction devices that are easy to handle, such as for instance a computer mouse, i.e. by means of interaction devices that have two translational degrees of freedom. In contrast to special three-dimensional interaction devices having three degrees of freedom, this permits simpler and easier to learn interaction with a three-dimensional scenario, since an input device having fewer degrees of freedom is used for the interaction.

[0076] Naturally virtual objects may also be represented in the three-dimensional scenario such that the coordinates of the objects in the virtual scenario appear from the point of view of the user as if the virtual objects are disposed behind the visualization surface of the representation device.

[0077] FIG. 5 depicts a workstation device 200 for depicting a three-dimensional virtual scenario and for interacting with the virtual scenario.

[0078] The workstation device 200 has a representation device 100 having a representation unit 110.

[0079] The representation unit 110 has a first representation region 111 and a second representation region 112, wherein the second representation region is angled, relative to the first representation region, toward the user such that the two representation regions form an included angle α 115.

[0080] With their angled position relative to a viewer position 195, i.e. the eye position of the viewer, the first representation region 111 of the representation unit 110 and the second representation region 112 of the representation unit 110 cover a representation space 130 for the three-dimensional virtual scenario.

[0081] The representation space 130 is thus the spatial volume in which the visible three-dimensional virtual scene is represented.

[0082] A viewer who uses the seat 190 while using the workstation device 200, in addition to using the representation space 130 for the three-dimensional virtual scenario, can also use a workstation region 140 on which additional touch-sensitive or conventional displays may be disposed.

[0083] The included angle α 115 may be dimensioned such that all virtual objects in the representation space 130 are disposed within arm's reach of the user of the workstation device 200. There is good adaptation to the arm's reach of the user in particular with an included angle α that is between 90 degrees and 150 degrees. The included angle α may for instance also be adapted to the individual requirements of an individual user and may thus fall below or exceed the range of 90 degrees to 150 degrees. In one exemplary embodiment, the included angle α is 120 degrees.

[0084] For monitoring air space, the three-dimensional virtual scenario may be depicted for instance such that the second representation region 112 of the representation unit 110 is the virtually displayed surface of the earth or a reference surface in the space.

[0085] Thus the inventive workstation device is suitable in particular for lengthier, low-fatigue processing of three-dimensional virtual scenarios with integrated spatial representation of geographically referenced data, such as e.g. aircraft, waypoints, control zones, threat spaces, terrain topographies, and weather events, with simple intuitive interaction options.

[0086] The workstation device as described in the foregoing and in the following thus permits a large stereoscopic representation volume or a representation region. Furthermore, the workstation device permits a virtual reference surface to be positioned in the virtual three-dimensional scenario, for instance a terrain surface, in the same plane as the representation region or touch unit actually present.

[0087] FIG. 6 is a schematic depiction of a method for selecting virtual objects in a three-dimensional virtual scenario having the following steps: reproducing a marking element in a three-dimensional virtual scenario 601, moving the marking element with two degrees of freedom along a virtual surface area 602, determining a connecting line from one eye of the viewer to the marking element 603, and selecting a virtual object in the three-dimensional virtual scenario by positioning the marking element in a certain region on the virtual surface area 604.

[0088] The object may be selected in that the marking element is moved on the virtual surface area such that the connecting line from the eye of the viewer to the marking element intersects a virtual location of the virtual object to be selected in the three-dimensional virtual scenario.

[0089] FIG. 7 depicts a computer-readable medium 701 having a computer program element for controlling a representation device as described in the foregoing and in the following that is embodied for performing the method for selecting virtual objects in a three-dimensional virtual scenario when the computer program element is executed on a processor of a machine.

[0090] The foregoing disclosure has been set forth merely to illustrate the invention and is not intended to be limiting. Since modifications of the disclosed embodiments incorporating the spirit and substance of the invention may occur to persons skilled in the art, the invention should be construed to include everything within the scope of the appended claims and equivalents thereof