Method and apparatus for presenting two and three-dimensional computer applications within a 3D meta-visualization

[0002] 1. Field of the Invention

[0003] The present invention relates generally to computer system applications, and, more particularly, to a method and apparatus for presenting two and three-dimensional computer applications in a 3D meta-visualization for organizing the display of such applications.

[0004] 2. Description of the Related Art

[0005] Since the inception of personal computers, the sophistication of their processors has increased significantly. Initially, personal computers were typically limited to running only one computer application at a time, thus severely limiting the computer user's ability to perform multiple computer tasks in an efficient manner. However, with the increase in the sophistication of these processors, today's personal computers are able to run a multitude of computer applications simultaneously (i.e., multi-task), thus significantly increasing the productivity of the computer user.

[0006] To manage the vast number of computer applications running simultaneously, the personal computers of today typically run a Windows®-based operating system, which places each computer application in a corresponding application window. Such a Windows®-based operating system could be, for example, Windows® 95 by Microsoft Corp., which is a operating system used by a significant portion of personal computers today.

[0007] Referring to FIG. 1, a personal computer having such a Windows® environment maps a given computer application into an application window 110 on a computer display 130, such that the user interface with the computer application is encapsulated within the window 110. The computer user has the option via controls 140-160 to manipulate each respective application window 110 on the display 130. Such manipulation could result from actuating the MINIMIZE button 140 to minimize the size of the application window 110; actuating the MAXIMIZE button 150 to maximize the size of the application window 110; or closing the computer application altogether via the CLOSE button 160. The user can move the application window 110 by “dragging” the window 110 by its title bar 170 via a user-input device, such as a computer mouse (not shown). Additionally, the user can adjust the size of the application window 110 (i.e., make it larger or smaller) by manipulating a control 180 on the lower right-hand corner of the window 110 via the user-input device.

[0008] Although the current Windows® environment aids the computer user to manage a plurality of computer applications to some extent, it suffers from disadvantages as well. When a significant number of application windows 110 are present on the display 130, they tend to overlap (i.e., some of the windows obscure the view of the other windows). Quite often, a computer user will have to move, minimize or close one or more of the windows 110 that obscure a desired application window in order to view the desired window. This process of “shuffling” the windows 110 can be burdensome and time consuming to the user, thereby decreasing their efficiency in performing a particular computer task. Additionally, with the present Windows® environment, a user cannot group or categorize the application windows 110 according to a particular project or task to be performed by the user. Accordingly, the user, while only desiring to use a subset of the applications running on the computer, might have to “sift” through the undesired application windows 110 to retrieve the desired applications to perform a particular task.

[0009] The present invention is directed to overcoming, or at least reducing the effects of, one or more of the problems set forth above.

[0010] In one aspect of the present invention, a method is provided for organizing computer applications in a three-dimensional perspective. The method includes creating at least one two-dimensional bitmap for at least one computer application; creating at least one three-dimensional geometry; mapping bits from the at least one two-dimensional bitmap to the at least one three-dimensional geometry; and displaying the at least one three-dimensional geometry with the bits mapped thereon.

[0011] In another aspect of the present invention, an apparatus includes a processor capable of running at least one computer application. The apparatus further includes a sub-processor capable of creating at least one two-dimensional bitmap for at least one computer application running on the processor, creating at least one three-dimensional geometry, and mapping bits from the at least one two-dimensional bitmap to the at least one three-dimensional geometry. The apparatus further includes a display capable of displaying the at least one three-dimensional geometry with the at least one computer application mapped thereon.

[0012] Other objects and advantages of the invention will become apparent upon reading the following detailed description and upon reference to the drawings in which:

[0013] FIG. 1 illustrates a typical prior art Windows® environment for managing various computer program applications on a computer display;

[0014] FIG. 2 is a block diagram of an apparatus for implementing the organization of multiple two and three-dimensional computer applications in a 3D meta-visualization;

[0015] FIG. 3 shows a conventional prior art process for the display of a 2D and/or 3D application on a display screen;

[0016] FIG. 4 shows a plurality of application windows that are mapped into a 3D meta-visualization in accordance with one embodiment of the present invention;

[0017] FIG. 5 illustrates a process for mapping the 2D or 3D applications in a 3D meta-visualization in accordance with the present invention;

[0018] FIG. 6 shows the mapping of a plurality of applications on a three-dimensional geometry in accordance with another embodiment of the present invention;

[0019] FIG. 7 shows a plurality of three-dimensional geometries upon which the application windows are mapped to permit distinct groupings of computer applications in accordance with yet another embodiment of the present invention; and

[0020] FIG. 8 depicts a flowchart implemented by a processor for mapping the various computer applications in a 3D meta-visualization in accordance with one embodiment of the present invention.

[0021] While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawings and are herein described in detail. It should be understood, however, that the description herein of specific embodiments is not intended to limit the invention to the particular forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.

[0022] Illustrative embodiments of the invention are described below. In the interest of clarity, not all features of an actual implementation are described in this specification. It will of course be appreciated that in the development of any such actual embodiment, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which will vary from one implementation to another. Moreover, it will be appreciated that, even if such a development effort is complex and time-consuming, it would nonetheless be a routine undertaking for those of ordinary skill in the art having the benefit of this disclosure.

[0023] Turning now to the drawings, and specifically referring to FIG. 2, a conventional apparatus 200 is shown. The apparatus 200 may be modified and programmed as discussed below to implement the invention to organize various two- and three-dimensional computer applications within a 3D meta-visualization. The apparatus 200 may be a conventional personal computer (“PC”) available from a variety of manufacturers. The apparatus 200 comprises a processor 210, such as a Pentium® processor from Intel Corp., which can run the various two and three-dimensional computer applications based upon instructions from computer application programs stored in a storage device 220. The storage device 220 may be embodied as a combination of various devices, such as a disk drive for accessing the computer applications from a disk (e.g., a conventional floppy disk, CD-ROM, digital video disc “DVD”, etc.), a hard drive, or the like. Alternatively, the computer applications may be accessed by the apparatus 200 via a communications medium (not shown) from some other computer source, such as a computer network, as opposed to being accessed from the storage device 220.

[0024] The different types of computer applications running on the processor 210 may be from a wide array of commercially available software. For example, the computer application may be text-based, such as word processing, spreadsheet, or e-mail applications, and, may also be two or three-dimensional graphics-based applications, such as computer games or design-related software. Of course, several other commercially available computer application programs could also run on the processor 210, and, thus, need not be limited to the aforementioned examples.

[0025] The processor 210 is capable of running a certain number of these computer applications simultaneously and displays such applications on a display 130 for visual presentation to a user. The apparatus 200 also includes a graphics rendering unit 250 for generating 2D or 3D graphics applications running on the processor 210 for output on the display 130. The 3D graphics rendering unit 250 has associated therewith a texture memory 260 for applying textural images to displayed graphics scenes, such as three-dimensional graphics scenes, for example. The manner in which the graphics rendering unit 250 and texture memory 260 create such graphics will be better understood as the detailed description proceeds.

[0026] The apparatus 200 further includes a user-input device 240, which permits the user to interact with the various computer applications running on the processor 210. In the illustrated embodiment, the user-input device 240 includes both a keyboard and a computer mouse. However, it will be appreciated that the user-input device 240 could be embodied as various other types of input devices, such as, for example, a trackball, touchpad, tablet, touchscreen, voice recognition via a microphone, joystick, etc. without departing from the spirit and scope of the present invention.

[0027] Turning now to FIG. 3, a conventional process for displaying a two or three-dimensional application on the display 130 is shown. The processor 210 shown in FIG. 2, with the 2D or 3D application running thereon, generates 2D or 3D graphics primitives, and sends these primitives to the graphics rendering unit 250. The graphics rendering unit 250 then renders the 2D or 3D graphics onto a bitmap 310. The bitmap 310 is a two-dimensional array of pixels, which in this particular example represents the 2D or 3D computer application that is to be displayed on the display 130. When necessary, the processor 210 will apply or “draw” the bits from the bitmap 310 to a frame buffer 330 for displaying the 2D or 3D application on the display 130. Such instances when the processor 210 transfers these bits from the bitmaps 310 to the frame buffer 330 is upon initialization of the 2D or 3D application or when the application needs to be updated on the display 130. Updating of the displayed application could occur as a result of a predetermined action performed by the user via the user-input device 240 or as required by the 2D or 3D application itself. The process by which the processor 210 creates the bitmap 310 for each corresponding 2D or 3D application and transfers specific bits to the frame buffer 330 for displaying the application is well known to those of ordinary skill in the art. Accordingly, a more detailed process will not be discussed herein to avoid unnecessarily obscuring the present invention. Alternatively, it will be appreciated that certain applications would not require the formation of the bitmap 310, but the graphics rendering unit 250 may transfer the bits directly to the frame buffer 330.

[0028] For generating the 3D graphics, the graphics rendering unit 250 performs mathematical calculations, also in accordance with the prior art, from the 3D graphics application running on the processor 210, to determine how points within a 3D space correspond relative to the positioning of a “virtual camera” within such 3D space. The graphics rendering unit 250 then maps (i.e., projects) these points, representing the 3D space and the objects contained therein, onto the 2D bitmap 310. That is, the graphics rendering unit 250 will project the 3D space onto a 2D surface (i.e., the bitmap 310) for eventual display on the display 130. When necessary, the graphics rendering unit 250 will recalculate the points within the 3D space for changing the 3D scene displayed on the display 130. Such recalculation may occur as a result of the 3D graphics program running on the processor 210, or interaction with the 3D application program via the user-input device 240. The specific process by which the graphics rendering unit 250 calculates the points within the 3D space, projects these points onto the bitmap 310, and applies bits from the bitmap 310 to the frame buffer 330 for display of the 3D graphics scene is well known to those of ordinary skill in the art. Accordingly, a more detailed process will not be discussed herein to avoid unnecessarily obscuring the present invention.

[0029] Turning now to FIG. 4, a perspective depicting the transformation of conventionally displayed application windows 110 to a 3D meta-visualization of the computer applications on the display 130 in accordance with the present invention is shown. The application windows 110 are conventionally generated on the display 130 via the process of FIG. 3, as previously discussed. The computer applications present within these conventional windows 110 could be either two or three-dimensional, and could essentially include any computer application program that is compatible with the processor 210. These application windows 110 would appear on the application workspace 120 of the computer display 130 similar to the arrangement as illustrated in FIG. 1.

[0030] To alleviate the aforementioned problems associated with the conventional Windows® environment, the present invention employs a technique for providing these conventional application windows 110 in a 3D meta-visualization. In accordance with one embodiment, these application windows 110 are mapped into a three-dimensional environment such that the contents of each window 110 are transformed into a respective “3D” application window 420 as shown on the display 130 of FIG. 4. To accomplish such 3D meta-visualization of the computer applications on the display 130, an “off-screen” rendering technique is employed where the bitmaps 310 of the two and three-dimensional applications are applied to the “3D” application windows 420.

[0031] In conventional 3D graphics, “texture” (i.e., images) from bitmaps in the texture memory 260 are mapped onto three-dimensional geometries, known as “primitives” generated by the graphics rendering unit 250. The process of applying this texture to such primitives, known as “rendering”, is well established in the art. Accordingly, the specifics of such process of rendering will not be discussed hererin to avoid unnecessarily obscuring the present invention.

[0032] FIG. 5 illustrates the technique of the present invention in one particular embodiment to apply the contents of the application windows 110 of FIG. 4 to the “3D” application windows 420 to implement such 3D meta-visualization. The bitmaps 310 from the two and three-dimensional computer applications (as obtained from the process of FIG. 3) are disposed in the texture memory 260 of the apparatus 200. The graphics rendering unit 250 generates a series of primitives for applying the bitmaps 310 of the two and three-dimensional computer applications that are to be mapped thereunto. In one embodiment, the primitives generated by the graphics rendering unit 250 are a series of “planes” as shown by the 3D application windows 420 in FIG. 4. However, it will be appreciated that the 3D graphics rendering unit 250 could generate other 3D geometries (i.e., different polygonal-shaped primitives) for mapping the applications thereon and not necessarily limited to the 3D planes as shown in FIG. 4.

[0033] Once the three-dimensional geometries are generated, the graphics rendering unit 250 applies the bits from a particular bitmap 310 (corresponding to a particular 2D or 3D computer application) to a surface of the 3D geometry (e.g., the plane 420) using the conventional graphics rendering technique as previously discussed. The graphics rendering unit 250 then transfers the “3D” application window 420 to the frame buffer 330 for subsequent display on the display 130.

[0034] The mapping between the bitmaps 310 (of one coordinate space) and the “3D” application windows 420 (of another coordinate space) by the graphics rendering unit 250 occurs bi-directionally. When the user interacts with the computer application of the “3D” window 420, the computer application itself will know the precise point of manipulation as if the user had interacted with the same computer application as if it appeared in a conventional application window 110. That is, as the graphics rendering unit 250 is able to map particular points onto the “3D” window 420, it can also map particular points of the “3D” window 420 back into the bitmap 310. Therefore, if the user interacts with the computer application at a particular point within the “3D” window 420, the graphics rendering unit 250 will be able to translate this point to the corresponding point of the associated bitmap 310 for processing by the corresponding computer application to that bitmap 310. The manner in which the graphics rendering unit 250 accomplishes such bi-directional mapping is well known to those of ordinary skill in the art. Accordingly, the specifics of such process will not be disclosed herein to avoid unnecessarily obscuring the present invention.

[0035] In typical operation, the user interacts with the “3D” window 420 utilizing the user-input device 240 (e.g., a pointing device such as a mouse). As practiced in the art of 3D graphics programming, the present invention can process the 2D events generated by the user-input device 240 and project their 2D position into the 3D graphical space to detect when the pointer (from the user-input device 240) is over the 3D object textured with the application window 110. This “ray-picking” process can identify not only the 3D object, but can also determine the pointer's position in the coordinates of the texture of the 3D geometry to which it is applied. Since this 2D texture coordinate corresponds directly to the 2D coordinates of the application window 110, the present invention can easily synthesize another pointer event, this time in the coordinate space of the application window 110, and send that event to that window, as is well known in the art of 2D graphical user interface programming. When the application window 110 receives this event, it will typically behave as if the pointer event occurred over the actual window 110, when in fact it originally occurred in the “3D” application window 420. If the event causes any change to the appearance of the application, then the bitmap 310 and correspondingly the texture is updated.

[0036] Turning now to FIG. 6, an alternative embodiment of the 3D meta-visualization is shown. In this particular embodiment, the contents of the bitmaps 310 could be mapped onto the surfaces 620 of a three-dimensional cube 610 by the graphics rendering unit 250. The user could view all the surfaces 620 of the cube 610, and, thus, all of the computer applications mapped thereon, by rotating the cube 610 in a predetermined direction via the user-input device 240. This may be accomplished by receiving a first predetermined input at the user-input device 240 to cause the cube 610 to rotate horizontally. Or, alternatively, receiving a second predetermined input that may cause the cube 610 to rotate vertically. Such rotation could be accomplished by animating the movement of the cube 610 either horizontally or vertically, which could be achieved using techniques that are well established in the art of computer graphics.

[0037] The 3D cube 610 could appear anywhere within an application workspace 120 of the display 130. The user could further have the option to move the cube 610 anywhere within the application workspace 120 by “dragging” the cube 610 via the user-input device 240, if so desired. The user could also cause one of the application windows 620 of the cube 610 to occupy the entire application workspace 120, by performing a predetermined input via the user-input device 240. For example, by performing a right mouse button click via the user-input device 240 on one of the surfaces 620 of the cube 710, may cause the computer application appearing on that surface 620 to occupy the entire application workspace 120 of the display 130. The user could then subsequently reallocate that particular application back to one of the surfaces 620 of the cube 610 by right “clicking” on the application (in the application workspace 120) and, then, right “clicking” again on a surface 620 of the cube 610.

[0038] The user could also be provided with the capability to have the cube 610 essentially “fold-out” into two-dimensions, such that the application workspace 120 of the display 130 is divided equally into six windows, with each of the six surfaces 620 of the cube 610 being present in each respective window. The user could then have the two-dimensional representation of the six application windows “fold-up” into the three-dimensional cube 610 for conserving work space on the display 130.

[0039] Referring to FIG. 7, instead of having one cube 610 displayed, an additional cube 710 can also be provided to arrange various other computer applications on its surfaces 720. Although the cubes 610, 710 are shown in the illustrated embodiment, it will be appreciated that other three-dimensional geometries could be used in lieu thereof and not necessarily be limited to a cube. It will be further appreciated that any number of cubes 610, 710 (or other 3D geometries) could be used and need not necessarily be limited to two.

[0040] The use of multiple cubes 610, 710 permits the user to group and, thus, organize the various computer applications running on the processor 210. For example, the applications appearing on the surfaces 620 of the cube 610 could be applications pertaining to the user's preparation of a trip expense report. That is, one of the surfaces 620 could include a spreadsheet application for itemizing various expenses during the trip. Another surface 620 may contain a word processing document for actually preparing the report. And, other surfaces 620 may include e-mails pertaining to the trip. With such an arrangement on the cube 610, the user would have quick access to all of the computer applications necessary to complete his or her trip report. That is, while the user is typing the report in the word processing application, he or she could quickly access the spreadsheet itemizing the various expenses of the trip, and, perhaps, cut and paste these itemized expenses from the spreadsheet to the word processing document with relative ease. The cube 710, on the other hand, could be used to organize the user's e-mail application. For example, one of the surfaces 720 of the cube 710 may be used to display the list of the user's “received” e-mail, while other surfaces 420 may include “sent”, “draft”, etc. e-mails.

[0041] Turning now to FIG. 8, a process 800 executed by the processor 210 for implementing a 3D meta-visualization in one particular embodiment is shown. The process 800 commences at step 810 where the processor 210 launches a plurality of computer applications as designated by the user via the user-input device 240. For example, the user may elect to simultaneously run on the processor 210 an e-mail application, a word processing application, and a graphics-based application, etc. A bitmap 310 for each respective launched computer application is created at step 820. At step 830, the processor 210 summons the graphics rendering unit 250 to create a 3D geometry to map a particular application to a particular surface of the geometry. The 3D geometry could take the form of any one of various three-dimensional geometries. For example, it may take the form of a plane as shown by the application windows 420 of FIG. 4. Or, perhaps, could take the form of a cube 610 as shown in FIG. 6. The user could be given the option to select the particular 3D geometry, to have the applications mapped thereupon, via a “pull-down” menu, if so desired.

[0042] At step 840, the graphics rendering unit 250 engages in an off-screen rendering process to apply (i.e., texture) the bitmap 310 that corresponds to a particular launched application onto the 3D geometry. The manner in which the processor 210 accomplishes such off-screen rendering is accomplished by disposing the bitmaps 310 into the texture memory 260 and applying bits from the desired bitmaps 310 to a 3D geometry created by the graphics rendering unit 250.

[0043] The 3D geometry is then displayed on the display 130 with the 2D and/or 3D computer applications mapped thereon at step 850. The user can view the different 2D or 3D applications on the 3D geometry by manipulating the orientation of the geometry on the display 130. For example, if the 3D object were a cube, the user could have the cube rotate either horizontally or vertically to view all sides of the cube. The manner in which the user could cause such manipulation of the cube could be by providing predetermined inputs to the processor 210 via the user-input device 240. If the user desires to manipulate the orientation of the 3D cube, the process continues to step 865, where the processor 210 re-orients the 3D geometry to the user's preference. If not, the process 800 continues to step 870, where it is determined whether the user desires to manipulate or interact with a particular computer application mapped on the 3D geometry. If not, at step 875, the desired task is processed by the processor 210 and the process 800 reverts back to step 860. If the user desires to interact with a particular computer application in step 870, the process 800 continues to step 880, where the user's requested task is transformed from the coordinate space of the 3D geometry to the coordinate space of the 2D bitmap 310. Subsequently, at step 890, the desired task is processed by sending a signal to the corresponding computer application.

[0044] The particular embodiments disclosed above are illustrative only, as the invention may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. Furthermore, no limitations are intended to the details of construction or design herein shown, other than as described in the claims below. It is therefore evident that the particular embodiments disclosed above may be altered or modified and all such variations are considered within the scope and spirit of the invention. Accordingly, the protection sought herein is as set forth in the claims below.